In this post, we will discuss an interesting technique for testing test coverage, and the associated Rust crate —
cov-mark
.
The two goals of the post are:
-
Share the knowledge about a specific testing approach.
-
Show a couple of Rust tricks for writing libraries.
This post is an independent sequel to A Trick for Test Maintenance one.
cov-mark
crate provides two macros, hit!
and check!
, which can be used like this:
fn safe_divide(dividend: u32, divisor: u32) -> u32 { if divisor == 0 { cov_mark::hit!(divide_by_zero); return 0; } dividend / divisor } #[test] fn test_safe_divide_by_zero() { cov_mark::check!(divide_by_zero); assert_eq!(safe_divide(92, 0), 0); }
hit!
takes a mark name as a parameter and, at runtime, records that this mark was hit. check!
verifies that the specified mark will be hit when executing the enclosing block.
It creates a guard object which checks the mark in Drop
.
If mark is not hit, check!
will fail the test.
Motivaiton
Such manual coverage marks are complementary to the traditional code coverage tools. The main benefit they bring to the table is that it becomes easier to co-evolve tests and code, marks help with maintenance.
Larger scale projects often suffer from the "infinite test suite problem" — there are just so many tests that finding a specific test for a given piece of code is challenging.
This, in turn, can lead to test bifurcation, where the same code has effectively two independent sets of tests, added at different times.
With marks, if you see hit!(some_mark)
in the code, you can quickly find the corresponding test by grepping for check!(some_mark)
.
This is especially helpful if you are contributing to a new (or a well forgotten) code base.
Another common problem is testing for things that shouldn’t happen. Let’s say that you want to write a test that checks that people, who opted-out of emails, don’t get emails. The test itself is straightforward:
-
Create a user profile with opt-out checked.
-
Do something that would trigger sending an email to the user.
-
Verify that no email was sent.
The problem with this is that there might actually be many reasons why email was not sent:
-
The user might have an empty email address.
-
The email system might be not initialized.
-
Sending emails might be disabled globally for whatever reason.
So, the test might pass by accident even if we forget to actually check the specific opt-out! cov-mark
allows you to check that, during the execution of this test, we’ve actually looked at the checkbox, and that that
was the reason why the email was not send.
These problems become much worse when code changes over time. Even if tests and code live in a single module initially, one day you might move the code but forget to move the tests. When writing new tests, you can test that the test fails if you comment-out tested code. But, if you later change something else, the test might magically become evergreen. Having explicit and tested coverage marks helps with catching such code drift at the moment it happen.
Finally, if you are writing something mission critical, explicit coverage marks might help with checking that every condition and control flow variation have associated tests.
In rust-analyzer , we have been using coverage marks from early on. We use them primarily to avoid the infinite test suite problem, so they are used sparingly, we don’t try to mark each and every branch. The experience has been positive so far — adding the marks is slightly annoying, but they do help with reading the code afterwards, and it has caught a couple of non-trivial problems during refactors.
Implementation
Here’s the full implementation of cov-mark
:
#[macro_export] macro_rules! hit { ($ident:ident) => {{ #[cfg(test)] { extern "C" { #[no_mangle] static $ident: $crate::__rt::AtomicUsize; } unsafe { $ident.fetch_add(1, $crate::__rt::Ordering::Relaxed); } } }}; } #[macro_export] macro_rules! check { ($ident:ident) => { #[no_mangle] static $ident: $crate::__rt::AtomicUsize = $crate::__rt::AtomicUsize::new(0); let _guard = $crate::__rt::Guard::new(&$ident); }; } #[doc(hidden)] pub mod __rt { pub use std::sync::atomic::{AtomicUsize, Ordering}; pub struct Guard { mark: &'static AtomicUsize, value_on_entry: usize, } impl Guard { pub fn new(mark: &'static AtomicUsize) -> Guard { let value_on_entry = mark.load(Ordering::Relaxed); Guard { mark, value_on_entry } } } impl Drop for Guard { fn drop(&mut self) { if std::thread::panicking() { return; } let value_on_exit = self.mark.load(Ordering::Relaxed); assert!( value_on_exit > self.value_on_entry, "mark was not hit" ) } } }
A hit!(mark_name)
is implemented using an atomic counter:
static mark_name: AtomicUsize = AtomicUsize::new(0);
This counter should be accessed by both hit!
and check!
macro, which might live in different modules.
There’s no pure Rust way to make the two to refer to the same name, without explicitly declaring it in some commonly accessible module.
Fortunately, Rust uses the C linkage model, so we can use the linker to do dependency injection for us.
The check!
macro defines a counter using a #[no_mangle]
attribute.
The hit!
macro then declares it using an extern "C"
block.
The linker then weaves the two together.
If we make a typo in the mark name, we’ll get a linker error.
Strictly speaking, this linker trick exists outside of the definition of the Rust language, but sometimes it can be used quite effectively to cut down on dependencies.
It has a couple of serious drawbacks though.
First, mark names must be globally unique.
Second, if we accidentally use an existing symbol name in hit!
, we will try to increment something which might not be an integer at all!
Strictly speaking, the unsafe
block inside the hit!
macro can cause undefined behavior.
However, hitting such a symbol clash by chance without triggering a linker error seems improbable, so I think it is acceptable to not make the caller type unsafe
.
If we really want to fix this, we can add a unique prefix to a symbol name, but that would require a procedural macro.
The body of hit!
is guarded with a #[cfg(test)]
, so no marks are present in the final artifact.
The check!
macro creates a guard object:
let _guard = $crate::__rt::Guard::new(&$ident);
We use
prefix to avoid unused variable error.
We however need to use
_guard
and not just
— a plain underscore will drop the guard immediately, and not at the end of the enclosing block.
Inside the Guard
, we check that the counter was incremented.
In general, especially when tests are run concurrently, we can’t guarantee that the mark will be hit exactly once.
There’s also a slight chance of false negative here if the mark is hit by an unrelated thread.
Inside Drop
, we also check if the current thread is already panicking.
This is a common pattern for asserting drops — we don’t want to mask the original test failure, if any (panicking during a panic aborts the process).
We use Relaxed
ordering to access the counter.
As we are accessing only a single memory location, modification order
is enough.
Note also that all "runtime" code used by our macros is hidden inside mod rt
module.
Specifically, the rt
module ( rt
stands for runtime) also re-export AtomicUsize
from the standard library.
I like to use this pattern to make sure that $crate::__rt
in macros refers to the names I want, without fear that the caller might shadow the std
name.
It also gives a nice overview of the code that is injected into the caller.
That’s all there is for the implementation.
Really, the code in cov-mark
crate itself is not that interesting, it’s the general pattern that matters.
Feel free to use the crate as is, or copy its code it and tweak it to your liking!
We offer consulting services and Rust trainings —contact us for a quote and consider subscribing to our Trainings newsletter ! We also organize a remote embedded Rust conference, Oxidize Global , that will be taking place in July 2020.
以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
我知道他想看什么
沙建军 / 中信出版社 / 2018-1 / 48.00
社交媒体迅速发展、信息快速迭代、时间碎片化;大数据、智能终端、物联网横空出世;移动支付、网红经济和传统营销失效,这些都让这个时代的媒体、内容、渠道、产品之间的边界越来越模糊,也从根本上改变了营销的逻辑,内容营销从热词变成趋势,变成营销的底层思维。未来一切都是媒体,形式也是内容。 本书作者通过对国内外36个内容营销的新近案例的故事化描述和透彻分析,提出“组织媒介化”“营销内容化”“内容情趣化”......一起来看看 《我知道他想看什么》 这本书的介绍吧!
RGB CMYK 转换工具
RGB CMYK 互转工具
HSV CMYK 转换工具
HSV CMYK互换工具