[Dev Diaries] Code Coverage?
As you probably know by now, I'm fairly obsessed with tools that give me metrics about quality: linting, docs, tests...
Unfortunately, code coverage is fairly hard to get with SPM in a way that is usable.
What?
Code coverage is the amount of code in the package that is covered with your tests. If you run your tests, are these lines run? Are those? Your tests pass, and that's fine, but have you forgotten to test anything?
You can enable it in Xcode using the option "gather code coverage" in the scheme you are running tests for, which allows you to find a decent visualization if you know where to look for it (a new gutter appears in your code editors).
In Swift Package Manager though, it's fairly obscure:
- first you have to use the
--enable-code-coverage
option of the testing phase - then you have to grab the output json path by using
--show-codecov-path
- then you get a collection of things that is unique to SPM, and therefore unusable elsewhere
Now, if you look closely at the output, you can see it's fairly close to the lcov
format, which is more or less a standard.
Let's make a script!
Because I'm very attached to my packages running on both Linux and MacOS, I need to grab the correct values from the environment (makes them dockerizable too).
I need:
- the output of the
swift test
phase llvm-cov
which is used by the swift toolchain and can extract usable information- A few frills here and there
Looking here and there if anything existed already, I stumbled upon a good writeup setting some of the bricks up. I would suggest reading this first if you want to get the nitty gritty details.
Reusing parts of this and making my own script that can spit either human readable or lcov-compatible, and can work either on Linux or MacOS, and dockerizable, here's what I end up with:
codecov.sh
spits something like:
Filename Regions Missed Regions Cover Functions Missed Functions Executed Lines Missed Lines Cover
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SEKRET.swift 67 20 70.15% 23 4 82.61% 157 27 82.80%
Extensions.swift 15 3 80.00% 2 0 100.00% 20 0 100.00%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 82 23 71.95% 25 4 84.00% 177 27 84.75%
codecov.sh lcov
spits the corresponding lcov output.
Hurray for automation!