Criterion Begone! (LLO Archive)
Created 2025-11-30, last modified 2025-11-30
Part of my archive of Layover Linux Official posts on Tumblr.
2025-08-30
I am happy to report that, with only a few side quests and distractions, I'm finally fully migrated off of Criterion for the test suite in Prone. This has pros and cons, but the pros are far more important to me!
For example, Criterion has better isolation between running tests. As far as I can tell, tests are run in some kind of spawned thread or subprocess, so it can better report errors from catastrophic failures. That's neat, but actually makes it harder to debug in GDB should I want to - not impossible, but reliably a pain for a feature I don't value higher than "that's neat."
In contrast, a cold build of the entire test suite is now 2 seconds of parallel builds and a 0.04s merge of the object files. And that's on my 2013 desktop. Criterion builds were taking over 4 seconds. This is in the worst case btw, but most of the time I'm iterating on a single test file and ONLY building that. Feedback loop is incredibly fast now, test times are giving me hints about what I need to optimize for runtime performance, and there's still low hanging fruit to make compiles even faster (shaving the headers a bit, index the parse tests like I did for lexing already).
It was a heavy migration, but put me in a much better spot for future work, like fixing some naming issues for the native function wrapper type.
