Faster Than Criterion (LLO Archive)
Created 2025-11-30, last modified 2025-11-30
Part of my archive of Layover Linux Official posts on Tumblr.
2025-08-26
Stayed up way too late, I'm gonna be fucking toasted in the morning, but I had to work out some details with a partner of mine to keep him safe from an unsafe situation.
Anyways, since I had to stay up anyways, I worked a bit on Prone, including trying out utest.h as a quicker-compiling alternative to Criterion. Verdict so far, is it fucking slaps. Obviously the test suite scale isn't in the same ballpark yet, but the early results with a few representative tests are 350ms vs 4.4s compile times. The execution time is excellent for both, but utest.h does by default print how many nanoseconds each test took, which is delightful and also a motivator for me to optimize more later. One thing I really want to do is switch the allocator abstraction to something that can be changed at runtime, so I can exercise (and now, see timings for) the untracked mode that's default in non-debug builds, like the REPL. When you're seeing test execution times in nanoseconds, it makes you really think about the overhead of the tracking allocator, and want to know how fast a release-optimized version would be.
So yeah, I'm going to be incrementally converting tests over to the new tech and eventually removing Criterion use from the codebase entirely. Criterion has nice features, but no juice worth the squeeze for my use case. And in the process, I got this working for all current types:
prn> op::is_equal("test", "other")
FALSE
prn> op::is_equal("test", "test")
TRUE
Operator parsing is gonna be blissful when it lands, I tell ya. But this is going to go a long way to fixing the parse tests as I migrate them over. They were based on the string representations of objects, and that's just... very fragile, especially since the repr code has holes in it. Overall, the testing is getting a lot more vigorous with this new round of changes, and that feels freaking good.
