In-House Tests (LLO Archive)

Created 2025-12-09, last modified 2025-12-09. Visibility: public

Part of my archive of Layover Linux Official posts on Tumblr.

2025-10-26

I always get my best programming compulsions when I'm too ill to stare at a laptop screen.

I'm more satisfied with utest.h as a test framework than anything else I've used before, but it's still a compromise for my use case. There are still some things where I would benefit from writing my own test framework.

• The output format is pretty verbose by default, making it annoying to zero in on test failures.
• It's awkward to extend in Prone-specific ways that would be immensely valuable. For example, I'm constantly having to write boilerplate for comparing dynamic objects. utest.h assert macros aren't designed to work in helper functions.
• Because it's header-only, utest.h is cheap for a small number of cases/assertions, but you end up with a ton of complex macro-expanded code being redundantly compiled after awhile. This and the previous problem could be solved with a small header separate from a compiled-once test harness full of function defs.
• It's hardcore, but there are certain benefits to having one test case per file. For example, your filename is implicitly your test case name. utest.h is a bit too heavy a header to pull this off. More reason for separate h/c.
• It would be really useful to be able to pick at compile time "run this one test", so that if your build has multiple failures, you can focus them down individually.
• I just generally dislike using ARGV to control the test suite behavior. It increases the weight of what we're building, and test order randomization should probably always be on.
• I very deeply don't care about C++ or MSVC support, but I'm paying for that complexity.
• I'm somewhat empowered to fix issues in my test harness by the fact that I've vendored utest.h, but the complexity makes it hard for me to understand what's actually going on. In practice, modifying this code is very intimidating.
• I don't use, or I'm already moving away from using, harness-level batching of subcases/fixtures. It's just better to write a test that contains its own setup/teardown/looping logic. A lot of the utest.h code is in support of these features.

Maybe I could think of more later, but you get the idea. I'm equally bothered by the things utest.h does do that I don't want, and the stuff it doesn't do that I do want. So it's not a priority to roll my own, but I do think it's inevitable.

2025-10-26

Hours later and I'm still laying in bed with vertigo and bikeshedding myself in the theater of the mind. While knowing damn well I'm one relatively small boring commit away from a much more important, totally unrelated milestone.

The real torture of a migraine isn't the headache, or the nausea, or ten minutes of not being able to see around the magic pixie brain sparkles. It's the feral, helpless boredom that slowly rips you apart from the frontal lobe backwards.

2025-10-27

A day later and I am securely back on my bullshit.

Not only have I eaten the vegetables I've needed to re Construct representation, at least for now, I'm getting to start tinkering with building a test suite harness that I actually own, and want to own. It's very similar from a usability standpoint most of the time, but as a nice bonus... it's already faster.

You can see because I've been using the Atoms tests as a proving ground, and I've kept the utest.h version in coexisting with their new in-house counterparts. Some tests have dramatic differences in execution time, like atoms_store being cut to a third of its former self. The reasons are pretty simple. We're reusing our comparison functions, and the macro wrappers around them are very thin. This generates less and better code.

It's been neat to figure out how utest.h was working, and which secrets to steal. Right now I'm borrowing its clock implementation, but I've got my own macros for registering tests, which works a lot more simply than utest.h can. This taught me some cool stuff about compiler-specific attributes like constructor in GCC.

One area where there's already difference is DV comparison. Dynamic values usually need a complex comparison, and need special printing logic, because in isolation they're just opaque u64s that are often just pointers to memory in a dead process. The dv_atom test in my custom test framework is literally doing more work than its old "equivalent", because the old one relied on a special case where the two DVs could be compared as numbers (the output was still kinda bad though). The new one still usually finishes faster because of removed overhead in other places.

These type-specific helpers are also going to be very helpful for typecodes, which can be printed as their string versions instead of opaque constants. Or in both forms next to each other! Sooo many tests are going to be easier to write, faster to compile and execute, and produce better output. I'm very excited!

Oh and one more thing. I currently control this with a compile time constant, but PRN_TEST_VERBOSE turns on showing successful test cases (still just one per line, taking half the vertical real estate of utest.h). By default, running the test suite only shows failing tests, which helps you cut through noise quicker.

CHARLOTTEWORLD Webring!