Notes

2017-05-11

It is typical, and often recommended, to use tests when writing programs. Tests allow you to ensure that your program functions as expected and facilitate refactoring code by expliciting its desired behavior. In this note, I would like to present a cheap way to multiply the number of tests you have.

Assume you want to multiply the size of your test suite of N tests by K, one (expensive) way is to write (K-1)N new tests. But if you find K places where your program can be changed and remain correct, you can re-use the N tests of your suite on the K tweaked programs, essentially running KN tests! (In fact, if the tweaks do not interfere, you only need log(K) of them.)

Plenty of handy frameworks are available to let you test your program easily on a bunch of inputs. But I claim that, in view of the previous paragraph, test frameworks should also let you test different variations of your program. So, test framework writers, at your editors!

This observation occured to me when working on the register allocator of my compiler. Indeed, my algorithm is supposed to work regardless of the order in which basic blocks are processed. During my refactoring I found it useful to stress the allocation code by using many different orders.