Performance Baselines

  • Before setting out, we need to measure the performance of simple solutions as a baseline to compare against

Measurement is, by far, the most important tool for writing fast programs.

Without measuring, you're just guessing. Consider a modern CPU, with multiple tiers of cache, branch prediction, and parallel execution -- not to mention how the many other components of your system, each complex itself, interact with each other -- it all renders intuition basically worthless.

The strength of my conviction about the importance of measuring comes from how often the measurements I take surprise me.

"I always get it wrong, it's difficult," Carl Cook, at the time a developer for the high frequency trading firm Optiver, said in his 2017 talk at CppCon, "When a Microsecond is an Eternity: High Performance Trading Systems in C++."

Cook also told the audience, who had gathered to discuss the intricacies of C++ programming, that measurement trumps coding ability:

I'm definitely not a C++ expert. there's people in this room who know 10x more about C++ than me. It's not necessarily a problem, though, you just need to be really good at measurement. You need to be really good at applying the scientific method. You need to understand the tools. If you can do that, and you can measure, and you can measure well, you can write very, very fast software without knowing the standard inside out.

In that spirit, we begin our work not by launching into the most advanced techniques, but with careful measurement of some simpler approaches.