Match the Measurement to the Decision
Roman Mars mentioned wanting to do a 99% Invisible episode on the Brannock Device. He was looking for a design story. I think it's a strategy one.
The Brannock Device is that metal foot-measuring tool bolted to the floor of every shoe store. One company in Syracuse, unchanged for nearly a century. It captures three dimensions: length, arch, width. Compresses them into one output: your shoe size. No power, no calibration, no updates. Costs $87, lasts 15 years.
Nike acquired a computer-vision startup, built an AR scanner capturing 13 data points at sub-2mm accuracy, and called the sizing system "antiquated." Nike Fit quietly faded from prominence. HP built a 3D scanning platform. Also gone. The Brannock Device is still on the floor.
The reason is structural. Its measurement granularity matches the decisions it informs. Shoes ship in roughly 100 size-width combinations. Sub-millimeter precision doesn't change which box comes off the shelf.
AI evaluation follows the same pattern. LLM benchmarks score models to the decimal point on academic tasks. But the decision most teams actually make is binary: does this model handle our workflow or not. A two-point MMLU gap doesn't change which model you deploy. The measurement granularity doesn't match the decision granularity, and the mismatch costs time, budget, and focus without improving the outcome.
Better measurement isn't more precise measurement. It's measurement that matches the granularity of the decision it informs.
Roman Mars discusses the Brannock Device in the 99% Invisible 15th Anniversary Special. If you're a product manager who thinks about design, 99PI is essential listening.
