Are measurement & creativity incompatible?

The old mantra “What gets measured gets managed” used to be reserved for suited middle managers looking to bring order to chaotic businesses. It’s also used as a warning. A warning that those who focusing on numbers rather than actual improvement can become fixated on data and lose sight of the original goal. It’s known as Goodhart’s Law and ROI is a great example. If you wanted to increase your ROI, just cut stuff. Focus on smaller audience, target specific locations, stick to known strategies and spend efficiently right up to your budget limit. Then calculate your ROI. Even better, compare it to the last campaign. Congratulations, you’ve chased efficiency, not reach. Your client’s brand didn’t reach a tipping point it needed for true fame, but you did increase a number on a spreadsheet.  

When measurement became the only game  

It’s a worthy warning, but Goodhart’s Law only really becomes possible when an ecosystem is mature enough to be measured in the first place. OOH is a prime example. For the longest time, much of the OOH supply chain wasn’t really measured. The measurements were physical. Did we spend all the budget? How many 6-sheets were posted? Did the media owner say they delivered to the media plan? The actual success metric was less tangible. Sales uplift was, and still is, annoyingly difficulty to attribute to OOH, specifically during an omnimedia campaign. Some ads are better than others, some brands seemed to resonate more, but the beautiful vagueness was accepted. That’s just how it was. 

Then came online and the annoying clarity of clicking a banner and directly tracking an affiliate link through to purchase. Suddenly, measuring clickthrough rates was a proxy for success. Suddenly, the measurement game was the only game. Clients wanted proof. Proof made them spend more.  

Online measurement and effectiveness definitely has its issues, but OOH, to a greater extent, benefitted from the best bits and side-stepping the worst. We now measure everything from real-time mobile proximity data to programmatic budget burn rate. Again, clients wanted proof. Proof made them spend more. Inevitably, once everything else has been measured, the laser Eye of Sauron has finally turned its gaze towards the role of the creative itself. It previously avoided any form of scientific rigour due to its notorious, well, er, ‘unmeasurabilityness’.  

But that’s not as true as some thought (or hoped). Any half-decent designer knows basic use of layout, alignment, colours, contrast, fonts, product placement and call-to-actions, can easily make an ad look interesting, classy, or cheap. It’s not 100% science, but we can detect the presence of elements and infer their relative effectiveness. Bonus points are added for deploying sneaky behavioural tricks. Add a face, emotions, a joke, alliteration, a known logo… even a mistake, and the creative becomes measurably more effective. Unsurprisingly, there are companies devoted to measuring and codifying creative effectiveness. System1 are doing a roaring trade testing creative on real people. Why? So agencies can give clients proof. 

Subjectivity is fine, but let’s get some science in 

With the fast advance of AI, we can train synthetic users to look at the creative and tell us how to improve it. We can even iterate the design in a feedback loop until it’s optimised for each audience. Meta have taken this a step further, they have created an AI simulation of the brain’s response. Meta Tribe v2 literally simulates a real-time MRI scan. If your creative doesn’t light up the right pleasure receptors, back into the feedback improvement loop it goes.  

As a designer and practicing geek, I fully support bringing some science to the subjectivity gunfight. No designer likes that nagging feeling when the creative isn’t as good as it can be. As long as the quest for optimisation doesn’t end up in Goodhart’s Law territory, we’re good. We’ve lost when the quest for perfection means we end up with effective but samey ad-slop. As the proverb goes “In the kingdom of the blind, the one-eyed man is king”. Advertising stands out when it’s different, when it challenges the norm. Measurement and uniqueness (aka creativity) are not incompatible, so let’s measure it. Let’s get that proof, because when clients get proof, we know what happened next…