I’ve spent nearly a decade hanging around the MMM space, and if there’s one thing I can tell you it’s that MMM starts at the model, and proves itself in the product and growth. An MMM without growth is like a car without petrol. It simply doesn’t work.
With the rise of open-source MMM and the flood of cheap new wrapper vendors coming around the space, we’re seeing increasingly that MMM is moving from a simple to a complex category to navigate with an over-abundance of choice. The problem is a lot of these new entrants are not solving core product problems that allow MMMs to unlock growth clearly and fundamentally from the market.
Unless we get the core of MMM working as a growth co-pilot, it’s doomed to fail. And as such, we must focus and elevate our expectations of MMM beyond modelling libraries if we are to make them work at scale as genuinely impactful products to marketers.
At its core, MMM is an observational technique. Claims of causal MMMs are ignoring the product problems that exist and are marketing fads that try to sucker in marketers. Marketers should see through this and start thinking through MMM from the perspective of ‘what model do I want to use?’
And marketers are falling for fads that are pushed by vendors rather than elevating the question to ‘what product will unlock and drive growth in my business?’
To give an example, I think incrementality testing is overused to power and calibrate MMMs and a fad most vendors are jumping on.
Let me be clear: incrementality testing is a great way to validate models and their performance, and to hold them to account. But having them power a model solely? That’s a short-sighted approach, pushed by inexperienced experimentation companies looking to triple down into econometrics to satiate growth ambitions. Of course, many companies, who may benefit from incrementality testing, are advocates for powering MMM only via this tool. Time will tell if the alliance between novice MMM vendors powered by platform-led research will play out well. There is nuance required here to make this work long-term.
But MMM has not been a question of if we can deploy a model for some time. That’s basic stuff. It’s also not a question of ‘can we run a few experiments to power a model’. Also really basic stuff.
Instead, people passionate about growth and MMM need to elevate the conversation to focus on solving the deep product problems in the space that have stopped Enterprise from getting genuine, deep and sustained value from MMM programs. The primary issues that plague MMM, and must be solved to actually get models to work, are quite simple to articulate but far harder to solve.
Firstly is the ability to generate clean, well-organised data without killing yourself in data engineering. We’ve all been there. MMM vendors demand data in clean, structured formats or data warehouses. We’ve all been told that’s easy enough to do, and then get killed by the hidden cost of hiring data engineers, maintaining data pipelines and ultimately seeing garbage data in and out.
Secondly is the ability to deploy a strong model that solves for the structural data issues that typically exist if you model in-house or can only test a model on one Enterprise’s data. These are the problems of multi-collinearity and endogeneity. These are structural problems in MMM. Because lots of data is weekly at best (though I wish daily became the standard), these issues exist. Either MMM must go to daily data to break this cycle further, or we must build smarter models that can defeat these issues.
Thirdly is how someone gets insights from the MMM product. Budget optimisations are one very simple part of the ecosystem. Optimisations, recommendations, tactical insights, reporting justifications, explanations are all key to doing MMM at scale. It’s clear from the volume of customer success teams and bespoke consulting that plagues the industry that we have not done enough to build world-class product in these areas.
So, how are we going to solve this?
I think the next generation of MMM companies need to forget the term and category MMM. It’s a limiter and is keeping us obsessed with the models, not with the customer and the product problem. What people want from MMMs is a simple product challenge: trusted, reliable and interpretable answers.
Instead we must champion the Growth category. After all, that’s the product proposition good MMM should deliver: Growth with a capital G.
For example, every MMM vendor should have a way to deliver clean data at scale. We built this some time ago with DataOS, which led to the explosion of MMM across the APAC region as every single marketer finally was able to unlock and unify their data into MMM. Blake Rand (now Global Head of Insights at Dominos Pizza Enterprises) said at the time: “It’s really easing that tension point around the data wrangling and ingestion that’s made these projects hard in the past.”
Every MMM vendor should have a sophisticated model that is transparent on testing and rigorous in its ability to deliver in the real world. It’s not enough to have a few shoddy experiments powering an MMM. There should be a robust data pipeline that is assessing out-of-sample fit, stability, collinearity and endogeneity issues. Furthermore, models should be automatically validating against experimentation done, instead of using experiments to hack models. Transparency is key and we must move to a set of modelling platforms that are backed by robust testing, delivered transparently. That’s why we have been focusing so hard on building robust testing suites that our customers can access. It’s simply too important not to do.
And every MMM vendor should be focused on making answers easier. When you’re building bespoke data architectures this is hard. But getting answers, having them clearly presented in ways that marketers can consume them, means we can finally have tooling that gets marketers to unpack and answer questions in real time. We’ve focused on making the UX beautiful and easy to use for a marketer. After all, products built and aimed only at the data scientist or analyst will only serve to create more work in teams that are already backlogged and worn out. We simply must elevate the product level – creating beautiful growth products everyone can use – to truly challenge and turn the MMM category into a growth category.
A great example of this comes from Cam Luby at Optus. “I really love looking at the GrowthOS platform, but when you’re looking at a chart first you need to do the analysis – what is this telling me? And then you have to formulate a hypothesis – what should I do about it?”
Shifting the dial must be platform first. It simply must be answers first. Our job is not to deliver a model, but to make answers and growth easier and faster for marketers who are beset by increasingly complex and fragmented media landscapes.
Of course, it’s easy for me to say all of this, but this is exactly the future of what we are delivering at Mutinex. It’s why myself and my team are obsessed with building a Growth Co-pilot to power the future of growth, with an end to end system that can collect, process and analyse your data in near real time. And it’s why I believe in a call to arms to the MMM industry to forget trying to ship basic models, and champion the thing we all need to deliver to ensure our industry is sustainable and our future bright.
Ultimately, the way we believe we can deliver growth is by reinventing MMM at the product level into a growth product. And we believe our job is to do that by making growth questions answerable through trust, speed and little to no cost.
Nothing else matters.