Media Mix Modeling and Optimization Use and Abuse - Limitations and What to Look For

To round off my series of posts on Media Mix Modeling and Media Mix Optimization it is only fair that we discuss the limitations of the approach and warn you of what to look for.
The best way to approach this topic is to understand its components and how they work (at a high level, of course):

1. media mix modeling has to do with being able to determine the contribution of each medium to a business outcome, more specificaly for instance, it is about being able to tie media spend to, say sales or response or web visits
2. media mix modeling is about issuing recommendations for spend allocation across media in order to

• attain a targeted business outcome (e.g. sales) with a minimal budget or

• maximize a business outcome given a predetermined budget

According to this simple classification, a MMM is necessary in order to implement MMO; also, it is clear that MMM is an analytical process that looks into the past whereas MMO projects into the future.
Armed with this simple classification it is easier to understand and discuss the following main limitations of Media Mix Analysis:

• back to the future - this type of analysis is great at describing the past, the more and more granular historical data that is used, the more robust and accurate the model and the better the description of how the different media contributed to the outcome. If a spending plan is fed to the model, it is possible to produce fairly accurate forecasts of the outcome of that plan and an optimization model based on this analysis is extremely likely to produce excellent results. The problem is that the future is not very likely to mirror the past and backward-looking nature of this models does not make them particularly reliable predictors of the future. These models provide great guidance for Marketing spending in industries and environments that are relatively stable and only in the medium term (fiscal quarter at most). The output of these models should be viewed more as directional guidance than as dictum and Marketers must be aware that this guidance can and should be overridden as business conditions change

• bad synergy - accounting for synergy between media is what distinguishes good models from bad ones: the former evaluate the strength of the synergy between media (positive or negative) and the corresponding lag, whereas the latter may ignore this aspect altogether. This is an extremely important element of a good model: mass supports direct and you'll need to know not only whether TV30 works better with DM than TV60 or Radio but also how much better, you will also need to know how far ahead of time do you need to plan your media flights. In the absence of this functionality your model is seriously lacking in value.

• closed system - this is almost like thermodynamics: closed systems are as great for analysis as they are far from reality, in the case of MMM, the models account for historical data and for forward-looking plan but have no way to make allowances for possible variations in the business environment. For example, competitive action may be entered by way of pricing or of spending but it is nearly impossible to predict and include a variable that accounts for the introduction of a new product by the competition. Another care is related to the relative importance of environmental factors: a model for retail built on data from 1995 to 2000 would likely indicate that macroeconomic factors had little impact on the outcome... the model would tell very differently in 2001 after the dot-com bubble burst and personal spending was drastically reduced.

The main take away from this post is: the best or perhaps the most prudent use of MMM and MMO is directional, consider that the models outline a a way to allocate your spending but that this recommendation is limited and may need to be revised as business conditions change. Following the recommendations of MMO to the letter or too closely may be a costly mistake.

Comments

[Jason Fernandes]

This is a really interesting article. Thanks for the great piece of information.