P L E A S E  W A I T...

Media Mix Modeling: How Nabler Built a Model to Measure Online Advertising Effectiveness and Optimize Ad Spend

 

The Client

The client, a US-based electronics and electrical equipment company, is one of the largest printer manufacturing companies worldwide manufacturing printers, desktops, fax machines, consumer/industrial machinery, and accessories.

 

The Objective

  • The client wanted to measure the online advertising effectiveness of different channels in the US. This analysis would help drive business acumen on Return on Ad Spend (ROAS) for the paid marketing channels.
  • The company’s annual media spend was around five million dollars (US). The client wanted to optimize this spending across paid marketing channels.
  • In addition, the client wanted insight into the delayed effect and saturation levels of the paid marketing channels.
 

The Challenges in Measuring Online Advertising Effectiveness

  1. The Client had six product categories, each having unique audiences and data patterns, including more than 10,000 SKUs
  2. The client had an omnichannel retail model: eCommerce website, offline stores, distributors, and other online retailers.
  3. The client had two years of data (January 2019 to May 2021) that had to be aggregated at a week level (~ 126 weeks) as the sales numbers were available at weekly granularity.
  4. The Covid-19 pandemic had a significant impact on the client’s sales, inventory, and media spend. Nabler needed an approach that would account for this while measuring online advertising effectiveness.
Data Unification Challenges:
  1. Sales data was available at an SKU level, but the weekly media spend data was only available at a category level.
  2. The data needed to be collected from multiple sources. This required complex joins and aggregations to stitch the data together.
  3. SKUs in the POS data needed to be mapped to a relevant category. The mapping was done based on rules/values provided by the client.
  4. There were other fields such as unit price and promotions that needed to be aggregated accordingly to category and week level data.
  5. In the case of distributors and smaller stores, POS data was unavailable. So, sell-in data was used as a proxy to derive end consumer sales.
Model Building Challenges:
  1. Media spend was sporadic for certain product categories and channels. Moreover, low media spends often resulted in inflated ROAS values, presenting model-building challenges in measuring online advertising effectiveness.
  2. Each category needed its own model and each model had to be tuned according to the significant variations in trend, seasonal patterns, Covid impact, and business outlook.
Data Overview
Challenge
 
 

The Nabler Approach to Measuring Online Advertising Effectiveness:

Model Dev Phase:

Nabler applied Media Mix Modelling (MMM) analysis technique to measure the effectiveness of the client’s online marketing and advertising campaigns. MMM was used to determine how various elements contribute to revenue.

Feature Engineering:

Nabler implemented an iterative process of selecting and transforming variables to create a predictive model using statistical modelling.

  • Holidays can influence user interest to purchase. So, to capture its effect, a holiday flag was created to indicate holidays on a particular week.
  • Sales were decomposed into “trend” and “seasonality” to measure the online advertising effectiveness of different paid marketing channels, and to isolate the revenue.

    Feature
  • Usually, a high percentage of the sales were accounted for by seasonality and trend. The remainder was modelled to capture the true effect of media channels.

    Feature
  • Seasonality refers to periodic patterns that recur every calendar year. It is regular and predictable.

    Feature
  • Trend indicates the cumulative impact of the client’s marketing, brand building, and strategic initiatives. It also includes the effect of macroeconomic conditions on consumer demand.
Ad Stock transformation:
  • Marketing exposures build awareness in consumers’ minds. It doesn’t disappear right after the consumers see the ad but rather remains in their memory and decays over time.
  • Nabler modelled this effect using Adstock transformation for each media spend.
  • Adstock transformations were done for some of the media channels to account for the carryover effect, which in turn, increased the model’s accuracy.

    Stock

From the chart, it is observed that transformed spending is greater than the actual spending as the result of the current week is attributed to the previous X weeks spend. The above spend has been transformed using a decay rate of 0.8 with a half-life of 4 weeks.

  • Decay rate is the percentage of effect each week that is carried over to the following week.
  • Half-life is time until the advertising effect reduces by 50%.
Saturation Effect:
  • After a certain point, the revenue through media spend would reach a limit and further media spending would have a near-zero incremental effect on the revenue. This is known as the saturation effect of media spend.
  • To capture this effect, media spend was transformed using a shape function.
External Factors:
  • There were certain factors apart from the trend and seasonality, that influenced the sales.
  • So, Nabler included these external factors as variables to capture the true effect of media spend and to accurately measure online advertising effectiveness.
  • The pandemic influenced the purchase behaviour of the people. To account for this effect, variables such as “coronavirus cases” and “vaccination” were created. From the below chart, it is seen that the sales have decreased considerably since the start of the pandemic.

    Impact
  • The pandemic also caused supply chain issues across all industries. It is also observed from the below chart that there is a decrease in the inventory levels during the pandemic.

    Impact
Algorithms Used in Modelling
  • The ideal model to measure online advertising effectiveness should not only be accurate but also must perform well in terms of explaining the effect of media spend.
  • Upon iterating through multiple algorithms, Robyn, a semi-automated open-sourced Marketing Mix Modelling package from Facebook Marketing Science, was chosen as the best performing algorithm.
  • Ridge regression with cross-validation was used to address multicollinearity among many regressors and prevent overfitting.
  • A multi-objective evolutionary algorithm was used for hyperparameter optimization. Multiple iterations were done to identify hyperparameters for ad stock transformations and shape transformations.

    Algorithm
The Results:

Using the above machine learning model to measure the online advertising effectiveness, Nabler achieved the following results

  1. ROAS and revenue contribution by media channel
    The client was able to measure the effectiveness of each paid marketing channel to overall sales/revenue for each product category and identify the contribution of all the features that drive the sales.Result Result
  2. Saturation Levels
    The client was able to identify the revenue saturation levels for different media channels.

    Result
  3. Gradient-based optimization to get ideal budget spend distribution
    • The optimizer used learnings from the model and provided an optimal budget for each media channel, thus maximizing the revenue.
    • The client was able to observe a 1.83x lift in ROAS contribution from paid media channels using the budget optimizer.

      Result
 

Want to learn more? Let's Talk.

Contact Us