A large dataset with multiple dimensions and metrics stretched across millions of rows can look tiresome to the untrained human eye but can have significant signals when looked through the right techniques. A careful inspection can reveal patterns in the dataset of creating intra-interacting cohesive groups characterized by various parameters like user behavior, conversion patterns, historical traits, etc. Segmentation has become a prerequisite for any data deep dives and is widely used in every vertical, be it retail, e-commerce, banking, aviation, or astronomy, for invaluable insights about data structure.
Segmentation, as mentioned, involves identifying interesting latent traits in the data. There are multiple techniques of doing segmentation, below is a brief about the three most popular methods:
Using Segmentation In Web Analytics
Here is a quick overview of how the analytics team at Nabler successfully deployed a segmentation methodology (clustering) to achieve a significant success in outcome.
The client was among North America‘s largest retail chain. The occasion was ‘Black Friday’, one of the peak seasons for sales, discounts, rush, and shopping frenzy, be it online or offline. The big retailers plan for this day way in advance (as the financial year begins) as most make around 60% of their annual revenue in these few days itself. Our client was no exception. They gave us a task; quite a succinct one: “What should we promote on our website this ‘Black Friday’ to get the most out of the day?”
- Defining the approach : The analytics team went to the drawing board to put together an approach for the problem. The problem was sophisticated and challenging which required multiple steps and some smart data science techniques such as logistic regression, causality checks, etc.
- Necessity of segmentation : Before we started, we agreed upon using some segmentation technique to make their massively large dataset digestible. This was absolutely essential as we wanted to create buckets instead of trying to attack the problem at the very macro level.
- Analyzing past metrics : A list of metrics that we looked at were Visits and Page Views, Engagement Metrics, Type of Page, Last Touch Channel, Orders, Revenue, Conversion Rates, Qualitative Characteristic of the product, to mention a few broken down by promoted and non-promoted products during ‘Black Friday’ last year (that was the maximum data that was available).
- Finding the right segments : With this data our first approach as required for any modeling exercise was outlier and anomaly detection. With the data cleansing done, graphical inspections made and some descriptive statistics we had a fair idea of how varied a data we were dealing with which bolstered the requirement to find homogenous segments within the heterogeneous dataset so that when we run the more sophisticated algorithms we do not suffer from the problem of unaccounted variance.
- Using segmented data for forecasting : Thus segmentation here was used more as an intermediate process to refine and feed into models to improve modelling efficiency. Let me give a very quick example of how the segments looked, like a significant chuck of visitors spending more than ‘x’ minutes on the website browsing through ‘y1 – y2’ (range) number of pages of ‘A’ type of product would be much more easier and scalable to model than to take the complete gamut of unsegmented data and struggling to achieve modelling efficiency. The segmented dataset was then used to model and produce different forecasts for the upcoming big day.
Because of the segmented view approach the client had a greater clarity. They were able to target campaigns in much customized fashion, setting personalized goals and revenue targets, based on the various characteristics that defined each segment. With our predicted traffic on website on the upcoming ‘Black Friday’ and revenue targets forecasted with the customized approach the client experienced a phenomenal lift in revenue of 17% compared to last year (marginally overshooting our expectations at 14%). A battle well won, another feather on the cap earned.