The Likelihood to Buy score is the result of the sum of all the activities of the person within the last 90 days, factored with a decay.
Each event performed by the person is associated with an Importance (weight) and a Lifespan (decay), which are the scoring rules. MadKudu does not work with default scoring rules but instead automatically suggests custom scoring rules based on the analysis of the behavior of your past conversions.
However, we may want to customize the weight of some events to improve the performance of the model and to suit your business need.
This means the configuration of the model essentially includes
- configuring the importance (weight) and lifespan (decay) of each event in the Feature evaluation tab See how to edit your Likelihood to Buy scoring model.
- then the thresholds of points defining the very high, high, medium and low segment, in the Ensembling tab.
In the Data Science Studio, the Feature evaluation tab allows to understand how events correlate to conversion and to tweak the weights and the decay of the events used in the model.
How to read the feature evaluation graph?
The feature evaluation graph displays the Lift of each event with regards to conversion.
The Lift is the ratio between the conversion rate of people who did the event to the overall average conversion rate of the training dataset. The higher, the more the event is correlated to conversion. However, this depends on statistical significance. The graph is separated into 4 sections:
- The events with the most statistical significance (at least 100 people in the training dataset have performed this event). This means there is enough data to be able to drive conclusions from the lift. The weights and decays automatically suggested are reasonable to keep.
- then the events with little statistical significance (between 10 to 100 people in the training dataset have performed this event). This means there isn't enough data to be able to drive conclusions from the lift but it still gives an indication if the events are important or not.
- then the events with no statistical significance (less than 10 people in the training dataset have performed this event). This means there isn't enough data to be able to drive conclusions from the lift as to whether or not these events are important or not.
- then the events with no lift available (no one in the training dataset has performed this event). This means assigning weights to these events will be taken into account in the model, but you won't be able to see their impact on the model performance.
How to read the feature evaluation table?
The Features section lists the different events and aggregated events, and how much each of them weight in the scoring.
- Event: user-friendly name of the event performed by a user, mapped from the behavioral data from your integration (in app.madkudu.com > mapping > event mapping).
- Tips: events like "Email - At least X activities in the last X day(s)" are aggregated events. Learn more.
- Importance (in points): weight of the event
- Lifespan (in days): the decay of the event in days. For example: if lifespan = 90 ays, this means tha 90 days after doing this event, this event has no contribution to the lead or account likelihood to buy score.
When clicking on "Advanced mode", the table expands to show the historical data analysis
Columns related to the Event mapping:
- Activity type: when building the event mapping we categorize events ("meta events") in different segments (Web Activity, Marketing Activity, Product Usage, Sales Activity, Email Activity...)
- Negative User Activity: when building the event mapping we also define if the event is more of a "negative action" (deleted account, unsubscribe from newsletter, declined invitation ...) than a positive event (showing that the user is engaging with the product / company).
- We would typically assign negative weights to negative user activities
Columns related to Historical analysis (based on the people in the training dataset and their activity 3 months before), providing information and guidance on how the factor loading and decay are and should be configured:
- Suggested importance = the recommended weight (in points) to attribute to this event, which is calculated based on the lift.
- Suggested Lifespan = the recommended Lifespan to attribute to this event, which is calculated based on how frequently an event is done. (The more frequent, the lower the lifespan should be)
- Average for converted: how many times was this event performed on average by a person that converted
- Average for non-converted: how many times was this event performed on average by a person that did not convert
- Did X: how many people performed this event
- If Did X >= 100 we estimate that the sample of people is large enough to derive conclusions (statistically significant)
- If Did X < 100 we estimate that the sample of people is too small to derive conclusions
- Did not do X: how many people did not perform this event
- Did X conversion rate: conversion rate of people who did this event
- Did not do X conversion rate: conversion rate of people who did not do this event
- Lift: Ratio between the conversion rate of people who did the event to the overall average conversion rate of the training dataset
- when lift > 0 it means that someone performing this event is more likely to convert
- when lift < 0 it means that someone performing this event is less likely to convert
- Recall conversions: proportion of conversions that did this event
- Recall non-conversions: Proportion of non-converters that did this event
- Average for converted * factor loading: multiplication of the event weight and the average number of occurrences of this event per conversion. This gives us an idea of how many points would be assigned to someone who usually does this event with that many occurrences.
How are the weights and decays configured?
The weight and decays automatically suggested are derived from a calculation based on the lift of the event, how many people have done this event (did X) and how often is the event performed by converted (average for converted).
However, you can manually change the weights and decays to tweak the model according to your Sales team feedback, your analysis, or business needs.
- You'd like to make sure that registering to a webinar does not bump the person automatically to a high likelihood to buy, therefore you would want to decrease the weight of the event "Registered to Webinar" to be under the threshold of the medium/high segment (see in Ensembling tab)
- You'd like to make sure that people who requested a demo are scored low after 30 days if they have only performed this action, therefore you would want to decrease the decay of the event "Requested a demo" to 30 days.