The Likelihood to Buy score is the result of the sum of all the activities of the person within the last 90 days, factored with a decay.
Each event performed by the person is associated with a weight and a decay, which are the scoring rules. MadKudu does not work with default scoring rules but instead automatically suggests custom scoring rules based on the analysis of the behavior of your past conversions.
However, we may want to customize the weight of some events to improve the performance of the model and to suit your business need. See how to edit your Likelihood to Buy scoring model.
In the Data Science Studio, the Feature evaluation tab allows to understand how events correlate to conversion and to tweak the weights and the decay of the events used in the model.
How to read the feature evaluation graph?
The feature evaluation graph displays the Lift of each event with regards to conversion.
The Lift is the ratio between the conversion rate of people who did the event to the overall average conversion rate of the training dataset. The higher, the more the event is correlated to conversion. However, this depends on statistical significance. The graph is separated into 4 sections:
- The events with the most statistical significance (at least 100 people in the training dataset have performed this event). This means there is enough data to be able to drive conclusions from the lift. The weights and decays automatically suggested are reasonable to keep.
- then the events with little statistical significance (between 10 to 100 people in the training dataset have performed this event). This means there isn't enough data to be able to drive conclusions from the lift but it still gives an indication if the events are important or not.
- then the events with no statistical significance (less than 10 people in the training dataset have performed this event). This means there isn't enough data to be able to drive conclusions from the lift as to whether or not these events are important or not.
- then the events with no lift available (no one in the training dataset has performed this event). This means assigning weights to these events will be taken into account in the model, but you won't be able to see their impact on the model performance.
How to read the feature evaluation table?
To read the "Feature" table:
Columns related to Events:
- Activity type: when building the event mapping we categorize events ("meta events") in different segments (Web Activity, Marketing Activity, Product Usage, Sales Activity, Email Activity...)
- Negative User Activity: when building the event mapping we also define if the event is more of a "negative action" (deleted account, unsubscribe from newsletter, declined invitation ...) than a positive event (showing that the user is engaging with the product / company).
- We would typically assign negative weights to negative user activities
- Meta Event: event performed by a user, mapped from your original event. We use the term "meta" to reflect the layer of mapping that we have done on top of the events you send us,which can just be a renaming but also a grouping of different events.
To understand what is behind those meta events, please go to app.madkudu.com > Mapping > Event mapping.
Columns related to Parameters (configuration of the model):
- Factor Loading: weight of the event
- Decay: the decay of the event in days
Columns related to Historical analysis (based on the people in the training dataset and their activity 3 months before), providing information and guidance on how the factor loading and decay are and should be configured:
- Average for converted: how many times was this event performed on average by a person that converted
- Average for non-converted: how many times was this event performed on average by a person that did not convert
- Did X: how many people performed this event
- If Did X >= 100 we estimate that the sample of people is large enough to derive conclusions (statistically significant)
- If Did X < 100 we estimate that the sample of people is too small to derive conclusions
- Did not do X: how many people did not perform this event
- Did X conversion rate: conversion rate of people who did this event
- Did not do X conversion rate: conversion rate of people who did not do this event
- Lift: Ratio between the conversion rate of people who did the event to the overall average conversion rate of the training dataset
- when lift > 0 it means that someone performing this event is more likely to convert
- when lift < 0 it means that someone performing this event is less likely to convert
- Recall conversions: proportion of conversions that did this event
- Recall non-conversions: Proportion of non-converters that did this event
- Average for converted * factor loading: multiplication of the event weight and the average number of occurrences of this event per conversion. This gives us an idea of how many points would be assigned to someone who usually does this event with that many occurrences.
How are configured the weights and decays?
The weight and decays automatically suggested are derived from a calculation based on the lift of the event, how many people have done this event (did X) and how often is the event performed by converted (average for converted).
However, you can manually change the weights and decays to tweak the model according to your Sales team feedback, your analysis, or business needs.
- You'd like to make sure that registering to a webinar does not bump the person automatically to a high likelihood to buy, therefore you would want to decrease the weight of the event "Registered to Webinar" to be under the threshold of the medium/high segment (see in Ensembling tab)
- You'd like to make sure that people who requested a demo are scored low after 30 days if they have only performed this action, therefore you would want to decrease the decay of the event "Requested a demo" to 30 days.