We have a NEW User role in the MadKudu app instances called "Architect", which grants permissions for you to: 

• Create your mapping configuration(s) in the app
• Configure your predictive models
• Add new models into the Data Science Studio for discovery or deployment

With great power, comes great responsibility - so this document aims to empower you with the right tools to play around with this new user role.

There are several things you can do with this new "access":

1. You can remove any false positives or negatives in the model.
2. You can boost new target segments in the model.
3. You can measure your current model performance against a target segment.
4. You can modify the Signals that are returned to your Sales team to make explaining the scores more relevant. [Coming soon!]
5. You can do data discovery around a specific audience segmentation to determine next steps: new model, overrides update, signals update, etc. [Coming soon!]

How-to: remove any false positives or negatives in the model

Objective: If you encounter feedback from Sales where there are certain scores that don't make sense - either the lead is not a good fit but scored as qualified or the lead is a good fit but showing up as unqualified - the best thing to do first is to check out if there are any overrides implemented in your model that are forcing certain leads to go into a specific segment. 

There are of course many other ways to do this (example: configuring the trees, etc) but below is a typical first step to understand any false positives/false negatives.

Step-by-step guide:

1. Access the Data Science Studio through the Customer Fit Model (App > Predictions > Customer Fit > Diagnostics > Model).
2. Check out the Overrides tab and review where the "problematic" lead could fall under. 
3. Create a computation with that override definition by going on Customer Profile > Computations > New Computation. Ensure that you select "Customer Available (the computation will be visible in Customer Fit Insights)" tag. Click on Release and Recompute and wait for the processing to be completed.
4. Once completed, go back to the Data Science Studio tab and check out the Univariate Analysis for that particular computation. Here, you want to see what proportion of leads contribute to conversions. If lift is > 0.5 (best to be above 1) and with a significant number of leads/conversions, it means that the override is "performing". If not, iterate on the override until you find a better lift. 
5. Once you find a better override definition, recreate the override in the Overrides tab.
6. Validate the performance of this update in the Validation tab. The benchmark is to get 20-30% of population to 60-80% of conversions as well as a linear conversion rate across each segment. 

How-to: boost new target segments in the model

Objective: If you receive feedback from your leadership or sales team to go after a specific segment but Sales is seeing leads in those segments as unqualified, since historically they have not been performing, you could potentially apply an override to boost that target segment to qualified.

Step-by-step guide:

1. Access the Data Science Studio through the Customer Fit Model (App > Predictions > Customer Fit > Diagnostics > Model).
2. Create a computation with that target segment definition by going on Customer Profile > Computations > New Computation. Ensure that you select "Customer Available (the computation will be visible in Customer Fit Insights)" tag. Click on Release and Recompute and wait for the processing to be completed.
3. Once completed, go back to the Data Science Studio tab and check out the Univariate Analysis for that particular computation. Here, you want to see what proportion of leads contribute to conversions. If lift is > 0.5 (best to be above 1) and with a significant number of leads/conversions, it means that the target segment is "performing". If not, this just means that historically you may not have had that many leads or even conversions in those target segment so it would be important to take a self-assessment here if you still want to override this target segment as qualified. 
4. Create the override with the target segment definition.
5. Validate the performance of this update in the Validation tab. The benchmark is to get 20-30% of population to 60-80% of conversions as well as a linear conversion rate across each segment. 

How-to: measure your current model performance against a target segment

Objective: If you would like to measure your current model performance against specific target segments (example: geos, product lines, industries, etc.), you could potentially do this BEFORE modifying the model so that you can determine the appropriate next steps. 

Step-by-step guide:

1. Access the Audience Mapping in the app (App > Mapping > Audience Mapping).
2. Create an audience with the right filters. If you don't see a particular field from an object, please make sure you have pulled the field. Access this by looking at Integrations > Connector > Pull. 
3. Wait for the new audience processing to be completed (this typically takes 24hours). 
4. Once the audience has been created, you can then filter the model performance report in App > Predictions > Customer Fit to that audience and look at the recall of the model on the audience. Based on the results here, you can then determine if you want to update the model, create new models or do nothing. :)