Pre-requisites

• Page only available to Admin and Architect user

Disclaimer: this is a beta feature, tell us what you think at product@madkudu.com

The Processes page tool (Settings > Processes) helps you understand when MadKudu is processing and scoring your data, and helps you see if MadKudu encountered any issue between the pull of your data, all the way to the sync (push) of the scores to your destinations in batch.

Please refer to this article to understand the difference between batch and real-time scoring.

Data pipeline

MadKudu processes your data in a specific order to send back scores to your integration. 

The data pipeline is executed continuously and usually runs end to end within 4h to 12h, or more depending on

• the volume of data (leads, contacts, accounts, events) from your integrations to process
• the number of models and the complexity of customizations to process. 

There are different reasons why a process may be failing or delayed (pipeline congested, server unavailable, connection interrupted ...etc)  but the batch will retry continuously to process the steps.  

Processes

The different steps of the batch data pipeline and scoring process are in the following order

1. merge: pulls data from your systems integrated into our platform (see Integrations tab) and import them into our data warehouse 
2. map: execute the Event mapping, Audience mapping and Conversion mapping
3. augment: enrich leads, contacts, and accounts with demographic, firmographic, technographic Enrichment from MadKudu's partners (Clearbit, HG, PredictLeads)
4. compute: execute the Attribute mapping, gathering all the additional enrichment available in your CRM and other integrations if any.
5. aggregations: execute the Aggregations (for LTB model)
6. analysis: execute all the live models configured for your organization to score leads, contacts, and accounts. 
7. sync: push the scores to your integrations (by batches)

Views and Janitor are additional processes for cleaning up our data warehouse.

Below is the representation of this data pipeline executed to transform your data into predictions or segmentations back to your CRM (or other systems) 

Last success

The last success column indicates when was the last time this process has run. 

The last success of the Sync process indicates when was the last time MadKudu has tried to score records in your systems. 

Status

The different statuses can be 

• idle: not running at the moment, waiting for its turn to process
• processing: currently being executed 
• queued: process in stand-by 
• failed: an error occurred, the process was not fully executed
• processing (last error: ): last try failed, the process is being executed at the moment

Connector status

The section Connectors - Pull indicates when was the last time MadKudu pulled data from your integration. MadKudu usually pulls every 5-10 min. 

Integration error codes

Frequent errors meaning MadKudu is unable to access your system. Please update the user permission or credentials

• Invalid credentials
  
• INVALID_LOGIN: Invalid username, password, security token; or user locked out.
• Access denied
• This app hasn't been granted all required scopes to make this call

API limit 

• Too close to the API limit: MadKudu is unable to pull additional data because of the API limit of the integration. Consider increasing your API limits or the daily API bandwidth

For other errors, please refer to the error message you receive for more information.