All transformations must be validated before they can can be merged to production.

Validating Transformations

Narrator runs a series basic tests to ensure all transformations will process correctly. After you save a transformation, you'll be able to run validation.

These tests check the core assumptions that are used when updating the activity stream, customer, and enrichment tables. When these assumptions fail then the data doesn't process correctly, so the validation step will help catch processing issues before you merge to production.


Successful Validation for an Activity Transformation


Validation tests required

Test NameDetails
Check for LIMIT ClausesLIMIT is not used to limit the data output of the transformation
Check for Missing ColumnsAll required columns are available
Check for NULL Values in Required Columnsactivity_id, ts, and activity values are never NULL for Activity Transformations, enriched_activity_id , enriched_ts are never NULL for Enrichment Transformations, customer is never null for Customer Transformations
Check for ID Duplicationactivity_id or enriched_activity_id is unique. For Customer Transformations, the test checks that customer is unique.

Note: Additional tests can be manually run on the transformation from the Validation Log tab. More information on the additional validations can be found here.

Data Sample Size for Validation

By default, Narrator uses the last 3 months of data to run the validation tests. Users can control the validation months by going to Company Settings > Processing Configuration. Learn more.

Custom Validation / Custom Alerts

Set up a custom alert to get notified if assumptions about the data changes. You can define a custom SQL query and set up an email alert if the query returns rows or doesn't return rows. This is a helpful way to ensure the assumptions you made when defining the transformation stay true over time.

You can schedule your custom alert to run the query regularly using the cron schedule and it can be managed from the processing page (under the Alerts tab).


Custom Validation Set Up


How to: Use Custom Validations (Custom Alerts)

Watch this step-by-step tutorial to set up a custom validation.

Troubleshooting tips

Still have questions?

Our data team is here to help! Here are a couple ways to get in touch...

πŸ’¬ Chat us from within Narrator

πŸ’Œ Email us at [email protected]

πŸ—“ Or schedule a 15 minute meeting with our data team