Validation

All transformations must be validated before they can can be merged to production.

Validating Transformations

Narrator runs a series basic tests to ensure all transformations will process correctly. After you save a transformation, you'll be able to run validation.

These tests check the core assumptions that are used when updating the activity stream, customer, and enrichment tables. When these assumptions fail then the data doesn't process correctly, so the validation step will help catch processing issues before you merge to production.

Successful Validation for an Activity TransformationSuccessful Validation for an Activity Transformation

Successful Validation for an Activity Transformation



Tests

Validation tests required

Test Name

Details

Check for LIMIT Clauses

LIMIT is not used to limit the data output of the transformation

Check for Missing Columns

All required columns are available

Check for NULL Values in Required Columns

activity_id, ts, and activity values are never NULL for Activity Transformations, enriched_activity_id , enriched_ts are never NULL for Enrichment Transformations, customer is never null for Customer Transformations

Check for ID Duplication

activity_id or enriched_activity_id is unique. For Customer Transformations, the test checks that customer is unique.

Note: Additional tests can be manually run on the transformation from the Validation Log tab. More information on the additional validations can be found here.



Data Sample Size for Validation

By default, Narrator uses the last 3 months of data to run the validation tests. Users can control the validation months by going to Company Settings > Processing Configuration. Learn more.



Custom Validation / Custom Alerts

Set up a custom alert to get notified if assumptions about the data changes. You can define a custom SQL query and set up an email alert if the query returns rows or doesn't return rows. This is a helpful way to ensure the assumptions you made when defining the transformation stay true over time.

You can schedule your custom alert to run the query regularly using the cron schedule and it can be managed from the processing page (under the Alerts tab).

Custom Validation Set UpCustom Validation Set Up

Custom Validation Set Up

πŸ“˜

How to: Use Custom Validations (Custom Alerts)

Watch this step-by-step tutorial to set up a custom validation.



Troubleshooting tips




Still have questions?


Our data team is here to help! Here are a couple ways to get in touch...

πŸ’¬ Chat us from within Narrator

πŸ’Œ Email us at [email protected]

πŸ—“ Or schedule a 15 minute meeting with our data team


Did this page help you?