Runtime Optimizations for Large Datasets

Large datasets will run even faster with the recent optimizations in Narrator. When running a query on a large dataset, you may notice two helpful defaults appear:

  1. Time filter for last 3 months
  2. "But Only Within" condition added to the Last Before Append Relationship

Both of these query modifications create minimal impact to the dataset, but reduce total query time by a significant amount. Of course, these defaults can be easily removed if desired. More details about each improvement can be found below.

Time filter for last 3 months

When querying a large dataset, a time filter will be automatically applied to the dataset to limit the results to the last 3 months of data only.


Default Time Filter for Large Datasets

Why did we make this change?

  • Shorter Query Times
    This will speed up processing time significantly. When applied, you'll see a notification appear at the top right of the screen notifying you of the change.

  • Easier Dataset Iteration

    We recommend building your dataset with this filter on, and then removing it once the dataset structure is complete if you wish to run the dataset on the entirety of the historical data. This should significantly improve your workflow and prevent long query wait times, especially during dataset development and iteration.

Default condition added to LAST BEFORE relationship

When using the LAST BEFORE relationship, you'll notice a within time condition added by default. This change will speed up query time, while also adding more precision to the activity join logic so that you're only appending relevant activities that happened before.


Default within time condition applied to LAST BEFORE relationship

Learn more about the LAST BEFORE append relationship

Why did we make this change?

  • Align with the common use case for "Before" relationships
    The LAST BEFORE relationship is often used to understand the activity that happened just before the cohort activity.

    Ex. What docs page did someone visit before submitting a support ticket?

    Most times, we're trying to specify a close linkage between the cohort activity and the activity that preceded it. But the LAST BEFORE relationship does not differentiate between activities that happened a long time ago and activities that happened close to the cohort activity. For example, do we care about the last docs page that someone looked at before submitting a support ticket if they viewed the page 6 months prior? Probably not.
    So adding a within time filter by default will better align with the intended use case for this relationship.

  • Shorter Query Times

    The within time filter reduces the amount of data that needs to be processed when using the LAST BEFORE append relationship. And adopting a shorter time window will significantly reduce query times, making datasets process faster overall.

    Learn more about the LAST BEFORE append relationship

User feedback helps Narrator improve. ❤️
We'd love to hear what you think. Email us @ [email protected]