Announcements
Back to All
improved

Cost saving changes for large processing updates

over 2 years ago by Brittany Davis

We've added an independent processing schedule for transformations with "Full Data Resync" update types, so that large processing updates can run less frequently to reduce warehouse costs.



What's Changed?

You'll now see two tasks in the processing view that control updates to tables in your activity schema.

  1. Run Transformations: Processing schedule for transformations with update types set to: Incremental (default) and Insert Missing Rows Only
  2. Run Async Transformations: Processing schedule for transformations with update types set to: Full Data Resync
2314




Frequently Asked Questions

  • How does this save costs?
  • Where can I control the processing type for each transformation?
  • What are the default schedules for processing?
  • Where can I set a custom processing schedule?
  • Which transformations are "Full Data Resync" by default?



FAQ: How does this save costs?

Some update types require more warehouse resources than others. The default update type for activities (incremental) is very efficient, and requires fewer resources than other update types. So, these updates can be run more frequently without incurring unnecessary costs.

On the other hand, the "Full Data Resync" update type requires more warehouse resources because it deletes and fully reprocesses all the data on each update. For example, the customer table is updated using a Full Data Resync. Now users can modify the processing schedule so that large update types can be run on a less frequent basis.



FAQ: Where can I control the processing type for each transformation?

Update types can be changed on the "Processing Configuration" tab of each transformation. Learn more about Narrator's processing types.

1894



FAQ: What are the default schedules for processing?

  • Run Transformations: Every 30 minutes
  • Run Async Transformations: Every day



FAQ: Where can I set a custom processing schedule?

Use the gear next to each task on the processing page to configure the schedule. Select "Custom" to specify custom schedule.

2092



FAQ: Which transformations are "Full Data Resync" by default?

The customer transformation is the only table that is set to "Full Data Resync" by default.




User feedback helps Narrator improve. ❤️
We'd love to hear what you think. Email us @ [email protected]