We've added an independent processing schedule for transformations with "Full Data Resync" update types, so that large processing updates can run less frequently to reduce warehouse costs.
You'll now see two tasks in the processing view that control updates to tables in your activity schema.
- Run Transformations: Processing schedule for transformations with update types set to:
Insert Missing Rows Only
- Run Async Transformations: Processing schedule for transformations with update types set to:
Full Data Resync
- How does this save costs?
- Where can I control the processing type for each transformation?
- What are the default schedules for processing?
- Where can I set a custom processing schedule?
- Which transformations are "Full Data Resync" by default?
Some update types require more warehouse resources than others. The default update type for activities (incremental) is very efficient, and requires fewer resources than other update types. So, these updates can be run more frequently without incurring unnecessary costs.
On the other hand, the "Full Data Resync" update type requires more warehouse resources because it deletes and fully reprocesses all the data on each update. For example, the customer table is updated using a Full Data Resync. Now users can modify the processing schedule so that large update types can be run on a less frequent basis.
Update types can be changed on the "Processing Configuration" tab of each transformation. Learn more about Narrator's processing types.
- Run Transformations: Every 30 minutes
- Run Async Transformations: Every day
Use the gear next to each task on the processing page to configure the schedule. Select "Custom" to specify custom schedule.
The customer transformation is the only table that is set to "Full Data Resync" by default.
User feedback helps Narrator improve. ❤️
We'd love to hear what you think. Email us @ [email protected]