Skip to main content
Version: Next

Changes Refresh Mode

The changes refresh mode applies incremental inserts, updates, and deletes from a Change Data Capture (CDC) source. Unlike append, changes mode reflects modifications and deletions in the acceleration, keeping it consistent with sources where rows mutate over time.

Use changes when:

  • The source supports CDC (e.g., a database with a transaction log).
  • Rows in the source are updated or deleted, not just inserted.
  • The acceleration must reflect the current state of the source row-for-row.

Configuration​

refresh_mode: changes requires a CDC-capable data connector. Spice supports CDC via PostgreSQL Logical Replication, DynamoDB Streams, Apache Kafka, and Debezium. See Supported Data Connectors for details.

datasets:
  - from: debezium:cdc.public.customer_orders
    name: customer_orders
    acceleration:
      enabled: true
      refresh_mode: changes
      engine: duckdb
      mode: file

The Debezium connector streams change events from a Kafka topic produced by Debezium. Each event is applied to the acceleration in order, preserving inserts, updates, and deletes from the source.

Behavior​

  • The acceleration is bootstrapped from the source snapshot, then continuously updated from the change stream.
  • refresh_check_interval, refresh_cron, on-demand refresh, refresh_data_window, and retention_period do not apply — updates are driven by the change stream rather than periodic polling.
  • refresh_sql can only modify selected columns in changes mode and cannot apply row filters.