Spice v2.4.0 (Oct 27, 2026)

Spice v2.4.0 is a feature release that expands data ingestion, caching, and AI decisions, and makes Spice Cayenne the default accelerator. It also upgrades the query engine to Apache DataFusion v55 and Apache Arrow v59.3. Datasets and views that do not set acceleration.engine now use Cayenne instead of Arrow, except on native Windows. Other accelerators are deprecated and will be removed in v2.5. In tests, Cayenne served uncached primary-key lookups about twice as fast, and query latency stayed steady during full refreshes.

SQL queries and the new OpenAI-compatible /v1/decisions API can now ask TypeSafe or chat models for decisions. With refresh_mode: changes, accelerated S3 datasets read new objects from S3 when SQS delivers S3 event notifications. This release also adds SQL results-cache warmup, Enterprise acceleration snapshots, and new data connectors and formats.

Read Breaking Changes before you upgrade. SQL results-cache entries now expire only by TTL, not when a dataset refreshes. Cayenne ignores on_conflict and keeps one row for each primary key. Some datasets that loaded on Arrow fail to load on Cayenne unless they set engine: arrow.

Highlights in v2.4.0 include:

• Spice Cayenne by Default: Datasets without acceleration.engine now use Cayenne instead of Arrow, except on native Windows. Cayenne adds automatic primary-key handling, persistent secondary indexes, and adaptive tuning (preview).
• DataFusion v55: the query engine upgrades to DataFusion v55 and Arrow v59.3. In DataFusion benchmarks, the TopK query suite ran about 43% faster. Ballista v55, Vortex v0.86, and Iceberg v0.11 also ship in this release.
• Faster Cayenne Lookups and Refreshes: in tests, uncached Cayenne primary-key lookups ran about twice as fast. P99 query latency during full refreshes fell from 130–690 ms to 44–50 ms.
• AI Decisions in SQL and HTTP: five SQL decision functions and an OpenAI-compatible POST /v1/decisions API, with TypeSafe models or any chat model.
• S3 Event-Driven Refresh: with refresh_mode: changes, accelerated S3 datasets read new objects when SQS delivers S3 event notifications. A periodic listing loads objects whose notification did not arrive.
• Acceleration Snapshots (Enterprise): serve a dataset from published snapshots without its source connector. Snapshots use the shared state location and expire automatically.
• SQL Results-Cache Warmup: record query shapes, then replay them after a restart so that the first queries use a warm cache.
• New Data Connectors and Formats: query Hugging Face datasets, ORC files, and Iceberg tables without a catalog.

What's New in v2.4.0

Spice Cayenne by Default

Spice Cayenne now accelerates datasets and views that omit acceleration.engine. Before this release, they used Arrow. Native Windows keeps Arrow. On native Windows, run Spice in Docker or WSL to use Cayenne. Storage still defaults to mode: memory. Set mode: file for persistent acceleration. To move an existing acceleration, read the Cayenne migration guide.

Automatic ON CONFLICT handling for Primary Keys

A Cayenne dataset with a primary_key now keeps one row for each key in every refresh mode, without an on_conflict setting. This table shows which version of a repeated key Cayenne keeps:

Dataset	Version kept for a repeated key
No time_column	The last arrival.
time_column with full or append refresh	The newest time. The last arrival wins a tie. A NULL time is oldest.
refresh_mode: changes	Each change applies in source order.

With a time_column, a late row with an older time does not replace a newer stored row. Sources such as Iceberg, where writers can commit out of time order, keep the newest version of each key.

datasets:
  - from: s3://my-bucket/orders/
    name: orders
    time_column: updated_at
    params:
      file_format: parquet
    acceleration:
      enabled: true
      mode: file
      primary_key: id

Cayenne no longer uses on_conflict to select its conflict policy. For read-write datasets whose writes must stay in the acceleration, set access: read_write and acceleration.write_mode: acceleration. The source need not support writes. This setting cannot accompany a change-stream refresh (refresh_mode: changes, or a CDC source whose default refresh mode is changes).

An UPDATE that creates a primary-key collision fails before it changes rows. See duplicate primary keys and the migration guidance.

Runtime-Wide Adaptive Tuning (Preview)

Cayenne can now change its own settings to meet runtime-wide targets for freshness, replication lag, query latency, and throughput. Set runtime.adaptive_tuning to enable it for Cayenne datasets and catalogs. Adaptive tuning is in preview.

runtime:
  adaptive_tuning: enabled
  params:
    target_freshness: 30s

datasets:
  - from: postgres:public.orders
    name: orders
    acceleration:
      enabled: true
      mode: file
      refresh_mode: changes
      primary_key: id

Adaptive tuning accepts four target objectives and one pacing setting in runtime.params. Cayenne changes its settings in small steps to move each measured value toward its target:

• target_replication_lag: Desired maximum age of the newest applied source commit. Applies only to change-stream (refresh_mode: changes) datasets.
• target_freshness: Desired maximum delay from a source commit until its data becomes queryable. Applies only to change-stream datasets.
• target_query_latency: Desired P99 latency of the queries that read each Cayenne dataset.
• target_qph: Desired throughput, in queries per hour (QPH), of queries that read Cayenne datasets.
• target_convergence_window: Time budget for tuning to approach its targets. Default: 60s.

Adaptive tuning defaults to disabled. Targets apply only when adaptive tuning is enabled. A refresh_mode: full dataset does not use adaptive tuning. Each refresh replaces the whole table, so Cayenne turns off background compaction (cayenne_compaction_background_interval_ms: 0). The tuner runs on the compaction task, so it does not run for that dataset. The runtime logs a warning for each such dataset. See Cayenne self-tuning and Cayenne Tuning Settings.

Indexes, Initial Loads, and Compaction

• Persistent secondary indexes: File-mode Cayenne stores index runs with its acceleration. Writes maintain the indexes, and acceleration snapshots retain them. Dynamic join filters can probe secondary indexes.
• Keyed initial loads: Cayenne can build the primary-key filter during the first load within its memory budget. Otherwise, it retains the existing warmup path. Bulk loads avoid per-row routing and a subsequent whole-table fold.
• Maintained aggregates: Views share a compact index of per-key contributions. Rebuilds capture concurrent writes and apply them after the scan. A bounded query pool assigns 10% to aggregates without the former 512 MiB cap. An unbounded pool retains that cap.
• Append-only compaction: For unkeyed datasets without position deletes, Cayenne encodes merged files outside the write lock. Publication still takes the lock. Datasets with position deletes retain the full-rewrite path.
• Configuration updates: A restart applies changed write settings to an existing acceleration. Identity changes can rebuild eligible full-refresh accelerations from their source. See the Cayenne documentation.

Apache DataFusion v55 and Query Engine Upgrades

Spice now runs on Apache DataFusion v55 and Apache Arrow v59.3. This release also upgrades Apache Ballista to v55, Vortex to v0.86.1, and Apache Iceberg to v0.11.0. The numbers in this section come from each project's own benchmarks, not from Spice tests.

Apache DataFusion v55

The DataFusion v55 release adds these changes:

• TopK pruning: As the threshold of an ORDER BY ... LIMIT query tightens, Parquet scans check the unread row groups again. They skip groups that cannot change the result. Compound ORDER BY keys are also supported. The topk_tpch suite ran about 43% faster in total.
• Join planning: The optimizer converts eligible inner joins to semi joins. It removes redundant sides of LEFT and RIGHT joins and orders filter predicates by estimated cost. ClickBench Q21 ran 10–13% faster.
• Planning speed: The optimizer collapses chained projections and rewrites subqueries in place. One planning case fell from 623 ms to 155 ms, and TPC-DS optimization fell from 220 ms to 170 ms.
• Functions: IN lists with few primitive values evaluate up to 14.7× faster in microbenchmarks. approx_distinct over 50,000 groups fell from 1,723 ms to 17 ms. More string functions keep dictionary encoding.
• Aggregation: Multi-column GROUP BY keeps column-oriented storage for all key types, such as fixed-size binary UUIDs. Grouping 200 million UUIDs ran about 5% faster and used less memory.
• Parquet scans: Scans skip nested fields that the table schema does not declare. They also skip the page index when a file has none. ClickBench Q1 on a single file ran 38% faster.
• Spills: A sort under memory pressure spills its largest stream again in smaller batches instead of failing. Merges also limit their number of input streams.
• SQL: EXPLAIN accepts PostgreSQL-style options and FORMAT pgjson. New array functions add, subtract, scale, sum, and average array elements.

Spice carries these changes through its query plans and keeps statistics across its plan wrappers. Cayenne does not split Vortex scans below 10 MiB, to avoid repeated footer reads.

Apache Ballista v55

Spice.ai Enterprise feature. See Distributed Query.

Spice distributed query runs on a Spice fork of Apache Ballista v55. The Ballista v55 release adds these upstream changes:

• Adaptive query execution by default: The scheduler replans between stages with row counts from completed stages. All 99 TPC-DS queries complete and match single-process DataFusion in the Ballista test suite.
• Fewer tasks: One task can cover several input partitions, up to the available virtual cores of its executor. This change reduces planning, serialization, and dispatch work.
• Shuffle: Remote shuffle reads receive blocks without a copy. Sort shuffle writes one file for each task.
• Protocol version: Clients receive a clear error when their major version differs from the version of the scheduler.
• Benchmark: On TPC-H at scale factor 1,000, Ballista ran the 22 queries in 502.4 seconds. Apache Spark 4.1.3 took 898.2 seconds on the same cluster.

Schedulers and executors must run the same version, so upgrade all cluster components together.

Vortex v0.79.0 to v0.86.1

Spice v2.3.3 used the Vortex v0.79.0 fork. This upgrade covers every upstream release from v0.79.0 through v0.86.1:

• Types and arithmetic: Vortex adds native Map arrays, union arrays, and decimal addition, subtraction, multiplication, and division.
• Row selection: Piecewise-sequence indices represent contiguous selections without an index entry for every row.
• Scans and expressions: Layout scans get a physical plan and expression optimization. Filters can pass through scalar functions with more than one argument. Filters on wide lists read only the selected range of child elements.
• Compression: Binary arrays support FSST compression with variable-length offsets. OnPair becomes a stable encoding with a dictionary in storage.
• File metadata: Files can store custom metadata. Readers cache decoded type descriptions.
• Correctness and validation: Vortex corrects NULL handling for dictionary predicates, BETWEEN bounds, and empty arrays. Readers validate footer offsets, compression metadata, and array indices.

See the Vortex release notes for v0.80.0 through v0.86.0 and v0.86.1.

Apache Iceberg v0.11.0

Spice updates its Iceberg reader and catalog integrations from v0.10.1 to the v0.11.0 fork for DataFusion v55 and Arrow v59. The upstream v0.11 changes extend reads and catalog compatibility:

• Iceberg v3 reads: The reader applies deletion vectors from Puffin files. It carries row identifiers and sequence numbers through scans.
• Metadata-only scans: Projections of metadata only do not read data columns. Positional-delete processing buffers runs instead of one key for each row.
• REST catalogs: Clients negotiate the endpoints that the server advertises. They use OAuth2 authentication for each session.

The upgrade also aligns Iceberg storage with OpenDAL v0.58.

Performance

Spice Cayenne serves lookups, loads, and refreshes faster in this release. Each number below comes from the test in the linked pull request. The tests ran on different machines, so do not compare numbers between them.

Spice Cayenne

• Steady query latency during refreshes: Full and append refreshes now write on dedicated refresh threads by default. Before, a large refresh write ran on the threads that serve queries. In a test, a client sent 500 point-lookup joins per second to a Cayenne table of 2.2 million rows and about 8 GB. P99 latency during each full-refresh commit fell from 130–690 ms to the steady 44–50 ms. With runtime.params.dedicated_thread_pool: disabled, refresh writes run as before. See #15262.
• Primary-key point lookups: Cayenne serves uncached primary-key point lookups about twice as fast. On a file-mode table of 10 million rows, P50 latency improved 2.11–2.41× at 1 to 128 clients. See #14433.
• Lookups after the first load: Uncached primary-key lookups right after the first full refresh are faster. On a table of 10 million rows, P50 latency at one client fell from 6.0 ms to 1.1 ms. At 16 clients, P99 latency fell from 16.3 ms to 3.9 ms. See #14314.
• First refresh after a load: Cayenne builds the primary-key filter during the first load, so the first append refresh does not rebuild it. On a keyed table of 40.2 million rows, the first append refresh fell from a median of 4.2 s to 0.11 s. See #15099.
• Keyed bulk loads: A first load into an empty keyed table no longer routes each row separately. On an Iceberg table of 40.2 million rows, the slowest first load fell from 21.0 s to 13.5 s. Iceberg scans also decode at the session batch size instead of 1,024 rows. See #15076 and #15072.
• Append visibility on clustered tables: Cayenne orders each scan partition of a whole-table rewrite separately, then merges them. On an append table of 20 million rows with cayenne_cluster_by, P99 time from append to query visibility fell from 17.2 s to 3.0 s. See #14488.
• Append-only ingestion: Small-file compaction no longer stalls writers on tables without a primary key or position deletes. In an OTLP log-ingestion test, export P99 latency fell from about 1.2 s to 0.12 s. Throughput rose from about 210,000 to 775,000 records per second. See #15227.
• Metastore writes: Cayenne batches manifest rewrites in its SQLite metastore and runs every write on one writer connection. At 128 clients, P99 latency of a 1,000-file manifest rewrite fell from 5.39 s to 374 ms. See #14369.
• Filtered key deletes: A filtered delete on a keyed table now reads only the key and filter columns. In a test, P99 latency of a key-only delete fell from 27.7 ms to 8.9 ms. See #14374.

Caching

• Caching mode under sustained misses: Cayenne datasets with refresh_mode: caching now batch cache replacements. In a test with 500 lookups per second over 2.5 million random keys, request P99 latency fell from 363 ms to 9 ms. Dropped cache fills fell from 122,508 to 0. See #15253.
• Caching mode without a primary key: Lookups stay fast when the cache replaces entries continuously on a dataset without a primary key. In a two-core test at 300 lookups per second, P99 latency fell from 1,377 ms to 9.3 ms. See #15362.
• SQL results-cache expiry: Expiry work now grows with the number of expired entries, not with the size of the cache. With 3 million entries, insert P99.9 latency fell from 10.8–15.7 ms to 70–84 µs. See #14919.

Object-Store Listings

• Metadata-only queries: GROUP BY, DISTINCT, MIN, and MAX over partition or metadata columns no longer scan whole files. Spice answers them from the file listing, or from the first record of each file. count, sum, and avg still scan the files. See #14116.
• Listing pruning: Filters on _last_modified and other metadata columns now prune the file listing, so Spice does not open the excluded objects. Timestamp bounds joined by AND prune the listing. Filters with OR or NOT read the full listing. See #14265 and #14303.

AI Decisions in SQL and HTTP

SQL queries and HTTP clients can now ask a model for yes-or-no answers, probabilities, labels, and scores. This release adds five SQL decision functions and an OpenAI-compatible POST /v1/decisions API. TypeSafe decision models return calibrated probabilities. Any chat model can also answer decisions, but its probabilities are uncalibrated estimates.

Add a TypeSafe model to the Spicepod:

models:
  - from: typesafe:jev
    name: support_model
    params:
      typesafe_api_key: ${secrets:TYPESAFE_API_KEY}

Then call a decision function in SQL. Each function selects its model with the model argument:

Function	Result
ai_if(input, condition)	BOOLEAN: true when the probability of yes exceeds 0.5.
ai_probability(input, condition)	DOUBLE: a probability from 0 to 1.
ai_classify(input, labels)	VARCHAR: one of the supplied labels.
ai_score(input, instructions, levels)	DOUBLE: a probability-weighted score.
ai_decide(input, questions)	STRUCT: answers, probabilities, and confidence.

SELECT
    id,
    ai_classify(body, ['billing', 'technical', 'account'],
                model => 'support_model') AS team,
    ai_probability(body, 'The customer requests a refund',
                   model => 'support_model') AS refund_probability
FROM tickets
WHERE status = 'open';

See AI decisions.

S3 Event-Driven Refresh

With refresh_mode: changes, accelerated S3 datasets read new objects from S3 when SQS delivers S3 event notifications. Set refresh_mode: changes and s3_changes_queue_url together. The queue must serve only this dataset, and from must be a prefix, not a single object or a glob:

datasets:
  - from: s3://my-bucket/events/
    name: events
    params:
      file_format: parquet
      s3_region: us-east-1
      s3_changes_queue_url: ${secrets:events_queue_url}
    acceleration:
      enabled: true
      mode: file
      refresh_mode: changes

A periodic listing of the prefix (s3_changes_backfill_interval, default 1h) loads objects whose notification did not arrive. Spice loads each object key one time. By default, rows from deleted objects stay in the acceleration (s3_on_object_removed: ignore).

Acceleration Snapshots

Spice.ai Enterprise feature. See Acceleration Snapshots.

A dataset can now serve published acceleration snapshots without its original source connector. Set file_format: snapshot, and set from to the snapshot location:

datasets:
  - from: s3://my-bucket/spice/snapshots/
    name: orders
    params:
      file_format: snapshot
      s3_region: us-east-1

Snapshots now expire after 28 days, except for the current and newest snapshot of each dataset. A reader that cannot load a newer snapshot keeps serving its loaded snapshot and tries again on the next poll. Each failure increments dataset_acceleration_refresh_errors{mode="snapshot", reason="snapshot_download_failed"}. OSS builds refuse snapshot datasets and do not publish snapshots. See Serve a dataset from published snapshots and Shared State and Snapshot Migration.

One Location for Shared Runtime State

runtime.state sets one object-store location for the state that the runtime keeps across restarts and instances. Each of these features stores its state in its own subdirectory:

• Cluster state
• SQL results-cache warmup
• Shared HTTP rate control (Spice.ai Enterprise)
• Acceleration snapshots (Spice.ai Enterprise)

runtime:
  state:
    location: s3://my-bucket/spice/
    params:
      s3_region: us-east-1

datasets:
  - from: postgres:orders
    name: orders
    acceleration:
      enabled: true
      mode: file
      snapshots: enabled

Earlier per-feature locations, such as runtime.scheduler.state_location, still work but are deprecated. See Shared State and Snapshot Migration.

SQL Results Cache and HTTP Caching

SQL Results-Cache Warmup

After a restart, the first queries can now read from a warm SQL results cache. The cache records query shapes and stores them. When Spice starts again, it replays the stored shapes after the first full or append refresh of each accelerated dataset:

runtime:
  caching:
    sql_results:
      enabled: true
      warmup: on_first_refresh

Spice records the first 10 distinct shapes of cacheable, unauthenticated queries. Queries that differ only in equality-filter values share one shape. During replay, warmup reads up to 1,024 distinct combinations of filter-column values for each shape. It reruns the shape with each combination until the cache is full. Datasets do not report ready until warmup completes. The runtime does not start when a Spicepod sets both warmup and cache_key_type: sql.

Cache Backend

The SQL results, search, and embedding caches now use the Spice sharded cache. The runtime accepts engine: moka and engine: pingora for compatibility, but these values no longer select a backend.

Caching Refresh Mode

These changes apply to datasets with refresh_mode: caching, such as HTTP datasets:

• Shared cache-miss fetches: Concurrent cache misses for the same key now share one origin fetch.
• stale-if-error: Cached HTTP responses now follow the RFC 5861 stale-if-error rules.

New Data Connectors and Formats

Hugging Face Datasets

The Hugging Face data connector queries and accelerates Parquet, CSV, TSV, JSON, and ORC files from the Hub:

datasets:
  - from: hf://datasets/stanfordnlp/imdb/plain_text/
    name: imdb
    acceleration:
      enabled: true

ORC and Object Metadata

Listing connectors now support file_format: orc. Enable metadata columns to query the location and modification time of each row's source object:

datasets:
  - from: s3://my-bucket/events/
    name: events
    params:
      file_format: orc
      s3_region: us-east-1
    metadata:
      _location: enabled
      _last_modified: enabled

For example, query objects modified since October 1:

SELECT DISTINCT _location, _last_modified
FROM events
WHERE _last_modified >= TIMESTAMP '2026-10-01 00:00:00';

Apache Iceberg Without a Catalog

Apache Iceberg datasets can read a table directly from its root directory on local or object storage. Spice finds the current metadata file on every query and every refresh, so it reads new commits without a restart:

datasets:
  - from: iceberg:s3://my-bucket/warehouse/db/taxis
    name: taxis
    params:
      iceberg_s3_region: us-east-1

To read one fixed table version, set iceberg_metadata_path to a metadata file. A pinned dataset is read-only, so it fails to load with access: read_write:

datasets:
  - from: iceberg:s3://my-bucket/warehouse/db/taxis
    name: taxis_v3
    params:
      iceberg_s3_region: us-east-1
      iceberg_metadata_path: /metadata/00003-8f2c.metadata.json

Iceberg Append Refresh Follows Table Snapshots

An Iceberg dataset with refresh_mode: append and a time_column now loads the rows that each new commit adds, whatever their time values. This applies to Cayenne datasets and to datasets without a primary key. Before this release, an append refresh read only rows later than the stored maximum time minus refresh_append_overlap. A writer that committed out of time order could put rows outside that window. The refresh now saves the last Iceberg snapshot that it loaded, and the next refresh reads the data files that later snapshots added. See append refresh.

Operations and Observability

The runtime can now send structured logs through the existing OTLP exporter:

runtime:
  telemetry:
    otel_exporter:
      endpoint: otel-collector:4317
      signals: [metrics, logs]
  query:
    log_errors: execution

Log records carry severity, typed attributes, and the query trace ID. The runtime batches delivery on a dedicated export runtime. signals defaults to [metrics].

runtime.query.log_errors	Behavior
none (default)	Memory refusals at WARN, other query failures at DEBUG.
execution	Also logs execution failures at WARN.
all	Also logs syntax and planning failures at INFO.

Failure records contain error_code and datasets. They do not contain SQL text. See runtime.query.log_errors and observability.

Other changes:

• Adaptive HTTP rate control: HTTP rate control now reduces admission after upstream failures and stays within the configured request limits. In Spice.ai Enterprise, instances that share runtime.state also share per-second and per-minute limits. See HTTP rate control.
• Open-file limits on Unix: The runtime raises its soft open-file limit to the hard limit at startup. On macOS, kern.maxfilesperproc also bounds the result. If the hard limit remains too low, the Cayenne error names the effective limit and the remedy.
• Shared-state metrics: state_store_request_duration_ms{feature, operation}, state_store_conflicts_total{feature}, and state_store_cleanup_deleted_total{feature} report requests, conditional-write conflicts, and cleanup.
• Prometheus scrapes: Derived summary families emit one HELP and TYPE block per family. Strict parsers no longer reject the scrape for repeated metadata.
• Datadog dashboard: New panels cover snapshot freshness, Cayenne segment caches, and pod health.

Security Improvements

• Brotli decoder (CVE-2026-111497): The DuckDB that Spice bundles no longer writes out of bounds when it decodes a malformed Brotli-compressed Parquet page. Most Parquet reads in Spice do not go through DuckDB. The fix matters where DuckDB decodes a Parquet file that an untrusted party can supply.
• Flight sessions: Authentication precedes session creation. Sessions remain bound to their creating principal. TLS handshakes have a 10-second deadline, and Flight SQL statement updates count against the Flight write limit.
• HTTP paths: The HTTP connector rejects request_path values that URL parsing would rewrite outside their allowed paths. This covers encoded dot segments and control-character rewrites.

Bug Fixes

The fixes with the widest effect are below. The changelog lists every fix.

• Wrong rows from Cayenne queries: Cayenne queries no longer return wrong rows from maintained views, Vortex IN lists, or shared dynamic filters. See #14427.
• NOT IN and NULL values: Queries with NOT IN no longer lose their NULL semantics after a join reorder or the Cayenne sort-merge rewrite. See #14429.
• Wrong results from DataFusion: The DataFusion upgrade includes upstream fixes for wrong results from filter pushdown, expression simplification, and planning. See #14430.
• Federated pushdown: Federated queries no longer send some casts and filters to a source that evaluates them differently from Spice. Affected sources include DuckDB, PostgreSQL, MySQL, BigQuery, DynamoDB, Cosmos DB, and MongoDB. See #14484, #14601, and #14419.
• Accelerated HTTP datasets: An HTTP error response no longer replaces the rows of an accelerated HTTP dataset. See #13538.
• Unavailable sources at startup: A dataset whose source is unreachable at startup now retries. An existing acceleration keeps serving queries while its source is unavailable. See #14623 and #14624.
• Stale plans and results after a reload: Queries no longer use cached plans or results after a hot reload replaces a catalog or unloads a dataset. See #13914 and #14365.
• CDC positions at shutdown: Change-data-capture sources now record their positions before shutdown closes the accelerations. See #14702.
• Tool calls: Tool-use streams no longer end on a stale tool_calls finish. /v1/responses runs Spice tools and honors tool_recursion_limit when it streams. See #14548 and #14930.
• Vector search on chunked columns: vector_search returns its top k rows by score on chunked and multi-vector columns. See #15056.

Dependency Updates

Dependency / Component	v2.3.3	v2.4.0
Apache DataFusion	v54.1.0	v55.2.0
Apache Arrow	v58.3.0	v59.3.0
LF+AI Vortex	v0.79.0	v0.86.1
Apache Iceberg	v0.10.1	v0.11.0
Apache Ballista	v54.0.0	v55.0.0
Turso	v0.7.2	v0.8.2
ADBC	v0.23.0	v0.24.0
Rust toolchain	v1.97.1	v1.98.1

Breaking Changes

Accelerator Selection and Primary-Key Policies

Spice Cayenne is now the default accelerator on supported platforms. Datasets and views without acceleration.engine switch from Arrow to Cayenne. Native Windows keeps Arrow. If datasets rely on another accelerator's behavior, review the Cayenne migration guide before the upgrade.

Other accelerators are deprecated for removal in v2.5. This applies to Arrow, DuckDB, SQLite, Turso, and PostgreSQL acceleration, and to the DuckDB vector engine. They remain available in v2.4. A startup warning groups affected components by accelerator.

On native Windows, run Spice in Docker or WSL to use Cayenne before v2.5.

Some datasets that loaded on Arrow fail to load on Cayenne. These configurations loaded on Arrow when they did not set engine. They now fail to load unless the dataset sets engine: arrow:

• partition_by in memory mode, or with no mode
• A column of type Interval, Duration, FixedSizeBinary, Union, or RunEndEncoded, such as a PostgreSQL interval column
• A Debezium or cdc: source with no key
• A numeric time_column with retention_period
• maintained_aggregates with MIN or MAX and no primary key

To retain Arrow temporarily in v2.4, name it explicitly:

acceleration:
  enabled: true
  engine: arrow

Plan acceleration migration before v2.5. Do not delete acceleration-only data to change accelerators. Keep those datasets on v2.4 until a supported migration can preserve their data. The migration guide describes other limitations and vector-score changes.

on_conflict is deprecated for removal in v3.0. Cayenne accepts the field but no longer applies its policy. A primary key selects automatic version resolution. Before an upgrade, review drop policies and policies that reject conflicts. Other accelerators retain their existing policies in v2.4.

on_conflict no longer routes writes to the acceleration, for all accelerators. In v2.3, a read-write dataset that set on_conflict sent its writes to the acceleration, so its source did not need to accept writes. In v2.4, only write_mode selects where writes go. Such a dataset does not load if its source cannot accept writes. To keep writes in the acceleration, set write_mode: acceleration:

datasets:
  - from: s3://my-bucket/orders/
    name: orders
    access: read_write
    params:
      file_format: parquet
    acceleration:
      enabled: true
      mode: file
      primary_key: id
      write_mode: acceleration

SQL Results-Cache TTL and Freshness

SQL results-cache entries now expire only by TTL. Full and append refreshes, CDC batches, and retention passes no longer invalidate them. Queries can return data from before the latest refresh until item_ttl expires. The stale_while_revalidate_ttl setting can extend this period. Review both settings before the upgrade. Reduce them if results must reflect refreshed data sooner. See Cached Results and Dataset Refresh.

HTTP Rate-Control Wait Timeouts

If rate_control_acquire_timeout is unset, an HTTP request now waits up to the connector's client_timeout for rate-control capacity. Requests previously waited without a limit. Set rate_control_acquire_timeout: 0 under the dataset's params to retain unbounded waits. Otherwise, choose a duration that matches the request budget, such as 30s.

Datasets that share an HTTP origin share one rate controller. They must use the same max_concurrent_requests, requests_per_second_limit, requests_per_minute_limit, rate_control_jitter_min, rate_control_jitter_max, rate_control_failure_threshold, and rate_control_window values. If the values differ, the dataset fails to load with an error that names these settings. See rate control parameters.

Iceberg REST Compatibility

GET /v1/namespaces/{namespace}/tables/{table} now returns HTTP 400 with BadRequestException for accelerated datasets, views, and other unsupported datasets. The endpoint no longer synthesizes metadata for those datasets. For schema discovery, use SQL DESCRIBE or information_schema.columns. For queries, use /v1/sql or Arrow Flight SQL. Eligible unaccelerated Iceberg datasets expose their source metadata, and clients need their own storage credentials.

Instance Identity and Cluster Upgrades

Instance identity resolves in this order: nonempty SPICE_INSTANCE_ID, HOSTNAME, COMPUTERNAME, then the host name. A scheduler's default changes from host:port to the host name. Executors also register under the instance identity.

If several schedulers or executors share a host, give each a distinct SPICE_INSTANCE_ID. Upgrade schedulers before executors. See distributed query. Older schedulers cannot fan out task-history requests to newer identity-based registrations. All cluster components must use compatible Ballista protocol versions.

PostgreSQL replication slots keep their names when a nonempty identity environment variable already selected the instance name. Without those variables, the host-name fallback creates a new slot. An empty SPICE_INSTANCE_ID also changes the previous slot name. After the new slot works, drop the unused old slot to stop its WAL retention. See PostgreSQL replication:

SELECT pg_drop_replication_slot('<old slot name>');

Shared State and Snapshot Migration

The top-level snapshots section and runtime.scheduler.state_location and params are deprecated. They continue to work, and when set they take priority over runtime.state for their feature. Prefer runtime.state, with per-dataset overrides (acceleration.snapshots_location, acceleration.snapshots_params) when necessary.

For upgrades from v2.4.0-rc.1, move state from the shared root into these paths:

State	Subdirectory
Cluster scheduler	cluster/
Rate control	rate-control/
SQL results-cache warmup	results-cache/

An explicit legacy scheduler location retains its earlier layout until migrated. In rc.1, Azure state ignored the URL path and used the container root. Move that state into the configured path and feature subdirectory.

The configured store must support writes and conditional writes. The runtime refuses newer state formats and stores that ignore write conditions. Check storage permissions before the upgrade.

Snapshots now expire automatically after 28 days, except for each dataset's current and newest snapshot. Rollback cannot select a pruned snapshot. Review external retention requirements before the upgrade.

Upgrade snapshot readers before, or alongside, their writers. Cayenne writes format 2 of the metastore slice when a snapshot contains partition children or pending write-back markers. Older readers reject that format. Other slices retain format 1.

Cayenne cannot publish acceleration snapshots for datasets with cayenne_datalake_location. Review snapshot settings for those datasets.

For a serving snapshot reader, bootstrap_on_failure_behavior: retry no longer retries a newer snapshot indefinitely inside one refresh. The reader reports the failure and retries at the next poll. Startup behavior does not change.

Cayenne Tuning Settings

Replace retired per-dataset and per-catalog tuning settings with runtime settings:

Previous setting	Replacement
cayenne_tuning: adaptive	runtime.adaptive_tuning: enabled
cayenne_tuning: auto	runtime.adaptive_tuning: disabled
cayenne_goal_replication_lag	runtime.params.target_replication_lag
cayenne_goal_freshness	runtime.params.target_freshness
cayenne_goal_query_latency	runtime.params.target_query_latency
cayenne_goal_convergence_window	runtime.params.target_convergence_window
cayenne_goal_qph	runtime.params.target_qph

The runtime no longer applies the old names. It warns with their replacements. Targets are runtime-wide. runtime.adaptive_tuning accepts only enabled or disabled. A Boolean such as true fails Spicepod loading.

Upgrading

Review Breaking Changes before upgrading. Upgrade all distributed cluster components together, with schedulers before executors.

Upgrade snapshot readers before, or alongside, their writers so they can read the newer Cayenne metastore slice format.

To upgrade to v2.4.0, use one of these methods:

CLI:

spice upgrade v2.4.0

Homebrew:

brew upgrade spiceai/spiceai/spice

Homebrew follows the version published in the tap.

Docker:

docker pull spiceai/spiceai:2.4.0

See Docker Hub for available tags.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.4.0

AWS Marketplace:

Spice is available in the AWS Marketplace. Marketplace availability follows its published versions.

Cookbook Updates

• New Decisions API recipe: call /v1/decisions with TypeSafe Jev and GPT-6 Luna.

The Spice Cookbook provides recipes to help you get started with Spice.

Contributors

• @bjchambers
• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov

What's Changed

Changelog

• fix(runtime): discard cached logical plans when a hot reload replaces a catalog (fixes #13910) by @claudespice in #13914
• fix(acceleration): let a schema repair correct a checkpoint without resetting the freshness clock (fixes #13817) by @claudespice in #13894
• fix(search): filter a chunked Elasticsearch delete on a field that can match the key (fixes #13714) by @claudespice in #13926
• fix(search): classify a partially non-finite embedding as unindexable on every backend (fixes #13872) by @claudespice in #13902
• fix: stabilize GitHub tests and bound GraphQL registration (fixes #13762) by @lukekim in #13939
• fix(postgres): decode versioned JSONB binary replication values by @phillipleblanc in #13962
• fix(cayenne): release the keyset bytes an abandoned PK checkout accounted (fixes #13668) by @grokspice in #13925
• fix(duckdb): deny the regexp built-ins DuckDB cannot answer faithfully (fixes #13809) by @claudespice in #13871
• fix(search): drop the chunks a shortened row no longer produces from a chunked index (refs #13717) by @claudespice in #13960
• Reduce Cayenne allocations during primary-key validation and filtering by @lukekim in #14009
• fix: clarify OpenDAL S3 retry warnings by @lukekim in #14040
• feat(cayenne): materialize multi-reference CTEs on the query path by @lukekim in #13918
• fix(smb): pad an empty CREATE buffer so Samba lists the share root (fixes #13293) by @grokspice in #14050
• feat(mcp): support MCP specification 2026-07-28 (dual-era) by @lukekim in #14043
• feat(connectors): add ORC listing format via in-repo FileFormat by @lukekim in #14075
• feat(cloud-connect): add a GetDatasets command that answers the /v1/datasets document (refs #13369) by @grokspice in #14051
• Suppress Cayenne startup logs when no Cayenne dataset is configured by @Jeadie in #14042
• fix(cache): re-bind parameter values when revalidating a stale result (fixes #14099) by @bjchambers in #14100
• fix(deps): remediate open Dependabot alerts by @phillipleblanc in #14111
• feat(caching): Make caching_stale_if_error RFC-5861 compliant (with stale-if-error header) by @Jeadie in #14134
• perf(runtime-table): defer cache-eviction key extraction to entries a delete actually names by @Jeadie in #14138
• fix(runtime): report the acceleration.ready_state deprecation once per component (fixes #13749) by @claudespice in #14006
• fix(runtime): write the inferred Arrow sort order under the prefixed key its validation accepts (fixes #14023) by @claudespice in #14032
• fix(connectors): Fix JSON/Orca files using metadata columns by @Jeadie in #14115
• fix(turso): build a dictionary column directly so a dictionary over a list, map or boolean value reads back (fixes #13033) by @grokspice in #14181
• fix(github): paginate nested GraphQL connections and pace to GitHub's rate limits by @lukekim in #14179
• fix(vortex): stop an IN list holding a NULL from panicking the scan by @krinart in #14163
• fix(runtime-table): stop rebuilding SessionContext on every cache fetch by @Jeadie in #14141
• fix(runtime): count a first load as still loading in the Dataset load summary (fixes #13974) by @claudespice in #14020
• fix(duckdb): push regexp_count down again at a rendering that counts as the kernel does (fixes #13870) by @claudespice in #14153
• fix(cayenne): build the DELETE sink inside the execution-time write lock (fixes #13828) by @claudespice in #14218
• fix(cluster): decide what a Flight message carries by its IPC header, not its body length (refs #13737) by @claudespice in #14212
• fix(runtime): install the Spice function deny-list on the PostgreSQL catalog connector (refs #13664) by @claudespice in #14225
• perf(cayenne): share inline-cache view entries by Arc instead of cloning them per scan by @krinart in #14191
• feat(cache): Spice sharded cache as the sole LruCache engine by @lukekim in #14206
• feat: TypeSafe Jev System One evaluation provider by @lukekim in #14215
• fix(cayenne): tag file statistics bounds as the column's Arrow type (fixes #14280) by @phillipleblanc in #14283
• fix(cayenne): coalesce inline writes into one batch by @sgrebnov in #14279
• fix(cayenne): count each Arrow allocation once in the inline-cache gauge by @krinart in #14272
• feat(s3): SQS event-driven changes for refresh_mode: changes by @lukekim in #14121
• feat(caching): single-flight coalesce concurrent cache-miss fetches by @Jeadie in #14142
• fix(caching): make caching_stale_if_error detect transient HTTP failures on real schemas by @krinart in #14161
• Use Cayenne secondary indexes for dynamic join filters by @phillipleblanc in #14282
• fix(cli): surface the full error chain for spice chat connection failures by @krinart in #14289
• fix(runtime-table): serialize a direct write against acceleration snapshot creation (fixes #13548) by @claudespice in #14310
• fix(duckdb): screen regexp_like and regexp_replace as regexp_count is screened (fixes #14148) by @claudespice in #14321
• fix(http): honor retry budget without an extra origin request by @phillipleblanc in #14313
• Fix SchemaCastScanExec's schema conversion in fn partition_statistics by @Jeadie in #14258
• fix(cluster): recognise a Flight keepalive by its empty envelope, not by what its header declares (fixes #13737) by @claudespice in #14327
• perf(http): defer zero-TTL acceleration lookup until origin failure by @phillipleblanc in #14302
• fix(cayenne): keep a key visible when it is re-inserted over a stale-insert tombstone by @sgrebnov in #14312
• perf(cayenne): read only key and filter columns in filtered key deletes by @sgrebnov in #14374
• fix(cayenne): let small protected-snapshot merges run during a long large-tier merge by @sgrebnov in #14296
• perf(cayenne): serve primary-key lookups on a freshly loaded table in ~1 ms by @lukekim in #14314
• fix(cayenne): skip min/max statistics for nested columns (fixes #14368) by @sgrebnov in #14392
• fix(turso): keep a quantified comparison out of Turso SQL (fixes #14041) by @grokspice in #14393
• perf(cayenne): batch metastore manifest rewrites, upsert in place, and run every write on one writer connection by @lukekim in #14369
• fix(runtime-table): ignore zero-row stale fallback batches by @phillipleblanc in #14331
• Prune object-store file listing by _last_modified predicates by @Jeadie in #14265
• Reading only partition or metadata columns needlessly scans all file contents by @Jeadie in #14116
• fix(duckdb): keep a concat over a binary operand out of the federated plan (fixes #13915) by @claudespice in #14333
• fix(federation): keep a correlated subquery predicate above a join of two sources (refs #8220) by @claudespice in #14372
• fix(caching): snapshot staleness before the origin fetch for stale_if_error by @Jeadie in #14263
• perf(snapshots): skip unchanged snapshot metadata with a conditional GET by @sgrebnov in #14409
• fix(search): prune the rest of a key group from an Elasticsearch chunked index (refs #13717) by @claudespice in #14320
• fix(graphql): debit a LIMIT by the rows a page returned, not the declared page size (fixes #14308) by @claudespice in #14353
• fix(snapshots): resolve snapshot entries relative to the metadata location (#14425) by @sgrebnov in #14426
• Lower GitHub connector default concurrency limit to 4 by @lukekim in #14431
• fix(cache): don't report an invalidation cancelled by runtime shutdown as a failure by @grokspice in #14417
• fix(runtime): keep the built-in date_part so both weekday spellings agree (fixes #13920) by @claudespice in #14154
• fix(cayenne): include the in-memory CDC tier when an overwrite or a retention pass covers the whole table by @lukekim in #14428
• fix: keep NOT IN null-aware through the join reorder and the Cayenne sort-merge rewrite by @lukekim in #14429
• fix(cayenne): discard a compaction whose snapshot an overwrite replaced mid-pass by @lukekim in #14432
• fix(cayenne): stop maintained views, Vortex IN lists and dynamic-filter sharing from returning wrong rows by @lukekim in #14427
• fix(cayenne): hide spilled rows on CDC upsert fallback by @bjchambers in #14416
• fix(cayenne): judge a filtered key delete's captured sources by the index captured with them (refs #13913) by @claudespice in #14455
• fix(cayenne): correctness fixes for the goal-driven adaptive controller, with a closed-loop simulation harness by @lukekim in #14443
• fix(cdc): keep the newest source commit timestamp on a coalesced change batch by @lukekim in #14463
• fix(cayenne): draw a sequence for a current-snapshot append so per-key OCC can order it (fixes #13685) by @claudespice in #14360
• fix(http): name a configured model's load failure instead of reporting it not found (fixes #13303) by @claudespice in #14395
• fix(duckdb): keep inferred source indexes off change-stream accelerations so upserts commit under concurrent reads (refs #13929) by @claudespice in #14396
• fix(duckdb): keep a text cast over a binary operand out of the federated plan (fixes #14355) by @claudespice in #14448
• fix(llms): honor tool_choice required and allowed_tools on mistral.rs-hosted models instead of panicking (fixes #14230) by @claudespice in #14460
• fix(runtime): stop a replaced dataset configuration's load from registering over the new one (fixes #1458) by @claudespice in #14367
• fix(install): stop asking for sudo on a first install into a fresh HOME (fixes #14445) by @claudespice in #14495
• perf(cayenne): serve primary-key point lookups in half the time by @lukekim in #14433
• fix(deps): bump DataFusion for upstream fixes to wrong results from filter pushdown, simplification and planning by @lukekim in #14430
• fix(runtime): size every internal DataFusion session from the CPU budget by @bjchambers in #14412
• fix(runtime): serve nested, zoned and half-float columns from the Iceberg catalog API (fixes #4815) by @claudespice in #14480
• fix(acceleration): keep cached results when a snapshot refresh finds no newer snapshot by @sgrebnov in #14497
• fix(cayenne): stop the small-file compaction of a position-mode PK table from deadlocking on its own write lock (fixes #14420) by @claudespice in #14481
• fix(runtime-table): report refresh bytes for the rows each batch holds by @Jeadie in #14469
• fix(spark,databricks): keep Spice-only functions out of SQL sent to Spark Connect and Databricks SQL Warehouse (refs #13664) by @claudespice in #14498
• fix(vortex): size a cached footer by what it retains, not its serialized bytes (fixes #12917) by @claudespice in #14502
• fix(cayenne,telemetry): Register accelerated sink dataset immediately from existing acceleration by @peasee in #13955
• fix(cayenne): keep a rewrite's count inexact when it retains a late protected snapshot (fixes #14383) by @claudespice in #14385
• fix(postgres): release a shared slot's hold on a table its publication cannot drop (fixes #13032) by @claudespice in #14527
• fix(cayenne): size the build side before sort-merging a non-Cayenne outer join by @krinart in #14520
• fix(deps): move the DataFusion pin past the four unparser fixes, and guard each of them (fixes #13022) by @grokspice in #14570
• fix(dynamodb, cosmosdb, mongodb): push filters down only where the source evaluates them as SQL does by @lukekim in #14419
• feat(snapshots): serve a dataset from published snapshots with from: s3://… and file_format: snapshot by @lukekim in #14529
• perf(cayenne): keep the per-shard PK index across checkpoint flushes and back off futile bakes; fix q17 and two wrong-results races (fixes #14235) by @lukekim in #14555
• fix(graphql): show the parse-failure bytes in JSON decode previews by @Jeadie in #14530
• Defer source-first cache fallback planning by @phillipleblanc in #14556
• perf(cayenne): order whole-table rewrites per scan partition, then merge by @bjchambers in #14488
• fix(graphql): retry an inferred credential refusal instead of failing the refresh by @Jeadie in #14539
• fix(cayenne): serve a widened table's files from their persisted statistics (refs #13829) by @claudespice in #14316
• fix(acceleration): checkpoint DuckDB's write-ahead log before a snapshot copies its file (fixes #13912) by @claudespice in #14325
• fix(models): apply a tool_choice that forces a call to one round of the tool-use loop (fixes #14459) by @claudespice in #14635
• fix(runtime): don't end a tool-use stream on a stale tool_calls finish (fixes #13309) by @claudespice in #14548
• fix(smb): name listed locations under the share so directory datasets load (fixes #14060) by @claudespice in #14549
• fix(models): name a configured model's load failure in ai() and streaming nsql (fixes #14394) by @claudespice in #14632
• fix(cayenne): record sharded CDC keys in the table-wide PK index by @lukekim in #14603
• fix(sqlite): keep TRY_CAST and every cast SQLite answers differently out of the federated plan (fixes #14398) by @claudespice in #14496
• fix(snapshots): retry a failed snapshot attempt with backoff by @sgrebnov in #14676
• fix(snapshots): large snapshots no longer fail bootstrap on slow connection by @sgrebnov in #14571
• fix(refresh): start a refresh's source scan only when the sink reads it, so the full-text index keeps its rows (fixes #14619) by @claudespice in #14663
• Evaluate any chat model via /v1/evaluate by @lukekim in #14568
• fix(cayenne): stop snapshotting datasets with a datalake tier, whose restored copies deleted each other's files by @lukekim in #14583
• fix(cayenne): count each Arrow allocation once for the mem-tier limit and checkpoint write sizing by @krinart in #14675
• fix(runtime): retry a dataset whose source is unreachable at startup (fixes #14609) by @bjchambers in #14623
• fix(cayenne): archive only the snapshot directories a dataset snapshot references (fixes #14605) by @sgrebnov in #14627
• fix(smb): serve every share on a host from the one store registered for it (fixes #14550) by @grokspice in #14689
• fix(runtime): discard cached plans and results when a dataset is unloaded (refs #14251) by @claudespice in #14365
• feat(snapshots): reload snapshot-mode datasets from S3 event notifications (SQS) by @lukekim in #14335
• fix(cayenne): report runs per size tier when protected-snapshot compaction declines (fixes #13622) by @claudespice in #14476
• fix(cayenne): re-run a post-write compaction pass a concurrent append asked for (fixes #13906) by @claudespice in #14479
• fix(http): keep the request URL out of HTTP connector errors (fixes #13534) by @claudespice in #14490
• fix(runtime): load every localpod dataset that reads from one parent at startup (fixes #13087) by @claudespice in #14532
• fix(duckdb): push regexp_replace down again for a one-digit group reference, and refresh the Postgres ClickBench q35 plan (refs #13966) by @claudespice in #14552
• fix(duckdb): keep a cast into binary out of the federated plan (refs #14397) by @claudespice in #14633
• fix(runtime): keep finished async query jobs finished, and delete expired ones by @lukekim in #14585
• fix(federation): keep DataFusion's cast built-ins out of every federated plan (fixes #14444) by @claudespice in #14484
• fix(runtime): decide every Spark/built-in function collision by name, and refuse an undecided one (fixes #14361) by @grokspice in #14400
• fix(runtime-component): skip an inferred Cayenne index on a floating-point column instead of failing the load (fixes #14590) by @grokspice in #14599
• fix(cache): serve stale cached results during frequent table updates (stale_while_revalidate_ttl) by @sgrebnov in #14708
• Update Turso to 0.8.1 and retry write conflicts a metastore statement raises by @lukekim in #14680
• fix(snapshots): stop a snapshot dataset's load when a reload replaces it by @lukekim in #14673
• fix(cayenne): hand non-partition filters to every partition scan (fixes #12959) by @claudespice in #14370
• fix(postgres-accel): resolve secret references in the sidecar connection parameters (fixes #13296) by @claudespice in #14544
• fix(cayenne): drain the in-memory CDC tier before a full rewrite scans it (fixes #14450) by @claudespice in #14486
• feat(caching): warm SQL results cache on first refresh from persisted plan shapes by @lukekim in #14178
• fix(federation): keep a fractional-to-integer cast out of the plans pushed to DuckDB, PostgreSQL, MySQL and BigQuery (fixes #14482) by @grokspice in #14601
• fix(federation): keep arrow_typeof and its plan-introspection siblings local on every backend (fixes #14334) by @grokspice in #14695
• fix(cayenne): back off futile bakes under a violated query goal too by @lukekim in #14736
• fix(snapshots): bootstrap readers from late snapshot publications by @phillipleblanc in #14468
• fix(cayenne): apply retention_sql to a mode: memory acceleration (fixes #14045) by @claudespice in #14344
• fix(duckdb, sqlite): keep the first copy of a key a write repeats under on_conflict: drop (fixes #14629) by @claudespice in #14748
• fix(cache): release a removed dataset's memory when the plan cache discards its plans (fixes #14251) by @claudespice in #14760
• fix(llms): explain an Anthropic model's refusal of a sampling control instead of passing the bare 400 through (fixes #13564) by @grokspice in #14700
• fix(snapshots): allow cayenne_file_path and cayenne_metadata_dir on file_format: snapshot datasets (fixes #14696) by @sgrebnov in #14698
• feat(cayenne): keep the secondary index current across every write by @bjchambers in #14593
• fix(http): stop a non-2xx response body replacing an accelerated table's rows (fixes #13515) by @claudespice in #13538
• fix(chat-api): answer a tool-call-only assistant turn instead of panicking (fixes #13207) by @grokspice in #14232
• fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492) by @claudespice in #14670
• fix(spiceai, duckdb, sharepoint): name the Spicepod key a missing-parameter error asks for (refs #14446) by @claudespice in #14769
• fix(cayenne): keep the per-shard PK index when the table-wide index is discarded by @lukekim in #14604
• Upgrade to DataFusion 55.1 and Arrow 59.3 by @krinart in #14612
• Upgrade to iceberg-rust to 0.11.0 and DF 55.2 by @sgrebnov in #14771
• fix(cayenne): write an append into an empty unkeyed table as one load by @phillipleblanc in #14784
• fix(mysql_replication): don't re-send a live member's delivered commits after a reconnect by @lukekim in #14751
• fix(acceleration): accept time_format iso8601 when the time_column is already a timestamp by @lukekim in #14777
• fix(cache): store SQL results that go stale mid-query by @lukekim in #14710
• fix(cayenne): serve filtered and global maintained aggregates by @lukekim in #14761
• perf(cayenne): build the checkpoint's tombstone union after releasing the capture locks by @lukekim in #14759
• fix(cayenne): keep maintenance from deleting files a snapshot is archiving by @sgrebnov in #14789
• fix(snapshots): never overwrite shared snapshot metadata, and publish more than once to a file:// location by @lukekim in #14582
• perf(cdc): build deferred change rows in the reader while the apply runs by @lukekim in #14746
• fix(runtime,data_components): wait for change-data-capture sources to record their positions before shutdown closes the accelerations (fixes #14523) by @grokspice in #14702
• fix(graphql): detect non-JSON responses and treat gateway errors as retryable by @lukekim in #14781
• fix(runtime): require keys for s3_auth: key, honor gs:// state location params, and leave newer rate-control state alone by @lukekim in #14584
• fix(listing): skip zero-byte objects (S3 folder markers) in format-selected listings by @sgrebnov in #14822
• Avoid unnecessary repartitioning in indexed dynamic-filter joins by @bjchambers in #14799
• feat(cayenne): keep one row per primary key automatically and deprecate on_conflict by @bjchambers in #14726
• fix(object-store): stop reading a response body at its first error by @phillipleblanc in #14831
• fix(listing): keep a partition predicate as a residual filter on the _location fast path by @grokspice in #14790
• fix(federation): keep date, timestamp and interval values local on SQLite reached through ADBC or ODBC (fixes #14753) by @claudespice in #14840
• perf(runtime): skip EnsureRequirements while planning a point lookup by @lukekim in #14807
• feat(acceleration): make Cayenne the default acceleration engine by @phillipleblanc in #14837
• fix(cayenne): keep a join's LIMIT when the oversized-join rewrite makes it a sort-merge join by @lukekim in #14805
• fix(cayenne): fail contradictory write settings once, and order NULL times below every time by @bjchambers in #14847
• fix(runtime): serve an existing acceleration while its source is unavailable (fixes #14610) by @bjchambers in #14624
• Prune object-store file listing by metadata columns (#14264) by @Jeadie in #14303
• fix(vortex): fold partition values into the file-pruning predicate; test null-equal joins under mode:file by @lukekim in #14797
• feat(rate-control): adaptive, bounded and cluster-coordinated HTTP rate controls by @Jeadie in #14143
• fix(sql): align date_part('dow') with EXTRACT(dow) Sunday=0 by @lukekim in #14796
• fix(runtime): carry API-key principal into MCP tools/call by @lukekim in #14828
• fix(llms): un-mute the LLM integration suite and make it pass (fixes #6399; Bedrock deferred to #13558) by @Jeadie in #13656
• perf(cayenne): hold maintained-aggregate retraction state in a compact shared index by @lukekim in #14762
• fix(deps): keep a hash join's order once it has a dynamic filter, so vector_search plans again by @Jeadie in #14857
• feat(cayenne): persist secondary index runs, recorded in the metastore by @bjchambers in #14594
• fix(cayenne): fence schema statistics and control maintenance tests by @bjchambers in #14856
• fix(github): retry nested GraphQL pages in place and fail incomplete nested connections by @sgrebnov in #14862
• fix(cayenne): load an append dataset that has retention_sql and orders versions by time by @phillipleblanc in #14878
• fix(cdc): Cayenne replication needs only a primary key, not on_conflict by @phillipleblanc in #14880
• feat(connector-huggingface): Hugging Face datasets data connector (hf://datasets/...) by @lukekim in #14877
• fix(iceberg): Iceberg REST clients read the real table or get an error, never an empty one by @lukekim in #14588
• fix(federation): ask the executor for the SELECT 1 placeholder of an empty projection by @krinart in #14851
• fix(cayenne): float statistics must not leave NaN out of pruning or metadata answers (fixes #14719) by @claudespice in #14792
• fix(caching): a GET and a POST to one HTTP path no longer share a cache entry (fixes #14768) by @claudespice in #14884
• fix(anthropic): drop empty text blocks from assistant messages by @Jeadie in #14834
• fix(cayenne): ask the table whether it takes a versioned append, instead of predicting it by @lukekim in #14882
• fix(cayenne): a hash join over a secondary index reads the indexed table's files once (fixes #14933) by @lukekim in #14934
• fix(sharepoint): list drive items when the listing contains a folder (fixes #2797) by @phillipleblanc in #14946
• fix(runtime): log each failing dataset's load failure (fixes #14918) by @phillipleblanc in #14939
• fix(deps): move the DataFusion pin past spiceai/datafusion#256, so FlightSQL Substrait plans read string columns (fixes #14935) by @lukekim in #14944
• fix(cayenne): publish a keyed partition's append as a protected overlay (fixes #14947) by @lukekim in #14955
• fix(acceleration): a failed acceleration build no longer keeps the dataset down until restart (fixes #14917) by @phillipleblanc in #14942
• fix(postgres): reject a changes dataset without a primary key when it loads by @phillipleblanc in #14960
• fix(cluster): treat secrets the scheduler declines as not found on executors by @phillipleblanc in #14957
• fix(flight): return query and data errors as InvalidArgument, not Internal by @phillipleblanc in #14956
• fix(duckdb,sqlite): correct pushdown for string_agg ORDER BY, concat NULLs, and unsupported fns by @lukekim in #14798
• fix(llms): stream Anthropic responses that open with a thinking block by @krinart in #14920
• fix(arrow): serve lookups from enabled indexes; a repeated key disables the index with a warning by @sgrebnov in #14968
• feat(nsql): accept reasoning_effort on POST /v1/nsql by @lukekim in #14852
• fix(listing): scan file_extension '*' tables through ListingTable, so Glue joins broadcast again (fixes #14890) by @krinart in #14972
• fix(llms): load and serve an OpenAI model when a reply names an unknown service tier (fixes #14916) by @krinart in #14927
• feat(cayenne): always persist secondary index runs by @sgrebnov in #14979
• fix(cache): expire results-cache entries in insertion order instead of walking every LRU list by @bjchambers in #14919
• feat: decisions in SQL (ai_if, ai_probability, ai_classify, ai_score, ai_decide) and an OpenAI-compatible /v1/decisions by @lukekim in #14838
• fix(search): keep named arguments on search calls nested in rrf() (fixes #14923) by @phillipleblanc in #14975
• fix(rate-control): hold instances that set different limits for one origin to the lowest (fixes #14913) by @krinart in #14932
• fix(vortex): don't push down temporal casts Vortex can't evaluate by @sgrebnov in #14985
• fix(acceleration): recommend mode: file for CDC on_conflict migrations to Cayenne by @bjchambers in #14928
• fix(http): reject a request_path that URL parsing would rewrite by @bjchambers in #14965
• fix(clickhouse): load LowCardinality, DateTime64, Array, Map, Tuple, Enum, 128-bit integer and IP columns by @phillipleblanc in #14953
• Add anthropic_workspace_id param for Anthropic models by @Jeadie in #14829
• fix(acceleration): keep retention-evicted rows out of the on_zero_results source fallback by @lukekim in #14783
• fix(drasi): forward CDC creates and snapshots as upserts by @ewgenius in #14791
• fix(datafusion-optimizer-rules): answer only plain partition columns from the listing by @lukekim in #14806
• fix(connector-huggingface): retry Hub API responses that break off part way, and harden the live-Hub check by @lukekim in #15009
• fix(models): /v1/responses runs Spice tools and honors tool_recursion_limit when streaming (fixes #14905, #14631) by @krinart in #14930
• fix(cayenne): refuse acceleration data stored under a different partition_by by @phillipleblanc in #15019
• fix: Cayenne deletion-index bounds and reload fencing, scan-statistics single-flight, UDF filter pushdown, Server-Timing, results-cache determinism, and view loading by @lukekim in #14993
• fix(cayenne): load write-back without on_conflict, and refuse pk_conflict_detection: none where it loses data (fixes #14886, #14889) by @bjchambers in #14891
• feat(cayenne): runtime-wide adaptive_tuning and target_* settings by @peasee in #14844
• fix(snapshots): give each snapshot publish its own object name, so publishes in the same second stop overwriting each other by @phillipleblanc in #15028
• fix(glue): load Iceberg tables from S3 buckets with dots in the name by @phillipleblanc in #15053
• fix(runtime): give each runtime.state feature its own folder by @phillipleblanc in #15036
• fix(json): reject incomplete arrays and avoid repeated parsing by @phillipleblanc in #15054
• fix(cayenne): keep secondary indexes complete across acceleration snapshots by @sgrebnov in #15068
• feat(rerank): use evaluation models (TypeSafe) in rerank(), with per-query strategy by @Jeadie in #14782
• fix(cayenne): reclaim the snapshot directories a snapshot reader's reload supersedes by @sgrebnov in #15069
• feat(iceberg): read a table from a pinned metadata file without a catalog by @sgrebnov in #15081
• Fixes for MCP server by @Jeadie in #15041
• fix(search): vector_search returns its top k rows by score on chunked and multi-vector columns (fixes #14819) by @claudespice in #15056
• fix(postgres): keep applying CDC changes after a source column type change, and never store a rounded value (refs #14718) by @claudespice in #15037
• perf(iceberg): decode Iceberg scans at the session batch_size by @bjchambers in #15072
• fix(rate-control): admit waiting requests in arrival order at the charge in force (fixes #14912) by @krinart in #14931
• fix(rate-control): compare shared-origin settings as they resolve (fixes #14914) by @krinart in #14937
• fix(decisions): Fixes for /v1/decisions by @Jeadie in #15040
• perf(cayenne): bulk-load a keyed table without per-row routing or a whole-table fold by @bjchambers in #15076
• fix(cayenne): list a new snapshot's files once, when it is published by @bjchambers in #15077
• fix(duckdb): unparse string literals with backslash-quote exactly by @lukekim in #15083
• fix(vortex): keep casts Vortex cannot evaluate out of the scan by @phillipleblanc in #15096
• feat(index): operation-aware CDC index maintenance via IndexChangeSet by @Jeadie in #14340
• feat(iceberg): read a table root without a catalog, finding its current metadata on every query by @sgrebnov in #15095
• fix(runtime): fail recovery of an unsaved running query by @phillipleblanc in #15092
• perf(cayenne): build a keyed table's primary-key filter during its first load by @phillipleblanc in #15099
• feat(cache): age SQL results on TTL instead of invalidating them on refresh by @bjchambers in #15033
• fix(deps): estimate selective IN lists and OR-of-equalities from NDV by @bjchambers in #14140
• fix(flightsql): prepared-statement GetFlightInfo schema matches the stream when parameters are bound by @lukekim in #15106
• fix(telemetry): report the configured CPU count as host.cpu.count by @lukekim in #15108
• fix(runtime-table): prepare every partition of a partitioned Cayenne table after a refresh by @bjchambers in #15111
• feat(snapshots): store snapshots under runtime.state, built by the shared state-location builder by @sgrebnov in #15117
• fix(federation): match NULL rows in pushed-down INTERSECT and EXCEPT (fixes #14941) by @sgrebnov in #15110
• fix(models): allowed_tools shape, provider error statuses, decision response model (fixes #14910) by @krinart in #14977
• feat(cayenne): let deletion_mode auto pick position deletes for small primary-key tables by @lukekim in #15042
• fix(iceberg): keep nested child namespaces when loading an Iceberg catalog by @lukekim in #15105
• fix(flight): authenticate before binding sessions, and bound TLS handshakes by @phillipleblanc in #15128
• fix(models): return Spice tool calls in /v1/responses output so clients can replay them by @krinart in #15126
• feat(runtime): check runtime.state at startup, and record its state format by @phillipleblanc in #15138
• feat(acceleration): apply changed acceleration settings on restart by @lukekim in #15094
• fix(models): enable Luna chat and NSQL support by @krinart in #15166
• fix(acceleration): read every accelerator partition in a caching point lookup by @phillipleblanc in #15177
• fix(cayenne): run snapshot-directory sweeps when a snapshot releases its file-deletion hold by @sgrebnov in #15179
• feat(telemetry): export runtime logs over OTLP and log failed queries by level by @lukekim in #15176
• fix(iceberg): find the committed Hadoop metadata version by @phillipleblanc in #15142
• feat(iceberg): append refresh follows Iceberg snapshots so late-arriving rows are not missed by @sgrebnov in #15147
• Drive accelerator refresh writes on the refresh runtime by @sgrebnov in #15262
• fix(models): answer ai_* decisions from Bedrock models instead of failing them as refusals by @krinart in #15270
• fix(cache): coalesce Cayenne cache replacements so caching mode keeps up with sustained misses by @krinart in #15253
• feat(cluster): lend a distributed task's slot while it waits on an HTTP source by @peasee in #15123
• fix(cayenne): evict a folded snapshot's cached listing once its superseded files are removed by @bjchambers in #15303
• fix(cache): delete replaced Cayenne cache entries by primary key, not by scanning the table by @krinart in #15312
• fix(cayenne): list snapshot directories under the listing fence by @bjchambers in #15304
• fix(cayenne): checkpoint before a durable CDC write only when a source callback is installed by @phillipleblanc in #15315
• fix(iceberg): read tables whose last commit is schema-only (ADD/RENAME COLUMN) by @lukekim in #15334
• fix(runtime): skip the refresh_append_overlap warning for Iceberg append datasets by @sgrebnov in #15340
• fix(spiced): raise the soft open file limit to the hard limit at startup by @bjchambers in #15339
• fix(iceberg): keyed DuckDB/SQLite append refreshes read by time so late rows don't overwrite newer versions by @sgrebnov in #15346
• fix(acceleration): restore the strict high-water mark in append dedupe by @sgrebnov in #15358
• fix(observability): export a pinned query's execution log records under its trace id by @krinart in #15363
• fix(caching): let an empty origin response replace a cached HTTP entry by @krinart in #15365
• feat: one instance identity for every shared-state feature by @phillipleblanc in #15135
• feat(snapshots): per-dataset snapshot settings; deprecate top-level snapshots and runtime.scheduler state settings by @phillipleblanc in #15136
• fix(state): make every shared-state write in object storage conditional by @phillipleblanc in #15137
• fix(mysql): read the shared binlog on a thread of its own by @lukekim in #15181
• fix(datafusion): keep placeholder type inference linear over deep expressions by @lukekim in #15187
• fix(sql): cast float literals to decimal through their exact spelling by @lukekim in #15082
• fix(federation): keep a Spice-only function local without unfederating the plan around it (fixes #13770) by @krinart in #15078
• fix(snapshots): count every failed snapshot in dataset_acceleration_snapshot_failure_count by @sgrebnov in #15212
• feat(state): clean up unused shared state and keep 14 days of snapshots by @phillipleblanc in #15199
• feat(state): add state-store request, conflict and cleanup-delete metrics by @phillipleblanc in #15203
• fix(sql): keep float token spellings when planning refresh_sql by @lukekim in #15235
• feat(state): one ownership primitive for scheduler membership, job ownership and snapshot writers by @phillipleblanc in #15196
• fix(search): reject empty or whitespace-only text on /v1/search by @lukekim in #15119
• feat(snapshots): keep a dataset's snapshots for 28 days by @phillipleblanc in #15241
• fix(snapshots): release the snapshot writer lease on graceful shutdown by @sgrebnov in #15256
• feat(monitoring): snapshot freshness, Cayenne segment cache and pod health panels on the Datadog dashboard by @sgrebnov in #15309
• fix(observability): report snapshot reader refresh failures in metrics and label them mode=snapshot by @sgrebnov in #15310
• fix(cayenne): leave the Turso metastore log out of snapshots by @phillipleblanc in #15318
• fix(object_store_occ): never let a local version repeat an earlier version's ETag by @claudespice in #15437
• fix(cayenne): complete, version, and validate a dataset's metastore slice (fixes #13241) by @claudespice in #14238
• fix(acceleration): fail and retry a refresh whose source goes silent mid-stream by @sgrebnov in #15457
• fix: emit one HELP/TYPE per derived Prometheus summary family by @lukekim in #15416
• fix: serve append accelerations with snapshots enabled while the source is down by @sgrebnov in #15463
• perf(cayenne): compact an append-only table's small files without stalling its writers by @lukekim in #15227
• fix(cayenne): keep lookups fast under sustained replacement on tables without a primary key by @krinart in #15362
• fix(snapshots): report a snapshot reader that cannot load a newer snapshot by @sgrebnov in #15472
• fix(cayenne): rematerialize autocommit UPDATE inside write_lock by @lukekim in #15490
• feat: deprecate non-Cayenne acceleration engines and the DuckDB vector engine by @bjchambers in #15496
• fix(cayenne): count logical nulls so IS NULL keeps rows in inline and RAM-tier data by @lukekim in #15186
• fix(cayenne): filter a scan view's rows against the deletions it captured by @lukekim in #15169
• fix(snapshots): stop a restarted snapshot reader from serving a partial table by @sgrebnov in #15536
• fix(cluster): keep executor shuffle fetches on cluster mTLS by @phillipleblanc in #15484
• Add table-bound ChangeSink ownership and backends by @phillipleblanc in #14704
• chore: Update Turso crate to 0.8.2 by @spiceemma in #15022
• chore(deps): raise the datafusion-table-providers pin to include the NUMERIC result-column fix by @phillipleblanc in #14254
• feat(key-index): immutable secondary index runs over compound Arrow keys by @bjchambers in #14592
• feat(object_store_occ): add transactional WAL and MVCC snapshots by @lukekim in #14732
• Route CDC refresh and Cayenne cache writes through the table-bound ChangeSink by @phillipleblanc in #14788
• Reapply "fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492)" by @krinart in #14888
• fix(duckdb): pull in the bundled Brotli decoder fix (CVE-2026-111497) by @phillipleblanc in #15580

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.3.3...v2.4.0