Skip to main content

Spice v2.4.3 (Nov 17, 2026)

ยท 20 min read
Sergei Grebnov
Member of Technical Staff at Spice AI

Spice v2.4.3 is now available! ๐Ÿ› ๏ธ

Spice v2.4.3 is a security patch and hardening release. It upgrades dependencies to fix 27 security advisories. These include SSRF and denial of service in the MCP server and client, denial of service in HTTP/2, and command injection in the FTP connector. The install scripts now check the sha256 of each download before they install it. A ${ secrets:โ€ฆ } reference that does not resolve now fails the dataset, instead of loading it with an empty value.

The release also upgrades DataFusion to 55.3.0 and ADBC to 0.25. It fixes bugs in file and S3 datasets, HTTP caching mode, and Spicepod reload. All users should upgrade. Some datasets that loaded in v2.4.2 now fail with an error that names the cause. Before you upgrade, read Breaking Changes.

What's New in v2.4.3โ€‹

DataFusion 55.3.0โ€‹

The runtime now uses DataFusion 55.3.0. The upgrade fixes these query bugs:

  • Sort order with some filters. A query with a filter on a lossy cast or on wrapping arithmetic of an ORDER BY column now returns its rows in order. Before this release, the planner removed the sort for these filters.
  • LEAD and LAG with IGNORE NULLS. These window functions now return the correct row.
  • Sort-merge outer joins. An outer join with a filter that runs as a sort-merge join no longer fails with declared as non-nullable but contains null values.
  • Other fixes. The upgrade also fixes sort-merge joins on float keys and the output order of outer joins. It fixes row counts from async UDFs, the IPC framing of spill files, and the memory accounting of grouped MIN and MAX.

With hive_partitioning_enabled: true, a __HIVE_DEFAULT_PARTITION__ directory now reads as NULL. Hive writes a NULL partition value to a directory with this name. Some file accelerations need an action after the upgrade. See Breaking Changes.

Dependency Security Updatesโ€‹

This release upgrades dependencies to fix 27 security advisories. The advisories affect these areas:

Two behavior changes come with these upgrades:

  • An FTP download that the server reports as failed now returns an error. Before this release, it could return truncated data.
  • The MCP server returns the JSON-RPC error -32602 with HTTP 200 for an unknown tool. Before this release, it returned HTTP 400. The MCP specification requires HTTP 200.

Install Scripts Check Downloadsโ€‹

The install scripts now check the sha256 of each download before they extract it. This applies to install.sh, install-spiced.sh, and Install.ps1. If the sha256 does not match, the install stops and installs nothing. For a 2.4.x release, the scripts compare the download with the sha256 that GitHub recorded for the release asset. Before this release, the scripts did not check downloads.

The install scripts also have these fixes:

  • Shell syntax in an install directory. install.sh, install-spiced.sh, and install-nightly.sh no longer run an install directory that contains shell syntax as a command. Before this release, SPICE_INSTALL_DIR='/tmp/x; touch /tmp/pwned' ran touch /tmp/pwned, as root when the install used sudo.
  • Missing /usr/local/bin. install-spiced.sh now asks for sudo before it creates a missing install directory. Before this release, a user who was not root stopped with Permission denied. This occurred on a Mac with Apple silicon and no /usr/local/bin, for example.
  • Windows PATH. Install.ps1 no longer fails when the user PATH is not set. It now compares whole PATH entries. Before this release, a similar directory such as .spice\bin2 stopped the installer from adding .spice\bin.

spice install <version> and spice upgrade --version <version> no longer fail with Release not found for a release older than the 30 newest releases. The CLI now finds the release by its tag.

The new download checks can stop some installs that worked before. See Breaking Changes.

Secret References That Do Not Resolve Fail the Datasetโ€‹

A dataset or catalog with a ${ secrets:โ€ฆ } reference that does not resolve now fails to load. The error names the parameter and the reference, and it never shows a secret value. Before this release, the runtime replaced the reference with an empty value and loaded the dataset with that value.

This applies only to dataset and catalog params. Other datasets continue to load. When a secret store returns an error, the runtime retries the load. An empty or whitespace file_format now also fails, with an error that lists the valid formats.

If a secret is optional on purpose, define it with an empty value, such as FMT= in the environment or a .env file. A secret with an empty value counts as resolved. For details, see Secret Stores.

Buffered /v1/sql Results Stay Within the Memory Limitโ€‹

A /v1/sql response that the runtime buffers now counts against runtime.query.memory_limit. A result that does not fit in the limit returns HTTP 503 with The query result is too large to buffer in memory for this response. Before this release, one request could use memory outside the limit. In a test with a 256 MiB limit, one text/plain request raised peak memory from 0.7 GB to 7.4 GB.

The runtime buffers text/plain, CSV, the vnd.* formats, JSON with union columns, and the final result of a BEGIN โ€ฆ COMMIT transaction. Other JSON responses stream and do not change.

ADBC 0.25โ€‹

The ADBC data connector and the ADBC catalog now use ADBC 0.25, from Apache Arrow ADBC 25. The runtime includes the ADBC driver manager. The upgrade from ADBC 0.24 brings these changes to it:

  • A broken driver fails when it loads, not at its first call. On Linux and macOS, the driver manager now resolves every symbol of a driver library when it loads the library. A driver with a missing symbol now fails to load. The dataset or catalog fails with Failed to load ADBC driver '<driver>', and the error gives the reason from the system loader. Before this release, such a driver loaded, and the process could crash at the first call into the driver.
  • Query cancellation comes from upstream ADBC. ADBC 0.25 cancels a running statement without a wait for the lock that the statement holds. Spice carried the same fix as a patch in its ADBC fork through ADBC 0.24. Cancellation works as in v2.4.2. When the runtime stops reading an ADBC query, for example after a client cancels it, the runtime cancels the statement in the driver. Then it returns the connection to the pool.
  • One fork patch. The Spice fork of ADBC now carries one patch. The patch limits ADBC to the Arrow major version that the runtime uses.

The runtime does not include ADBC drivers, and the upgrade does not change a driver that you install. ADBC 25 also releases new C and Go drivers, version 1.13.0. These drivers have their own fixes, for example:

  • PostgreSQL driver. It binds and reads uuid values as the Arrow arrow.uuid extension type. It maps more SQLSTATE codes to ADBC errors, and it reads all results after a query. It also fixes ingestion of some negative decimal values.
  • SQLite driver. It formats extreme date and time values correctly. It counts rows_affected only for statements that change rows.

For the full list, see the ADBC 25 release notes.

File and S3 Dataset Bug Fixesโ€‹

This release fixes bugs in file and object-store datasets, such as S3, when their source files change. The fixes apply to federated and accelerated datasets.

  • CSV columns bind by header name. A CSV or TSV dataset with has_header: true (the default) now reads each column by its header name. Before this release, the runtime read columns by position. When a file changed its column order, values loaded under the wrong column names. A CSV with has_header: false still binds by position.
  • A deleted or emptied source fails the refresh. When a refresh finds no data files for a dataset, it now fails, and the acceleration keeps its last data. The dataset status changes to Error, and the log names the dataset and path: No data files are yet available for dataset โ€ฆ. Before this release, a full refresh loaded zero rows into the acceleration. This covers an emptied directory or prefix, a prefix with only its folder marker, and a deleted single S3 object. A header-only CSV file or a zero-row Parquet file still loads as zero rows.
  • A renamed or removed column fails the refresh. A refresh that reads every file now fails when no source file has a column of the dataset. The acceleration keeps its last data. Before this release, the refresh loaded NULL values for that column. This check covers Parquet, ORC, JSON, CSV, and TSV files. CSV and TSV files need a header row.
  • JSON sources and federated queries. A missing JSON key fails the refresh only when the files also have new keys, which is the pattern of a rename. A JSON source with zero records can still clear the acceleration. A federated query returns NULL for a missing column. It logs one warning that names the dataset and the column.
  • The file watcher sees more types of writes. With file_watcher: enabled, the acceleration now refreshes after an atomic replace (write a temporary file, then rename it). It also refreshes after appends that arrive close together and after a large in-place rewrite. Before this release, the watcher missed later writes after a rename. It also dropped the last write of a burst and could refresh from an empty or partial file. The watcher now watches the parent directory and waits 400 ms after the last write before it refreshes.

Other Bug Fixesโ€‹

  • HTTP caching mode. An HTTP dataset with refresh_mode: caching now fetches the request keys that are not in the cache. Before this release, a query that named both cached and uncached keys returned only the cached rows. This applied to an IN list or an OR of the same column on request_path, request_query, or request_body. A query that finds some of its keys in the cache also obeys max_request_partitions. A query over the limit fails with the same error as a query with no cached keys, and it sends no request.
  • Views after a Spicepod reload. A view now reads the dataset that the Spicepod declares after a reload replaces that dataset, and it fails when a reload removes the dataset. Before this release, the view continued to read the old version of the dataset. This also applies to a view that reads the dataset in a subquery or through another view.
  • /v1/ready after a Spicepod reload. /v1/ready no longer returns 503 for a component that a reload removed before it loaded.
  • SQL parameters in search functions. A query with a parameter such as $1 in a text_search, vector_search, or rrf argument no longer fails with a placeholder error. The runtime now puts the parameter values into the arguments before it plans the query. The plan cache does not keep these plans. For the search functions, see Search.
  • Refresh requests with an empty body. POST /v1/datasets/{name}/acceleration/refresh with an empty body and Content-Type: application/json no longer fails with HTTP 400. This occurred when a proxy added that header to a request with no body. An empty or whitespace body now gives the default refresh. Malformed JSON still returns 400, and an unknown refresh_mode still returns 422.
  • time_column: last_modified. A dataset with time_column: last_modified no longer fails with time_column 'last_modified' was not found. Spice renamed this metadata column to _last_modified in v2.0. The runtime now uses _last_modified when the data has no last_modified column, and it logs a deprecation warning. time_format defaults to timestamptz for _last_modified.
  • Delta Lake struct filters. A WHERE filter on a struct field no longer fails with Error during planning: Field a not found in struct. This applied to a federated Delta Lake dataset on a table with column mapping (delta.columnMapping.mode set to name or id).
  • Maps inside struct columns. A key lookup into a map inside a struct column, such as som['mm']['k'], no longer fails on a federated Parquet file dataset. The query failed with Internal error: PhysicalExpr Column references column 'som' โ€ฆ.

Dependency Updatesโ€‹

Dependency / Componentv2.4.2v2.4.3
Apache DataFusionv55.2.0v55.3.0
ADBCv0.24.0v0.25.0
rmcpv3.3.0v3.10.0
h2v0.4.14v0.4.20
zstdv0.13.3v0.13.5
zstd-safev7.2.4v7.4.1
suppaftpv6.3.0v12.2.1
quinn-protov0.11.16v0.11.19
wasmtimev48.0.7v48.0.8
calaminev0.37.0v0.38.0
pulldown-cmarkv0.13.4v0.13.6
zip 3v3.0.0Removed
crypto-bigintv0.5.5v0.7.11
ctutilsv0.4.2v0.4.8
tokiov1.52.3v1.54.1
libz-sysv1.1.29v1.1.33

For the changes in DataFusion and ADBC, see DataFusion 55.3.0 and ADBC 0.25. For the advisories that the other upgrades fix, see Dependency Security Updates.

Breaking Changesโ€‹

Each breaking change in v2.4.3 is part of a security fix or a correctness fix. Most of them replace a silent fallback with an error that names the cause. Two of them change a result without an error: the HTTP status for an unknown MCP tool, and the value of a Hive default partition.

Securityโ€‹

Secret references that do not resolve fail the dataset. A dataset or catalog with a ${ secrets:โ€ฆ } reference in params that does not resolve now fails to load. It no longer loads with an empty value. Other datasets continue to load. If a secret is optional on purpose, define it with an empty value, such as FMT=. An empty file_format value also fails.

Install scripts stop on a download that they cannot check.

  • An install stops when the script cannot get the expected sha256, for example when the GitHub API request fails or hits its rate limit. For a 2.4.x release, the script always sends this request, also for a pinned version such as bash -s 2.4.3. To fix it, set GITHUB_TOKEN. To install with no check, set SPICE_INSTALL_ALLOW_UNVERIFIED=1.
  • An install of v1.3.1 or older stops, because GitHub recorded no sha256 for these releases. Set SPICE_INSTALL_ALLOW_UNVERIFIED=1 to install them.
  • An install always stops on a sha256 mismatch. No setting changes this.
  • Install.ps1 writes its output with Write-Output, not Write-Host. A failed install now throws an error. Before this release, it exited with status 0.

Large buffered /v1/sql results return HTTP 503. A CSV, text/plain, or vnd.* response from /v1/sql that does not fit in runtime.query.memory_limit now returns HTTP 503. Before this release, it returned HTTP 200 and used memory outside the limit. To get a large result over HTTP, request JSON. The runtime streams JSON when the result has no union columns.

MCP and FTP behavior after the dependency upgrades.

  • The MCP server returns HTTP 200 with the JSON-RPC error -32602 for an unknown tool. Before this release, it returned HTTP 400.
  • An FTP download that the server reports as failed now returns an error. Before this release, it could return truncated data.

Correctnessโ€‹

Hive __HIVE_DEFAULT_PARTITION__ reads as NULL. With hive_partitioning_enabled: true, rows under a col=__HIVE_DEFAULT_PARTITION__/ directory now return NULL for that partition column. Partition columns are now nullable. WHERE col IS NULL matches these rows, and WHERE col = '__HIVE_DEFAULT_PARTITION__' does not.

An existing mode: file acceleration of such a dataset fails to refresh after the upgrade. The error is Invalid batch column โ€ฆ has null but schema specifies non-nullable. The dataset continues to serve its existing data. To fix it:

  • With refresh_mode: full, set on_schema_change: sync_all_columns.
  • With refresh_mode: append or changes, rebuild the acceleration.

Other Hive datasets continue to refresh. They log a startup warning and report the partition column as NOT NULL until you rebuild the acceleration.

File datasets fail on missing files or columns. These checks keep a refresh from replacing the data in an acceleration with an empty or NULL result.

  • A refresh that finds no data files now fails, and the acceleration keeps its last data. A federated query of a dataset with no data files now returns an error that names the dataset and path. Before this release, both returned zero rows.
  • A refresh that reads every file now fails when no file has a column of the dataset. Before this release, the refresh loaded NULL values.

Upgradingโ€‹

To upgrade to v2.4.3, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.4.3 image:

docker pull spiceai/spiceai:2.4.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.4.3

AWS Marketplace:

Spice is available in the AWS Marketplace.

Cookbook Updatesโ€‹

No new cookbook recipes.

The Spice Cookbook provides recipes to help you get started with Spice.

Contributorsโ€‹

What's Changedโ€‹

Changelogโ€‹

  • fix(tpch): Update expected answers for TPCH by @peasee in #16063
  • ci(testoperator): keep the snapshots GitHub refuses to let GITHUB_TOKEN push from a release commit by @grokspice in #16190
  • [release/2.4] fix(cache): fetch missing keys on partial caching-mode hits by @lukekim in #16154
  • fix: key lookup into a map inside a struct column on Parquet scans (2.4) by @lukekim in #15992
  • fix(file): watch the parent directory and debounce trailing writes [release/2.4] by @lukekim in #16127
  • fix(file): treat a missing listing source as a scan error [release/2.4] by @lukekim in #16126
  • [release/2.4] fix: alias deprecated last_modified time_column to _last_modified by @lukekim in #16207
  • [release/2.4] fix(delta_lake): filter on struct fields of column-mapped tables by @lukekim in #16217
  • ci(testoperator): configure apt in the bench, throughput and schema jobs by @grokspice in da46ce3
  • [release/2.4] fix(http): accept an empty refresh body with Content-Type: application/json by @lukekim in #16313
  • [release/2.4] test(bench): validate TPC-H SF1 against the official TPC answers; authenticate DynamoDB spicepods with their keys by @krinart in #16349
  • [release/2.4] build(deps): upgrade ADBC to 0.25 by @sgrebnov in #16354
  • [release/2.4] build(deps): upgrade DataFusion to 55.3.0 by @sgrebnov in #16369
  • fix(cache): enforce max_request_partitions on partial cache hits by @sgrebnov in #16355
  • [2.4] fix(file): bind headered CSV columns by name (backport #15454) by @lukekim in #16353
  • [release/2.4] fix(deps): upgrade vulnerable dependencies flagged by cargo-deny by @sgrebnov in #16365
  • fix(cli): look up a release by its tag, not on the first page of the list by @grokspice in #16310
  • ci(install): install v1.11.6, the newest v1.11 release, as the previous stable version by @lukekim in #16352
  • fix(install): pass runAsRoot's arguments to sudo as they are, and create spiced's install dir after the sudo decision by @lukekim in #16343
  • fix(install): add the Spice CLI to an unset user PATH, and match PATH entries whole by @lukekim in #16344
  • fix(install): check every download's sha256 before installing it, and its attestation with gh by @lukekim in #16342
  • fix(reload): views follow replaced datasets; readiness ignores removed components by @phillipleblanc in #15662
  • [2.4] fix: fail datasets whose secret references do not resolve (backport #16390) by @lukekim in #16392
  • [release/2.4] v2.4.3: backport CI environment fix (#16202), bump version to 2.4.3 by @sgrebnov in #16397
  • [2.4] fix(http): reserve buffered /v1/sql results from the query memory pool (#15847) by @lukekim in #16398
  • [2.4] Bind SQL parameters into table function arguments before planning (#15048) by @lukekim in #16410

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.4.2...v2.4.3