Skip to main content

Spice v2.4.2 (Nov 9, 2026)

· 15 min read
Sergei Grebnov
Sergei Grebnov
Member of Technical Staff at Spice AI

Spice v2.4.2 is now available! 🛠️

Spice v2.4.2 is a patch release that continues security hardening and fixes Snowflake key-pair authentication. Security fixes limit the nesting depth, the expanded size, and the memory use of SQL queries, and HTTP datasets refuse redirects to other internal hosts. The release also fixes several SQL, Spice Cayenne, and runtime issues.

What's New in v2.4.2​

SQL Query Limits Protect the Runtime​

Spice now limits the nesting depth, the expanded size, and the memory use of SQL queries. A query over a limit fails with HTTP 400 on /v1/sql, or with InvalidArgument on Flight SQL. The runtime continues to serve other queries. Before this release, any client that could send SQL could stop the runtime with one query. A deeply nested query overflowed the stack, and the process stopped with fatal runtime error: stack overflow, aborting. Other queries made one function or one plan allocate memory without a limit.

The checks apply to HTTP /v1/sql, /v1/nsql, Flight SQL statements and prepared statements, Flight SQL updates, and transactions. The checks cover these shapes:

  • Long expressions. A chain of +, ||, AND, or OR terms deeper than 512 levels fails. This includes a /v1/search where condition.
  • Nested queries. Subqueries, or common table expressions (CTEs) that each read the CTE before them, fail past 64 levels.
  • CTE expansion. A query fails when its CTE references expand to more than 4096 table references. The planner makes a copy of the plan of a CTE for each reference to it.
  • Nested data types. A data type such as ARRAY<ARRAY<…>> fails past 32 levels. A type such as INT[][]… fails past 64 levels.
  • Arrow type strings. An Arrow type string, such as an arrow_cast argument or a column type that a dataset declares, fails past 64 levels.
  • Grouping sets. A GROUP BY that expands to more than 4096 grouping sets fails. CUBE of 13 expressions makes 8192 sets. 4096 is also the limit in PostgreSQL.
  • Search keywords. A /v1/search request with more than 512 keywords fails with HTTP 400.

These scalar functions now keep their output within runtime.query.memory_limit, or within a fixed cap:

  • lpad and rpad charge the whole output buffer to the memory limit. This includes an empty fill string and negative lengths. A query that does not fit fails with Resources exhausted.
  • array_repeat and list_repeat charge the real size of a list, struct, binary, boolean, or NULL element. Before this release, they charged 1 byte for each copy of such an element.
  • flatten_json_properties, json_tree, and flatten_json stop at 32 × max_bytes of output for one document. max_depth is limited to 128. A small JSON schema with repeated allOf or $ref entries no longer expands exponentially.
  • truncate cuts text by characters, not bytes. Non-ASCII text made the query fail with Query driver task ended unexpectedly.

Flight SQL metadata requests also have limits. GetSqlInfo info IDs and GetTables table types are limited to 1024 entries. GetTables and GetDbSchemas filter patterns are limited to 1024 bytes.

The limits are:

ShapeLimit
Expression depth, such as a chain of AND, OR, +, or ||512 levels
Query nesting: subqueries, or CTEs that read each other64 levels
Table references after CTE expansion4096
SQL data type nesting with ARRAY, STRUCT, MAP, and similar keywords32 levels
SQL data type nesting with []64 levels
Arrow type string nesting64 levels
GROUP BY grouping sets4096 (CUBE of up to 12 expressions)
/v1/search keywords512
Flight SQL GetSqlInfo info IDs and GetTables table types1024 entries
Flight SQL GetTables and GetDbSchemas filter patterns1024 bytes

CTEs that do not read each other are not limited by depth. Table references outside of CTEs do not count toward the expansion limit.

Credentials Stay out of API Responses and Logs​

GET /v1/datasets and GET /v1/catalogs no longer return credentials from the from value of a component. Before this release, any API key that could list components could read a password in a source URL, such as https://user:password@host/file.csv. This included a read-only key. The runtime now removes user:password@ from each URL. It replaces values of query keys such as password, secret, token, and sig with ***. With status=true, /v1/datasets also removes credentials from error_message. Sources with no credentials do not change.

The runtime also removes credentials from these errors and log lines:

  • ADBC connection errors. Setup errors no longer contain adbc_uri. The runtime replaces each configured secret in the driver message with ***. Before this release, the full URI went to the warning log and to /v1/datasets?status=true.
  • MCP transport errors. The error no longer contains the URL of the MCP server. A URL can contain a credential.
  • .env files. The warning for a malformed line gives only the line number, not the line content.

HTTP Datasets Refuse Redirects to Other Internal Hosts​

An http or https dataset in a file format, such as CSV or Parquet, no longer follows a redirect to an internal host. An internal host is a different host that is a loopback, private, or link-local address, or that resolves to one. The dataset fails with this error:

Refused to connect to <target>: the HTTPS data connector does not follow a redirect to a private, loopback, or link-local address

The host of the dataset itself is always allowed, also when it is internal. Redirects that stay on that host still work. SPICE_URL_TABLE_ALLOW_PRIVATE_HOSTS=true turns off the check.

Snowflake Key-Pair Authentication Fix​

Snowflake datasets with key-pair authentication now load correctly. Before this release, such a dataset stopped at Initializing with this error:

Could not automatically determine the process-level CryptoProvider from jsonwebtoken crate features.

Other Bug Fixes​

  • IS [NOT] DISTINCT FROM with AND or OR. These queries no longer return wrong boolean results or fail with an integer type error. For example, SELECT true IS DISTINCT FROM true OR true now returns true, not false. The comparison now binds before AND or OR.
  • COUNT(DISTINCT …) on list or struct columns. A query with two or more COUNT(DISTINCT …) aggregates no longer fails with Resources exhausted when one of them reads a list or struct column. Before this release, the query failed at about 50,000 rows and requested tens of GB. Each stored value held a reference to its whole input batch. This applied to federated and Cayenne queries, with or without GROUP BY.
  • Spice Cayenne file mode. At startup, a file-mode acceleration checks that the data files in its metastore exist and have the expected size. If they do not match, the acceleration reloads from the source or fails to load, and it does not serve incomplete data. Accelerations on S3 and in memory mode are not checked.
  • Queries after a burst of stale cache hits. With stale_while_revalidate_ttl set on the SQL results cache, new client queries no longer wait behind background revalidations. In a test, 600 stale hits made the next uncached query wait 321.8 seconds, and 6.3 seconds with this fix. At most one quarter of the query admission limit runs revalidations at the same time. A stale hit over that limit gets the cached result with no revalidation, and results_cache_stale_swr_count counts it. The runtime cancels a revalidation after 5 minutes and records the outcome timed_out.
  • truncate on numbers. truncate no longer returns wrong values for integers and decimals near the limits of their type. For example, truncate(100, arrow_cast(50, 'Int8')) returned 56, and the correct result is 0. A result that does not fit its type is now an error.
  • SQL scalar functions with no arguments. A from: sql scalar function with no arguments no longer fails on every call.
  • Certificate errors in the CLI. spice HTTP commands, such as status and refresh, report a TLS certificate failure as a TLS error. In v2.4.1, the release build reported it as The Spice runtime is unavailable.
  • AWS session tokens. The runtime no longer sends an empty AWS session token together with static keys. A secret that is not set gives an empty token. This applies to S3, Delta Lake, and AWS SDK connectors.

Dependency Updates​

  • arrow-rs (Spice fork): 59.3.0 → 59.3.2. This update fixes RUSTSEC-2026-0442 in run-end-encoded array validation, and it adds the 64-level limit for Arrow type strings.
  • azure_security_keyvault_secrets: 0.10.0 → 1.0.1. This update fixes RUSTSEC-2026-0438 in the Azure Key Vault secret store. azure_identity and azure_core move to the 1.x line for that store.
  • sqlparser: 0.62.0, vendored with the precedence fix from apache/datafusion-sqlparser-rs#2436.
  • DataFusion (Spice fork): the pin adds the fixes for COUNT(DISTINCT …) memory and for nested SQL data types.

Breaking Changes​

Query limits. Queries and requests over the new limits fail with HTTP 400 or Flight InvalidArgument. Most of them stopped the runtime before this release, but some ran in v2.4.1. Examples are a GROUP BY CUBE of 13 expressions, a /v1/search request with more than 512 keywords, and a query whose CTEs expand to more than 4096 table references. For each limit, see SQL Query Limits Protect the Runtime.

HTTP dataset redirects. An http or https dataset in a file format fails when its server redirects to a different internal host. To allow it, set SPICE_URL_TABLE_ALLOW_PRIVATE_HOSTS=true.

JSON flattening. flatten_json, json_tree, and flatten_json_properties stop at 32 × max_bytes of output for one document (256 MiB by default). They return the rows up to that point, with no error. The limit is on output, so a small document with long, deeply nested keys can reach it. To get all rows, increase max_bytes.

Upgrading​

To upgrade to v2.4.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.4.2 image:

docker pull spiceai/spiceai:2.4.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.4.2

AWS Marketplace:

Spice is available in the AWS Marketplace.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook provides recipes to help you get started with Spice.

Contributors​

What's Changed​

Changelog​

  • [release/2.4] fix(sql): honour IS [NOT] DISTINCT FROM precedence before AND/OR (backport #15935) by @lukekim in #15936
  • fix(snowflake): select the jsonwebtoken crypto provider at startup by @sgrebnov in #15988
  • fix: reject over-deep SQL expressions and search keyword lists by @phillipleblanc in #15348
  • Reject over-deep expressions before they overflow the query stack by @phillipleblanc in #15433
  • Reject over-deep query nesting before planning by @phillipleblanc in #15722
  • fix(sql): reject queries whose CTE references expand to too many table references by @phillipleblanc in #15766
  • fix: bound nested SQL data types before parsing by @phillipleblanc in #15764
  • fix(sql): limit the nesting depth of SQL data types by @phillipleblanc in #15705
  • fix(query): reject GROUP BY that expands past 4096 grouping sets before planning by @phillipleblanc in #15843
  • fix: limit Arrow type-string nesting depth on every parse path by @phillipleblanc in #15845
  • fix(datafusion): charge lpad/rpad output buffers to the query memory limit by @phillipleblanc in #15846
  • fix(runtime): reserve array_repeat output by element size by @phillipleblanc in #15885
  • fix(udfs): bound JSON function expansion and output; fix truncate overflow and non-ASCII text by @phillipleblanc in #15912
  • fix(flight): bound Flight SQL metadata request list and pattern sizes by @phillipleblanc in #15848
  • fix(http): remove URL credentials from from in /v1/datasets and /v1/catalogs by @lukekim in #15834
  • fix(adbc): keep the connection URI out of ADBC setup errors by @lukekim in #15835
  • fix(http): refuse dataset redirects to other internal hosts by @phillipleblanc in #15877
  • [2.4] fix(cayenne): don't serve a file-mode acceleration whose data files no longer match its metastore by @lukekim in #15943
  • [release/2.4]fix(datafusion): charge each nested COUNT(DISTINCT) value for its own bytes (2.4) by @lukekim in #15986
  • fix(cache): bound stale-while-revalidate background revalidations and give them a deadline by @phillipleblanc in #15820
  • fix(cache): stop a timed-out stale-while-revalidate query that still waits for admission by @phillipleblanc in #15880
  • fix(cli): use rustls for the HTTP client so certificate errors are recognized (2.4 backport of #15811) by @lukekim in #15984
  • fix(deps): bump spiceai/arrow-rs to upstream 59.3.2 (2.4) by @lukekim in #16041
  • fix(deps): bump azure_security_keyvault_secrets to 1.0.1 on release/2.4 (RUSTSEC-2026-0438) by @lukekim in #16043
  • [release/2.4] fix(deps): replace temporary datafusion branch with spiceai-55-release-2.4 by @sgrebnov in #16046
  • Update spicepod.schema.json by @github-actions in #15828
  • ci: add rustfs-credentials action for GitHub OIDC credentials to RustFS by @phillipleblanc in #15976
  • ci: move CI credentials to OIDC and environment secrets by @phillipleblanc in #16010
  • ci: get RustFS credentials from the GitHub OIDC token by @phillipleblanc in #16005
  • test: remove CI tests that need a live Dremio server by @phillipleblanc in #15973
  • ci: authenticate BigQuery tests with workload identity federation by @sgrebnov in #16009
  • ci: authenticate CI calls to Anthropic with workload identity federation by @lukekim in #15954
  • ci: pin MongoDB 8.0 integration tests to 8.0.4 by @krinart in #15958
  • ci: run the released-runtime spice chat E2E test against xAI by @Jeadie in #15974
  • ci(cuda): least-privilege tokens and drop the MinIO copy by @sgrebnov in #16020
  • ci: scope workflow jobs to GitHub environments by @Jeadie in #15966
  • ci: pass secrets to run scripts through env, and use the read-write Postgres for load runs by @Jeadie in #15975
  • [release/2.4] ci: move CI credentials to OIDC and environment secrets (backport) by @sgrebnov in #16035
  • ci: fail a tag build when the tag does not match version.txt by @sgrebnov in #16021
  • ci(docker): build release images on GitHub-hosted runners by @sgrebnov in #16022
  • ci(release): cancel an older build of the same tag by @peasee in #16032
  • [release/2.4] ci: hosted Docker release builds and release hardening by @sgrebnov in #16044
  • ci: Use OIDC-based AWS roles for short-lived AWS credentials by @Jeadie in #16039
  • [release/2.4] Revert workload identity for models and GitHub App keys in Key Vault/KMS by @Jeadie in #16052
  • v2.4.2: remaining security backports, version bump to 2.4.2 by @sgrebnov in #16055

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.4.1...v2.4.2