Announcing the release of Spice v1.3.0! 🏎️
Spice v1.3.0 accelerates data and AI applications with significantly improved query performance, reliability, and expanded Databricks integration. New support for the Databricks SQL Statement Execution API enables direct SQL queries on Databricks SQL Warehouses, complementing Mosaic AI model serving and embeddings (introduced in v1.2.2) and existing Databricks catalog and dataset integrations. This release upgrades to DataFusion v46, optimizes results caching performance, and strengthens security with least-privilege sandboxed improvements.
-
Databricks SQL Statement Execution API Support: Added support for the Databricks SQL Statement Execution API, enabling direct SQL queries against Databricks SQL Warehouses for optimized performance in analytics and reporting workflows.
Example spicepod.yml configuration:
datasets:
- from: databricks:spiceai.datasets.my_awesome_table
name: my_awesome_table
params:
mode: sql_warehouse
databricks_endpoint: ${env:DATABRICKS_ENDPOINT}
databricks_sql_warehouse_id: ${env:DATABRICKS_SQL_WAREHOUSE_ID}
databricks_token: ${env:DATABRICKS_TOKEN}
For details, see the Databricks Data Connector documentation.
-
Improved Results Cache Performance & Hashing Algorithm: Spice now supports an alternative results cache hashing algorithm, ahash, in addition to siphash, being the default. Configure it via:
runtime:
results_cache:
hashing_algorithm: ahash
The hashing algorithm determines how cache keys are hashed before being stored, impacting both lookup speed and protection against potential DOS attacks.
Using ahash improves performance for large queries or query plans. Combined with results cache optimizations, it reduces 99th percentile request latency and increases total requests/second for queries with large result sets (100k+ cached rows). The following charts show performance tested against the TPCH Query #17 on a scale factor 5 dataset (30+ million rows, 5GB):
| Latency | Req/sec |
|---|
 |  |
Note: ahash was not available in v1.2.2, so it is excluded from comparisons.
To learn more, refer to the Results Cache Hashing Algorithm documentation.
-
SQL Query Performance: Optimized the critical SQL query path, reducing overhead and improving response times for simple queries by 10-20%.
-
DuckDB Acceleration: Fixed a bug in the DuckDB acceleration engine causing query failures under high concurrency when querying datasets accelerated into multiple DuckDB files.
-
Container Security: The container image now runs as a non-root user with enhanced sandboxing and includes only essential dependencies for a slimmer, more secure image.
Spice.ai is built on the DataFusion query engine. The v46 release brings:
-
Faster Performance 🚀: DataFusion 46 introduces significant performance enhancements, including a 2x faster median() function for large datasets without grouping, 10–100% speed improvements in FIRST_VALUE and LAST_VALUE window functions by avoiding sorting, and a 40x faster uuid() function. Additional optimizations, such as a 50% faster repeat() string function, accelerated chr() and to_hex() functions, improved grouping algorithms, and Parquet row group pruning with NOT LIKE filters, further boost overall query efficiency.
-
New range() Table Function: A new table-valued function range(start, stop, step) has been added to make it easy to generate integer sequences — similar to PostgreSQL’s generate_series() or Spark’s range(). Example: SELECT * FROM range(1, 10, 2);
-
UNION [ALL | DISTINCT] BY NAME Support: DataFusion now supports UNION BY NAME and UNION ALL BY NAME, which align columns by name instead of position. This matches functionality found in systems like Spark and DuckDB and simplifies combining heterogeneously ordered result sets.
Example:
SELECT col1, col2 FROM t1
UNION ALL BY NAME
SELECT col2, col1 FROM t2;
See the DataFusion 46.0.0 release notes for details.
Spice.ai adopts the latest minus one DataFusion release for quality assurance and stability. The upgrade to DataFusion v47 is planned for Spice v1.4.0 in June.
The container image now always runs as a non-root user (UID/GID 65534) with minimal dependencies, resulting in a smaller, more secure image. Standard Linux tools, including bash, are no longer included.
Kubernetes Deployments:
-
Use of the v1.3.0+ Helm chart is required, which includes a securityContext ensuring the sandbox user has required file access.
-
For deployments using a lower version than the v1.3.0 Helm chart, add the following securityContext to the pod specification:
securityContext:
runAsUser: 65534
runAsGroup: 65534
fsGroup: 65534
See the Docker Sandbox Guide for details on how to update custom Docker images to restore the previous behavior.
- Added Accelerated Views: Pre-calculate and materialize data derived from one or more underlying datasets.
The Spice Cookbook now includes 67 recipes to help you get started with Spice quickly and easily.
To upgrade to v1.3.0, use one of the following methods:
CLI:
Homebrew:
brew upgrade spiceai/spiceai/spice
Docker:
Pull the spiceai/spiceai:1.3.0 image:
docker pull spiceai/spiceai:1.3.0
For available tags, see DockerHub.
Helm:
helm repo update
helm upgrade spiceai spiceai/spiceai
See the full list of changes at: v1.2.2...v1.3.0
