Skip to main content

Spice v1.2.2 (May 13, 2025)

Β· 4 min read
Jack Eadie
Token Plumber at Spice AI

Announcing the release of Spice v1.2.2! 🌟

Spice v1.2.2 introduces support for Databricks Mosaic AI model serving and embeddings, alongside the existing Databricks catalog and dataset integrations. It adds configurable service ports in the Helm chart and resolves several bugs to improve stability and performance.

Highlights in v1.2.2​

  • Databricks Model & Embedding Provider: Spice integrates with Databricks Model Serving for models and embeddings, enabling secure access via machine-to-machine (M2M) OAuth authentication with service principal credentials. The runtime automatically refreshes tokens using databricks_client_id and databricks_client_secret, ensuring uninterrupted operation. This feature supports Databricks-hosted large language models and embedding models.

    models:
    - from: databricks:databricks-llama-4-maverick
    name: llama-4-maverick
    params:
    databricks_endpoint: dbc-46470731-42e5.cloud.databricks.com
    databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
    databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}

    embeddings:
    - from: databricks:databricks-gte-large-en
    name: gte-large-en
    params:
    databricks_endpoint: dbc-42424242-4242.cloud.databricks.com
    databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
    databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}

    For detailed setup instructions, refer to the Databricks Model Provider documentation.

  • Configurable Helm Chart Service Ports: The Helm chart now supports custom ports for flexible network configurations for deployments. Specify non-default ports in your Helm values file.

  • Resolved Issues:

    • MCP Nested Tool Calling: Fixed a bug preventing nested tool invocation when Spice operates as the MCP server federating to MCP clients.

    • Dataset Load Concurrency: Corrected a failure to respect the dataset_load_parallelism setting during dataset loading.

    • Acceleration Hot-Reload: Addressed an issue where changes to acceleration enable/disable settings were not detected during hot reload of Spicepod.yaml.

Contributors​

Breaking Changes​

No breaking changes.

Cookbook Updates​

Updated cookbooks:

The Spice Cookbook now includes 68 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.2.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.2.2 image:

docker pull spiceai/spiceai:1.2.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed​

Dependencies​

  • No major dependency changes.

Changelog​

- Update spark-connect-rs to override user agent string by @ewgenius in https://github.com/spiceai/spice/pull/5798
- Merge pull request by @ewgenius in https://github.com/spiceai/spice/pull/5796
- Pass the default user agent string to the Databricks Spark, Delta, and Unity clients by @ewgenius in https://github.com/spiceai/spice/pull/5717
- bump to 1.2.2 by @Jeadie in https://github.com/spiceai/spice/pull/none
- Helm chart: support for service ports overrides by @sgrebnov in https://github.com/spiceai/spice/pull/5774
- Update spice cli login command with client-id and client-secret flags for Databricks by @ewgenius in https://github.com/spiceai/spice/pull/5788
- Fix bug where setting Cache-Control: no-cache doesn't compute the cache key by @phillipleblanc in https://github.com/spiceai/spice/pull/5779
- Update to datafusion-contrib/datafusion-table-providers#336 by @phillipleblanc in https://github.com/spiceai/spice/pull/5778
- Lru cache: limit single cached record size to u32::MAX (4GB) by @sgrebnov in https://github.com/spiceai/spice/pull/5772
- Fix LLMs calling nested MCP tools by @Jeadie in https://github.com/spiceai/spice/pull/5771
- MySQL: Set the character_set_results/character_set_client/character_set_connection session variables on connection setup by @Sevenannn in https://github.com/spiceai/spice/pull/5770
- Control the parallelism of acceleration refresh datasets with runtime.dataset_load_parallelism by @phillipleblanc in https://github.com/spiceai/spice/pull/5763
- Fix Iceberg predicates not matching the Arrow type of columns read from parquet files by @phillipleblanc in https://github.com/spiceai/spice/pull/5761
- fix: Use decimal_cmp for numerical BETWEEN in SQLite by @peasee in https://github.com/spiceai/spice/pull/5760
- Support product name override in databricks user agent string by @ewgenius in https://github.com/spiceai/spice/pull/5749
- Databricks U2M Token Provider support by @ewgenius in https://github.com/spiceai/spice/pull/5747
- Remove HTTP auth from LLM config and simplify Databricks models logic by using static headers by @Jeadie in https://github.com/spiceai/spice/pull/5742
- clear plan cache when dataset updates by @kczimm in https://github.com/spiceai/spice/pull/5741
- Support Databricks M2M auth in LLMs + Embeddings by @Jeadie in https://github.com/spiceai/spice/pull/5720
- Retrieve Github App tokens in background; make TokenProvider not async by @Jeadie in https://github.com/spiceai/spice/pull/5718
- Make 'token_providers' crate by @Jeadie in https://github.com/spiceai/spice/pull/5716
- Databricks AI: Embedding models & LLM streaming by @Jeadie in https://github.com/spiceai/spice/pull/5715

See the full list of changes at: v1.2.1...v1.2.2

Spice v1.2.1 (May 6, 2025)

Β· 5 min read
Sergei Grebnov
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.2.1! πŸ”₯

Spice v1.2.1 includes several data connector fixes and improves query performance for accelerated views. This release also introduces Databricks Service Principal (M2M OAuth) authentication and expands parameterized queries.

Highlights in v1.2.1​

  • Databricks Service Principal Support: Databricks datasets and catalogs now support Machine-to-Machine (M2M) OAuth authentication via Service Principals, enabling secure machine connections to Databricks.

Example spicepod.yaml:

datasets:
- from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
name: my_delta_lake_table
params:
mode: delta_lake
databricks_endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com
databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}

For details, see documentation for:

  • Databricks Data Connector

  • Databricks Unity Catalog Connector

  • Iceberg Data Connector: Now supports cross-account table access via the AWS Glue Catalog Connector and fixes an issue when querying data from append mode datasets.

  • Iceberg Catalog API: Full compatibility with the Iceberg HTTP REST Catalog API to consume Spice datasets from Iceberg Catalog clients.

For details, see documentation for:

  • Iceberg Data Connector

  • S3 Data Connector

  • Improved Parameterized Query Support: Expanded type inference for placeholders in:

    • IN list expressions
    • LIKE patterns
    • SIMILAR TO patterns
    • LIMIT clauses
    • Subqueries

New Contributors πŸŽ‰β€‹

Contributors​

Breaking Changes​

No breaking changes.

Cookbook Updates​

New recipes for:

The Spice Cookbook now includes 68 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.2.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.2.1 image:

docker pull spiceai/spiceai:1.2.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed​

Dependencies​

  • No major dependency changes.

Changelog​

- Fix: Specify metric type as a dimension for testoperator by [@peasee](https://github.com/peasee) in [#5630](https://github.com/spiceai/spiceai/pull/5630)
- Fix: Add option to run dispatch schedule by [@peasee](https://github.com/peasee) in [#5631](https://github.com/spiceai/spiceai/pull/5631)
- Infer placeholder datatype for InList, Like, and SimilarTo by [@kczimm](https://github.com/kczimm) in [#5626](https://github.com/spiceai/spiceai/pull/5626)
- Add QA analytics for 1.2.0 by [@phillipleblanc](https://github.com/phillipleblanc) in [#5640](https://github.com/spiceai/spiceai/pull/5640)
- Fix: Use SPICED_COMMIT for spiced_commit_sha by [@peasee](https://github.com/peasee) in [#5632](https://github.com/spiceai/spiceai/pull/5632)
- New crates/tools by [@Jeadie](https://github.com/Jeadie) in [#5121](https://github.com/spiceai/spiceai/pull/5121)
- Update openapi.json by [@github-actions](https://github.com/github-actions) in [#5643](https://github.com/spiceai/spiceai/pull/5643)
- Enable metrics reporting for models benchmarks (evals) by [@sgrebnov](https://github.com/sgrebnov) in [#5639](https://github.com/spiceai/spiceai/pull/5639)
- Implement CatalogBuilder, add app and runtime references to catalog component, add runtime reference to connector params by [@ewgenius](https://github.com/ewgenius) in [#5641](https://github.com/spiceai/spiceai/pull/5641)
- Fix eventing bug in LLM progress; Add tool and worker progress by [@Jeadie](https://github.com/Jeadie) in [#5619](https://github.com/spiceai/spiceai/pull/5619)
- Handle small precision differences in TPCH answer validation by [@phillipleblanc](https://github.com/phillipleblanc) in [#5642](https://github.com/spiceai/spiceai/pull/5642)
- Add TokenProviderRegistry to the runtime by [@ewgenius](https://github.com/ewgenius) in [#5651](https://github.com/spiceai/spiceai/pull/5651)
- Provide ModelContextLayer for evals by [@Jeadie](https://github.com/Jeadie) in [#5648](https://github.com/spiceai/spiceai/pull/5648)
- Databricks data_components refactor. Databricks Spark connect - add set_token method and writable spark session by [@ewgenius](https://github.com/ewgenius) in [#5654](https://github.com/spiceai/spiceai/pull/5654)
- Extract AWS Glue warehouse for cross-account Iceberg tables by [@phillipleblanc](https://github.com/phillipleblanc) in [#5656](https://github.com/spiceai/spiceai/pull/5656)
- Refactor Dataset component by [@phillipleblanc](https://github.com/phillipleblanc) in [#5660](https://github.com/spiceai/spiceai/pull/5660)
- Fix Iceberg API returning 404 when schema contains a Dictionary by [@phillipleblanc](https://github.com/phillipleblanc) in [#5665](https://github.com/spiceai/spiceai/pull/5665)
- Fix dependencies: downgrade swagger-ui to v8; force zip to 2.3.0 by [@kczimm](https://github.com/kczimm) in [#5664](https://github.com/spiceai/spiceai/pull/5664)
- Add DuckDB indexes spicepod, additional dispatches by [@peasee](https://github.com/peasee) in [#5633](https://github.com/spiceai/spiceai/pull/5633)
- Update readme: update data federation link by [@nuvic](https://github.com/nuvic) in [#5673](https://github.com/spiceai/spiceai/pull/5673)
- Support metadata columns for object-store based data connectors by [@phillipleblanc](https://github.com/phillipleblanc) in [#5661](https://github.com/spiceai/spiceai/pull/5661)
- Add model name to LLM judges, and add model_graded_scoring task by [@Jeadie](https://github.com/Jeadie) in [#5655](https://github.com/spiceai/spiceai/pull/5655)
- Add SF1000 TPCH test spicepods for delta lake by [@Sevenannn](https://github.com/Sevenannn) in [#5606](https://github.com/spiceai/spiceai/pull/5606)
- Validate Github Connector resource existence before building the github connector graphql table by [@Sevenannn](https://github.com/Sevenannn) in [#5674](https://github.com/spiceai/spiceai/pull/5674)
- Remove hard-coded embedding performance tests in CI by [@Sevenannn](https://github.com/Sevenannn) in [#5675](https://github.com/spiceai/spiceai/pull/5675)
- Databricks M2M auth for spark connect data connector by [@ewgenius](https://github.com/ewgenius) in [#5659](https://github.com/spiceai/spiceai/pull/5659)
- Enable federated data refresh support for accelerated views by [@sgrebnov](https://github.com/sgrebnov) in [#5677](https://github.com/spiceai/spiceai/pull/5677)
- Add pods watcher integration test by [@Sevenannn](https://github.com/Sevenannn) in [#5681](https://github.com/spiceai/spiceai/pull/5681)
- Add m2m support for databricks delta connector by [@ewgenius](https://github.com/ewgenius) in [#5680](https://github.com/spiceai/spiceai/pull/5680)
- Update end_game.md by [@sgrebnov](https://github.com/sgrebnov) in [#5684](https://github.com/spiceai/spiceai/pull/5684)
- Update StaticTokenProvider to use SecretString instead of raw str value by [@ewgenius](https://github.com/ewgenius) in [#5686](https://github.com/spiceai/spiceai/pull/5686)
- Add M2M Auth support for Databricks catalog connector by [@ewgenius](https://github.com/ewgenius) in [#5687](https://github.com/spiceai/spiceai/pull/5687)
- Update UX to disable acceleration federation by [@sgrebnov](https://github.com/sgrebnov) in [#5682](https://github.com/spiceai/spiceai/pull/5682)
- Improve placeholder inference (LIMIT & Expr::InSubquery) by [@phillipleblanc](https://github.com/phillipleblanc) in [#5692](https://github.com/spiceai/spiceai/pull/5692)
- Tweak default log to ignore aws_config::imds::region by [@phillipleblanc](https://github.com/phillipleblanc) in [#5693](https://github.com/spiceai/spiceai/pull/5693)
- Make Spice properly Iceberg Catalog API compatible for load table API by [@phillipleblanc](https://github.com/phillipleblanc) in [#5695](https://github.com/spiceai/spiceai/pull/5695)
- Use deterministic queries for Databricks m2m catalog tests by [@ewgenius](https://github.com/ewgenius) in [#5696](https://github.com/spiceai/spiceai/pull/5696)
- Support retrieving the latest Iceberg table on table scan by [@phillipleblanc](https://github.com/phillipleblanc) in [#5704](https://github.com/spiceai/spiceai/pull/5704)

Full Changelog: v1.2.0...v1.2.1

Spice v1.2.0 (Apr 28, 2025)

Β· 12 min read
Evgenii Khramkov
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.2.0! πŸš€

Spice v1.2.0 is a significant update. It upgrades DataFusion to v45 and Arrow to v54. This release brings faster query performance, support for parameterized queries in SQL and HTTP APIs, and the ability to accelerate views. Several bugs have been fixed and dependencies updated for better stability and speed.

DataFusion v45 Highlights​

Spice.ai is built on the DataFusion query engine. The v45 release brings:

  • Faster Performance πŸš€: DataFusion is now the fastest single-node engine for Apache Parquet files in the clickbench benchmark. Performance improved by over 33% from v33 to v45. Arrow StringView is now on by default, making string and binary data queries much faster, especially with Parquet files.

  • Better Quality πŸ“‹: DataFusion now runs over 5 million SQL tests per push using the SQLite sqllogictest suite. There are new checks for logical plan correctness and more thorough pre-release testing.

  • New SQL Functions ✨: Added show functions, to_local_time, regexp_count, map_extract, array_distance, array_any_value, greatest, least, and arrays_overlap.

See the DataFusion 45.0.0 release notes for details.

Spice.ai upgrades to the latest minus one DataFusion release to ensure adequate testing and stability. The next upgrade to DataFusion v46 is planned for Spice v1.3.0 in May.

What's New in v1.2.0​

  • Parameterized Queries: Parameterized queries are now supported with the Flight SQL API and HTTP API. Positional and named arguments via $1 and :param syntax are supported, respectively. Logical plans for SQL statements are cached for faster repeated queries.

    Example Cookbook recipes:

    See the API Documentation for additional details.

  • Accelerated Views: Views, not just datasets, can now be accelerated. This provides much better performance for views that perform heavy computation.

    Example spicepod.yaml:

    views:
    - name: accelerated_view
    acceleration:
    enabled: true
    engine: duckdb
    primary_key: id
    refresh_check_interval: 1h
    sql: |
    select * from dataset_a
    union all
    select * from dataset_b

    See the Data Acceleration documentation.

  • Memory Usage Metrics & Configuration: Runtime now tracks memory usage as a metric, and a new runtime memory_limit parameter is available. The memory limit parameter applies specifically to the runtime and should be used in addition to existing memory usage configuration, such as duckdb_memory_limit. Memory usage for queries beyond the memory limit will spill to disk.

    See the Memory Reference for details.

  • New Worker Component: Workers are new configurable compute units in the Spice runtime. They help manage compute across models and tools, handle errors, and balance load. Workers are configured in the workers section of spicepod.yaml.

    Example spicepod.yaml:

    workers:
    - name: round-robin
    description: |
    Distributes requests between 'foo' and 'bar' models in a round-robin fashion.
    models:
    - from: foo
    - from: bar
    - name: fallback
    description: |
    Tries 'bar' first, then 'foo', then 'baz' if earlier models fail.
    models:
    - from: foo
    order: 2
    - from: bar
    order: 1
    - from: baz
    order: 3

    See the Workers Documentation for details.

  • Databricks Model Provider: Databricks models can now be used with from: databricks:model_name.

    Example spicepod.yaml:

    models:
    - from: databricks:llama-3_2_1_1b_instruct
    name: llama-instruct
    params:
    databricks_endpoint: dbc-46470731-42e5.cloud.databricks.com
    databricks_token: ${ secrets:SPICE_DATABRICKS_TOKEN }

See the Databricks model documentation.

  • spice chat CLI Improvements: The spice chat command now supports an optional --temperature parameter. A one-shot chat can also be sent with spice chat <message>.

  • More Type Support: Added support for Postgres JSON type and DuckDB Dictionary type.

  • Other Improvements:

    • New image tags let you pick memory allocators for different use-cases: jemalloc, sysalloc, and mimalloc.
    • Better error handling and logging for chat and model operations.

Contributors​

Cookbook Updates​

New recipes for:

The Spice Cookbook now includes 68 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.2.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.2.0 image:

docker pull spiceai/spiceai:1.2.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed​

Dependencies​

Spice is now built with Rust 1.85.0 and Rust 2024.

Changelog​

- Update end_game.md (#5312) by @peasee in https://github.com/spiceai/spiceai/pull/5312
- feat: Add initial testoperator query validation (#5311) by @peasee in https://github.com/spiceai/spiceai/pull/5311
- Update Helm + Prepare for next release (#5317) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5317
- Update spicepod.schema.json (#5319) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5319
- add integration test for reading encrypted PDFs from S3 (#5308) by @kczimm in https://github.com/spiceai/spiceai/pull/5308
- Stop `load_components` during runtime shutdown (#5306) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5306
- Update openapi.json (#5321) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5321
- feat: Implement record batch data validation (#5331) by @peasee in https://github.com/spiceai/spiceai/pull/5331
- Update QA analytics for v1.1.1 (#5320) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5320
- fix: Update benchmark snapshots (#5337) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5337
- Enforce pulls with Spice v1.0.4 (#5339) by @lukekim in https://github.com/spiceai/spiceai/pull/5339
- Upgrade to DataFusion 45, Arrow 54, Rust 1.85 & Edition 2024 (#5334) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5334
- feat: Allow validating testoperator in benchmark workflow (#5342) by @peasee in https://github.com/spiceai/spiceai/pull/5342
- Upgrade `delta_kernel` to 0.9 (#5343) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5343
- deps: Update odbc-api (#5344) by @peasee in https://github.com/spiceai/spiceai/pull/5344
- Fix schema inference for Snowflake tables with large number of columns (#5348) by @ewgenius in https://github.com/spiceai/spiceai/pull/5348
- feat: Update testoperator dispatch for validation, version metric (#5349) by @peasee in https://github.com/spiceai/spiceai/pull/5349
- fix: validate_results not validate (#5352) by @peasee in https://github.com/spiceai/spiceai/pull/5352
- revert to previous pdf-extract; remove test for encrypted pdf support (#5355) by @kczimm in https://github.com/spiceai/spiceai/pull/5355
- Stablize the test `verify_similarity_search_chat_completion` (#5284) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5284
- Turn off `delta_kernel::log_segment` logging and refactor log filtering (#5367) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5367
- Upgrade to DuckDB 1.2.2 (#5375) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5375
- Update Readme - fix broken and outdated links (#5376) by @ewgenius in https://github.com/spiceai/spiceai/pull/5376
- Upgrade dependabot dependencies (#5385) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5385
- fix: Remove IMAP oauth (#5386) by @peasee in https://github.com/spiceai/spiceai/pull/5386
- Bump Helm chart to 1.1.2 (#5389) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5389
- Refactor accelerator registry as part of runtime. (#5318) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5318
- Include `vnd.spiceai.sql/nsql.v1+json` response examples (openapi docs) (#5388) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5388
- docs: Update endgame template with SpiceQA, update qa analytics (#5391) by @peasee in https://github.com/spiceai/spiceai/pull/5391
- Make graceful shutdown timeout configurable (#5358) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5358
- docs: Update release criteria with note on max columns (#5401) by @peasee in https://github.com/spiceai/spiceai/pull/5401
- Update openapi.json (#5392) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5392
- FinanceBench: update scorer instructions and switch scoring model to `gpt-4.1` (#5395) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5395
- feat: Write OTel metrics for testoperator (#5397) by @peasee in https://github.com/spiceai/spiceai/pull/5397
- Update nsql openapi title (#5403) by @ewgenius in https://github.com/spiceai/spiceai/pull/5403
- Track `ai_inferences_count` with used tools flag. Extensible runtime request context. (#5393) by @ewgenius in https://github.com/spiceai/spiceai/pull/5393
- Include newly detected view as changed view (#5408) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5408
- Track used_tools in ai_inferences_with_spice_count as number (#5409) by @ewgenius in https://github.com/spiceai/spiceai/pull/5409
- Update openapi.json (#5406) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5406
- Tweak enforce pulls with Spice (#5411) by @lukekim in https://github.com/spiceai/spiceai/pull/5411
- Allow `flightsql` and `spiceai` connectors to override flight max message size (#5407) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5407
- Retry model graded scorer once on successful, empty response (#5405) by @Jeadie in https://github.com/spiceai/spiceai/pull/5405
- use span task name in 'spice trace' tree, not span_id (#5412) by @Jeadie in https://github.com/spiceai/spiceai/pull/5412
- Rename to `track_ai_inferences_with_spice_count` in all places (#5410) by @ewgenius in https://github.com/spiceai/spiceai/pull/5410
- Update qa_analytics.csv (#5421) by @peasee in https://github.com/spiceai/spiceai/pull/5421
- Remove the filter for the `list_datasets` tool in the AI inferences metric count. (#5417) by @ewgenius in https://github.com/spiceai/spiceai/pull/5417
- fix: Testoperator uses an exact API key for benchmark metric submission (#5413) by @peasee in https://github.com/spiceai/spiceai/pull/5413
- feat: Enable testoperator metrics in workflow (#5422) by @peasee in https://github.com/spiceai/spiceai/pull/5422
- Upgrade mistral.rs (#5404) by @Jeadie in https://github.com/spiceai/spiceai/pull/5404
- Include all FinanceBench documents in benchmark tests (#5426) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5426
- Handle second Ctrl-C to force runtime termination (#5427) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5427
- Add optional `--temperature` parameter for `spice chat` CLI command (#5429) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5429
- Remove `with_runtime_status` from the `RuntimeBuilder` (#5430) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5430
- Fix spice chat error handling (#5433) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5433
- Add more test models to FinanceBench benchmark (#5431) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5431
- support 'from: databricks:model_name' (#5434) by @Jeadie in https://github.com/spiceai/spiceai/pull/5434
- Upgrade Pulls with Spice to v1.0.6 and add concurrency control (#5442) by @lukekim in https://github.com/spiceai/spiceai/pull/5442
- Upgrade DataFusion table providers (#5443) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5443
- Test spice chat in e2e_test_spice_cli (#5447) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5447
- Allow for one-shot chat request using `spice chat <message>` (#5444) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5444
- Enable parallel data sampling for NSQL (#5449) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5449
- Upgrade Go from v1.23.4 to v1.24.2 (#5462) by @lukekim in https://github.com/spiceai/spiceai/pull/5462
- Update PULL_REQUEST_TEMPLATE.md (#5465) by @lukekim in https://github.com/spiceai/spiceai/pull/5465
- Enable captured outputs by default when spiced is started by the CLI (spice run) (#5464) by @lukekim in https://github.com/spiceai/spiceai/pull/5464
- Parameterized queries via Flight SQL API (#5420) by @kczimm in https://github.com/spiceai/spiceai/pull/5420
- fix: Update benchmarks readme badge (#5466) by @peasee in https://github.com/spiceai/spiceai/pull/5466
- delay auth check for binding parameterized queries (#5475) by @kczimm in https://github.com/spiceai/spiceai/pull/5475
- Add support for `?` placeholder syntax in parameterized queries (#5463) by @kczimm in https://github.com/spiceai/spiceai/pull/5463
- enable task name override for non static span names (#5423) by @Jeadie in https://github.com/spiceai/spiceai/pull/5423
- Allow parameter queries with no parameters (#5481) by @kczimm in https://github.com/spiceai/spiceai/pull/5481
- Support unparsing UNION for distinct results (#5483) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5483
- add rust-toolchain.toml (#5485) by @kczimm in https://github.com/spiceai/spiceai/pull/5485
- Add parameterized query support to the HTTP API (#5484) by @kczimm in https://github.com/spiceai/spiceai/pull/5484
- E2E test for spice chat <message> behavior (#5451) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5451
- Renable and fix huggingface models integration tests (#5478) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5478
- Update openapi.json (#5488) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5488
- feat: Record memory usage as a metric (#5489) by @peasee in https://github.com/spiceai/spiceai/pull/5489
- fix: update dispatcher to run all benchmarks, rename metric, update spicepods, add scale factor (#5500) by @peasee in https://github.com/spiceai/spiceai/pull/5500
- Fix ILIKE filters support (#5502) by @ewgenius in https://github.com/spiceai/spiceai/pull/5502
- fix: Update test spicepod locations and names (#5505) by @peasee in https://github.com/spiceai/spiceai/pull/5505
- fix: Update benchmark snapshots (#5508) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5508
- fix: Update benchmark snapshots (#5512) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5512
- Fix Delta Lake bug for: Found unmasked nulls for non-nullable StructArray field "predicate" (#5515) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5515
- fix: working directory for duckdb e2e test spicepods (#5510) by @peasee in https://github.com/spiceai/spiceai/pull/5510
- Tweaks to README.md (#5516) by @lukekim in https://github.com/spiceai/spiceai/pull/5516
- Cache logical plans of SQL statements (#5487) by @kczimm in https://github.com/spiceai/spiceai/pull/5487
- Fix `content-type: application/json` (#5517) by @Jeadie in https://github.com/spiceai/spiceai/pull/5517
- Validate postgres results in testoperator dispatch (#5504) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5504
- fix: Update benchmark snapshots (#5511) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5511
- Fix results cache by SQL with prepared statements (#5518) by @kczimm in https://github.com/spiceai/spiceai/pull/5518
- Add initial support for views acceleration (#5509) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5509
- fix: Update benchmark snapshots (#5527) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5527
- Support switching the memory allocator Spice uses via `alloc-*` features. (#5528) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5528
- fix: Update benchmark snapshots (#5525) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5525
- Add test spicepod for tpch mysql-duckdb[file acceleration] (#5521) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5521
- Fix nightly arm build - change tag `-default` to `-models` (#5529) by @ewgenius in https://github.com/spiceai/spiceai/pull/5529
- LLM router via `worker` spicepod component (#5513) by @Jeadie in https://github.com/spiceai/spiceai/pull/5513
- Apply Spice advanced acceleration logic and params support to accelerated views (#5526) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5526
- Enable DatasetCheckpoint logic for accelerated views (#5533) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5533
- Fix public '.model' name for router workers (#5535) by @Jeadie in https://github.com/spiceai/spiceai/pull/5535
- feat: Add Runtime memory limit parameter (#5536) by @peasee in https://github.com/spiceai/spiceai/pull/5536
- For fallback worker, check first item in `chat/completion` stream. (#5537) by @Jeadie in https://github.com/spiceai/spiceai/pull/5537
- Move rate limit check to after parameterized query binding (#5540) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5540
- Update spicepod.schema.json (#5545) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5545
- Accelerate views: refresh_on_startup, ready_state, jitter params support (#5547) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5547
- Add integration test for accelerated views (#5550) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5550
- Don't install make or expect on spiceai-macos runners (#5554) by @lukekim in https://github.com/spiceai/spiceai/pull/5554
- `event_stream` crate for emitting events from tracing::Span; used in v1/chat/completions streaming. (#5474) by @Jeadie in https://github.com/spiceai/spiceai/pull/5474
- Fix typo in method (#5559) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5559
- Run test operator every day and current and previous commits (#5557) by @lukekim in https://github.com/spiceai/spiceai/pull/5557
- Add aws_allow_http parameter for delta lake connector (#5541) by @Sevenannn in https://github.com/spiceai/spiceai/pull/5541
- feat: Add branch name to metric dimensions in testoperator (#5563) by @peasee in https://github.com/spiceai/spiceai/pull/5563
- fix: Update the tpch benchmark snapshots for: ./test/spicepods/tpch/sf1/federated/odbc[databricks].yaml (#5565) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5565
- fix: Split scheduled dispatch into a separate job (#5567) by @peasee in https://github.com/spiceai/spiceai/pull/5567
- fix: Use outputs.SPICED_COMMIT (#5568) by @peasee in https://github.com/spiceai/spiceai/pull/5568
- fix: Use refs in testoperator dispatch instead of commits (#5569) by @peasee in https://github.com/spiceai/spiceai/pull/5569
- fix: actions/checkout ref does not take a full ref (#5571) by @peasee in https://github.com/spiceai/spiceai/pull/5571
- fix: Testoperator dispatch (#5572) by @peasee in https://github.com/spiceai/spiceai/pull/5572
- Respect `update-snapshots` when running all benchmarks manually (#5577) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5577
- Use FETCH_HEAD instead of ${{ inputs.ref }} to list commits in setup_spiced (#5579) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5579
- Add additional test scenarios for benchmarks (#5582) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5582
- fix: Update the tpch benchmark snapshots for: test/spicepods/tpch/sf1/accelerated/databricks[delta_lake]-duckdb[file].yaml (#5590) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5590
- fix: Update the tpch benchmark snapshots for: test/spicepods/tpch/sf1/accelerated/mysql-duckdb[file].yaml (#5591) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5591
- Fix Snowflake data connector rows ordering (#5599) by @sgrebnov in https://github.com/spiceai/spiceai/pull/5599
- fix: Update benchmark snapshots (#5595) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5595
- fix: Update the tpch benchmark snapshots for: test/spicepods/tpch/sf1/accelerated/databricks[delta_lake]-arrow.yaml (#5594) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5594
- fix: Update benchmark snapshots (#5589) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5589
- fix: Update benchmark snapshots (#5583) by @app/github-actions in https://github.com/spiceai/spiceai/pull/5583
- Downgrade DuckDB to 1.1.3 (#5607) by @phillipleblanc in https://github.com/spiceai/spiceai/pull/5607
- Add prepared statement integration tests (#5544) by @kczimm in https://github.com/spiceai/spiceai/pull/5544

Full Changelog: v1.1.2...v1.2.0

Spice v1.1.2 (Apr 14, 2025)

Β· 3 min read
Phillip LeBlanc
Co-Founder and CTO of Spice AI

Announcing the release of Spice v1.1.2! πŸ”—

Spice v1.1.2 improves Delta Lake Data Connector performance, introduces new Accept headers for the /v1/sql and /v1/nsql endpoints to include query metadata with results, and resolves an issue with the Snowflake Data Connector when handling wide tables (>600 columns).

The official Tableau Connector for Spice.ai v0.1 has been released, making it easy to connect to both self-hosted Spice.ai and Spice Cloud instances using Tableau.

What's New in v1.1.2​

Query Spice.ai using Tableau

  • Delta Lake Data Connector: Upgraded delta_kernel to v0.9, and optimized scan operations, reducing query execution time by up to 20% on large datasets.

  • Snowflake Data Connector: Fixed a bug that caused failures when loading tables with more than 600 columns.

  • Query Metadata (SQL and NSQL): Added support for the application/vnd.spiceai.sql.v1+json Accept header on the /v1/sql endpoint, and the application/vnd.spiceai.nsql.v1+json Accept header on the /v1/nsql endpoint, enabling responses to include metadata such as the executed SQL query and schema alongside results.

Example:

curl -XPOST "http://localhost:8090/v1/nsql" \
-H "Content-Type: application/json" \
-H "Accept: application/vnd.spiceai.nsql.v1+json" \
-d '{
"query": "What’s the highest tip any passenger gave?"
}' | jq

Example response:

{
"row_count": 1,
"schema": {
"fields": [
{
"name": "highest_tip",
"data_type": "Float64",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
}
],
"metadata": {}
},
"data": [
{
"highest_tip": 428.0
}
],
"sql": "SELECT MAX(\"tip_amount\") AS \"highest_tip\"\nFROM \"spice\".\"public\".\"taxi_trips\""
}

For details, see the SQL Query API and NSQL API documentation.

Contributors​

Breaking Changes​

No breaking changes in this release.

Cookbook Updates​

No major cookbook additions.

The Spice Cookbook now includes 65 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.1.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.1.2 image:

docker pull spiceai/spiceai:1.1.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed​

Dependencies​

Changelog​

  • Backport - Fix schema inference for Snowflake tables with large number of columns #5348 by @ewgenius in #5350
  • Upgrade delta_kernel to 0.9 (#5343) by @phillipleblanc in #5356
  • Add basic support for application/vnd.spiceai.sql.v1+json format (#5333) by @sgrebnov in #5333
  • Convert DataFusion filters to Delta Kernel predicates by @phillipleblanc in #5362
  • revert to previous pdf-extract; remove test for encrypted pdf support by @kczimm in #5355
  • Turn off delta_kernel::log_segment logging and refactor log filtering by @phillipleblanc in #5367
  • Extend application/vnd.spiceai.sql.v1+json with schema and row_count fields by @sgrebnov in #5365
  • Make separate vnd.spiceai.sql.v1+json and vnd.spiceai.nsql.v1+json MIME types by @sgrebnov in #5382

Full Changelog: v1.1.1...v1.1.2

Spice v1.1.1 (Apr 7, 2025)

Β· 5 min read
Phillip LeBlanc
Co-Founder and CTO of Spice AI

Announcing the release of Spice v1.1.1! πŸ“Š

Spice v1.1.1 introduces several key updates, including a new Component Metrics System, improved Delta Data Connector performance, improved MCP tool descriptions, and expanded runtime results caching options. This release also adds detailed MySQL connection pool metrics for better observability. Component Metrics are Prometheus-compatible and accessible via the metrics endpoint.

Highlights v1.1.1​

  • Component Metrics System: A new system for monitoring components, starting with MySQL connection pool metrics. These metrics provide insights into MySQL connection performance and can be selectively enabled in the dataset configuration. Metrics are exposed in Prometheus format via the metrics endpoint.

For more details, see the Component Metrics documentation.

  • Results Caching Enhancements: Added a cache_key_type option for runtime results caching. Options include:
    • plan (Default): Uses the query's logical plan as the cache key. Matches semantically equivalent queries but requires query parsing.
    • sql: Uses the raw SQL string as the cache key. Provides faster lookups but requires exact string matches. Use sql for predictable queries without dynamic functions like NOW().

Example spicepod.yaml configuration:

runtime:
results_cache:
enabled: true
cache_max_size: 128MiB
cache_key_type: sql # Use SQL for the results cache key
item_ttl: 1s

For more details, see the runtime configuration documentation.

  • Delta Data Connector: Improved scan performance for faster query performance.

  • MCP Tools: Improved descriptions for built-in MCP tools to improve usability.

  • MySQL Component Metrics: Added detailed metrics for monitoring MySQL connections, such as connection count and pool activity.

Example spicepod.yaml configuration:

datasets:
- from: mysql:my_table
name: my_dataset
metrics:
- name: connection_count
enabled: true
- name: connections_in_pool
enabled: true
- name: active_wait_requests
enabled: true
params:
mysql_host: localhost
mysql_tcp_port: 3306
mysql_user: root
mysql_pass: ${secrets:MYSQL_PASS}

For more details, see the MySQL Data Connector documentation.

  • spice.js SDK: The spice.js SDK has been updated to v2.0.1 and includes several important security updates.

New Contributors πŸŽ‰β€‹

Contributors​

Breaking Changes​

No breaking changes in this release.

Cookbook Updates​

The Spice Cookbook now includes 65 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.1.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.1.1 image:

docker pull spiceai/spiceai:1.1.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed​

Dependencies​

  • No major dependency changes.

Changelog​

- fix: Testoperator DuckDB, SQLite, Postgres, Spicecloud by [@peasee](https://github.com/peasee) in [#5190](https://github.com/spiceai/spiceai/pull/5190)
- Update Helm Chart and SECURITY.md to v1.1.0 by [@lukekim](https://github.com/lukekim) in [#5223](https://github.com/spiceai/spiceai/pull/5223)
- Update version.txt to v1.1.1-unstable by [@lukekim](https://github.com/lukekim) in [#5224](https://github.com/spiceai/spiceai/pull/5224)
- Update Cargo.lock to v1.1.1-unstable by [@lukekim](https://github.com/lukekim) in [#5225](https://github.com/spiceai/spiceai/pull/5225)
- Add tests for `verify_schema_source_path` in `ListingTableConnector` by [@phillipleblanc](https://github.com/phillipleblanc) in [#5221](https://github.com/spiceai/spiceai/pull/5221)
- Reduce noise from debug logging by [@phillipleblanc](https://github.com/phillipleblanc) in [#5227](https://github.com/spiceai/spiceai/pull/5227)
- Improve `openai_test_chat_messages` integration test reliability by [@Sevenannn](https://github.com/Sevenannn) in [#5222](https://github.com/spiceai/spiceai/pull/5222)
- Verify the checkpoints existence before shutting down runtime in integration tests directly querying checkpoint by [@Sevenannn](https://github.com/Sevenannn) in [#5232](https://github.com/spiceai/spiceai/pull/5232)
- Fix CORS support for json content-type api by [@sgrebnov](https://github.com/sgrebnov) in [#5241](https://github.com/spiceai/spiceai/pull/5241)
- Fix ModelGradedScorer error: The 'metadata' parameter is only allowed when 'store' is enabled. by [@sgrebnov](https://github.com/sgrebnov) in [#5231](https://github.com/spiceai/spiceai/pull/5231)
- fix: Use `pulls-with-spice-action` and switch to `spiceai-macos` runners by [@peasee](https://github.com/peasee) in [#5238](https://github.com/spiceai/spiceai/pull/5238)
- Use v1.0.3 pulls with spice action by [@lukekim](https://github.com/lukekim) in [#5244](https://github.com/spiceai/spiceai/pull/5244)
- feat: Build ODBC binaries, run testoperator on ODBC by [@peasee](https://github.com/peasee) in [#5237](https://github.com/spiceai/spiceai/pull/5237)
- Bump timeout for several integration test runtime load_components & readiness check by [@Sevenannn](https://github.com/Sevenannn) in [#5229](https://github.com/spiceai/spiceai/pull/5229)
- Validate port is available before binding port for docker container in integration tests by [@Sevenannn](https://github.com/Sevenannn) in [#5248](https://github.com/spiceai/spiceai/pull/5248)
- Update datafusion-table-providers to fix the schema for PostgreSQL materialized views by [@ewgenius](https://github.com/ewgenius) in [#5259](https://github.com/spiceai/spiceai/pull/5259)
- Verify flight server is ready for flight integration tests by [@Sevenannn](https://github.com/Sevenannn) in [#5240](https://github.com/spiceai/spiceai/pull/5240)
- fix: Publish to MinIO inside of matrix on build_and_release by [@peasee](https://github.com/peasee) in [#5258](https://github.com/spiceai/spiceai/pull/5258)
- fix: TPCDS on zero results benchmarks by [@peasee](https://github.com/peasee) in [#5263](https://github.com/spiceai/spiceai/pull/5263)
- Use model as a judge scorer for Financebench by [@sgrebnov](https://github.com/sgrebnov) in [#5264](https://github.com/spiceai/spiceai/pull/5264)
- Fix FinanceBench llm scorer secret name by [@sgrebnov](https://github.com/sgrebnov) in [#5276](https://github.com/spiceai/spiceai/pull/5276)
- Implements support for `runtime.results_cache.cache_key_type` by [@phillipleblanc](https://github.com/phillipleblanc) in [#5265](https://github.com/spiceai/spiceai/pull/5265)
- fix: Testoperator MS SQL, query overrides, dispatcher by [@peasee](https://github.com/peasee) in [#5279](https://github.com/spiceai/spiceai/pull/5279)
- refactor: Delete old benchmarks by [@peasee](https://github.com/peasee) in [#5283](https://github.com/spiceai/spiceai/pull/5283)
- Imporve embedding column parsing performance test by [@Sevenannn](https://github.com/Sevenannn) in [#5268](https://github.com/spiceai/spiceai/pull/5268)
- Add Support for AWS Session Token in S3 Data Connector by [@kczimm](https://github.com/kczimm) in [#5243](https://github.com/spiceai/spiceai/pull/5243)
- Implement Component Metrics system + MySQL connection pool metrics by [@phillipleblanc](https://github.com/phillipleblanc) in [#5290](https://github.com/spiceai/spiceai/pull/5290)
- Add default descriptions to built-in MCP tools by [@lukekim](https://github.com/lukekim) in [#5293](https://github.com/spiceai/spiceai/pull/5293)
- fix: Vector search with cased columns by [@peasee](https://github.com/peasee) in [#5295](https://github.com/spiceai/spiceai/pull/5295)
- Run delta kernel scan in a blocking Tokio thread. by [@phillipleblanc](https://github.com/phillipleblanc) in [#5296](https://github.com/spiceai/spiceai/pull/5296)
- Expose the `mysql_pool_min` and `mysql_pool_max` connection pool parameters by [@phillipleblanc](https://github.com/phillipleblanc) in [#5297](https://github.com/spiceai/spiceai/pull/5297)
- use patched pdf-extract by [@kczimm](https://github.com/kczimm) in [#5270](https://github.com/spiceai/spiceai/pull/5270)

Full Changelog: v1.1.0...v1.1.1