Spice v1.2.1 (May 6, 2025)

Announcing the release of Spice v1.2.1! 🔥

Spice v1.2.1 includes several data connector fixes and improves query performance for accelerated views. This release also introduces Databricks Service Principal (M2M OAuth) authentication and expands parameterized queries.

Highlights in v1.2.1

• Databricks Service Principal Support: Databricks datasets and catalogs now support Machine-to-Machine (M2M) OAuth authentication via Service Principals, enabling secure machine connections to Databricks.

Example spicepod.yaml:

datasets:
  - from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
    name: my_delta_lake_table
    params:
      mode: delta_lake
      databricks_endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com
      databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
      databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}

For details, see documentation for:

• Databricks Data Connector

• Databricks Unity Catalog Connector

• Iceberg Data Connector: Now supports cross-account table access via the AWS Glue Catalog Connector and fixes an issue when querying data from append mode datasets.

• Iceberg Catalog API: Full compatibility with the Iceberg HTTP REST Catalog API to consume Spice datasets from Iceberg Catalog clients.

For details, see documentation for:

• Iceberg Data Connector

• S3 Data Connector

• Improved Parameterized Query Support: Expanded type inference for placeholders in:

  • IN list expressions
  • LIKE patterns
  • SIMILAR TO patterns
  • LIMIT clauses
  • Subqueries

New Contributors 🎉

• @nuvic made their first contribution in #5673

Contributors

• @peasee
• @ewgenius
• @karifabri
• @phillipleblanc
• @kczimm
• @Jeadie
• @sgrebnov
• @Sevenannn
• @nuvic

Breaking Changes

No breaking changes.

Cookbook Updates

New recipes for:

• Language Model Evaluations: Use Spice.ai OSS to evaluate language models.
• LLM as a Judge: Use LLM judge models to evaluate the performance of other language models.

The Spice Cookbook now includes 68 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v1.2.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.2.1 image:

docker pull spiceai/spiceai:1.2.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed

Dependencies

• No major dependency changes.

Changelog

- Fix: Specify metric type as a dimension for testoperator by [@peasee](https://github.com/peasee) in [#5630](https://github.com/spiceai/spiceai/pull/5630)
- Fix: Add option to run dispatch schedule by [@peasee](https://github.com/peasee) in [#5631](https://github.com/spiceai/spiceai/pull/5631)
- Infer placeholder datatype for InList, Like, and SimilarTo by [@kczimm](https://github.com/kczimm) in [#5626](https://github.com/spiceai/spiceai/pull/5626)
- Add QA analytics for 1.2.0 by [@phillipleblanc](https://github.com/phillipleblanc) in [#5640](https://github.com/spiceai/spiceai/pull/5640)
- Fix: Use SPICED_COMMIT for spiced_commit_sha by [@peasee](https://github.com/peasee) in [#5632](https://github.com/spiceai/spiceai/pull/5632)
- New crates/tools by [@Jeadie](https://github.com/Jeadie) in [#5121](https://github.com/spiceai/spiceai/pull/5121)
- Update openapi.json by [@github-actions](https://github.com/github-actions) in [#5643](https://github.com/spiceai/spiceai/pull/5643)
- Enable metrics reporting for models benchmarks (evals) by [@sgrebnov](https://github.com/sgrebnov) in [#5639](https://github.com/spiceai/spiceai/pull/5639)
- Implement CatalogBuilder, add app and runtime references to catalog component, add runtime reference to connector params by [@ewgenius](https://github.com/ewgenius) in [#5641](https://github.com/spiceai/spiceai/pull/5641)
- Fix eventing bug in LLM progress; Add tool and worker progress by [@Jeadie](https://github.com/Jeadie) in [#5619](https://github.com/spiceai/spiceai/pull/5619)
- Handle small precision differences in TPCH answer validation by [@phillipleblanc](https://github.com/phillipleblanc) in [#5642](https://github.com/spiceai/spiceai/pull/5642)
- Add TokenProviderRegistry to the runtime by [@ewgenius](https://github.com/ewgenius) in [#5651](https://github.com/spiceai/spiceai/pull/5651)
- Provide ModelContextLayer for evals by [@Jeadie](https://github.com/Jeadie) in [#5648](https://github.com/spiceai/spiceai/pull/5648)
- Databricks data_components refactor. Databricks Spark connect - add set_token method and writable spark session by [@ewgenius](https://github.com/ewgenius) in [#5654](https://github.com/spiceai/spiceai/pull/5654)
- Extract AWS Glue warehouse for cross-account Iceberg tables by [@phillipleblanc](https://github.com/phillipleblanc) in [#5656](https://github.com/spiceai/spiceai/pull/5656)
- Refactor Dataset component by [@phillipleblanc](https://github.com/phillipleblanc) in [#5660](https://github.com/spiceai/spiceai/pull/5660)
- Fix Iceberg API returning 404 when schema contains a Dictionary by [@phillipleblanc](https://github.com/phillipleblanc) in [#5665](https://github.com/spiceai/spiceai/pull/5665)
- Fix dependencies: downgrade swagger-ui to v8; force zip to 2.3.0 by [@kczimm](https://github.com/kczimm) in [#5664](https://github.com/spiceai/spiceai/pull/5664)
- Add DuckDB indexes spicepod, additional dispatches by [@peasee](https://github.com/peasee) in [#5633](https://github.com/spiceai/spiceai/pull/5633)
- Update readme: update data federation link by [@nuvic](https://github.com/nuvic) in [#5673](https://github.com/spiceai/spiceai/pull/5673)
- Support metadata columns for object-store based data connectors by [@phillipleblanc](https://github.com/phillipleblanc) in [#5661](https://github.com/spiceai/spiceai/pull/5661)
- Add model name to LLM judges, and add model_graded_scoring task by [@Jeadie](https://github.com/Jeadie) in [#5655](https://github.com/spiceai/spiceai/pull/5655)
- Add SF1000 TPCH test spicepods for delta lake by [@Sevenannn](https://github.com/Sevenannn) in [#5606](https://github.com/spiceai/spiceai/pull/5606)
- Validate Github Connector resource existence before building the github connector graphql table by [@Sevenannn](https://github.com/Sevenannn) in [#5674](https://github.com/spiceai/spiceai/pull/5674)
- Remove hard-coded embedding performance tests in CI by [@Sevenannn](https://github.com/Sevenannn) in [#5675](https://github.com/spiceai/spiceai/pull/5675)
- Databricks M2M auth for spark connect data connector by [@ewgenius](https://github.com/ewgenius) in [#5659](https://github.com/spiceai/spiceai/pull/5659)
- Enable federated data refresh support for accelerated views by [@sgrebnov](https://github.com/sgrebnov) in [#5677](https://github.com/spiceai/spiceai/pull/5677)
- Add pods watcher integration test by [@Sevenannn](https://github.com/Sevenannn) in [#5681](https://github.com/spiceai/spiceai/pull/5681)
- Add m2m support for databricks delta connector by [@ewgenius](https://github.com/ewgenius) in [#5680](https://github.com/spiceai/spiceai/pull/5680)
- Update end_game.md by [@sgrebnov](https://github.com/sgrebnov) in [#5684](https://github.com/spiceai/spiceai/pull/5684)
- Update StaticTokenProvider to use SecretString instead of raw str value by [@ewgenius](https://github.com/ewgenius) in [#5686](https://github.com/spiceai/spiceai/pull/5686)
- Add M2M Auth support for Databricks catalog connector by [@ewgenius](https://github.com/ewgenius) in [#5687](https://github.com/spiceai/spiceai/pull/5687)
- Update UX to disable acceleration federation by [@sgrebnov](https://github.com/sgrebnov) in [#5682](https://github.com/spiceai/spiceai/pull/5682)
- Improve placeholder inference (LIMIT & Expr::InSubquery) by [@phillipleblanc](https://github.com/phillipleblanc) in [#5692](https://github.com/spiceai/spiceai/pull/5692)
- Tweak default log to ignore aws_config::imds::region by [@phillipleblanc](https://github.com/phillipleblanc) in [#5693](https://github.com/spiceai/spiceai/pull/5693)
- Make Spice properly Iceberg Catalog API compatible for load table API by [@phillipleblanc](https://github.com/phillipleblanc) in [#5695](https://github.com/spiceai/spiceai/pull/5695)
- Use deterministic queries for Databricks m2m catalog tests by [@ewgenius](https://github.com/ewgenius) in [#5696](https://github.com/spiceai/spiceai/pull/5696)
- Support retrieving the latest Iceberg table on table scan by [@phillipleblanc](https://github.com/phillipleblanc) in [#5704](https://github.com/spiceai/spiceai/pull/5704)

Full Changelog: v1.2.0...v1.2.1