Skip to main content
Sergei Grebnov
Senior Software Engineer at Spice AI
View all authors

Spice v1.2.1 (May 6, 2025)

ยท 5 min read
Sergei Grebnov
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.2.1! ๐Ÿ”ฅ

Spice v1.2.1 includes several data connector fixes and improves query performance for accelerated views. This release also introduces Databricks Service Principal (M2M OAuth) authentication and expands parameterized queries.

Highlights in v1.2.1โ€‹

  • Databricks Service Principal Support: Databricks datasets and catalogs now support Machine-to-Machine (M2M) OAuth authentication via Service Principals, enabling secure machine connections to Databricks.

Example spicepod.yaml:

datasets:
- from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
name: my_delta_lake_table
params:
mode: delta_lake
databricks_endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com
databricks_client_id: ${secrets:DATABRICKS_CLIENT_ID}
databricks_client_secret: ${secrets:DATABRICKS_CLIENT_SECRET}

For details, see documentation for:

  • Databricks Data Connector

  • Databricks Unity Catalog Connector

  • Iceberg Data Connector: Now supports cross-account table access via the AWS Glue Catalog Connector and fixes an issue when querying data from append mode datasets.

  • Iceberg Catalog API: Full compatibility with the Iceberg HTTP REST Catalog API to consume Spice datasets from Iceberg Catalog clients.

For details, see documentation for:

  • Iceberg Data Connector

  • S3 Data Connector

  • Improved Parameterized Query Support: Expanded type inference for placeholders in:

    • IN list expressions
    • LIKE patterns
    • SIMILAR TO patterns
    • LIMIT clauses
    • Subqueries

New Contributors ๐ŸŽ‰โ€‹

Contributorsโ€‹

Breaking Changesโ€‹

No breaking changes.

Cookbook Updatesโ€‹

New recipes for:

The Spice Cookbook now includes 68 recipes to help you get started with Spice quickly and easily.

Upgradingโ€‹

To upgrade to v1.2.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.2.1 image:

docker pull spiceai/spiceai:1.2.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changedโ€‹

Dependenciesโ€‹

  • No major dependency changes.

Changelogโ€‹

- Fix: Specify metric type as a dimension for testoperator by [@peasee](https://github.com/peasee) in [#5630](https://github.com/spiceai/spiceai/pull/5630)
- Fix: Add option to run dispatch schedule by [@peasee](https://github.com/peasee) in [#5631](https://github.com/spiceai/spiceai/pull/5631)
- Infer placeholder datatype for InList, Like, and SimilarTo by [@kczimm](https://github.com/kczimm) in [#5626](https://github.com/spiceai/spiceai/pull/5626)
- Add QA analytics for 1.2.0 by [@phillipleblanc](https://github.com/phillipleblanc) in [#5640](https://github.com/spiceai/spiceai/pull/5640)
- Fix: Use SPICED_COMMIT for spiced_commit_sha by [@peasee](https://github.com/peasee) in [#5632](https://github.com/spiceai/spiceai/pull/5632)
- New crates/tools by [@Jeadie](https://github.com/Jeadie) in [#5121](https://github.com/spiceai/spiceai/pull/5121)
- Update openapi.json by [@github-actions](https://github.com/github-actions) in [#5643](https://github.com/spiceai/spiceai/pull/5643)
- Enable metrics reporting for models benchmarks (evals) by [@sgrebnov](https://github.com/sgrebnov) in [#5639](https://github.com/spiceai/spiceai/pull/5639)
- Implement CatalogBuilder, add app and runtime references to catalog component, add runtime reference to connector params by [@ewgenius](https://github.com/ewgenius) in [#5641](https://github.com/spiceai/spiceai/pull/5641)
- Fix eventing bug in LLM progress; Add tool and worker progress by [@Jeadie](https://github.com/Jeadie) in [#5619](https://github.com/spiceai/spiceai/pull/5619)
- Handle small precision differences in TPCH answer validation by [@phillipleblanc](https://github.com/phillipleblanc) in [#5642](https://github.com/spiceai/spiceai/pull/5642)
- Add TokenProviderRegistry to the runtime by [@ewgenius](https://github.com/ewgenius) in [#5651](https://github.com/spiceai/spiceai/pull/5651)
- Provide ModelContextLayer for evals by [@Jeadie](https://github.com/Jeadie) in [#5648](https://github.com/spiceai/spiceai/pull/5648)
- Databricks data_components refactor. Databricks Spark connect - add set_token method and writable spark session by [@ewgenius](https://github.com/ewgenius) in [#5654](https://github.com/spiceai/spiceai/pull/5654)
- Extract AWS Glue warehouse for cross-account Iceberg tables by [@phillipleblanc](https://github.com/phillipleblanc) in [#5656](https://github.com/spiceai/spiceai/pull/5656)
- Refactor Dataset component by [@phillipleblanc](https://github.com/phillipleblanc) in [#5660](https://github.com/spiceai/spiceai/pull/5660)
- Fix Iceberg API returning 404 when schema contains a Dictionary by [@phillipleblanc](https://github.com/phillipleblanc) in [#5665](https://github.com/spiceai/spiceai/pull/5665)
- Fix dependencies: downgrade swagger-ui to v8; force zip to 2.3.0 by [@kczimm](https://github.com/kczimm) in [#5664](https://github.com/spiceai/spiceai/pull/5664)
- Add DuckDB indexes spicepod, additional dispatches by [@peasee](https://github.com/peasee) in [#5633](https://github.com/spiceai/spiceai/pull/5633)
- Update readme: update data federation link by [@nuvic](https://github.com/nuvic) in [#5673](https://github.com/spiceai/spiceai/pull/5673)
- Support metadata columns for object-store based data connectors by [@phillipleblanc](https://github.com/phillipleblanc) in [#5661](https://github.com/spiceai/spiceai/pull/5661)
- Add model name to LLM judges, and add model_graded_scoring task by [@Jeadie](https://github.com/Jeadie) in [#5655](https://github.com/spiceai/spiceai/pull/5655)
- Add SF1000 TPCH test spicepods for delta lake by [@Sevenannn](https://github.com/Sevenannn) in [#5606](https://github.com/spiceai/spiceai/pull/5606)
- Validate Github Connector resource existence before building the github connector graphql table by [@Sevenannn](https://github.com/Sevenannn) in [#5674](https://github.com/spiceai/spiceai/pull/5674)
- Remove hard-coded embedding performance tests in CI by [@Sevenannn](https://github.com/Sevenannn) in [#5675](https://github.com/spiceai/spiceai/pull/5675)
- Databricks M2M auth for spark connect data connector by [@ewgenius](https://github.com/ewgenius) in [#5659](https://github.com/spiceai/spiceai/pull/5659)
- Enable federated data refresh support for accelerated views by [@sgrebnov](https://github.com/sgrebnov) in [#5677](https://github.com/spiceai/spiceai/pull/5677)
- Add pods watcher integration test by [@Sevenannn](https://github.com/Sevenannn) in [#5681](https://github.com/spiceai/spiceai/pull/5681)
- Add m2m support for databricks delta connector by [@ewgenius](https://github.com/ewgenius) in [#5680](https://github.com/spiceai/spiceai/pull/5680)
- Update end_game.md by [@sgrebnov](https://github.com/sgrebnov) in [#5684](https://github.com/spiceai/spiceai/pull/5684)
- Update StaticTokenProvider to use SecretString instead of raw str value by [@ewgenius](https://github.com/ewgenius) in [#5686](https://github.com/spiceai/spiceai/pull/5686)
- Add M2M Auth support for Databricks catalog connector by [@ewgenius](https://github.com/ewgenius) in [#5687](https://github.com/spiceai/spiceai/pull/5687)
- Update UX to disable acceleration federation by [@sgrebnov](https://github.com/sgrebnov) in [#5682](https://github.com/spiceai/spiceai/pull/5682)
- Improve placeholder inference (LIMIT & Expr::InSubquery) by [@phillipleblanc](https://github.com/phillipleblanc) in [#5692](https://github.com/spiceai/spiceai/pull/5692)
- Tweak default log to ignore aws_config::imds::region by [@phillipleblanc](https://github.com/phillipleblanc) in [#5693](https://github.com/spiceai/spiceai/pull/5693)
- Make Spice properly Iceberg Catalog API compatible for load table API by [@phillipleblanc](https://github.com/phillipleblanc) in [#5695](https://github.com/spiceai/spiceai/pull/5695)
- Use deterministic queries for Databricks m2m catalog tests by [@ewgenius](https://github.com/ewgenius) in [#5696](https://github.com/spiceai/spiceai/pull/5696)
- Support retrieving the latest Iceberg table on table scan by [@phillipleblanc](https://github.com/phillipleblanc) in [#5704](https://github.com/spiceai/spiceai/pull/5704)

Full Changelog: v1.2.0...v1.2.1

Spice v1.0.6 (Mar 17, 2025)

ยท 3 min read
Sergei Grebnov
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.0.6 โšก

Spice v1.0.6 improves stability for DuckDB acceleration, Iceberg Data/Catalog connector improvements when using AWS Glue, and fixes an issue with the ready_state: on_registration federation fallback when using DuckDB. In addition, redundant data refreshes on startup are avoided for accelerations with persistent data.

Highlights in v1.0.6โ€‹

  • Iceberg Data/Catalog Connector Improvements: Improves Iceberg data & catalog connector reliability, including bug fixes for AWS Glue API rate-limiting and compatibility, REST API pagination support, explicit AWS credential handling, and support for AWS STS role assumption.

  • Fixes On-Registration Fallback when using DuckDB: Previously, when using DuckDB as a data accelerator and the ready_state: on_registration configuration, queries made during the initial data refresh did not properly fallback to the federated source. This is now fixed.

  • DuckDB downgraded for Stability: DuckDB has been downgraded to v1.1.3 due to a regression in memory handling tracked by duckdb/duckdb issue #16640. Once resolved and validated, Spice will re-upgrade to v1.2.x.

  • Expanded Integration Tests: Additional integration tests covering federated accelerator behavior and graceful shutdown processes have been added.

  • Optimized Data Refresh for Persistent Accelerations: Changed behavior in v1.0.6. When using persistent (file-mode) acceleration without a defined refresh interval, Spice performs a full refresh at startup only if no previously accelerated data is available. This ensures efficient startup behavior by avoiding unnecessary refreshes. This logic applies only to full refreshes when no refresh interval is specified.

To maintain the previous behavior and always refresh on every startup, set:

acceleration:
refresh_on_startup: always

Contributorsโ€‹

  • @peasee
  • @phillipleblanc
  • @sgrebnov
  • @lukekim
  • @Sevenannn

Breaking Changesโ€‹

Starting from v1.0.6 when using persistent (file-mode) acceleration without a defined refresh interval, Spice performs a full refresh at startup only if no previously accelerated data is available. To maintain the previous behavior and always refresh on every startup, set:

acceleration:
refresh_on_startup: always

Cookbook Updatesโ€‹

No new recipes.

Upgradingโ€‹

To upgrade to v1.0.6, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.6 image:

docker pull spiceai/spiceai:1.0.6

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changedโ€‹

Dependenciesโ€‹

Changelogโ€‹

  • Implement proper ready_state: on_registration for federation enabled accelerators by @phillipleblanc in #5019
  • Add indexes and primary keys mismatch detection for DuckDB Acceleration by @sgrebnov in #5045
  • Add comprehensive integration tests for the ready_state behavior by @phillipleblanc in #5042
  • Add test Spicepod for acceleration with constraints by @sgrebnov in #4891
  • Add test Spicepod for DuckDB append acceleration with constraints by @sgrebnov in #4898
  • Add DuckDB graceful shutdown test to E2E CI tests by @sgrebnov in #5047
  • Update duckdb_append_with_pk_and_indexes.yaml (work for duckdb 1.1.x) by @sgrebnov in #5067
  • fix: Downgrade to DuckDB 1.1.3 by @peasee in #5055
  • fix: Acceleration federation integration test by @peasee in #5070
  • Improvements to Iceberg Catalog/Data Connector by @phillipleblanc in #5071
  • Add Results-Cache-Status to indicate query result came from cache by @phillipleblanc in #4809
  • fix: Spice.ai schema inference by @peasee in #4674
  • Add refresh_on_startup Spicepod configuration param by @phillipleblanc and @sgrebnov in #5086
  • Test restart behavior of DuckDB file acceleration against glue iceberg table by @Sevenannn #5075
  • Run Iceberg Data Connector - DuckDB File mode integration test by @Sevenannn #5069
  • Integration test for glue iceberg catalog by @Sevenannn #5077

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.0.5...v1.0.6

Spice v1.0.5 (Mar 11, 2025)

ยท 3 min read
Sergei Grebnov
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.0.5 ๐ŸงŠ

Spice v1.0.5 expands Iceberg support with the introduction of the Iceberg Data Connector, in addition to the existing Iceberg Catalog Connector. This new connector enables direct dataset creation and configuration for specific Iceberg objects, enabling federated and accelerated SQL queries on Apache Iceberg tables.

Performance improvements include object-store optimized Parquet pruning in append mode, where object-store metadata is now leveraged alongside Hive partitioning to optimize file pruning. This results in faster and more efficient queries.

DuckDB has been upgraded to v1.2.0, along with additional stability improvements, including improved graceful shutdown and the ability to configure the DuckDB memory limit.

Additional updates include support for the Arrow Map type.

Highlights in v1.0.5โ€‹

  • New Iceberg Data Connector: Enables direct dataset creation and querying of Iceberg tables.

    Example usage in spicepod.yaml:

    datasets:
    - from: iceberg:https://iceberg-catalog-host.com/v1/namespaces/my_namespace/tables/my_table
    name: my_table
    params:
    # Same as Iceberg Catalog Connector
    acceleration:
    enabled: true

    For detailed setup instructions, authentication options, and configuration parameters, refer to the Iceberg Data Connector documentation.

  • Improved Parquet pruning in append mode: Uses object-store metadata for more efficient file pruning.

  • DuckDB upgrade to v1.2.0 with improved graceful shutdown: Read the DuckDB v1.2.0 announcement for details, including breaking changes for map and list_reduce. Graceful shutdown of DuckDB has been improved for better stability across restarts.

  • Configurable DuckDB memory limit: Use the duckdb_memory_limit parameter to set the DuckDB acceleration memory limit:

    - from: spice.ai:path.to.my_dataset
    name: my_dataset
    acceleration:
    params:
    duckdb_memory_limit: '2GB'
    enabled: true
    engine: duckdb
    mode: file

Contributorsโ€‹

  • @peasee
  • @phillipleblanc
  • @sgrebnov
  • @lukekim

Breaking Changesโ€‹

Upgradingโ€‹

To upgrade to v1.0.5, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.5 image:

docker pull spiceai/spiceai:1.0.5

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changedโ€‹

Dependenciesโ€‹

Changelogโ€‹

  • fix: Update OpenAI model health check by @peasee in #4849
  • fix: Allow metrics endpoint setting in CLI by @peasee in #4939
  • DuckDB acceleration: fix Decimal with zero scale support by @sgrebnov in #4922
  • Introduce runtime shutdown state by @sgrebnov in #4917
  • Add support for Flight and HTTP endpoints configuration to Spice CLI (run and sql) by @sgrebnov and @lukekim in #4913
  • Fix Datafusion resources deallocation during shutdown by @sgrebnov in #4912
  • DuckDB: fix error handling during record batch insertion by @sgrebnov in #4894
  • DuckDB: add support for Map Arrow type for DuckDB acceleration by @sgrebnov in #4887
  • Upgrade to DuckDB v1.2.0 by @sgrebnov in #4842
  • Gracefully shutdown the runtime and deallocate static resources by @sgrebnov in #4879
  • Implement an Iceberg Data Connector by @phillipleblanc in #4941
  • Don't trace canceled dataset refresh during runtime termination by @sgrebnov in #4958
  • Use metadata column last_modified when specified as a time_column by @phillipleblanc in #4970
  • Add duckdb_memory_limit param support for DuckDB acceleration by @sgrebnov in #4971
  • Add Iceberg dataset integration test by @phillipleblanc in #4950

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.0.4...v1.0.5

Spice v1.0.2 (Feb 4, 2025)

ยท 5 min read
Sergei Grebnov
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.0.2 ๐ŸŽ“

Spice v1.0.2 adds support for running local filesystem-hosted DeepSeek models including R1 (cloud-hosted via DeepSeek API was already supported) and improves the developer experience for debugging AI chat tasks along with several bug fixes. The HuggingFace and Filesystem-Hosted models providers have both graduated to Release Candidates (RC) and the Spice.ai Cloud Platform catalog provider has graduated to Beta.

Highlights in v1.0.2โ€‹

  • spice trace New spice trace CLI command that outputs a detailed breakdown of traces and tasks, including tool usage and AI completions.

    Examples:

    trace> spice trace ai_chat
    [61cc6bd0e571c783] ( 2593.77ms) ai_chat
    โ”œโ”€โ”€ [69362c30f238076f] ( 0.36ms) tool_use::get_readiness
    โ”œโ”€โ”€ [b6b17f1a9a6b86dc] ( 982.21ms) ai_completion
    โ”œโ”€โ”€ [c30d692c6c41c5ee] ( 0.06ms) tool_use::list_datasets
    โ””โ”€โ”€ [ce18756d5fef0df0] ( 1605.12ms) ai_completion

    trace> spice trace ai_chat --trace-id 61cc6bd0e571c783

    trace> spice trace ai_chat --id chatcmpl-AvXwmPSV1PMyGBi9dLfkEQTZPjhqz

    The spice trace CLI simply outputs data available in the runtime.task_history table which can also be queried by SQL.

    To learn more, see:

  • Filesystem-Hosted Models Provider: Graduated to Release Candidate (RC). To learn more, see the Filesystem-Hosted Models Provider Documentation.

  • HuggingFace Models Provider: Graduated to Release Candidate (RC). To learn more, see the HuggingFace Models Provider Documentation.

  • Spice.ai Cloud Platform Catalog: Graduated to Beta.

Contributorsโ€‹

  • @phillipleblanc
  • @johnnynunez
  • @Sevenannn
  • @sgrebnov
  • @peasee
  • @Jeadie
  • @lukekim

New Contributorsโ€‹

Breaking Changesโ€‹

No breaking changes.

Cookbook Updatesโ€‹

Upgradingโ€‹

To upgrade to v1.0.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.2 image:

docker pull spiceai/spiceai:1.0.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changedโ€‹

Dependenciesโ€‹

No major dependency changes.

Changelogโ€‹

- Update release branch naming by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4539
- ready for arm buildings by @johnnynunez in https://github.com/spiceai/spiceai/pull/4502
- Bump helm chart version to 1.0.1 by @Sevenannn in https://github.com/spiceai/spiceai/pull/4542
- Include 1.0.1 as supported version in security.md by @Sevenannn in https://github.com/spiceai/spiceai/pull/4545
- Update CI to build on hosted windows runners by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4540
- docs: Update Windows install by @peasee in https://github.com/spiceai/spiceai/pull/4551
- Fix spark spicepod for test operator by @Sevenannn in https://github.com/spiceai/spiceai/pull/4555
- Improve hugging face model chat error by @Sevenannn in https://github.com/spiceai/spiceai/pull/4554
- fix: Update Windows E2E install by @peasee in https://github.com/spiceai/spiceai/pull/4557
- feat: Add Spice Cloud Catalog Spicepod, release Alpha by @peasee in https://github.com/spiceai/spiceai/pull/4561
- Fix huggingface embedding errors by @Sevenannn in https://github.com/spiceai/spiceai/pull/4558
- feat: Load table schemas through REST for Spice Cloud Catalog by @peasee in https://github.com/spiceai/spiceai/pull/4563
- Add upgrade instruction in release note by @Sevenannn in https://github.com/spiceai/spiceai/pull/4548
- Add federated source information to refresh errors by @sgrebnov in https://github.com/spiceai/spiceai/pull/4560
- docs: Update ROADMAP.md by @peasee in https://github.com/spiceai/spiceai/pull/4566
- Merge mistral upstream by @Jeadie in https://github.com/spiceai/spiceai/pull/4562
- Fix windows build by @Sevenannn in https://github.com/spiceai/spiceai/pull/4574
- feat: Update Spice Cloud Catalog errors, release as Beta by @peasee in https://github.com/spiceai/spiceai/pull/4575
- docs: Add TOC to README.md by @peasee in https://github.com/spiceai/spiceai/pull/4538
- Updates to spiceai/mistral.rs by @Jeadie in https://github.com/spiceai/spiceai/pull/4580
- Improve refresh error tracing by @sgrebnov in https://github.com/spiceai/spiceai/pull/4576
- Add HTTP consistency & overhead to testoperator dispatch tool by @Jeadie in https://github.com/spiceai/spiceai/pull/4556
- Fix append mode refresh with MySQL Data Connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/4583
- fix: Retry flaky tests by @peasee in https://github.com/spiceai/spiceai/pull/4577
- Fix E2E models test build on macOS runners by @sgrebnov in https://github.com/spiceai/spiceai/pull/4585
- spice trace chat support in CLI by @Jeadie in https://github.com/spiceai/spiceai/pull/4582
- Include hf test specs, enable ready_wait in workflow by @Sevenannn in https://github.com/spiceai/spiceai/pull/4584
- Add paths verification when loading models by @sgrebnov in https://github.com/spiceai/spiceai/pull/4591
- Add generation_config.json support for Filesystem models by @sgrebnov in https://github.com/spiceai/spiceai/pull/4592
- Promote Filesystem model provider to RC by @sgrebnov in https://github.com/spiceai/spiceai/pull/4593
- docs: Add models grading criteria by @peasee in https://github.com/spiceai/spiceai/pull/4550
- Fix typo in Alpha Release Criteria (models) by @sgrebnov in https://github.com/spiceai/spiceai/pull/4588
- fix: Retry AI integration tests by @peasee in https://github.com/spiceai/spiceai/pull/4595
- Run LLM integration tests on Macs; add running local models by @Jeadie in https://github.com/spiceai/spiceai/pull/4495
- Update version to 1.0.2 by @sgrebnov in https://github.com/spiceai/spiceai/pull/4594
- feat: Schedule testoperator by @peasee in https://github.com/spiceai/spiceai/pull/4503
- Improve UX of downloading GGUF from HF by @Jeadie in https://github.com/spiceai/spiceai/pull/4601
- Improve spice trace CLI command by @sgrebnov https://github.com/spiceai/spiceai/pull/4629
- Improve the UX of using huggingface models & embeddings by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4623
- GGUF, hide metadata by @Jeadie in https://github.com/spiceai/spiceai/pull/4631
- Promote hugging face to rc by @Sevenannn in https://github.com/spiceai/spiceai/pull/4626
- Endgame Issue template improvements by @lukekim in https://github.com/spiceai/spiceai/pull/4647
- feat: setup sccache for PR checks by @peasee in https://github.com/spiceai/spiceai/pull/4652
- Run build_and_release_cuda.yml when crates/llms/Cargo.toml changes by @Jeadie in https://github.com/spiceai/spiceai/pull/4648
- Update E2E installation tests to match model runtime version by @sgrebnov in https://github.com/spiceai/spiceai/pull/4653
- fix: Postgres LargeUtf8 is equal to Utf8 by @peasee in https://github.com/spiceai/spiceai/pull/4664
- Fix eager string formatting in mistral.rs by @Jeadie in https://github.com/spiceai/spiceai/pull/4665
- Better error for spicepod parsing by @Sevenannn in https://github.com/spiceai/spiceai/pull/4632
- Update datafusion-table-providers (MySQL improvements) by @sgrebnov in https://github.com/spiceai/spiceai/pull/4670
- Handle delta tables partitioned by a date column with large date values by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4672

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.1...v1.0.2

Resourcesโ€‹

Communityโ€‹

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

Spice v0.19.3-beta (Oct 28, 2024)

ยท 4 min read
Sergei Grebnov
Senior Software Engineer at Spice AI

Announcing the release of Spice v0.19.3-beta ๐Ÿ“ˆ

Spice v0.19.3-beta improves the performance and stability of data connectors and accelerators, including faster queries across multiple federated sources by optimizing how filters are applied. Anthropic has also been added as a LLM model provider.

Highlights in v0.19.3โ€‹

DataFusion Fixes: Resolved bugs in DataFusion and DataFusion Table Providers, expanding TPC-DS coverage and correctness.

GitHub Data Connector Beta Milestone: The GitHub Data Connector has graduated to Beta after extensive testing, stability, and performance improvements.

Anthropic Models Provider: Anthropic has been added as an LLM provider, including support for streaming.

Example spicepod.yml:

models:
- from: anthropic:claude-3-5-sonnet-20240620
name: claude_3_5_sonnet
params:
anthropic_api_key: ${ secrets:SPICE_ANTHROPIC_API_KEY }

Breaking changesโ€‹

None.

Contributorsโ€‹

  • @Jeadie
  • @Sevenannn
  • @phillipleblanc
  • @peasee
  • @sgrebnov
  • @nlamirault
  • @barracudarin
  • @lukekim
  • @slyons

New Contributorsโ€‹

What's Changedโ€‹

- Make Anthropic OpenAI compatible. by @Jeadie in https://github.com/spiceai/spiceai/pull/3087
- Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/3200
- Bump version to 1.0.0-rc.1 by @Sevenannn in https://github.com/spiceai/spiceai/pull/3202
- Fix clickhouse schema inference for non-default database by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3201
- Update endgame template by @Sevenannn in https://github.com/spiceai/spiceai/pull/3198
- Upgrade dependencies by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3197
- fix: dataset refresh defaults properties to None by @peasee in https://github.com/spiceai/spiceai/pull/3205
- Upgrade OTEL to v0.26 and make seconds based metrics reported precisely by @sgrebnov in https://github.com/spiceai/spiceai/pull/3203
- use `text_embedding_inference::Infer` for more complete embedding solution by @Jeadie in https://github.com/spiceai/spiceai/pull/3199
- Add S3 parquet file - arrow accelerator e2e test by @Sevenannn in https://github.com/spiceai/spiceai/pull/3154
- feat: Add script to setup clickbench on mysql by @peasee in https://github.com/spiceai/spiceai/pull/3176
- Update helm chart version to v0.19.2 by @Sevenannn in https://github.com/spiceai/spiceai/pull/3210
- Add sample dataset option in `v1/nsql`. by @Jeadie in https://github.com/spiceai/spiceai/pull/3105
- Split spiced_docker build across architectures by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3206
- feat(helm): do not install demo dataset by default by @nlamirault in https://github.com/spiceai/spiceai/pull/3207
- Split integration test across build/run steps by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3215
- feat(helm): Refactoring Kubernetes labels by @nlamirault in https://github.com/spiceai/spiceai/pull/3208
- Define 'tool_recursion_limit' for LLMs, and limit internal tool calling recursion. by @Jeadie in https://github.com/spiceai/spiceai/pull/3214
- Improve filters pushdown for federated queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/3183
- Implement native schema inference for PostgreSQL by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3209
- docs: Update release criteria by @peasee in https://github.com/spiceai/spiceai/pull/3219
- Run SQLite acceleration TPC-DS tests using smaller scale by @sgrebnov in https://github.com/spiceai/spiceai/pull/3227
- bind the serviceAccount if a name is given or if we're creating one by @barracudarin in https://github.com/spiceai/spiceai/pull/3228
- Only emit channel send error log when its not a closed channel error by @Jeadie in https://github.com/spiceai/spiceai/pull/3230
- Enable Parquet Exec filter pushdown in Spice by @Sevenannn in https://github.com/spiceai/spiceai/pull/3216
- Add snapshots for SQLite TPC-DS benchmark (file mode) by @sgrebnov in https://github.com/spiceai/spiceai/pull/3234
- docs: Add SDK release checks to endgame by @peasee in https://github.com/spiceai/spiceai/pull/3256
- Implement `localpod` Data Connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3249
- Revert "Enable Parquet Exec filter pushdown in Spice (#3216)" by @Sevenannn in https://github.com/spiceai/spiceai/pull/3244
- refactor: Use existing action for detecting changes by @peasee in https://github.com/spiceai/spiceai/pull/3255
- feat: Add GitHub integration test by @peasee in https://github.com/spiceai/spiceai/pull/3226
- Add get_readiness tool to retrieve status of all registered components by @lukekim in https://github.com/spiceai/spiceai/pull/3035
- Improve CLI error output when REPL can't connect to the Flight endpoint by @slyons in https://github.com/spiceai/spiceai/pull/3188
- Fixing FTP link in Endgame by @slyons in https://github.com/spiceai/spiceai/pull/3267
- Update version to 0.19.3-beta by @sgrebnov in https://github.com/spiceai/spiceai/pull/3269
- add service type and annotation customizations in https://github.com/spiceai/spiceai/pull/3268

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v0.19.2-beta...v0.19.3-beta

Resourcesโ€‹

Communityโ€‹

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.