Spice v1.0.5 (Mar 11, 2025)

March 11, 2025 · 4 min read

Member of Technical Staff at Spice AI

Announcing the release of Spice v1.0.5 🧊

Spice v1.0.5 expands Iceberg support with the introduction of the Iceberg Data Connector, in addition to the existing Iceberg Catalog Connector. This new connector enables direct dataset creation and configuration for specific Iceberg objects, enabling federated and accelerated SQL queries on Apache Iceberg tables.

Performance improvements include object-store optimized Parquet pruning in append mode, where object-store metadata is now leveraged alongside Hive partitioning to optimize file pruning. This results in faster and more efficient queries.

DuckDB has been upgraded to v1.2.0, along with additional stability improvements, including improved graceful shutdown and the ability to configure the DuckDB memory limit.

Additional updates include support for the Arrow Map type.

Highlights in v1.0.5

New Iceberg Data Connector: Enables direct dataset creation and querying of Iceberg tables.

Example usage in spicepod.yaml:
```
datasets:
  - from: iceberg:https://iceberg-catalog-host.com/v1/namespaces/my_namespace/tables/my_table
    name: my_table
    params:
      # Same as Iceberg Catalog Connector
    acceleration:
      enabled: true
```
For detailed setup instructions, authentication options, and configuration parameters, refer to the Iceberg Data Connector documentation.
Improved Parquet pruning in append mode: Uses object-store metadata for more efficient file pruning.
DuckDB upgrade to v1.2.0 with improved graceful shutdown: Read the DuckDB v1.2.0 announcement for details, including breaking changes for map and list_reduce. Graceful shutdown of DuckDB has been improved for better stability across restarts.

Configurable DuckDB memory limit: Use the duckdb_memory_limit parameter to set the DuckDB acceleration memory limit:

- from: spice.ai:path.to.my_dataset
   name: my_dataset
   acceleration:
     params:
       duckdb_memory_limit: '2GB'
     enabled: true
     engine: duckdb
     mode: file

Contributors

@peasee
@phillipleblanc
@sgrebnov
@lukekim

Breaking Changes

DuckDB v1.2.0 has breaking changes.

Upgrading

To upgrade to v1.0.5, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.5 image:

docker pull spiceai/spiceai:1.0.5

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed

Dependencies

duckdb-rs: Upgraded from 1.1.1 to 1.2.0

Changelog

fix: Update OpenAI model health check by @peasee in #4849
fix: Allow metrics endpoint setting in CLI by @peasee in #4939
DuckDB acceleration: fix Decimal with zero scale support by @sgrebnov in #4922
Introduce runtime shutdown state by @sgrebnov in #4917
Add support for Flight and HTTP endpoints configuration to Spice CLI (run and sql) by @sgrebnov and @lukekim in #4913
Fix Datafusion resources deallocation during shutdown by @sgrebnov in #4912
DuckDB: fix error handling during record batch insertion by @sgrebnov in #4894
DuckDB: add support for Map Arrow type for DuckDB acceleration by @sgrebnov in #4887
Upgrade to DuckDB v1.2.0 by @sgrebnov in #4842
Gracefully shutdown the runtime and deallocate static resources by @sgrebnov in #4879
Implement an Iceberg Data Connector by @phillipleblanc in #4941
Don't trace canceled dataset refresh during runtime termination by @sgrebnov in #4958
Use metadata column last_modified when specified as a time_column by @phillipleblanc in #4970
Add duckdb_memory_limit param support for DuckDB acceleration by @sgrebnov in #4971
Add Iceberg dataset integration test by @phillipleblanc in #4950

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.0.4...v1.0.5

Spice v1.0.4 (Feb 17, 2025)

February 17, 2025 · 3 min read

Jack Eadie

Member of Technical Staff at Spice AI

Announcing the release of Spice v1.0.4 🏎️

Spice v1.0.4 improves partition pruning for Delta Lake tables, significantly increasing scan efficiency and reducing overhead. xAI tool calling is more robust and the spice trace CLI command now provides expanded, detailed output for deeper analysis. Additionally, a bug has been fixed to correctly apply column name case-sensitivity in refresh SQL, indexes, and primary keys.

Highlights in v1.0.4

Improved Append-Based Refresh When using an append-based acceleration where the time_column format differs from the physical partition, two new dataset configuration options, time_partition_column and time_partition_format can be configured to improve partition pruning and exclude irrelevant partitions during the refreshes.

For example, when the time_column format is timestamp and the physical data partition is date such as below:

my_delta_table/
├── _delta_log/
├── date_col=2023-12-31/
├── date_col=2024-02-04/
├── date_col=2025-01-01/
└── date_col=2030-06-15/

Partition pruning can be optimized using the configuration:

datasets:
  - from: delta_lake://my_delta_table
    name: my_delta_table
    time_column: created_at # A fine-grained timestamp
    time_format: timestamp
    time_partition_column: date_col # Data is physically partitioned by `date_col`
    time_partition_format: date
sgrebnov marked this conversation as resolved.

Expanded spice trace output: The spice trace CLI command now includes additional details, such as task status, and optional flags --include-input and --include-output for detailed tracing.

Example spice trace output:

TREE                   STATUS DURATION   TASK
a97f52ccd7687e64       ✅       673.14ms ai_chat
  ├── 4eebde7b04321803 ✅         0.04ms tool_use::list_datasets
  └── 4c9049e1bf1c3500 ✅       671.91ms ai_completion

Example spice trace --include-input --include-output output:

TREE                   STATUS DURATION   TASK                    OUTPUT
a97f52ccd7687e64       ✅       673.14ms ai_chat                 The capital of New York is Albany.
  ├── 4eebde7b04321803 ✅         0.04ms tool_use::list_datasets []
  └── 4c9049e1bf1c3500 ✅       671.91ms ai_completion           [{"content":"The capital of New York is Albany.","refusal":null,"tool_calls":null,"role":"assistant","function_call":null,"audio":null}]

Contributors

@Jeadie
@peasee
@phillipleblanc
@Sevenannn
@sgrebnov
@lukekim

Breaking Changes

No breaking changes.

Cookbook Updates

No new recipes.

Upgrading

To upgrade to v1.0.4, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.4 image:

docker pull spiceai/spiceai:1.0.4

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed

Dependencies

No major dependency changes.

Changelog

- Do not return underlying content of chunked embedding column by default during tool_use::document_similarity by @Jeadie in https://github.com/spiceai/spiceai/pull/4802
- Fix Snowflake Case-Sensitive Identifiers support by @sgrebnov in https://github.com/spiceai/spiceai/pull/4813
- Prepare for 1.0.4 by @sgrebnov in https://github.com/spiceai/spiceai/pull/4801
- Add support for a time_partition_column by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4784
- Prevent the automatic normalization of refresh_sql columns to lowercase by @sgrebnov in https://github.com/spiceai/spiceai/pull/4787
- Implement partition pruning for Delta Lake tables by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4783
- Fix constraint verification for columns with uppercase letters by @sgrebnov in https://github.com/spiceai/spiceai/pull/4785
- Add truncate command for spice trace by @peasee in https://github.com/spiceai/spiceai/pull/4771
- Implement Cache-Control: no-cache to bypass results cache by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4763
- Prompt user to download runtime when running spice sql by @Sevenannn in https://github.com/spiceai/spiceai/pull/4747
- Add vector search tracing by @peasee in https://github.com/spiceai/spiceai/pull/4757
- Update spice trace output format by @Jeadie in https://github.com/spiceai/spiceai/pull/4750
- Fix tool call arguments in Grok messages by @Jeadie in https://github.com/spiceai/spiceai/pull/4741

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.3...v1.0.4

Spice v1.0.3 (Feb 10, 2025)

February 10, 2025 · 3 min read

Phillip LeBlanc

Co-Founder and CTO of Spice AI

Announcing the release of Spice v1.0.3 🛠️

Spice v1.0.3 provides several bug fixes, including a fix for the initial data load period when a retention policy has been set, and a new unsupported_type_action: string parameter to auto-convert unsupported types to strings.

Highlights in v1.0.3

PostgreSQL Data Connector: New unsupported_type_action: string parameter that auto-converts unsupported types such as JSONB to strings.

Contributors

@phillipleblanc
@Sevenannn
@sgrebnov
@peasee
@Jeadie
@lukekim

Breaking Changes

No breaking changes.

Cookbook Updates

Updated Kubernetes Deployment Recipe
Updated Data Retention Recipe

Upgrading

To upgrade to v1.0.3, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.3 image:

docker pull spiceai/spiceai:1.0.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed

Dependencies

No major dependency changes.

Changelog

- For local models, use 'content=""' instead of None by @Jeadie and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4646
- Perplexity Sonar LLM component by @Jeadie and @lukekim in https://github.com/spiceai/spiceai/pull/4673
- Update async openai fork & support reasoning effort parameter by @Sevenannn and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4679
- Web search tool by @Jeadie and @lukekim in https://github.com/spiceai/spiceai/pull/4687
- Setup tpc-extension by @ewgenius and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4690
- fix: Use PostgreSQL interval style for Spice.ai by @peasee and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4716
- Fix spice upgrade command by @Sevenannn and @sgrebnov in https://github.com/spiceai/spiceai/pull/4699
- Fix bug: Ensure refresh only retrieves data within the retention period by @sgrebnov and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4717
- Implement unsupported_type_action: string for Postgres JSONB support by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4719
- Fix the get latest release logic by @Sevenannn and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4721
- add 'accelerated_refresh' to 'spice trace' allowlist by @Jeadie and @phillipleblanc in https://github.com/spiceai/spiceai/pull/4711
- Update version to 1.0.3 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4731
- Truncate embedding columns within sampling tool by @Jeadie in https://github.com/spiceai/spiceai/pull/4722
- Validate primary key columns during accelerated dataset initialization by @sgrebnov in https://github.com/spiceai/spiceai/pull/4736

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.2...v1.0.3

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]

Spice v1.0.2 (Feb 4, 2025)

February 4, 2025 · 7 min read

Sergei Grebnov

Member of Technical Staff at Spice AI

Announcing the release of Spice v1.0.2 🎓

Spice v1.0.2 adds support for running local filesystem-hosted DeepSeek models including R1 (cloud-hosted via DeepSeek API was already supported) and improves the developer experience for debugging AI chat tasks along with several bug fixes. The HuggingFace and Filesystem-Hosted models providers have both graduated to Release Candidates (RC) and the Spice.ai Cloud Platform catalog provider has graduated to Beta.

Highlights in v1.0.2

spice trace New spice trace CLI command that outputs a detailed breakdown of traces and tasks, including tool usage and AI completions.

Examples:

trace> spice trace ai_chat
[61cc6bd0e571c783] ( 2593.77ms) ai_chat
  ├── [69362c30f238076f] (    0.36ms) tool_use::get_readiness
  ├── [b6b17f1a9a6b86dc] (  982.21ms) ai_completion
  ├── [c30d692c6c41c5ee] (    0.06ms) tool_use::list_datasets
  └── [ce18756d5fef0df0] ( 1605.12ms) ai_completion

trace> spice trace ai_chat --trace-id 61cc6bd0e571c783

trace> spice trace ai_chat --id chatcmpl-AvXwmPSV1PMyGBi9dLfkEQTZPjhqz

The spice trace CLI simply outputs data available in the runtime.task_history table which can also be queried by SQL.

To learn more, see:

Filesystem-Hosted Models Provider: Graduated to Release Candidate (RC). To learn more, see the Filesystem-Hosted Models Provider Documentation.
HuggingFace Models Provider: Graduated to Release Candidate (RC). To learn more, see the HuggingFace Models Provider Documentation.
Spice.ai Cloud Platform Catalog: Graduated to Beta.

Contributors

@phillipleblanc
@johnnynunez
@Sevenannn
@sgrebnov
@peasee
@Jeadie
@lukekim

New Contributors

@johnnynunez made their first contribution in github.com/spiceai/spiceai/pull/4502

Breaking Changes

No breaking changes.

Cookbook Updates

Added Filesystem-Hosted Model Provider Recipe

Upgrading

To upgrade to v1.0.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.2 image:

docker pull spiceai/spiceai:1.0.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

What's Changed

Dependencies

No major dependency changes.

Changelog

- Update release branch naming by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4539
- ready for arm buildings by @johnnynunez in https://github.com/spiceai/spiceai/pull/4502
- Bump helm chart version to 1.0.1 by @Sevenannn in https://github.com/spiceai/spiceai/pull/4542
- Include 1.0.1 as supported version in security.md by @Sevenannn in https://github.com/spiceai/spiceai/pull/4545
- Update CI to build on hosted windows runners by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4540
- docs: Update Windows install by @peasee in https://github.com/spiceai/spiceai/pull/4551
- Fix spark spicepod for test operator by @Sevenannn in https://github.com/spiceai/spiceai/pull/4555
- Improve hugging face model chat error by @Sevenannn in https://github.com/spiceai/spiceai/pull/4554
- fix: Update Windows E2E install by @peasee in https://github.com/spiceai/spiceai/pull/4557
- feat: Add Spice Cloud Catalog Spicepod, release Alpha by @peasee in https://github.com/spiceai/spiceai/pull/4561
- Fix huggingface embedding errors by @Sevenannn in https://github.com/spiceai/spiceai/pull/4558
- feat: Load table schemas through REST for Spice Cloud Catalog by @peasee in https://github.com/spiceai/spiceai/pull/4563
- Add upgrade instruction in release note by @Sevenannn in https://github.com/spiceai/spiceai/pull/4548
- Add federated source information to refresh errors by @sgrebnov in https://github.com/spiceai/spiceai/pull/4560
- docs: Update ROADMAP.md by @peasee in https://github.com/spiceai/spiceai/pull/4566
- Merge mistral upstream by @Jeadie in https://github.com/spiceai/spiceai/pull/4562
- Fix windows build by @Sevenannn in https://github.com/spiceai/spiceai/pull/4574
- feat: Update Spice Cloud Catalog errors, release as Beta by @peasee in https://github.com/spiceai/spiceai/pull/4575
- docs: Add TOC to README.md by @peasee in https://github.com/spiceai/spiceai/pull/4538
- Updates to spiceai/mistral.rs by @Jeadie in https://github.com/spiceai/spiceai/pull/4580
- Improve refresh error tracing by @sgrebnov in https://github.com/spiceai/spiceai/pull/4576
- Add HTTP consistency & overhead to testoperator dispatch tool by @Jeadie in https://github.com/spiceai/spiceai/pull/4556
- Fix append mode refresh with MySQL Data Connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/4583
- fix: Retry flaky tests by @peasee in https://github.com/spiceai/spiceai/pull/4577
- Fix E2E models test build on macOS runners by @sgrebnov in https://github.com/spiceai/spiceai/pull/4585
- spice trace chat support in CLI by @Jeadie in https://github.com/spiceai/spiceai/pull/4582
- Include hf test specs, enable ready_wait in workflow by @Sevenannn in https://github.com/spiceai/spiceai/pull/4584
- Add paths verification when loading models by @sgrebnov in https://github.com/spiceai/spiceai/pull/4591
- Add generation_config.json support for Filesystem models by @sgrebnov in https://github.com/spiceai/spiceai/pull/4592
- Promote Filesystem model provider to RC by @sgrebnov in https://github.com/spiceai/spiceai/pull/4593
- docs: Add models grading criteria by @peasee in https://github.com/spiceai/spiceai/pull/4550
- Fix typo in Alpha Release Criteria (models) by @sgrebnov in https://github.com/spiceai/spiceai/pull/4588
- fix: Retry AI integration tests by @peasee in https://github.com/spiceai/spiceai/pull/4595
- Run LLM integration tests on Macs; add running local models by @Jeadie in https://github.com/spiceai/spiceai/pull/4495
- Update version to 1.0.2 by @sgrebnov in https://github.com/spiceai/spiceai/pull/4594
- feat: Schedule testoperator by @peasee in https://github.com/spiceai/spiceai/pull/4503
- Improve UX of downloading GGUF from HF by @Jeadie in https://github.com/spiceai/spiceai/pull/4601
- Improve spice trace CLI command by @sgrebnov https://github.com/spiceai/spiceai/pull/4629
- Improve the UX of using huggingface models & embeddings by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4623
- GGUF, hide metadata by @Jeadie in https://github.com/spiceai/spiceai/pull/4631
- Promote hugging face to rc by @Sevenannn in https://github.com/spiceai/spiceai/pull/4626
- Endgame Issue template improvements by @lukekim in https://github.com/spiceai/spiceai/pull/4647
- feat: setup sccache for PR checks by @peasee in https://github.com/spiceai/spiceai/pull/4652
- Run build_and_release_cuda.yml when crates/llms/Cargo.toml changes by @Jeadie in https://github.com/spiceai/spiceai/pull/4648
- Update E2E installation tests to match model runtime version by @sgrebnov in https://github.com/spiceai/spiceai/pull/4653
- fix: Postgres LargeUtf8 is equal to Utf8 by @peasee in https://github.com/spiceai/spiceai/pull/4664
- Fix eager string formatting in mistral.rs by @Jeadie in https://github.com/spiceai/spiceai/pull/4665
- Better error for spicepod parsing by @Sevenannn in https://github.com/spiceai/spiceai/pull/4632
- Update datafusion-table-providers (MySQL improvements) by @sgrebnov in https://github.com/spiceai/spiceai/pull/4670
- Handle delta tables partitioned by a date column with large date values by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4672

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.1...v1.0.2

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]

Spice v1.0.1 (Jan 27, 2025)

January 27, 2025 · 5 min read

Qianqian Liu

Software Engineer at Spice AI

Spice v1.0.1 focuses on an improved developer experience, with automatic CUDA GPU detection for local models, in addition to bug fixes. Notably, the Iceberg Catalog Connector now supports AWS Glue including Sig v4 authentication.

Highlights in v1.0.1

AWS Glue Support for Iceberg Catalog Connector: The Iceberg Catalog Connector now supports AWS Glue. Example spicepod.yaml configuration:

- from: iceberg:https://glue.ap-northeast-2.amazonaws.com/iceberg/v1/catalogs/123456789012/namespaces
  name: glue

spice upgrade CLI Command: The spice upgrade CLI command detects more edge cases for a smoother upgrade experience.
GPU Acceleration Detection: The Spice CLI now automatically detects and enables CUDA (NVIDIA GPUs) GPU acceleration when supported in addition to Metal (M-Series on macOS).
Python SDK: The Python SDK (spicepy) has updated to v3.0.0, aligning the SDK with the Runtime

Breaking changes

No breaking changes.

Dependencies

No major dependency changes.

Cookbook

Added DeepSeek Model Recipe
Added OpenAI LLM & Embeddings Recipe

Upgrading

To upgrade to v1.0.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.1 image:

docker pull spiceai/spiceai:1.0.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

Contributors

@Jeadie
@phillipleblanc
@ewgenius
@peasee
@Sevenannn
@sgrebnov
@lukekim

What's Changed

- Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/4459
- docs: 1.0 release notes by @peasee in https://github.com/spiceai/spiceai/pull/4440
- Create a release-only workflow that uses a previous run's artifacts by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4461
- Add publish-only CUDA workflow by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4462
- Fix the CUDA release workflow by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4463
- docs: Update SECURITY.md for stable by @peasee in https://github.com/spiceai/spiceai/pull/4465
- docs: Update endgame by @peasee in https://github.com/spiceai/spiceai/pull/4460
- docs: Promote HF and File model components by @peasee in https://github.com/spiceai/spiceai/pull/4457
- fix: E2E test release installation by @peasee in https://github.com/spiceai/spiceai/pull/4466
- Fix publish part of CUDA workflow by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4467
- Fix broken docs links in README by @ewgenius in https://github.com/spiceai/spiceai/pull/4468
- Update benchmark snapshots by @github-actions in https://github.com/spiceai/spiceai/pull/4474
- Update openapi.json by @github-actions in https://github.com/spiceai/spiceai/pull/4477
- Add instruction to force-install CPU runtime to v1.0 release notes by @sgrebnov in https://github.com/spiceai/spiceai/pull/4469
- feat: Add WIP testoperator dispatch workflow by @peasee in https://github.com/spiceai/spiceai/pull/4478
- Fix Bug: invalid REPL cursor position on Windows by @sgrebnov in https://github.com/spiceai/spiceai/pull/4480
- feat: Download latest spiced commit for testoperators by @peasee in https://github.com/spiceai/spiceai/pull/4483
- Add compute engine image by @lukekim in https://github.com/spiceai/spiceai/pull/4486
- fix: Testoperator git fetch depth by @peasee in https://github.com/spiceai/spiceai/pull/4484
- feat: New spicepods, testoperator improvements, TPCDS Q1 fix by @peasee in https://github.com/spiceai/spiceai/pull/4475
- Add 87 CUDA compatiblity to build CI by @Jeadie in https://github.com/spiceai/spiceai/pull/4489
- Use OpenAI golang client in `spice chat` by @Jeadie in https://github.com/spiceai/spiceai/pull/4491
- Verify `search` and `chat` on Windows as part of AI installation tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/4492
- feat: Add testoperator dispatch command by @peasee in https://github.com/spiceai/spiceai/pull/4479
- Run CUDA builds on non-GPU instances by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4496
- Use upgraded spice cli when performing runtime upgrade in spice upgrade by @Sevenannn in https://github.com/spiceai/spiceai/pull/4490
- Revert "Use OpenAI golang client in `spice chat` (#4491)" by @Jeadie in https://github.com/spiceai/spiceai/pull/4532
- Make Anthropic rate limit error message friendlier by @sgrebnov in https://github.com/spiceai/spiceai/pull/4501
- Update supported CUDA targets: add 87(cli), remove 75 by @sgrebnov in https://github.com/spiceai/spiceai/pull/4509
- Support AWS Glue for Iceberg catalog connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4517
- Package CUDA runtime libraries into artifact for Windows by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4497

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.0...v1.0.1

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]

Spice v1.0-stable (Jan 20, 2025)

January 20, 2025 · 11 min read

William Croxson

Member of Technical Staff at Spice AI

🎉 After 47 releases, Spice.ai OSS has reached production readiness with the 1.0-stable milestone!

The core runtime and features such as query federation, query acceleration, catalog integration, search and AI-inference have all graduated to stable status along with key component graduations across data connectors, data accelerators, catalog connectors, and AI model providers.

Highlights in v1.0-stable

Stable Data Connectors: The following data connectors have graduated to Stable:
- Delta Lake
- MySQL
- Dremio
- PostgreSQL
- Databricks (mode: delta_lake)
- DuckDB
- S3
Stable Data Accelerators: The following data accelerators have graduated to Stable:
- DuckDB
- Arrow
Unity Catalog Connector: Graduated to Stable.
Databricks (mode: spark_connect) Data Connector: Graduated to Beta.
Beta Catalog Connectors: The Iceberg and Databricks catalog connectors graduated to Beta.
OpenAI Model & Embeddings Provider: Graduated to Release Candidate (RC).
Alpha Model Providers: The Anthropic and xAI (Grok) model providers graduated to Alpha.

Breaking Changes

Default Runtime Version: The CLI will install the GPU accelerated AI-capable Runtime by default (if supported), when running spice install or spice run. To force-install the non-GPU version, run spice install ai --cpu.
Default OpenAI Model: The default OpenAI model has updated to gpt-4o-mini.
Identifier Normalization: Unquoted identifiers such as table names are no longer normalized to lowercase. Identifiers will now retain their exact case as provided.
Sandboxed Docker Image: The Runtime Docker Image now runs the spiced process as the nobody user in a minimal chroot sandbox.
Insecure S3 and ABFS endpoints: The S3 and ABFS connectors now enforce insecure endpoint checks, preventing HTTP endpoints unless allow_http is explicitly enabled. Refer to the documentation for details.

Dependencies

No major dependency changes.

Upgrading

To upgrade to v1.0.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.0.0 image:

docker pull spiceai/spiceai:1.0.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

Contributors

@peasee
@ewgenius
@Jeadie
@Sevenannn
@lukekim
@phillipleblanc
@sgrebnov

What's Changed

- feat: Update load test criteria, testoperator updates by @peasee in <https://github.com/spiceai/spiceai/pull/4311>
- Update helm for v1.0.0-rc.5 by @ewgenius in <https://github.com/spiceai/spiceai/pull/4313>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4318>
- Bump version to v1.0.0, update SECURITY.md by @ewgenius in <https://github.com/spiceai/spiceai/pull/4314>
- Initial criteria for models, embeddings by @Jeadie in <https://github.com/spiceai/spiceai/pull/4223>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/4321>
- Add dremio param for running load test by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4315>
- Promote Databricks (mode: delta_lake) connector to stable by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4328>
- Handle failed query in load test by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4327>
- feat: Use load test hours for baseline query sets by @peasee in <https://github.com/spiceai/spiceai/pull/4334>
- Fix typo in 1.0.0-rc.5 release notes by @ewgenius in <https://github.com/spiceai/spiceai/pull/4329>
- feat: add testoperator data consistency by @peasee in <https://github.com/spiceai/spiceai/pull/4319>
- docs: Release DuckDB connector stable by @peasee in <https://github.com/spiceai/spiceai/pull/4335>
- Fix DocumentDB -> DynamoDB by @lukekim in <https://github.com/spiceai/spiceai/pull/4339>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/4337>
- fix: Download hits.parquet from MinIO for benchmark by @peasee in <https://github.com/spiceai/spiceai/pull/4338>
- Update openapi.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4341>
- Remove evil averages by @lukekim in <https://github.com/spiceai/spiceai/pull/4343>
- Don't run builds on non-code changes by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4344>
- Remove streaming requirement from Databricks spark Beta and Spark connector Beta by @ewgenius in <https://github.com/spiceai/spiceai/pull/4345>
- Update s3 tpcds spicepods by @ewgenius in <https://github.com/spiceai/spiceai/pull/4346>
- Explicitly set required scale factor for throughput and load tests by @ewgenius in <https://github.com/spiceai/spiceai/pull/4347>
- Fix s3 tpcds dataset name by @ewgenius in <https://github.com/spiceai/spiceai/pull/4348>
- Promote Iceberg Catalog Connector to Beta by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4350>
- Update s3 clickbench benchmark snapshots by @ewgenius in <https://github.com/spiceai/spiceai/pull/4351>
- fix: DuckDB clickbench on zero results by @peasee in <https://github.com/spiceai/spiceai/pull/4349>
- Add integration test with snapshots for databricks catalog connector by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4353>
- refactor: Remove on zero results from benchmarks, add data consistency workflow by @peasee in <https://github.com/spiceai/spiceai/pull/4354>
- Fix Bug: No field named body_embedding when do vector search with refresh sql containing subset of columns by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4297>
- docs: Update roadmap by @peasee in <https://github.com/spiceai/spiceai/pull/4364>
- feat: Release accelerators stable by @peasee in <https://github.com/spiceai/spiceai/pull/4361>
- Add TPCH/TPCDS test spicepods for MySQL by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4365>
- Catch when an insecure (http) S3 and ABFS data connectors endpoint is used without specifying the `allow_http` parameter by @ewgenius in <https://github.com/spiceai/spiceai/pull/4363>
- Update ROADMAP - Iceberg catalog alpha for v1.0 by @ewgenius in <https://github.com/spiceai/spiceai/pull/4367>
- Promote databricks catalog and databricks (spark_connect) connector to beta by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4369>
- Update Roadmap - Iceberg beta by @ewgenius in <https://github.com/spiceai/spiceai/pull/4373>
- Build CUDA binaries for Linux by @Jeadie in <https://github.com/spiceai/spiceai/pull/4320>
- Promote Nvidia NIM as Alpha by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4380>
- Promote xai to alpha by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4381>
- Update stable criteria for object store based connectors by @ewgenius in <https://github.com/spiceai/spiceai/pull/4383>
- Testoperator: http consistency and overhead tests, fixes and ci by @ewgenius in <https://github.com/spiceai/spiceai/pull/4382>
- Promote S3 Data Connector to Stable by @ewgenius in <https://github.com/spiceai/spiceai/pull/4385>
- Download platform-supported CUDA binary version on Linux by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4356>
- Fix http consistency test workflow, add overhead workflow by @ewgenius in <https://github.com/spiceai/spiceai/pull/4387>
- feat: Add Postgres test spicepods by @peasee in <https://github.com/spiceai/spiceai/pull/4388>
- Fix typos + specific in model criteria; Make explicit alpha/beta tests for LLMS in `crates/llms/tests`.  by @Jeadie in <https://github.com/spiceai/spiceai/pull/4377>
- Fix federation bug for correlated subqueries of deeply nested Dremio tables by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4389>
- Fix http overhead workflow by @ewgenius in <https://github.com/spiceai/spiceai/pull/4390>
- Tweak model tests, fix embedding input by @ewgenius in <https://github.com/spiceai/spiceai/pull/4391>
- Promote Dremio to Stable quality by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4392>
- Add beta functionality tests for embedding models. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4352>
- docs: Release postgres connector stable by @peasee in <https://github.com/spiceai/spiceai/pull/4398>
- Increase timeout for model response in E2E tests by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4399>
- Disable ident normalization (i.e. `SELECT MyColumn from table` works) by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4400>
- Preserve schema metadata by @ewgenius in <https://github.com/spiceai/spiceai/pull/4402>
- Make models integration tests tracing less verbose by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4403>
- Fix `cuda` feature build on Windows by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4404>
- Promote MySQL to Stable by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4406>
- docs: Release Delta Lake and Unity catalog by @peasee in <https://github.com/spiceai/spiceai/pull/4405>
- Use `gpt-4o-mini` as a default model for openai provider by @ewgenius in <https://github.com/spiceai/spiceai/pull/4410>
- Fix streaming for Openai and Anthropic by @Jeadie in <https://github.com/spiceai/spiceai/pull/4409>
- Tweak model loading and missing tool errors messages by @ewgenius in <https://github.com/spiceai/spiceai/pull/4412>
- Spice CLI: fallback to CPU build for unsupported GPU Compute Capability by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4407>
- Build Windows CUDA binaries as part of `build_and_release` workflow by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4386>
- Update docs link by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4416>
- feat: Add CPU models install escape hatch by @peasee in <https://github.com/spiceai/spiceai/pull/4419>
- Handle OpenAI API Errors by @ewgenius in <https://github.com/spiceai/spiceai/pull/4417>
- Update spice cli to use `GH_TOKEN` or `GITHUB_TOKEN` env variables when calling releases api by @ewgenius in <https://github.com/spiceai/spiceai/pull/4175>
- Implement secure sandboxing for Docker image by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4411>
- Automatically install supported CUDA binary on Windows by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4420>
- Metrics for LLMs+ embeddings by @Jeadie in <https://github.com/spiceai/spiceai/pull/4418>
- Jeadie/25 01 17/beta perf by @Jeadie in <https://github.com/spiceai/spiceai/pull/4397>
- Pass GitHub token to all CI steps calling spice run by @ewgenius in <https://github.com/spiceai/spiceai/pull/4423>
- Run the models integration tests on PRs by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4421>
- Run CUDA builds in a separate workflow by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4430>
- Promote OpenAI models and embeddings providers to RC by @ewgenius in <https://github.com/spiceai/spiceai/pull/4432>
- Update link to retrieval-augmented generation (RAG) details by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4433>
- Unity catalog should strip parameter prefix before passing parameters to delta lake factory by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4436>
- Update quickstart traces to match current version by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4435>
- Update Supported Embeddings Providers Readme section by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4434>
- Local models can stream tools by @Jeadie in <https://github.com/spiceai/spiceai/pull/4429>
- fix: Use MetricsCollector::show() for HTTP testoperator commands by @peasee in <https://github.com/spiceai/spiceai/pull/4442>
- Fix run query action by @ewgenius in <https://github.com/spiceai/spiceai/pull/4444>
- Default to AI-enabled runtime for `spice run`/`spice install` by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4443>
- Change no spicepod.yaml log to warning by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4447>
- refactor: Update Catalog Connector error messages by @peasee in <https://github.com/spiceai/spiceai/pull/4441>
- Fix panic when converting OTel metrics by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4449>
- refactor: Update model errors by @peasee in <https://github.com/spiceai/spiceai/pull/4446>
- Update spiceai/mistral.rs to silence metadata logs by @ewgenius in <https://github.com/spiceai/spiceai/pull/4452>
- fix xAI; don't use openai defaults by @Jeadie in <https://github.com/spiceai/spiceai/pull/4450>
- Improves the UX of using huggingface models by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4451>
- Add GH Workflow to test `spice ai` runtime installation by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4448>
- fix: Use specific model errors where available by @peasee in <https://github.com/spiceai/spiceai/pull/4454>
- Detect and report unsupported embedding column type during dataset registration by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4456>
- Handle Errors by @Jeadie in <https://github.com/spiceai/spiceai/pull/4455>
- Catch and report negative openai_temperature error by @Sevenannn in <https://github.com/spiceai/spiceai/pull/4453>
- Clarify release check error message if it is caused by wrong GH token by @ewgenius in <https://github.com/spiceai/spiceai/pull/4458>

**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v1.0.0-rc.5...v1.0.0>

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]

Spice v1.0-rc.5 (Jan 13, 2025)

January 13, 2025 · 9 min read

Evgenii Khramkov

Member of Technical Staff at Spice AI

Announcing the release of Spice v1.0-rc.5 🛠️

Spice v1.0.0-rc.5 is the fifth release candidate for the first major version of Spice.ai OSS. This release focuses production readiness and critical bug fixes. In addition, a new DynamoDB data connector has been added along with automatic detection for GPU acceleration when running Spice using the CLI.

Highlights in v1.0-rc.5

Automatic GPU Acceleration Detection: Automatically detect and utilize GPU acceleration when running by CLI. Install AI components locally using the CLI command spice install ai. Currently supports NVIdia CUDA and Apple Metal (M-series).
DynamoDB Data Connector: Query AWS DynamoDB tables using SQL with the new DynamoDB Data Connector.

datasets:
  - from: dynamodb:users
    name: users
    params:
      dynamodb_aws_region: us-west-2
      dynamodb_aws_access_key_id: ${secrets:aws_access_key_id}
      dynamodb_aws_secret_access_key: ${secrets:aws_secret_access_key}
    acceleration:
      enabled: true

sql> describe users;
+----------------+-----------+-------------+
| column_name    | data_type | is_nullable |
+----------------+-----------+-------------+
| created_at     | Utf8      | YES         |
| date_of_birth  | Utf8      | YES         |
| email          | Utf8      | YES         |
| account_status | Utf8      | YES         |
| updated_at     | Utf8      | YES         |
| full_name      | Utf8      | YES         |
| ...                                      |
+----------------+-----------+-------------+

File Data Connector: Graduated to Stable.
Dremio Data Connector: Graduated to Release Candidate (RC).
Spice.ai, Spark, and Snowflake Data Connectors: Graduated to Beta.

Dependencies

No major dependency changes.

Contributors

@Jeadie
@phillipleblanc
@ewgenius
@peasee
@Sevenannn
@lukekim

What's Changed

* Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/4190
* Ensure non-nullity of primary keys in `MemTable`; check validity of initial data. by @Jeadie in https://github.com/spiceai/spiceai/pull/4158
* Bump version to v1.0.0 stable by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4191
* Fix metal + models download by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4193
* Update spice.ai connector beta roadmap by @ewgenius in https://github.com/spiceai/spiceai/pull/4194
* feat: verify on zero results snapshots by @peasee in https://github.com/spiceai/spiceai/pull/4195
* Add throughput test module to `test-framework` by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4196
* Update Spice.ai TPCH snapshots by @ewgenius in https://github.com/spiceai/spiceai/pull/4202
* Replace all usage of `lazy_static!` with `LazyLock` by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4199
* Fix model + metal download by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4200
* Run Clickbench for Dremio by @Sevenannn in https://github.com/spiceai/spiceai/pull/4138
* Update openapi.json by @github-actions in https://github.com/spiceai/spiceai/pull/4205
* Fix the typo in connector stable criteria by @Sevenannn in https://github.com/spiceai/spiceai/pull/4213
* feat: Add throughput test example by @peasee in https://github.com/spiceai/spiceai/pull/4214
* feat: calculate throughput test query percentiles by @peasee in https://github.com/spiceai/spiceai/pull/4215
* feat: Add throughput test to actions by @peasee in https://github.com/spiceai/spiceai/pull/4217
* Implement DynamoDB Data Connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4218
* 1.0 doc updates by @lukekim in https://github.com/spiceai/spiceai/pull/4181
* Improve clarity and concison of use-cases by @lukekim in https://github.com/spiceai/spiceai/pull/4220
* Remove macOS Intel build by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4221
* fix: Test operator throughput test workflow by @peasee in https://github.com/spiceai/spiceai/pull/4222
* DynamoDB: Automatically load AWS credentials from IAM roles if access key not provided by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4226
* File connector clickbench snapshots results by @ewgenius in https://github.com/spiceai/spiceai/pull/4225
* Spice.ai Catalog Connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4204
* feat: Add test framework metrics collection by @peasee in https://github.com/spiceai/spiceai/pull/4227
* Add badges for build/test status on README.md by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4228
* Release Dremio to RC by @Sevenannn in https://github.com/spiceai/spiceai/pull/4224
* feat: Add more test spicepods by @peasee in https://github.com/spiceai/spiceai/pull/4229
* feat: Add load test to testoperator by @peasee in https://github.com/spiceai/spiceai/pull/4231
* Add TSV format to all `object_store`-based connectors by @Jeadie in https://github.com/spiceai/spiceai/pull/4192
* Move test-framework to dev-dependencies for Runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4230
* Document limitation for correlated subqueries in TPCH for Spice.ai connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4235
* Changes for CUDA by @Jeadie in https://github.com/spiceai/spiceai/pull/4130
* fix: Collect batches from test framework, load test updates by @peasee in https://github.com/spiceai/spiceai/pull/4234
* Suppress opentelemetry_sdk warnings - they aren't useful by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4243
* fix: Set dataset status first, update test framework by @peasee in https://github.com/spiceai/spiceai/pull/4244
* feat: Re-enable defaults on test spicepods by @peasee in https://github.com/spiceai/spiceai/pull/4248
* Add usage for streaming local models; Fix spice chat usage bar TPS expansion by @Jeadie in https://github.com/spiceai/spiceai/pull/4232
* refactor: Use composite testoperator setup, add query overrides by @peasee in https://github.com/spiceai/spiceai/pull/4246
* Enable expand_views_at_output for DF optimizer and transform schema to expanded view types by @ewgenius in https://github.com/spiceai/spiceai/pull/4237
* Add throughput test spicepod for databricks delta mode connector by @Sevenannn in https://github.com/spiceai/spiceai/pull/4241
* Spark data connector - update and enable TPCH and TPCDS benchmarks by @ewgenius in https://github.com/spiceai/spiceai/pull/4240
* Increase the timeout minutes of load test to 10 hours by @Sevenannn in https://github.com/spiceai/spiceai/pull/4254
* Improve partition column counts error for delta table by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4247
* Add e2e test for databricks catalog connector (mode: delta_lake) by @Sevenannn in https://github.com/spiceai/spiceai/pull/4255
* Spark connector integration tests by @ewgenius in https://github.com/spiceai/spiceai/pull/4256
* Run benchmark test with the new test framework by @Sevenannn in https://github.com/spiceai/spiceai/pull/4245
* Configure databricks delta secrets to run load test by @Sevenannn in https://github.com/spiceai/spiceai/pull/4257
* Support `properties` for emitted telemetry by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4249
* feat: Add `ready_wait` test operator workflow input by @peasee in https://github.com/spiceai/spiceai/pull/4259
* Handle 'LargeStringArray' for embedding tables by @Jeadie in https://github.com/spiceai/spiceai/pull/4263
* `llms` tests for alpha/beta model criteria  by @Jeadie in https://github.com/spiceai/spiceai/pull/4261
* Configurable runner type for load and throughput tests by @ewgenius in https://github.com/spiceai/spiceai/pull/4262
* Handle NULL partition columns for Delta Lake tables by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4264
* Add integration test for Snowflake by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4266
* Add Snowflake TPCH queries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4268
* Handle `LargeStringArray` in `v1/search`. by @Jeadie in https://github.com/spiceai/spiceai/pull/4265
* Fix `build_cuda` in Update spiced_docker.yml by @Jeadie in https://github.com/spiceai/spiceai/pull/4269
* Run Snowflake benchmark in GitHub Actions by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4270
* Allow Snowflake query override for CI tests by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4271
* Don't run GPU builds for trunk by @Jeadie in https://github.com/spiceai/spiceai/pull/4272
* Fix InvalidTypeAction not working by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4273
* Add xAI key to llm integration tests by @Jeadie in https://github.com/spiceai/spiceai/pull/4274
* Update openai snapshots by @Jeadie in https://github.com/spiceai/spiceai/pull/4275
* Fix federation bug for correlated subqueries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4276
* Update end_game.md by @ewgenius in https://github.com/spiceai/spiceai/pull/4278
* Promote Snowflake to Beta by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4277
* Set version to 1.0.0-rc.5 by @ewgenius in https://github.com/spiceai/spiceai/pull/4283
* Update cargo.lock by @ewgenius in https://github.com/spiceai/spiceai/pull/4285
* Update spice.ai data connector snapshots by @ewgenius in https://github.com/spiceai/spiceai/pull/4281
* Promote the Spice.ai Data Connector to Beta by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4282
* Revert change to  `integration_models__models__search__openai_chunking_response.snap` by @Jeadie in https://github.com/spiceai/spiceai/pull/4279
* Allow for a subset of build artifacts to be published to minio by @Jeadie in https://github.com/spiceai/spiceai/pull/4280
* Promote File Data Connector to Stable by @ewgenius in https://github.com/spiceai/spiceai/pull/4286
* Add Iceberg to Supported Catalogs by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4287
* Update openapi.json by @github-actions in https://github.com/spiceai/spiceai/pull/4289
* Fix Spark benchmark credentials, add back overrides by @ewgenius in https://github.com/spiceai/spiceai/pull/4295
* Promote Spark Data Connector to Beta by @ewgenius in https://github.com/spiceai/spiceai/pull/4296
* Add Dremio throughput test spicepod by @Sevenannn in https://github.com/spiceai/spiceai/pull/4233
* Add error message for invalid databricks mode parameter by @Sevenannn in https://github.com/spiceai/spiceai/pull/4299
* Fix pre-release check to look for `build` string by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4300
* Promote databricks catalog connector (mode: delta_lake) to beta by @Sevenannn in https://github.com/spiceai/spiceai/pull/4301
* Properly delegate `load_table` to Rest Catalog by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4303
* Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/4302
* docs: Update ROADMAP.md by @peasee in https://github.com/spiceai/spiceai/pull/4306
* v1.0.0-rc.5 Release Notes by @ewgenius in https://github.com/spiceai/spiceai/pull/4298

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.0-rc.4...v1.0.0-rc.5

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]

Spice v1.0-rc.4 (Jan 6, 2025)

January 6, 2025 · 5 min read

Phillip LeBlanc

Co-Founder and CTO of Spice AI

Happy New Year 🎆!

Announcing the release of Spice v1.0-rc.4 🌟

Spice v1.0.0-rc.4 is the fourth release candidate for the first major version of Spice.ai OSS. This release continues the focus on production readiness. In addition, xAI has been added as a model provider.

Highlights in v1.0-rc.4

xAI Model Provider: Adds support for xAI hosted models.

models:
  - from: xai:grok2-latest
    name: xai
    params:
      xai_api_key: ${secrets:SPICE_XAI_API_KEY}

Spicepod Spec Version: Spicepod spec version v1 is now by default. v1beta1 will continue to work.

version: v1
kind: Spicepod
name: my_pod

GitHub Data Connector: Graduated to Stable.
PostgreSQL Data Accelerator: Graduated to Release Candidate (RC).

Cookbook

Added xAI model provider recipe.

Dependencies

No major dependency changes.

Contributors

@lukekim
@phillipleblanc
@peasee
@karifabri
@sgrebnov
@Jeadie
@ewgenius

What's Changed

- Update openapi.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4087>
- Update Helm chart for v1.0.0-rc.3 (v0.2.2) by @lukekim in <https://github.com/spiceai/spiceai/pull/4088>
- Rev version to v1.0.0-rc.4 by @lukekim in <https://github.com/spiceai/spiceai/pull/4090>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4089>
- Fix OpenAI Models Integration tests by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4084>
- fix: Update Postgres TPCDS and ClickBench queries by @peasee in <https://github.com/spiceai/spiceai/pull/4092>
- fix: Check Postgres acceleration schema on insert by @peasee in <https://github.com/spiceai/spiceai/pull/4094>
- Update v1.0.0-rc.3.md by @karifabri in <https://github.com/spiceai/spiceai/pull/4096>
- Update openapi.json by @github-actions in <https://github.com/spiceai/spiceai/pull/4093>
- First-class TSV for file data connector by @lukekim in <https://github.com/spiceai/spiceai/pull/4098>
- Allow Flight DoPut only for write api-keys by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4010>
- Only create tables `eval.runs` and `eval.results` when an eval is defined by @Jeadie in <https://github.com/spiceai/spiceai/pull/4099>
- Update Copyright year to include 2025 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4100>
- feat: add postgres clickbench accelerator, release postgres accelerator by @peasee in <https://github.com/spiceai/spiceai/pull/4111>
- Add spice binaries with metal to releases; detect metal device in `spice install/upgrade`. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4097>
- docs: Clarify connector release criteria by @peasee in <https://github.com/spiceai/spiceai/pull/4112>
- Update datafusion-federation to fix LIMIT with OFFSET handling in logical plan rewrite by @ewgenius in <https://github.com/spiceai/spiceai/pull/4115>
- Support Grok AI. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4113>
- Fix `spice chat` usage bar. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4119>
- Set unified max encoding and decoding message size for all flight client configurations across runtime by @ewgenius in <https://github.com/spiceai/spiceai/pull/4116>
- feat: Add the file connector as an appendable benchmark connector by @peasee in <https://github.com/spiceai/spiceai/pull/4120>
- Add `spice eval` command by @lukekim in <https://github.com/spiceai/spiceai/pull/4118>
- Support multi-level table nesting for Dremio by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4129>
- feat: run append TPCH benchmarks in workflow (Arrow, DuckDB) by @peasee in <https://github.com/spiceai/spiceai/pull/4131>
- Fix bug in Iceberg tables selecting a subset of columns by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4132>
- feat: Run append TPCDS benchmarks in workflow (Arrow, DuckDB) by @peasee in <https://github.com/spiceai/spiceai/pull/4141>
- Setup spice.ai clickbench by @ewgenius in <https://github.com/spiceai/spiceai/pull/4134>
- Data is streamed when reading from the GitHub connector (GraphQL tables) by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4142>
- Mark the GitHub Data Connector as Stable by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4143>
- Fix table quoting for Databricks Spark connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4145>
- Extend flight compute context for spice.ai connector with org and app names, to fix federated queries from different spice.ai data sources by @ewgenius in <https://github.com/spiceai/spiceai/pull/4144>
- Enforce Flight DoPut policies: Rate Limiting, Read Timeout, and Max Records per Batch by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4117>
- Fix bug Changes in catalog.yaml would require saving in spicepod.yaml to apply by @sgrebnov in <https://github.com/spiceai/spiceai/pull/4147>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/4137>
- Add `test-framework` crate to contain all common benchmark, E2E, integration testing logic. by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4157>
- Fix `platform_option` variable in `build_and_release.yml`. by @Jeadie in <https://github.com/spiceai/spiceai/pull/4154>
- feat: Add Clickbench append benchmark for DuckDB and Arrow by @peasee in <https://github.com/spiceai/spiceai/pull/4160>
- Upload artifacts to Minio on build_and_release by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/4159>
- feat: add on zero results benchmark by @peasee in <https://github.com/spiceai/spiceai/pull/4164>
- Update spice.ai connector tests by @ewgenius in <https://github.com/spiceai/spiceai/pull/4161>

**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v1.0.0-rc.3...v1.0.0-rc.4>
```text

## Resources

- [Getting started with Spice.ai](https://docs.spiceai.org/getting-started/)
- [Documentation](https://docs.spiceai.org/)

## Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

- Twitter: [@spice_ai](https://twitter.com/spice_ai)
- Slack: [spiceai.org/slack](https://spiceai.org/slack)
- Telegram: [Spice AI Discussion](https://t.me/spiceaichat)
- Reddit: [https://www.reddit.com/r/spiceai](https://www.reddit.com/r/spiceai)
- Email: [[email protected]](mailto:[email protected])

Spice v1.0-rc.3 (Dec 30, 2024)

December 30, 2024 · 9 min read

Luke Kim

Founder and CEO of Spice AI

Announcing the release of Spice v1.0-rc.3 🧊

Spice v1.0.0-rc.3 is the third release candidate for the first major version of Spice.ai OSS. This release continues the focus on production readiness and includes new Iceberg Catalog APIs, DuckDB improvements, and a new Iceberg Catalog Connector.

Highlights in v1.0-rc.3

Iceberg Catalog APIs: Spice now functions as an Iceberg Catalog provider, implementing a core subset of the Iceberg Catalog APIs. This enables Iceberg Catalog clients native discovery of datasets and schemas through Spice APIs.
GET /v1/namespaces - List all catalogs registered in Spice.
GET /v1/namespaces?parent=catalog - List schemas registered under a given catalog.
GET /v1/namespaces/:catalog_schema/tables - List tables registered under a given schema.
GET /v1/namespaces/:catalog_schema/tables/:table - Get the schema of a given table.
Iceberg Catalog Connector: The Iceberg Catalog Connector is a new integration to discover and query datasets from a remote Iceberg Catalog.

Example connecting to a remote Iceberg Catalog with tables stored in S3:

catalogs:
  - from: iceberg:https://my-iceberg-catalog.com/v1/namespaces
    name: ice
    params:
      iceberg_s3_access_key_id: ${secrets:ICEBERG_S3_ACCESS_KEY_ID}
      iceberg_s3_secret_access_key: ${secrets:ICEBERG_S3_SECRET_ACCESS_KEY}
      iceberg_s3_region: us-east-1

View the Iceberg Catalog Connector documentation for more details.

DuckDB Improvements: Added cosine_distance support for DuckDB-backed vector search, improved unnest nested type handling for array_element and lists, and optimized query performance.
SQLite Data Accelerator: Graduated to Release Candidate (RC).
File Data Accelerator: Graduated to Release Candidate (RC).

Breaking changes

API:v1/datasets/sample has been removed as it is not particularly useful, can be replicated via SQL, and via the tools endpoint POST v1/tools/:name.

Cookbook

New Language Model Evals Recipe showing how to measure the performance of a language model using LLM-as-Judge, configured entirely in the spice runtime.
New Iceberg Catalog Recipe showing how to use Spice to query Iceberg tables from an Iceberg catalog.

Dependencies

OpenTelemetry: Upgraded from 0.26.0 to 0.27.1
Go: Upgraded from 1.22 to 1.23 (CLI)

Contributors

@sgrebnov
@phillipleblanc
@peasee
@Jeadie
@Sevenannn
@lukekim
@ewgenius

What's Changed

- Add CI configuration for search benchmark dataset access by @sgrebnov in https://github.com/spiceai/spiceai/pull/3888
- Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/3895
- Upgrade dependencies by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3896
- chore: Update helm chart for RC.2 by @peasee in https://github.com/spiceai/spiceai/pull/3899
- Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/3903
- chore: Update MacOS test release install to macos-13 by @peasee in https://github.com/spiceai/spiceai/pull/3901
- Add usage to `spice chat` and fix `v1/models?status=true`. by @Jeadie in https://github.com/spiceai/spiceai/pull/3898
- chore: Bump versions for rc3 by @peasee in https://github.com/spiceai/spiceai/pull/3902
- docs: Update endgame with a step to verify dependencies in release notes by @peasee in https://github.com/spiceai/spiceai/pull/3897
- Ensure eval dataset input and ouput of correct length by @Jeadie in https://github.com/spiceai/spiceai/pull/3900
- `spice add/connect/dataset configure` should update spicepod, not overwrite it & upgrade to Go 1.23 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3905
- Bump opentelemetry from 0.26.0 to 0.27.1 by @dependabot in https://github.com/spiceai/spiceai/pull/3879
- Ensure trace_id is overridden for prior written spans by @Jeadie in https://github.com/spiceai/spiceai/pull/3906
- add 'role': 'assistant' for local models by @Jeadie in https://github.com/spiceai/spiceai/pull/3910
- Run tpcds benchmark for file connector by @Sevenannn in https://github.com/spiceai/spiceai/pull/3924
- Update to reference cookbook instead of quickstarts/samples by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3928
- Fix/remove flaky integration tests by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3930
- Implement `/v1/iceberg/namespaces` & `/v1/iceberg/config` APIs by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3923
- Add script for creating tpcds parquet files and spicepod for file connector by @Sevenannn in https://github.com/spiceai/spiceai/pull/3931
- Use `utoipa` to generate openapi.json and swagger for dev by @Jeadie in https://github.com/spiceai/spiceai/pull/3927
- `fuzzy_match`, `json_match`, `includes` scorer by @Jeadie in https://github.com/spiceai/spiceai/pull/3926
- Implement `/v1/iceberg/namespaces/:namespace` by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3933
- Implement `GET /v1/iceberg/namespaces/:namespace/tables` API by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3934
- Add custom Spice DuckDB dialect with cosine_distance support by @sgrebnov in https://github.com/spiceai/spiceai/pull/3938
- Fix NSQL error: `all columns in a record batch must have the same length` by @sgrebnov in https://github.com/spiceai/spiceai/pull/3947
- Don't include tools use in hf test model by @Jeadie in https://github.com/spiceai/spiceai/pull/3955
- Implement `GET /v1/namespaces/{namespace}/tables/{table}` API by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3940
- Update dependencies by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3967
- DuckDB: add support for nested types in Lists by @sgrebnov in https://github.com/spiceai/spiceai/pull/3961
- Add script to set up clickbench for file connector by @Sevenannn in https://github.com/spiceai/spiceai/pull/3945
- docs: Add connector stable criteria by @peasee in https://github.com/spiceai/spiceai/pull/3908
- Update Roadmp Dec 23, 2024 by @lukekim in https://github.com/spiceai/spiceai/pull/3978
- Improve CI testing for OpenAPI, new tool `spiceschema`, fix broken OpenAPI stuff. by @Jeadie in https://github.com/spiceai/spiceai/pull/3948
- remove `v1/datasets/sample` by @Jeadie in https://github.com/spiceai/spiceai/pull/3981
- feat: add SQLite ClickBench benchmark by @peasee in https://github.com/spiceai/spiceai/pull/3975
- Remove feature 'llms/mistralrs' by @Jeadie in https://github.com/spiceai/spiceai/pull/3984
- Add support for 'params.spice_tools: nsql' by @Jeadie in https://github.com/spiceai/spiceai/pull/3985
- Fix integration tests - add missing `format` query parameter in /v1/status requests by @ewgenius in https://github.com/spiceai/spiceai/pull/3989
- Enhance AI tools sampling logic for robust handling of large fields by @sgrebnov in https://github.com/spiceai/spiceai/pull/3959
- Fix subquery federation by @Sevenannn in https://github.com/spiceai/spiceai/pull/3991
- Fix unnest and add DuckDB support for `array_element` by @sgrebnov in https://github.com/spiceai/spiceai/pull/3995
- Add score value snapshotting to vector similarity search tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/3996
- Use Llama-3.2-3B-Instruct for Hugging Face integration testing by @sgrebnov in https://github.com/spiceai/spiceai/pull/3992
- Simplify `construct_chunk_query_sql` for DuckDB compatibility by @sgrebnov in https://github.com/spiceai/spiceai/pull/3988
- Update TPCH and TPCDS benchmarks for spice.ai connector by @ewgenius in https://github.com/spiceai/spiceai/pull/3982
- Correctly pass Hugging Face token in models integration tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/3997
- Fix: `on_zero_results` causes `TransactionContext Error: Catalog write-write conflict on create with "attachment_0"` by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3998
- Add DuckDB acceleration to search benchmarks by @sgrebnov in https://github.com/spiceai/spiceai/pull/4000
- Enable Postgres write via non-default `postgres-write` feature flag by @sgrebnov in https://github.com/spiceai/spiceai/pull/4004
- Allow search benchmark to write test results by @sgrebnov in https://github.com/spiceai/spiceai/pull/4008
- Make Flight DoPut atomic and commit write only on successful stream completion by @sgrebnov in https://github.com/spiceai/spiceai/pull/4002
- Create a `CatalogConnector` abstraction by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4003
- Fix `generate-openapi.yml` and add `.schema/openapi.json`. by @Jeadie in https://github.com/spiceai/spiceai/pull/3983
- Enable spice.ai tpcds bench workflow. Comment failing tpch queries. by @ewgenius in https://github.com/spiceai/spiceai/pull/4001
- feat: Add SQLite ClickBench overrides by @peasee in https://github.com/spiceai/spiceai/pull/4016
- Implement Iceberg Catalog Connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4053
- feat: Datafusion updates for SQLite fixes and release by @peasee in https://github.com/spiceai/spiceai/pull/4054
- docs: Add accelerator stable release criteria by @peasee in https://github.com/spiceai/spiceai/pull/4017
- Add dremio tpch / tpcds benchmark test by @Sevenannn in https://github.com/spiceai/spiceai/pull/4063
- Update docs, and make PR to `spiceai/docs` for new `openapi.json`. by @Jeadie in https://github.com/spiceai/spiceai/pull/4019
- Update openapi.json by @github-actions in https://github.com/spiceai/spiceai/pull/4065
- Fix dremio subquery rewrite by @Sevenannn in https://github.com/spiceai/spiceai/pull/4064
- Update generate-openapi.yml by @Jeadie in https://github.com/spiceai/spiceai/pull/4073
- docs: Add catalog criteria by @peasee in https://github.com/spiceai/spiceai/pull/4052
- fix `distinct_columns` in auto/nsql tool groups by @Jeadie in https://github.com/spiceai/spiceai/pull/4074
- Update openapi.json by @github-actions in https://github.com/spiceai/spiceai/pull/4075
- Update openapi.json by @github-actions in https://github.com/spiceai/spiceai/pull/4076
- Implement window_func_support_window_frame from DremioDialect by @Sevenannn in https://github.com/spiceai/spiceai/pull/4012
- Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/4079
- Promote file connector to rc by @Sevenannn in https://github.com/spiceai/spiceai/pull/4080
- Add Iceberg to README by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4085
- Fix '/v1/status' default format by @Jeadie in https://github.com/spiceai/spiceai/pull/4081

**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.0-rc.2...v1.0.0-rc.3

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai
Slack: spiceai.org/slack
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]

Spice v1.0-rc.2 (Dec 16, 2024)

December 16, 2024 · 11 min read

William Croxson

Member of Technical Staff at Spice AI

Announcing the release of Spice v1.0-rc.2 🔗

Spice v1.0.0-rc.2 is the second release candidate for the first major version of Spice.ai OSS. This release continues to build on the stability of Spice for production use, including key Data Connector graduations, bug fixes, and AI features.

Highlights in v1.0-rc.2

MS SQL and File Data Connectors: Graduated from Alpha to Beta.
GraphQL and Databricks Delta Lake Data Connectors: Graduated from Beta to Release Candidate.
gospice SDK Release: The Spice Go SDK has updated to v7.0, adding support for refreshing datasets and upgrading dependencies.
Azure AI Support: Added support for both LLMs and embedding models. Example spicepod.yml configuration:

embeddings:
  - name: azure
    from: azure:text-embedding-3-small
    params:
      endpoint: https://your-resource-name.openai.azure.com
      azure_api_version: 2024-08-01-preview
      azure_deployment_name: text-embedding-3-small
      azure_api_key: ${ secrets:SPICE_AZURE_API_KEY }
models:
  - name: azure
    from: azure:gpt-4o-mini
    params:
      endpoint: https://your-resource-name.openai.azure.com
      azure_api_version: 2024-08-01-preview
      azure_deployment_name: gpt-4o-mini
      azure_api_key: ${ secrets:SPICE_AZURE_TOKEN }

Accelerate subsets of columns: Spice now supports acceleration for specific columns from a federated source. Specify the desired columns directly in the Refresh SQL for more selective and efficient data acceleration.

Example spicepod.yaml configuration:

datasets:
  - from: s3://spiceai-demo-datasets/taxi_trips/2024/
    name: taxi_trips
    params:
      file_format: parquet
    acceleration:
      refresh_sql: SELECT tpep_pickup_datetime, tpep_dropoff_datetime, trip_distance, total_amount FROM taxi_trips

Breaking changes

Sharepoint Authentication Parameters: now use access tokens instead of authorization codes, using the sharepoint_bearer_token parameter. The sharepoint_auth_code parameter has been removed.

Data Connector Delimiters: now support / and ://, in addition to : in the from parameter of the dataset configuration. The following examples are equivalent:

from: postgres://my_postgres_table
from: postgres/my_postgres_table
from: postgres:my_postgres_table

Some data connectors, such as s3 which only accepts ://, place further restrictions on the allowed delimiter.

The file data connector has changed how it interprets the :// delimiter to reflect how most other URL parsers work, i.e. file://my_file_path. Previously, the file path was interpreted as /my_file_path. Now, it is interpreted as a relative path, i.e. my_file_path.

Spice Search limit: is now applied to the final search result, instead of previously being applied separately to each dataset involved in a search before aggregation.

Dependencies

Rust: Upgraded to 1.83

Contributors

@phillipleblanc
@ewgenius
@Jeadie
@sgrebnov
@peasee
@Sevenannn
@Advayp

New Contributors

@Advayp made their first contribution in github.com/spiceai/spiceai/pull/3862

What's Changed

- Fix install scripts to handle the RC release by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3718>
- Update helm chart to v1.0.0-rc.1 by @ewgenius in <https://github.com/spiceai/spiceai/pull/3720>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/3719>
- Add logic to ignore task cancellations due to runtime shutdown by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3717>
- Update to next relese version v1.0.0-rc.2 by @ewgenius in <https://github.com/spiceai/spiceai/pull/3721>
- Handle parsing OTel KeyValues from the `baggage` header by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3722>
- Update `llms` dependencies: `mistralrs`, `async-openai` by @Jeadie in <https://github.com/spiceai/spiceai/pull/3725>
- Support `jsonl` for object store by @Jeadie in <https://github.com/spiceai/spiceai/pull/3726>
- Fix NSQL models integration tests for HF by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3727>
- standardise 'csv_schema_infer_max_records' -> 'schema_infer_max_records'; include deprecation messages for dataset params by @Jeadie in <https://github.com/spiceai/spiceai/pull/3732>
- feat: Add script to generate TPC-H data for file connector by @peasee in <https://github.com/spiceai/spiceai/pull/3737>
- feat: Add file connector integration test by @peasee in <https://github.com/spiceai/spiceai/pull/3735>
- fix: Add explicit message for ODBC connector when not installed by @peasee in <https://github.com/spiceai/spiceai/pull/3736>
- Remove Box::leak in `create_accelerated_table` by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3739>
- docs: Update enhancement and PR template by @peasee in <https://github.com/spiceai/spiceai/pull/3740>
- feat: add file connector benchmark by @peasee in <https://github.com/spiceai/spiceai/pull/3734>
- docs: Release file connector beta by @peasee in <https://github.com/spiceai/spiceai/pull/3738>
- For embeddings, use `sentence_*_config.json`, download HF async, use TEI functions by @Jeadie in <https://github.com/spiceai/spiceai/pull/3724>
- Optimize build & release workflow for trunk builds by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3741>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/3752>
- Skip Spice cloud integration tests by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3755>
- Add `http_requests` metric and deprecate `http_requests_total` by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3748>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/3759>
- fix: Parquet file generation script by @peasee in <https://github.com/spiceai/spiceai/pull/3762>
- fix: Use InvalidConfiguration error for GraphQL query errors by @peasee in <https://github.com/spiceai/spiceai/pull/3763>
- Extend Spice Search integration and E2E tests to cover chunking by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3750>
- test: Add GraphQL integration tests from external sources by @peasee in <https://github.com/spiceai/spiceai/pull/3756>
- docs: Release GraphQL release candidate by @peasee in <https://github.com/spiceai/spiceai/pull/3764>
- Accelerate a subset of columns from source dataset in Refresh SQL by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3765>
- Run TPCDS benchmark for databricks delta mode by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3751>
- Update dependencies by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3747>
- Implement vector search benchmark initialization by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3774>
- Implement InvalidTypeAction for PostgreSQL Data Connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3767>
- fix: Check ODBC parameters are positive integers by @peasee in <https://github.com/spiceai/spiceai/pull/3777>
- Fix Delta DataType `Map` type mapping to arrow type by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3776>
- Update Databricks & Delta Lake Connector RC criteria by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3778>
- Add a `/v1/packages/generate` API to generate a Spicepod package from a GitHub repo. by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3782>
- Set `Spice-Target-Source` header for `spice add` by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3783>
- Call v1 spicerack API by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3784>
- Run models integration tests on self-hosted macOS runners by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3785>
- Fix OpenAI models integration tests by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3786>
- Integration test for Databricks delta_lake mode by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3779>
- Add `spice connect` for connecting to existing Spice.ai instances by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3790>
- Add `eval` spicepod component; basic HTTP api to run eval. by @Jeadie in <https://github.com/spiceai/spiceai/pull/3766>
- Release RC for databricks delta_lake mode by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3792>
- Include Huggingface model to E2E models tests by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3788>
- Enable `trace_id` & `parent_span_id` overrides for `v1/chat/completion` by @Jeadie in <https://github.com/spiceai/spiceai/pull/3791>
- Search benchmark: run search workload and measure result by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3793>
- Search benchmark: measure search precision by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3804>
- Use MinIO instead of S3 for benchmark tests by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3794>
- Update benchmark snapshots by @github-actions in <https://github.com/spiceai/spiceai/pull/3814>
- Only verify TPCH / TPCDS official query results for DuckDB by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3816>
- Fixes for the Debezium connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3819>
- Fix insert statement when all columns are constraint columns by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3820>
- docs: Move ODBC to Beta for current state of roadmap by @peasee in <https://github.com/spiceai/spiceai/pull/3823>
- Accept `:`, `/` or `://` as the delimiter for the data connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3821>
- Update dependencies by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3826>
- Enable `read_write` mode support for Postgres Data Connector by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3813>
- feat: add Databricks ODBC TPCDS benchmark by @peasee in <https://github.com/spiceai/spiceai/pull/3825>
- Change `spice.ai` data connector dataset path format to `<org>/<app>/datasets/<table_reference>` by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3828>
- fix: enable tpcds explain snapshotting by @peasee in <https://github.com/spiceai/spiceai/pull/3830>
- Azure AI support for both LLMs & embedding models by @Jeadie in <https://github.com/spiceai/spiceai/pull/3824>
- Add Github Workflow to run Search Benchmark by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3834>
- Fetch access token with Microsoft OAuth, and use access token to initiate Sharepoint data connector graph client by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3836>
- Initialize accelerator for datasets dynamically included by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3714>
- Update cargo.lock by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3838>
- feat: add MS SQL TPCH benchmark by @peasee in <https://github.com/spiceai/spiceai/pull/3833>
- Improve Azure AI models support by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3835>
- Primary key support for Arrow's `Memtable` by @Jeadie in <https://github.com/spiceai/spiceai/pull/3829>
- Update Tokenizer to 0.21 and mistral.rs by @Jeadie in <https://github.com/spiceai/spiceai/pull/3839>
- Fix models integration tests by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3843>
- Enable `spice login abfs` by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3844>
- update `crates/llms` dependencies to 'spiceai' branch by @Jeadie in <https://github.com/spiceai/spiceai/pull/3846>
- Make eval runs non-blocking; `spice.eval.{results, runs}` tables. by @Jeadie in <https://github.com/spiceai/spiceai/pull/3780>
- fix: Update GraphQL snapshots by @peasee in <https://github.com/spiceai/spiceai/pull/3849>
- Update to Rust 1.83 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3847>
- feat: add mssql integration test by @peasee in <https://github.com/spiceai/spiceai/pull/3848>
- Prepend user-specified user agent in flight repl by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3850>
- fix: trim CHAR in mssql by @peasee in <https://github.com/spiceai/spiceai/pull/3852>
- Fix column quoting for SpiceCloudPlatform dialect by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3857>
- Optimize builds by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/3861>
- Endgame template: Add recently added AI/ML quickstarts and samples by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3859>
- docs: Release MS SQL Beta by @peasee in <https://github.com/spiceai/spiceai/pull/3853>
- Fix nsql sampling for tables with embeddings by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3860>
- Make GH workflows with spiceai-macos runners more stable by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3863>
- fix: Remove GraphQL swapi test by @peasee in <https://github.com/spiceai/spiceai/pull/3867>
- create 1 `tokio::test` per test/model by @Jeadie in <https://github.com/spiceai/spiceai/pull/3696>
- handle `max_completion_tokens` vs `max_tokens` for openai vs azure by @Jeadie in <https://github.com/spiceai/spiceai/pull/3869>
- Search benchmark: write results to dataset by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3871>
- Create `evalconverter` that creates spice eval components. by @Jeadie in <https://github.com/spiceai/spiceai/pull/3864>
- Update quickstart in README.md by @ewgenius in <https://github.com/spiceai/spiceai/pull/3876>
- Remove reference to spiceai-smart-demo from the repo home by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3885>
- Trace `evals` accelerated tables updates in debug mode by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3884>
- Clarify confusing log message by @Advayp in <https://github.com/spiceai/spiceai/pull/3862>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/3840>
- Azure OpenAI models: make `endpoint` parameter required by @sgrebnov in <https://github.com/spiceai/spiceai/pull/3883>
- Use spiceai delta kernel fork, actionable message for delta checkpoint errors by @Sevenannn in <https://github.com/spiceai/spiceai/pull/3856>
- Add support for GGUF files in HF by @Jeadie in <https://github.com/spiceai/spiceai/pull/3875>

**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v1.0.0-rc.1...v1.0.0-rc.2>
```text

## Resources

- [Getting started with Spice.ai](https://docs.spiceai.org/getting-started/)
- [Documentation](https://docs.spiceai.org/)

## Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

- Twitter: [@spice_ai](https://twitter.com/spice_ai)
- Slack: [spiceai.org/slack](https://spiceai.org/slack)
- Telegram: [Spice AI Discussion](https://t.me/spiceaichat)
- Reddit: [https://www.reddit.com/r/spiceai](https://www.reddit.com/r/spiceai)
- Email: [[email protected]](mailto:[email protected])

Highlights in v1.0.5​

Contributors​

Breaking Changes​

Upgrading​

What's Changed​

Dependencies​

Changelog​

Highlights in v1.0.4​

Contributors​

Breaking Changes​

Cookbook Updates​

Upgrading​

What's Changed​

Dependencies​

Changelog​

Highlights in v1.0.3​

Contributors​

Breaking Changes​

Cookbook Updates​

Upgrading​

What's Changed​

Dependencies​

Changelog​

Resources​

Community​

Highlights in v1.0.2​

Contributors​

New Contributors​

Breaking Changes​

Cookbook Updates​

Upgrading​

What's Changed​

Dependencies​

Changelog​

Resources​

Community​

Highlights in v1.0.1​

Breaking changes​

Dependencies​

Cookbook​

Upgrading​

Contributors​

What's Changed​

Resources​

Community​

Highlights in v1.0-stable​

Breaking Changes​

Dependencies​

Upgrading​

Contributors​

What's Changed​

Resources​

Community​

Highlights in v1.0-rc.5​

Dependencies​

Contributors​

What's Changed​

Resources​

Community​

Highlights in v1.0-rc.4​

Cookbook​

Dependencies​

Contributors​

What's Changed​

Highlights in v1.0-rc.3​

Breaking changes​

Cookbook​

Dependencies​

Contributors​

What's Changed​

Resources​

Community​

Highlights in v1.0-rc.2​

Breaking changes​

Dependencies​

Contributors​

New Contributors​

What's Changed​

Highlights in v1.0.5

Contributors

Breaking Changes

Upgrading

What's Changed

Dependencies

Changelog

Highlights in v1.0.4

Contributors

Breaking Changes

Cookbook Updates

Upgrading

What's Changed

Dependencies

Changelog

Highlights in v1.0.3

Contributors

Breaking Changes

Cookbook Updates

Upgrading

What's Changed

Dependencies

Changelog

Resources

Community

Highlights in v1.0.2

Contributors

New Contributors

Breaking Changes

Cookbook Updates

Upgrading

What's Changed

Dependencies

Changelog

Resources

Community

Highlights in v1.0.1

Breaking changes

Dependencies

Cookbook

Upgrading

Contributors

What's Changed

Resources

Community

Highlights in v1.0-stable

Breaking Changes

Dependencies

Upgrading

Contributors

What's Changed

Resources

Community

Highlights in v1.0-rc.5

Dependencies

Contributors

What's Changed

Resources

Community

Highlights in v1.0-rc.4

Cookbook

Dependencies

Contributors

What's Changed

Highlights in v1.0-rc.3

Breaking changes

Cookbook

Dependencies

Contributors

What's Changed

Resources

Community

Highlights in v1.0-rc.2

Breaking changes

Dependencies

Contributors

New Contributors

What's Changed