Spice v1.0.4 (Feb 17, 2025)
Announcing the release of Spice v1.0.4 🏎️
Spice v1.0.4 improves partition pruning for Delta Lake tables, significantly increasing scan efficiency and reducing overhead. xAI tool calling is more robust and the spice trace CLI command now provides expanded, detailed output for deeper analysis. Additionally, a bug has been fixed to correctly apply column name case-sensitivity in refresh SQL, indexes, and primary keys.
Highlights in v1.0.4
- Improved Append-Based Refresh When using an append-based acceleration where the
time_columnformat differs from the physical partition, two new dataset configuration options,time_partition_columnandtime_partition_formatcan be configured to improve partition pruning and exclude irrelevant partitions during the refreshes.
For example, when the time_column format is timestamp and the physical data partition is date such as below:
my_delta_table/
├── _delta_log/
├── date_col=2023-12-31/
├── date_col=2024-02-04/
├── date_col=2025-01-01/
└── date_col=2030-06-15/
Partition pruning can be optimized using the configuration:
datasets:
- from: delta_lake://my_delta_table
name: my_delta_table
time_column: created_at # A fine-grained timestamp
time_format: timestamp
time_partition_column: date_col # Data is physically partitioned by `date_col`
time_partition_format: date
sgrebnov marked this conversation as resolved.
- Expanded
spice traceoutput: Thespice traceCLI command now includes additional details, such as task status, and optional flags--include-inputand--include-outputfor detailed tracing.
Example spice trace output:
TREE STATUS DURATION TASK
a97f52ccd7687e64 ✅ 673.14ms ai_chat
├── 4eebde7b04321803 ✅ 0.04ms tool_use::list_datasets
└── 4c9049e1bf1c3500 ✅ 671.91ms ai_completion
Example spice trace --include-input --include-output output:
TREE STATUS DURATION TASK OUTPUT
a97f52ccd7687e64 ✅ 673.14ms ai_chat The capital of New York is Albany.
├── 4eebde7b04321803 ✅ 0.04ms tool_use::list_datasets []
└── 4c9049e1bf1c3500 ✅ 671.91ms ai_completion [{"content":"The capital of New York is Albany.","refusal":null,"tool_calls":null,"role":"assistant","function_call":null,"audio":null}]
Contributors
- @Jeadie
- @peasee
- @phillipleblanc
- @Sevenannn
- @sgrebnov
- @lukekim
Breaking Changes
No breaking changes.
Cookbook Updates
No new recipes.
Upgrading
To upgrade to v1.0.4, use one of the following methods:
CLI:
spice upgrade
Homebrew:
brew upgrade spiceai/spiceai/spice
Docker:
Pull the spiceai/spiceai:1.0.4 image:
docker pull spiceai/spiceai:1.0.4
For available tags, see DockerHub.
Helm:
helm repo update
helm upgrade spiceai spiceai/spiceai
What's Changed
Dependencies
No major dependency changes.
Changelog
- Do not return underlying content of chunked embedding column by default during tool_use::document_similarity by @Jeadie in https://github.com/spiceai/spiceai/pull/4802
- Fix Snowflake Case-Sensitive Identifiers support by @sgrebnov in https://github.com/spiceai/spiceai/pull/4813
- Prepare for 1.0.4 by @sgrebnov in https://github.com/spiceai/spiceai/pull/4801
- Add support for a time_partition_column by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4784
- Prevent the automatic normalization of refresh_sql columns to lowercase by @sgrebnov in https://github.com/spiceai/spiceai/pull/4787
- Implement partition pruning for Delta Lake tables by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4783
- Fix constraint verification for columns with uppercase letters by @sgrebnov in https://github.com/spiceai/spiceai/pull/4785
- Add truncate command for spice trace by @peasee in https://github.com/spiceai/spiceai/pull/4771
- Implement Cache-Control: no-cache to bypass results cache by @phillipleblanc in https://github.com/spiceai/spiceai/pull/4763
- Prompt user to download runtime when running spice sql by @Sevenannn in https://github.com/spiceai/spiceai/pull/4747
- Add vector search tracing by @peasee in https://github.com/spiceai/spiceai/pull/4757
- Update spice trace output format by @Jeadie in https://github.com/spiceai/spiceai/pull/4750
- Fix tool call arguments in Grok messages by @Jeadie in https://github.com/spiceai/spiceai/pull/4741
**Full Changelog**: https://github.com/spiceai/spiceai/compare/v1.0.3...v1.0.4
