Spice v1.6.1 (Sep 1, 2025)
Announcing the release of Spice v1.6.1! ⚡
Spice 1.6.1 is a patch release that provides improved Kafka type inference and JSON flattening support, alongside several bug fixes.
What's New in v1.6.1
Improved Kafka Type Inference: Improve Kafka type inference by configuring the number of Kafka messages sampled during schema inference. Increasing the sample size can improve the robustness and reliability of inferred schemas, especially in cases where data contains optional fields or varying structures.
Example spicepod.yml:
dataset:
- from: kafka:orders_events
name: orders
params:
schema_infer_max_records: 100 # Default 1.
For details, see the Kafka Data Connector Documentation.
Improved Kafka JSON Support: Enable nested JSON Kafka messages to be represented in flattened JSON format for the dataset schema.
Example spicepod.yml:
dataset:
- from: kafka:orders_events
name: orders
params:
flatten_json: true # default false
For example, the object:
{
"order_id": "a1f2c3d4-1111-2222-3333-444455556666",
"customer": {
"id": 101,
"name": "Alice",
"premium": true,
"contact": {
"email": "[email protected]",
"phone": "555-1234"
}
},
"discount": 5.0,
"shipped": false
}
With flatten_json: true the result is:
+------------------------+-----------+-------------+
| column_name | data_type | is_nullable |
+------------------------+-----------+-------------+
| order_id | Utf8 | YES |
| customer.id | Int64 | YES |
| customer.name | Utf8 | YES |
| customer.premium | Boolean | YES |
| customer.contact.email | Utf8 | YES |
| customer.contact.phone | Utf8 | YES |
| discount | Float64 | YES |
| shipped | Boolean | YES |
+------------------------+-----------+-------------+
With flatten_json: false or ommitted the result is:
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+
| column_name | data_type | is_nullable |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+
| order_id | Utf8 | YES |
| customer | Struct([Field { name: "id", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "name", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "premium", data_type: Boolean, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "contact", data_type: Struct([Field { name: "email", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "phone", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]) | YES |
| discount | Float64 | YES |
| shipped | Boolean | YES |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+
For details, see the Kafka Data Connector Documentation.
Contributors
Breaking Changes
No breaking changes.
Cookbook Updates
No new cookbook recipes added in this release.
The Spice Cookbook includes 77 recipes to help you get started with Spice quickly and easily.
Upgrading
To upgrade to v1.6.1, use one of the following methods:
CLI:
spice upgrade
Homebrew:
brew upgrade spiceai/spiceai/spice
Docker:
Pull the spiceai/spiceai:1.6.1 image:
docker pull spiceai/spiceai:1.6.1
For available tags, see DockerHub.
Helm:
helm repo update
helm upgrade spiceai spiceai/spiceai
AWS Marketplace:
🎉 Spice is now available in the AWS Marketplace!
What's Changed
Changelog
- Fix metadata field issue by @Advayp in #6957
- Update datafusion and datafusion-table-providers crates (#6985) by @Jeadie in #6985
- Add flatten_json param support for Kafka connector (#6976) by @sgrebnov in #6976
- Add schema_inference_sample_count param support for Kafka connector (#6969) by @sgrebnov in #6969
- Add integration test for Kafka connector (#6965) by @sgrebnov in #6965
- Skip dataset health check for IcebergTableProvider datasets by @phillipleblanc in #6995
