Spice v0.17.2-beta (August 26, 2024)
· 6 min read
Announcing the release of Spice v0.17.2-beta 🏄
The v0.17.2-beta release focuses on improving data accelerator compatibility, stability, and performance. Expanded data type support for DuckDB, SQLite, and PostgreSQL data accelerators (and data connectors) enables significantly more data types to be accelerated. Error handling and logging has also been improved along with several bugs.
Highlights in v0.17.2-beta
Expanded Data Type Support for Data Accelerators: DuckDB, SQLite, and PostgreSQL Data Accelerators now support a wider range of data types, enabling acceleration of more diverse datasets.
Enhanced Error Handling and Logging: Improvements have been made to aid in troubleshooting and debugging.
Anonymous Usage Telemetry: Optional, anonymous, aggregated telemetry has been added to help improve Spice. This feature can be disabled. For details about collected data, see the telemetry documentation.
To opt out of telemetry:
-
Using the CLI flag:
spice run -- --telemetry-enabled false -
Add configuration to
spicepod.yaml:runtime:
telemetry:
enabled: false
Improved Benchmarking: A suite of performance benchmarking tests have been added to the project, helping to maintain and improve runtime performance; a top priority for the project.
Breaking Changes
None.
Contributors
- @Jeadie
- @y-f-u
- @phillipleblanc
- @sgrebnov
- @Sevenannn
- @peasee
- @ewgenius
What's Changed
Dependencies
- DataFusion: Upgraded from v40 to v41
Commits
- Pin actions/upload-artifact to v4.3.4 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2200>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/2202>
- Update to next release version, `v0.17.2-beta` by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2203>
- add accelerator beta criteria by @y-f-u in <https://github.com/spiceai/spiceai/pull/2201>
- update helm chart to 0.17.1-beta by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2205>
- add dockerignore to avoid copy target and test folder by @y-f-u in <https://github.com/spiceai/spiceai/pull/2206>
- add client timeout for deltalake connector by @y-f-u in <https://github.com/spiceai/spiceai/pull/2208>
- Upgrade tonic and opentelemetry-proto by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2223>
- Add index and resource tuning for postgres ghcr image to support postgres benchmark in sf1 by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2196>
- Remove embedding columns from `retrieved_primary_keys` in v1/search by @Jeadie in <https://github.com/spiceai/spiceai/pull/2176>
- use file as db_path_param as the param prefix is trimmed by @y-f-u in <https://github.com/spiceai/spiceai/pull/2230>
- use file for sqlite db path param by @y-f-u in <https://github.com/spiceai/spiceai/pull/2231>
- docs: Clarify the global requirement for local_infile when loading TPCH by @peasee in <https://github.com/spiceai/spiceai/pull/2228>
- Revert pinning actions/upload-artifact@v4 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2232>
- Runtime tools to chat models by @Jeadie in <https://github.com/spiceai/spiceai/pull/2207>
- Create `runtime.task_history` table for queries, and embeddings by @Jeadie in <https://github.com/spiceai/spiceai/pull/2191>
- chore: Update Databricks ODBC Bench to use TPCH SF1 by @peasee in <https://github.com/spiceai/spiceai/pull/2238>
- Replace `metrics-rs` with OpenTelemetry Metrics by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2240>
- fix: Remove dead code by @peasee in <https://github.com/spiceai/spiceai/pull/2249>
- Improve tool quality and add vector search tool by @Jeadie in <https://github.com/spiceai/spiceai/pull/2250>
- fix missing partition cols in delta lake by @y-f-u in <https://github.com/spiceai/spiceai/pull/2253>
- download file from remote for delta testing by @y-f-u in <https://github.com/spiceai/spiceai/pull/2254>
- feat: Set SQLite DB path to .spice/data by @peasee in <https://github.com/spiceai/spiceai/pull/2242>
- Support tools for chat completions in streaming mode by @ewgenius in <https://github.com/spiceai/spiceai/pull/2255>
- Load component `description` field from spicepod.yaml and include in LLM context by @ewgenius in <https://github.com/spiceai/spiceai/pull/2261>
- Add parameter for `connection_pool_size` in the Postgres Data Connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2251>
- Add primary keys to response of `DocumentSimilarityTool` by @Jeadie in <https://github.com/spiceai/spiceai/pull/2263>
- run queries bash script by @y-f-u in <https://github.com/spiceai/spiceai/pull/2262>
- Run benchmark test on schedule by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2277>
- feat: Add a reference to originating App for a Dataset by @peasee in <https://github.com/spiceai/spiceai/pull/2283>
- Tool use & telemetry productionisation. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2286>
- Fix cron in benchmarks.yml by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2288>
- Upgrade to DataFusion v41 by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2290>
- Chat completions adjustments and fixes by @ewgenius in <https://github.com/spiceai/spiceai/pull/2292>
- Define the new metrics Arrow schema based on Open Telemetry by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2295>
- OpenTelemetry Metrics Arrow exporter to `runtime.metrics` table by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2296>
- Calculate summary metrics from histograms for Prometheus endpoint by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2302>
- Add back Spice DF runtime_env during SessionContext construction by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2304>
- Add integration test for S3 data connector by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2305>
- Fix `secrets.inject_secrets` when secret not found. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2306>
- Intra-table federation query on duckdb accelerated table by @y-f-u in <https://github.com/spiceai/spiceai/pull/2299>
- Postgres federation on acceleration by @y-f-u in <https://github.com/spiceai/spiceai/pull/2309>
- sqlite intra table federation on acceleration by @y-f-u in <https://github.com/spiceai/spiceai/pull/2308>
- feat: Add `DataAccelerator::init()` for SQLite acceleration federation by @peasee in <https://github.com/spiceai/spiceai/pull/2293>
- Initial framework for collecting anonymous usage telemetry by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2310>
- Add gRPC action to trigger accelerated dataset refresh by @sgrebnov in <https://github.com/spiceai/spiceai/pull/2316>
- add `disable_query_push_down` option to acceleration settings by @y-f-u in <https://github.com/spiceai/spiceai/pull/2327>
- Remove `v1/assist` by @Jeadie in <https://github.com/spiceai/spiceai/pull/2312>
- bump table provider version to set the correct dialect for postgres writer by @y-f-u in <https://github.com/spiceai/spiceai/pull/2329>
- Send telemetry on startup by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2331>
- Calculate resource IDs for telemetry by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2332>
- Refactor `v1/search`: include WHERE condition, allow extra columns in projection. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2328>
- Add integration test for gRPC dataset refresh action by @sgrebnov in <https://github.com/spiceai/spiceai/pull/2330>
- Propagate errors through all `task_history` nested spans by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2337>
- Improve tools by @Jeadie in <https://github.com/spiceai/spiceai/pull/2338>
- update duckdb rs version to support more types: interval/duration/etc by @y-f-u in <https://github.com/spiceai/spiceai/pull/2336>
- feat: Add DuckDB accelerator init, attach databases for federation by @peasee in <https://github.com/spiceai/spiceai/pull/2335>
- Add query telemetry metrics by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2333>
- Add system prompts for LLMs; system prompts for tool using models. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2342>
- Fix benchmark test to keep running when there's failed queries by @Sevenannn in <https://github.com/spiceai/spiceai/pull/2347>
- Tools as a spicepod first class citizen. by @Jeadie in <https://github.com/spiceai/spiceai/pull/2344>
- Add `bytes_processed` telemetry metric by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2343>
- fix misaligned columns from delta lake by @y-f-u in <https://github.com/spiceai/spiceai/pull/2356>
- Emit telemetry metrics to `runtime.metrics`/Prometheus as well by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2352>
- Use UTC timezone for telemetry timestamps by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2354>
- Fix MetricType deserialization by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2358>
- Add dataset details to tool using LLMs; early check tables in vector search by @Jeadie in <https://github.com/spiceai/spiceai/pull/2353>
- Bump datafusion-federation/datafusion-table-providers dependencies by @phillipleblanc in <https://github.com/spiceai/spiceai/pull/2360>
- Update spicepod.schema.json by @github-actions in <https://github.com/spiceai/spiceai/pull/2362>
- fix: Disable DuckDB and SQLite federation by @peasee in <https://github.com/spiceai/spiceai/pull/2371>
- Fix system prompt in ToolUsingChat, fix builtin registration by @Jeadie in <https://github.com/spiceai/spiceai/pull/2367>
- fix: Use --profile release for benchmarks by @peasee in <https://github.com/spiceai/spiceai/pull/2372>
- nql parameter 'use' -> 'model' by @Jeadie in <https://github.com/spiceai/spiceai/pull/2366>
**Full Changelog**: <https://github.com/spiceai/spiceai/compare/v0.17.1-beta...v0.17.2-beta>
Resources
Community
Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.
- Twitter: @spice_ai
- Discord: https://discord.gg/kZnTfneP5u
- Telegram: Spice AI Discussion
- Reddit: https://www.reddit.com/r/spiceai
- Email: [email protected]
