Spice v0.13.2-alpha (June 3, 2024)

The v0.13.2-alpha release is focused on quality and stability with improvements to federated query push-down, telemetry, and query history.

Highlights

• Filesystem Data Connector: Adds the Filesystem Data Connector for directly using files as data sources.

• Federated Query Push-Down: Improved stability and schema compatibility for federated queries.

• Enhanced Telemetry: Runtime Metrics now include last update time for accelerated datasets, count of refresh errors, and new metrics for query duration and failures.

• Query History: Enabled query history logging for Arrow Flight queries in addition to HTTP queries.

Contributors

• @lukekim
• @y-f-u
• @ewgenius
• @phillipleblanc
• @Jeadie
• @Sevenannn
• @sgrebnov
• @gloomweaver
• @mach-kernel

What's Changed

• Update ROADMAP.md May 27, 2024 by @lukekim in https://github.com/spiceai/spiceai/pull/1535
• update helm chart version and use v0.13.1-alpha by @y-f-u in https://github.com/spiceai/spiceai/pull/1536
• version correction in v0.13.1 release note by @y-f-u in https://github.com/spiceai/spiceai/pull/1538
• update version to v0.14.0-alpha by @y-f-u in https://github.com/spiceai/spiceai/pull/1539
• Update spice_cloud - connect to cloud api by @ewgenius in https://github.com/spiceai/spiceai/pull/1523
• Update spice_cloud extension params, and remove logging by @ewgenius in https://github.com/spiceai/spiceai/pull/1541
• Update MSRV to 1.78 and remove unused Rust Version parameter in CI by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1540
• Improve llm UX in spicepod.yaml by @Jeadie in https://github.com/spiceai/spiceai/pull/1545
• Store local runtime metrics in Timestamp with nanoseconds precision and UTC time by @ewgenius in https://github.com/spiceai/spiceai/pull/1548
• Object store metadata Table provider by @Jeadie in https://github.com/spiceai/spiceai/pull/1518
• Remove clickhouse password requirement by @Sevenannn in https://github.com/spiceai/spiceai/pull/1547
• pretty print loaded rows number by @y-f-u in https://github.com/spiceai/spiceai/pull/1553
• Fix UNION ALL federated push down by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1550
• Update mistral, fix bugs and improve local file DX by @Jeadie in https://github.com/spiceai/spiceai/pull/1552
• Cast runtime.metrics schema, if remote (spiceai) data connector provided by @ewgenius in https://github.com/spiceai/spiceai/pull/1554
• Use proper MySQL dialect during federation push-down by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1555
• parallel load dataset when starting up by @y-f-u in https://github.com/spiceai/spiceai/pull/1551
• fix linter warning on Scanf return value by @y-f-u in https://github.com/spiceai/spiceai/pull/1556
• Update spice cloud connect api endpoint by @ewgenius in https://github.com/spiceai/spiceai/pull/1557
• Create new HTTP endpoint to create embeddings. by @Jeadie in https://github.com/spiceai/spiceai/pull/1558
• Query History support for Flight API by @sgrebnov in https://github.com/spiceai/spiceai/pull/1549
• Don't cache queries for runtime tables by @sgrebnov in https://github.com/spiceai/spiceai/pull/1561
• Fix schema incompatibility on federated push-down queries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1560
• move 'embeddings' to top-level concept in spicepod.yaml by @Jeadie in https://github.com/spiceai/spiceai/pull/1564
• object_store table provider for UTF8 data formats by @Jeadie in https://github.com/spiceai/spiceai/pull/1562
• Improve connectivity for JDBC clients, like Tableau by @sgrebnov in https://github.com/spiceai/spiceai/pull/1563
• Enable datasets from local filesystem by @Jeadie in https://github.com/spiceai/spiceai/pull/1584
• Adds benchmarking tests for Spice by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1577
• Push down correct timestamp expr to SQLite, add binary type mapping by @mach-kernel in https://github.com/spiceai/spiceai/pull/1566
• Add query_duration_seconds and query_failures metrics by @sgrebnov in https://github.com/spiceai/spiceai/pull/1575
• Use /app as a default workdir in spiceai docker image by @ewgenius in https://github.com/spiceai/spiceai/pull/1586
• Add support for both file:// and file:/ by @Jeadie in https://github.com/spiceai/spiceai/pull/1587
• put load_datasets as the latest step along with start_servers by @y-f-u in https://github.com/spiceai/spiceai/pull/1559
• Embedding columns (from embedding providers) are now run inside datafusion plans. by @Jeadie in https://github.com/spiceai/spiceai/pull/1576
• Support BinaryArray in DuckDB accelerations by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1595
• Add cache header to Flight API and Spice REPL indicator by @sgrebnov in https://github.com/spiceai/spiceai/pull/1591
• Add accelerated datasets refresh metrics by @sgrebnov in https://github.com/spiceai/spiceai/pull/1589
• update the error when starting spice sql with no runtime to be actionable by @digadeesh in https://github.com/spiceai/spiceai/pull/1597
• add odbc integration test by @y-f-u in https://github.com/spiceai/spiceai/pull/1590
• Fix bug in instantiating EmbeddingConnector by @Jeadie in https://github.com/spiceai/spiceai/pull/1592
• readme change to reflect new cli output by @y-f-u in https://github.com/spiceai/spiceai/pull/1602
• Update version v0.13.2 by @ewgenius in https://github.com/spiceai/spiceai/pull/1604
• Roadmap changes Jun 3, 2024 by @lukekim in https://github.com/spiceai/spiceai/pull/1609

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.13.1-alpha...v0.13.2

Resources

• Getting started with Spice.ai
• Documentation

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

• Twitter: @spice_ai
• Discord: https://discord.gg/kZnTfneP5u
• Telegram: Spice AI Discussion
• Reddit: https://www.reddit.com/r/spiceai
• Email: [email protected]