Announcing the release of Spice v1.10.0! ⚡
Spice v1.10.0 introduces a new Caching Acceleration Mode with stale-while-revalidate (SWR) semantics for disk-persisted, low-latency queries with background refresh. This release also adds the TinyLFU eviction policy for the SQL results cache, a preview of the DynamoDB Streams connector for real-time CDC, S3 location predicate pruning for faster partitioned queries, improved distributed query execution, and multiple security hardening improvements.
Low-Latency Queries with Background Refresh: This release introduces a new caching acceleration mode that implements the stale-while-revalidate (SWR) pattern. Queries return cached results immediately while data refreshes asynchronously in the background, eliminating query latency spikes during refresh cycles. Cached data persists to disk using DuckDB, SQLite, or Cayenne file modes.
Key Features:
- Stale-While-Revalidate (SWR): Returns cached data immediately while refreshing in the background, reducing query latency
- Disk Persistence: Cached results persist across restarts using DuckDB, SQLite, or Cayenne file modes
- Configurable Refresh: Control refresh intervals with
refresh_check_interval to balance freshness and source load
Recommendation: Use retention configuration with caching acceleration to ensure stale data is cleaned up over time.
Example spicepod.yaml configuration:
datasets:
- from: http://localhost:7400
name: cached_data
time_column: fetched_at
acceleration:
enabled: true
engine: duckdb
mode: file
refresh_mode: caching
refresh_check_interval: 10m
retention_check_enabled: true
retention_period: 24h
retention_check_interval: 1h
For more details, refer to the Data Acceleration Documentation.
Higher Cache Hit Rates for SQL Results Cache: A new TinyLFU cache eviction policy is now available for the SQL results cache. TinyLFU is a probabilistic cache admission policy that maintains higher hit rates than LRU while keeping memory usage predictable, making it ideal for workloads with varying query frequency patterns.
Example spicepod.yaml configuration:
runtime:
caching:
sql_results:
enabled: true
eviction_policy: tiny_lfu
For more details, refer to the Caching Documentation and the Moka TinyLFU Documentation for details of the algorithm.
Real-Time Change Data Capture for DynamoDB: The DynamoDB connector now integrates with DynamoDB Streams for real-time change data capture (CDC). This enables continuous synchronization of DynamoDB table changes into Spice for real-time query, search, and LLM-inference.
Key Features:
- Real-Time CDC: Automatically captures inserts, updates, and deletes from DynamoDB tables as they occur
- Table Bootstrapping: Performs an initial full table scan before streaming changes, ensuring complete data consistency
- Acceleration Integration: Works with
refresh_mode: changes to incrementally update accelerated datasets
Note: DynamoDB Streams must be enabled on your DynamoDB table. This feature is in preview.
Example spicepod.yaml configuration:
datasets:
- from: dynamodb:my_table
name: orders_stream
acceleration:
enabled: true
refresh_mode: changes
For more details, refer to the DynamoDB Connector Documentation.
Spice can now push metrics to an OpenTelemetry collector, enabling integration with platforms such as Jaeger, New Relic, Honeycomb, and other OpenTelemetry-compatible backends.
Key Features:
- Protocol Support: Supports the gRPC (default port 4317) protocol
- Configurable Push Interval: Control how frequently metrics are pushed to the collector
Example spicepod.yaml configuration for gRPC:
runtime:
telemetry:
enabled: true
otel_exporter:
endpoint: 'localhost:4317'
push_interval: '30s'
For more details, refer to the Observability & Monitoring Documentation.
S3 Location Predicate Pruning: The S3 data connector now supports location-based predicate pruning, dramatically reducing data scanned by pushing down location filter predicates to S3 listing operations. For partitioned datasets (e.g., year=2025/month=12/), Spice now skips listing irrelevant partitions entirely, significantly reducing query latency and S3 API costs.
AWS S3 Tables Write Support: Full read/write capability for AWS S3 Tables, enabling direct integration with AWS's managed table format for S3. Use standard SQL INSERT INTO to write data.
For more details, refer to the S3 Data Connector Documentation and Glue Data Connector Documentation.
Distributed query planning and execution have been significantly improved:
- Fixed executor registration in cluster mode for more reliable distributed deployments
- Improved hostname resolution for Flight server binding, enabling better executor discovery
- Distributed accelerator registration: Data accelerators now properly register in distributed mode
- Optimized query planning:
DistributeFileScanOptimizer improvements for faster planning with large datasets
For more details, refer to the Distributed Query Documentation.
Search capabilities have been improved with several performance and reliability enhancements:
- Fixed FTS query blocking: Full-text search queries no longer block unnecessarily, improving query responsiveness
- Optimized vector index operations: Eliminated unnecessary
list_vectors calls for better performance
- Improved limit pushdown:
IndexerExec now properly handles limit pushdown for more efficient searches
For more details, refer to the Search Documentation.
Multiple security improvements have been implemented:
- SQL Identifier Quoting: Hardened SQL identifier quoting across all database connectors (PostgreSQL, MySQL, DuckDB, etc.) to prevent SQL injection attacks through table or column names
- Token Redaction: Sensitive authentication tokens are now fully redacted in debug and error output, preventing accidental credential exposure in logs
- Path Traversal Prevention: Fixed tar extraction operations to prevent directory traversal vulnerabilities when processing archived files
- Input Sanitization: Added strict validation for
top_n_sample order_by clause parsing to prevent injection attacks
- Glue Credential Handling: Prevented automatic loading of AWS credentials from environment in Glue connector, ensuring explicit credential configuration
- Health probe metrics: Added health probe latency metrics for better observability
- CLI improvements: Fixed
.clear history command in the REPL to fully clear persisted history
No breaking changes.
No major cookbook updates.
The Spice Cookbook includes 82 recipes to help you get started with Spice quickly and easily.
To upgrade to v1.10.0, use one of the following methods:
CLI:
Homebrew:
brew upgrade spiceai/spiceai/spice
Docker:
Pull the spiceai/spiceai:1.10.0 image:
docker pull spiceai/spiceai:1.10.0
For available tags, see DockerHub.
Helm:
helm repo update
helm upgrade spiceai spiceai/spiceai
AWS Marketplace:
🎉 Spice is now available in the AWS Marketplace!
