Skip to main content

2 posts tagged with "opentelemetry"

OpenTelemetry topics and usage

View All Tags

Spice v2.0-rc.3 (Apr 21, 2026)

· 13 min read
Evgenii Khramkov
Evgenii Khramkov
Senior Software Engineer at Spice AI

Announcing the release of Spice v2.0-rc.3! ⚡

v2.0.0-rc.3 is the third release candidate for advanced testing of v2.0, building on v2.0.0-rc.2.

Highlights in this release candidate include:

  • HTTP Connector Enhancements with OAuth2 refresh-token authentication, query-parameter pagination, and map-to-array conversion for broader API compatibility
  • Databricks and Unity Catalog Reliability Improvements with resilience controls, improved UC-awareness, permission checks, and structured error reporting
  • Snowflake and ADBC Registration Performance Improvements with better observability during dataset registration
  • OpenTelemetry Exporter Improvements with exporter fixes and support for authenticated metrics export
  • Kafka, GitHub, and HTTP Connector Fixes including Kafka reliability improvements, GitHub GraphQL resilience updates, and HTTP JSON union/reload fixes

What's New in v2.0.0-rc.3​

HTTP Connector Enhancements​

The HTTP connector now supports more authentication and API response patterns, making it easier to integrate with modern REST APIs.

Key improvements:

  • OAuth2 Refresh-Token Authentication: Added support for OAuth2 refresh-token flows for APIs that issue short-lived access tokens.
  • Query-Parameter Pagination: Added pagination support using query parameters for APIs that expose page or cursor controls in the URL.
  • Map-to-Array Conversion: Added response transformation support for APIs that return map-shaped payloads that need to be normalized into arrays.
  • Improved JSON Union Handling: Better handling for heterogeneous JSON payloads during ingestion.
  • More Reliable Reloads: Fixed runtime behavior for HTTP-backed datasets during spicepod reloads.

Example configuration of an HTTP connector using the OAuth2 refresh token flow:

datasets:
  - from: https://api.example.com
    name: secure_data
    params:
      file_format: json
      allowed_request_paths: '/v1/**'
      auth_token_url: https://auth.example.com/oauth/token
      http_auth_refresh_token: ${secrets:my_refresh_token}
      http_auth_client_id: ${secrets:my_client_id}
      http_auth_client_secret: ${secrets:my_client_secret}
      auth_scopes: 'read:data offline_access'

Databricks and Unity Catalog Reliability Improvements​

Databricks and Unity Catalog integrations are now more resilient and provide clearer behavior in permission-constrained environments.

Key improvements:

  • Resilience Controls: Added controls to improve reliability when interacting with Databricks services.
  • Unity Catalog Awareness: Improved handling for Unity Catalog-specific behaviors and mixed deployment configurations.
  • Permission Prechecks: Databricks UC permission checks now distinguish explicit denials from ambiguous cases.
  • Structured Error Reporting: Advisory permission failures now surface with more actionable structured errors.
  • Classic SQL Warehouse Compatibility: Improved handling for foreign table scenarios with Classic SQL Warehouse combinations.
  • Task History Instrumentation: Added instrumentation to improve observability for Databricks-related operations.

Snowflake and ADBC Improvements​

Snowflake and ADBC-backed dataset registration is now faster and easier to observe.

Key improvements:

  • Faster Dataset Registration: Improved registration performance for Snowflake and ADBC datasets.
  • Better Observability: Added better instrumentation and visibility into registration workflows.
  • ADBC Alignment: Updated ADBC dependencies and integration points for improved compatibility.
  • Search Schema Fix: Fixed a full-text search schema mismatch issue with the ADBC connector.

OpenTelemetry and Observability Improvements​

Spice improves telemetry export reliability and authenticated metrics delivery for the OpenTelemetry integration.

Key improvements:

  • OTEL Exporter Fixes: Fixed issues in the OpenTelemetry exporter.
  • Authenticated Metrics Export: Added support for authorization headers in the OTEL metrics exporter.
  • Reduced Startup Noise: Suppressed unnecessary AWS SDK noise and improved OTEL-related initialization behavior.
  • Connector Initialization Reliability: Fixed issues that could block connector initialization in telemetry-related code paths.

Dependency and Toolchain Updates​

Dependency / ComponentVersion / Update
Rust toolchainv1.94.1 (from v1.93.1)
DataFusionv52.5.0-rc1
mistral.rs / candlemistral.rs v0.8.x, candle v0.10.1
ADBC Corev0.23

Other Improvements​

  • Improved Query Pushdown: Expanded sort and limit pushdown, including improved pushdown behavior for Oracle and MSSQL connectors.
  • Partitioned Query Planning Improvements: Improved PartitionedTableScanRewrite handling for ORDER BY, partition expressions, and fully qualified table references, while preventing incorrect bucketing partition pushdown to executors.
  • MongoDB SRV Support: Upgraded datafusion-table-providers with MongoDB SRV support.
  • Tantivy Logging: Search logging now defaults to warn unless very verbose logging is enabled.
  • Kafka Connector Fixes: Improved reliability for the Kafka connector behavior.
  • GitHub Connector Resilience: Improved commit fetching for dynamic and slash refs, and reduced GraphQL page sizes on gateway errors for the GitHub connector.
  • GitHub API Efficiency: Lowered default comment fetch sizes to reduce pressure on GitHub GraphQL APIs.
  • Embedding Validation: Added validation for embedding row_id columns during dataset initialization.
  • View Cache Invalidation: Cached plans are now cleared when views are updated.
  • Refresh SQL Dedup Fix: Fixed append refresh deduplication when refresh_sql selects a subset of columns.

Contributors​

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.0.0-rc.3, use one of the following methods:

CLI:

spice upgrade v2.0.0-rc.3

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.0.0-rc.3 image:

docker pull spiceai/spiceai:2.0.0-rc.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.0.0-rc.3

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

  • fix: Full Text Search schema mismatch with ADBC connector by @lukekim in #10235
  • docs: Update v2.0.0-rc.2 release notes with latest changes by @lukekim in #10238
  • Fix append refresh dedup failure when refresh_sql selects column subset by @sgrebnov in #10225
  • Revert "Properly mark dataset as Ready on Scheduler (#10215)" by @sgrebnov in #10242
  • Fix failing merge conflicts for benchmarks by @krinart in #10247
  • fix(github): fetch commits for dynamic and slash refs by @lukekim in #10233
  • Upgrade DataFusion to v52.5.0-rc1 by @lukekim in #10249
  • Merge develop to trunk (2026-04-09) by @claudespice in #10248
  • fix: Validate embedding row_id columns during dataset init (fixes #8226) by @claudespice in #10208
  • fix: Update tpch benchmark snapshots for federated/glue[csv].yaml by @app/github-actions in #10244
  • feat(databricks): add resilience controls, UC awareness, and task history instrumentation by @lukekim in #10246
  • fix: Make PartitionManager resilient to bare vs fully qualified table references by @sgrebnov in #10257
  • fix: Update tpch benchmark snapshots for accelerated/s3[parquet]-cayenne[file].yaml by @app/github-actions in #10256
  • Merge develop to trunk (2026-04-10) by @claudespice in #10251
  • Improve Snowflake/ADBC dataset registration performance and observability by @lukekim in #10266
  • Fixes for kafka connector by @krinart in #10263
  • fix(runtime): gate otel code tags, suppress aws sdk noise, and unblock connector init by @lukekim in #10260
  • fix(runtime): avoid regionless AWS SDK loads by @lukekim in #10271
  • Add versioned release install workflow coverage by @lukekim in #10276
  • fix(runtime): handle HTTP JSON unions and spicepod reloads by @lukekim in #10277
  • Databricks UC permission prechecks: explicit denial as permanent error, ambiguous cases advisory by @lukekim in #10274
  • Revert component status changes re-introduced by develop merge (#10248) by @sgrebnov in #10293
  • Fix broken CI workflows by @ewgenius in #10294
  • Group dependabot updates by ecosystem by @lukekim in #10296
  • fix(tests): Replace flaky S3 Vectors snapshot tests with structural validation by @lukekim in #10301
  • Update test_github_workflows snapshot by @lukekim in #10304
  • fix(ci): fix Bedrock runner mismatch and snapshot auto-merge failure by @ewgenius in #10306
  • feat(http): Add map-to-array conversion and query-parameter pagination by @lukekim in #10295
  • New crate: datafusion-ddl by @Jeadie in #10205
  • Make Databricks UC permission checks advisory with structured error reporting by @lukekim in #10283
  • build(deps): bump the github-actions-dependencies group with 4 updates by @app/dependabot in #10298
  • fix: Clear cached plans on view updates by @peasee in #10312
  • build(deps): bump the aws-sdk group with 7 updates by @app/dependabot in #10299
  • Code out of runtime. by @Jeadie in #10178
  • fix: Respect function registry denies for accelerated table filter pushdown by @peasee in #10311
  • fix: Don't block heartbeat when all slots acquired by @peasee in #10322
  • fix: strip only outer parens in get_table_partition_expr_from_ctx by @Jeadie in #10323
  • Upgrade datafusion-table-providers with MongoDB SRV support by @lukekim in #10317
  • fix: Avoid pushing down bucketing partition expressions into executors by @peasee in #10324
  • Upgrade datafusion-table-providers to d1b911a5 and bump adbc to 0.23 by @lukekim in #10329
  • fix: Update Search integration test snapshots by @app/github-actions in #10308
  • Handle foreign table + Classic sql warehouse combination gracefully by @krinart in #10318
  • New crate datafusion-flightsql by @Jeadie in #10201
  • Set tantivy=warn unless very verbose logging by @Jeadie in #10338
  • Remove image registry and image name options from spidapter by @ewgenius in #10241
  • build(deps): bump sysinfo from 0.37.2 to 0.38.4 by @app/dependabot in #10291
  • build(deps): bump futures from 0.3.31 to 0.3.32 by @app/dependabot in #10289
  • New crate 'datafusion-dml' by @Jeadie in #10334
  • Jeadie/26 04 16/spice sql by @Jeadie in #10343
  • Add Teraswitch/Pittsburgh apt mirrors + retry config for CI runners by @lukekim in #10349
  • Implement sort pushdown and fix pushdown gaps across providers by @lukekim in #10337
  • Merge develop to trunk (2026-04-16) by @claudespice in #10345
  • Update candle and mistral.rs lock-step pins by @lukekim in #10278
  • docs: fix status badges in README by @lukekim in #10350
  • Migrate secrets to vars by @krinart in #10354
  • Add limit pushdown and improve sort pushdown for Oracle and MSSQL by @sgrebnov in #10351
  • Fix ubuntu mirror configuration by @ewgenius in #10359
  • fix: Increase throughput test default ready_wait from 30s to 300s (fixes #8207) by @claudespice in #10344
  • Add auth headers support to OTEL metrics exporter by @lukekim in #10347
  • fix(github): shrink GraphQL page size on gateway errors; lower comment defaults by @lukekim in #10355
  • Relax apt mirror substitution failure to warning in CI action by @ewgenius in #10361
  • feat(http): Add OAuth2 refresh-token auth to HTTP connector by @lukekim in #10348
  • Upgrade Rust toolchain to 1.94.1 by @lukekim in #10353
  • Handle order by and sort in PartitionedTableScanRewrite by @Jeadie in #9656
  • Fix OTEL Exporter by @krinart in #10363
  • Pin spiceai candle / TEI forks to merged revs; drop local [patch] overrides by @lukekim in #10362

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.0.0-rc.2...v2.0.0-rc.3

Spice v1.10.0 (Dec 9, 2025)

· 18 min read
William Croxson
William Croxson
Senior Software Engineer at Spice AI

Announcing the release of Spice v1.10.0! ⚡

Spice v1.10.0 introduces a new Caching Acceleration Mode with stale-while-revalidate (SWR) semantics for disk-persisted, low-latency queries with background refresh. This release also adds the TinyLFU eviction policy for the SQL results cache, a preview of the DynamoDB Streams connector for real-time CDC, S3 location predicate pruning for faster partitioned queries, improved distributed query execution, and multiple security hardening improvements.

What's New in v1.10.0​

Caching Acceleration Mode​

Low-Latency Queries with Background Refresh: This release introduces a new caching acceleration mode that implements the stale-while-revalidate (SWR) pattern. Queries return cached results immediately while data refreshes asynchronously in the background, eliminating query latency spikes during refresh cycles. Cached data persists to disk using DuckDB, SQLite, or Cayenne file modes.

Key Features:

  • Stale-While-Revalidate (SWR): Returns cached data immediately while refreshing in the background, reducing query latency
  • Disk Persistence: Cached results persist across restarts using DuckDB, SQLite, or Cayenne file modes
  • Configurable Refresh: Control refresh intervals with refresh_check_interval to balance freshness and source load

Recommendation: Use retention configuration with caching acceleration to ensure stale data is cleaned up over time.

Example spicepod.yaml configuration:

datasets:
  - from: http://localhost:7400
    name: cached_data
    time_column: fetched_at
    acceleration:
      enabled: true
      engine: duckdb
      mode: file # Persist cache to disk
      refresh_mode: caching
      refresh_check_interval: 10m
      retention_check_enabled: true
      retention_period: 24h
      retention_check_interval: 1h

For more details, refer to the Data Acceleration Documentation.

TinyLFU Cache Eviction Policy​

Higher Cache Hit Rates for SQL Results Cache: A new TinyLFU cache eviction policy is now available for the SQL results cache. TinyLFU is a probabilistic cache admission policy that maintains higher hit rates than LRU while keeping memory usage predictable, making it ideal for workloads with varying query frequency patterns.

Example spicepod.yaml configuration:

runtime:
  caching:
    sql_results:
      enabled: true
      eviction_policy: tiny_lfu # default: lru

For more details, refer to the Caching Documentation and the Moka TinyLFU Documentation for details of the algorithm.

DynamoDB Streams Data Connector (Preview)​

Real-Time Change Data Capture for DynamoDB: The DynamoDB connector now integrates with DynamoDB Streams for real-time change data capture (CDC). This enables continuous synchronization of DynamoDB table changes into Spice for real-time query, search, and LLM-inference.

Key Features:

  • Real-Time CDC: Automatically captures inserts, updates, and deletes from DynamoDB tables as they occur
  • Table Bootstrapping: Performs an initial full table scan before streaming changes, ensuring complete data consistency
  • Acceleration Integration: Works with refresh_mode: changes to incrementally update accelerated datasets

Note: DynamoDB Streams must be enabled on your DynamoDB table. This feature is in preview.

Example spicepod.yaml configuration:

datasets:
  - from: dynamodb:my_table
    name: orders_stream
    acceleration:
      enabled: true
      refresh_mode: changes # Enable Streams capture

For more details, refer to the DynamoDB Connector Documentation.

OpenTelemetry Metrics Exporter​

Spice can now push metrics to an OpenTelemetry collector, enabling integration with platforms such as Jaeger, New Relic, Honeycomb, and other OpenTelemetry-compatible backends.

Key Features:

  • Protocol Support: Supports the gRPC (default port 4317) protocol
  • Configurable Push Interval: Control how frequently metrics are pushed to the collector

Example spicepod.yaml configuration for gRPC:

runtime:
  telemetry:
    enabled: true
    otel_exporter:
      endpoint: 'localhost:4317'
      push_interval: '30s'

For more details, refer to the Observability & Monitoring Documentation.

S3 Connector Improvements​

S3 Location Predicate Pruning: The S3 data connector now supports location-based predicate pruning, dramatically reducing data scanned by pushing down location filter predicates to S3 listing operations. For partitioned datasets (e.g., year=2025/month=12/), Spice now skips listing irrelevant partitions entirely, significantly reducing query latency and S3 API costs.

AWS S3 Tables Write Support: Full read/write capability for AWS S3 Tables, enabling direct integration with AWS's managed table format for S3. Use standard SQL INSERT INTO to write data.

For more details, refer to the S3 Data Connector Documentation and Glue Data Connector Documentation.

Faster Distributed Query Execution​

Distributed query planning and execution have been significantly improved:

  • Fixed executor registration in cluster mode for more reliable distributed deployments
  • Improved hostname resolution for Flight server binding, enabling better executor discovery
  • Distributed accelerator registration: Data accelerators now properly register in distributed mode
  • Optimized query planning: DistributeFileScanOptimizer improvements for faster planning with large datasets

For more details, refer to the Distributed Query Documentation.

Search Improvements​

Search capabilities have been improved with several performance and reliability enhancements:

  • Fixed FTS query blocking: Full-text search queries no longer block unnecessarily, improving query responsiveness
  • Optimized vector index operations: Eliminated unnecessary list_vectors calls for better performance
  • Improved limit pushdown: IndexerExec now properly handles limit pushdown for more efficient searches

For more details, refer to the Search Documentation.

Security Hardening​

Multiple security improvements have been implemented:

  • SQL Identifier Quoting: Hardened SQL identifier quoting across all database connectors (PostgreSQL, MySQL, DuckDB, etc.) to prevent SQL injection attacks through table or column names
  • Token Redaction: Sensitive authentication tokens are now fully redacted in debug and error output, preventing accidental credential exposure in logs
  • Path Traversal Prevention: Fixed tar extraction operations to prevent directory traversal vulnerabilities when processing archived files
  • Input Sanitization: Added strict validation for top_n_sample order_by clause parsing to prevent injection attacks
  • Glue Credential Handling: Prevented automatic loading of AWS credentials from environment in Glue connector, ensuring explicit credential configuration

Developer Experience Improvements​

  • Health probe metrics: Added health probe latency metrics for better observability
  • CLI improvements: Fixed .clear history command in the REPL to fully clear persisted history

Contributors​

Breaking Changes​

No breaking changes.

Cookbook Updates​

No major cookbook updates.

The Spice Cookbook includes 82 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.10.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.10.0 image:

docker pull spiceai/spiceai:1.10.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed​

Changelog​