Results Caching

Spice uses in-memory caching for query results, which is enabled by default for both the HTTP (/v1/sql) and Arrow Flight APIs.

Results caching improves performance for repeated requests and non-accelerated results, such as refresh data returned on zero results.

The cache uses a least-recently-used (LRU) replacement policy. You can configure the cache to set an item expiration duration, which defaults to 1 second.

version: v1
kind: Spicepod
name: app

runtime:
  results_cache:
    enabled: true
    cache_max_size: 128MiB
    item_ttl: 1s

Parameter name	Optional	Description
enabled	Yes	Defaults to true.
cache_max_size	Yes	Maximum cache size. Defaults to 128MiB.
eviction_policy	Yes	Cache replacement policy when the cache reaches cache_max_size. Defaults to lru, which is currently the only supported value.
item_ttl	Yes	Cache entry expiration duration (Time to Live). Defaults to 1 second.
cache_key_type	Yes	Determines how cache keys are generated. Defaults to plan. plan uses the query's logical plan, while sql uses the raw SQL query string.

Choosing a cache_key_type

• plan (Default): Uses the query's logical plan as the cache key. Matches semantically equivalent queries but requires query parsing.
• sql: Uses the raw SQL string as the cache key. Provides faster lookups but requires exact string matches. Queries with dynamic functions, such as NOW(), may produce unexpected results. Use sql only when results are predictable.

Use sql for the lowest latency with identical queries that do not include dynamic functions. Use plan for greater flexibility.

Cached Responses

The response includes a Results-Cache-Status header that indicates the cache status of the query:

Header value	Description
HIT	The query result was served from the cache.
MISS	The cache was checked, but the result was not found.
BYPASS	The cache was bypassed for this query (e.g., when cache-control: no-cache is specified).
header not present	The cache did not apply to this query (e.g., when caching is disabled or querying a system table).

Examples

Cached Response

$ curl -XPOST -i http://localhost:8090/v1/sql -d 'select * from taxi_trips limit 1;'
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
results-cache-status: HIT
vary: origin, access-control-request-method, access-control-request-headers
content-length: 416
date: Thu, 13 Feb 2025 03:05:39 GMT

Uncached Response

$ curl -XPOST -i http://localhost:8090/v1/sql -d 'select * from taxi_trips limit 1;'
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
results-cache-status: MISS
vary: origin, access-control-request-method, access-control-request-headers
content-length: 416
date: Thu, 13 Feb 2025 03:13:19 GMT

Bypassed Cache with cache-control: no-cache

$ curl -H "cache-control: no-cache" -XPOST -i http://localhost:8090/v1/sql -d 'select * from taxi_trips limit 1;'
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
results-cache-status: BYPASS
vary: origin, access-control-request-method, access-control-request-headers
content-length: 416
date: Thu, 13 Feb 2025 03:14:00 GMT

Cache Control

You can control caching behavior for specific queries using HTTP headers. The Cache-Control header helps skip the cache for a query while caching the results for subsequent queries.

HTTP/Flight API

The SQL query API endpoints (HTTP and Arrow Flight) support the standard HTTP Cache-Control header. The no-cache directive skips the cache for the current query but caches the results for future queries.

Other Cache-Control directives are not supported.

HTTP Example

# Default behavior (uses cache)
curl -XPOST http://localhost:8090/v1/sql -d 'SELECT 1'

# Skip cache for this query, but cache the results for future queries
curl -H "cache-control: no-cache" -XPOST http://localhost:8090/v1/sql -d 'SELECT 1'

Arrow FlightSQL Example

The following example skips the cache for a specific query using FlightSQL in Rust:

let sql_command = arrow_flight::sql::CommandStatementQuery {
    query: "SELECT 1".to_string(),
    transaction_id: None,
};
let sql_command_bytes = sql_command.as_any().encode_to_vec();

let mut request = FlightDescriptor::new_cmd(sql_command_bytes).into_request();

request
  .metadata_mut()
  .insert("cache-control", "no-cache");

// Send the request

spice sql CLI

The spice sql command accepts a --cache-control flag that follows the same behavior as the HTTP Cache-Control header:

# Default behavior (use cache if available)
spice sql
# Same as above
spice sql --cache-control cache
# Skip cache for this query, but cache the results for future queries
spice sql --cache-control no-cache