One post tagged with "security"

Announcing the release of Spice v1.10.1! 🚀

v1.10.1 is a patch release with Cayenne accelerator improvements including configurable compression strategies and improved partition ID handling, isolated refresh runtime for better query API responsiveness, and security hardening. In addition, the GO SDK, gospice v8 has been released.

What's New in v1.10.1​

Cayenne Accelerator Improvements​

Several improvements and bug fixes for the Cayenne data accelerator:

• Compression Strategies: The new cayenne_compression_strategy parameter enables choosing between zstd for compact storage or btrblocks for encoding-efficient compression.
• Improved Vortex Defaults: Aligned Cayenne to Vortex footer configuration for better compatibility.
• Partition ID Handling: Improved partition ID generation to avoid potential locking race conditions.

Example spicepod.yaml configuration:

datasets:
  - from: s3://my-bucket/data.parquet
    name: my_dataset
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      params:
        cayenne_compression_strategy: zstd # or btrblocks (default)

For more details, refer to the Cayenne Data Accelerator Documentation.

Isolated Refresh Runtime​

Refresh tasks now run on a separate Tokio runtime isolated from the main query API. This prevents long-running or resource-intensive refresh operations from impacting query latency and ensures the /health endpoint remains responsive during heavy refresh workloads.

Security Hardening​

Multiple security improvements have been implemented:

• Recursion Depth Limits: Added limits to DynamoDB and S3 Vectors integrations to prevent stack overflow from deeply nested structures, mitigating potential DoS attacks.
• Spicepod Summary API: The GET /v1/spicepods endpoint now returns summarized information instead of full spicepod.yaml representations, preventing potential sensitive information leakage.

Additional Improvements & Bug Fixes​

• Performance: Fixed double hashing of user supplied cache keys, improving cache lookup efficiency.
• Reliability: Fixed idle DynamoDB Stream handling for more stable CDC operations.
• Reliability: Added warnings when multiple partitions are defined for the same table.
• Performance: Eagerly drop cached records for results larger than max cache size.

Spice Go SDK v8​

The Spice Go SDK has been upgraded to v8 with a cleaner API, parameterized queries, and health check methods: gospice v8.0.0.

Key Features:

• Cleaner API: New Sql() and SqlWithParams() methods with more intuitive naming.
• Parameterized Queries: Safe, SQL-injection-resistant queries with automatic Go-to-Arrow type inference.
• Typed Parameters: Explicit type control with constructors like Decimal128Param, TimestampParam, and more.
• Health Check Methods: New IsSpiceHealthy() and IsSpiceReady() methods for instance monitoring.
• Upgraded Dependencies: Apache Arrow v18 and ADBC Go driver v1.3.0.

Example usage with a local Spice runtime:

import "github.com/spiceai/gospice/v8"

// Initialize client for local runtime
spice := gospice.NewSpiceClient()
defer spice.Close()

if err := spice.Init(
    gospice.WithFlightAddress("grpc://localhost:50051"),
); err != nil {
    panic(err)
}

// Parameterized query (safe from SQL injection)
reader, err := spice.SqlWithParams(
    ctx,
    "SELECT * FROM users WHERE id = $1 AND created_at > $2",
    userId,
    startTime,
)

Upgrade:

go get github.com/spiceai/gospice/[email protected]

For more details, refer to the Go SDK Documentation.

Contributors​

• @phillipleblanc
• @lukekim
• @peasee
• @krinart
• @jeadie
• @sgrebnov
• @mach-kernel

Breaking Changes​

• GET /v1/spicepods no longer returns the full spicepod.yaml JSON representation. A summary is returned instead. See #8404.

Cookbook Updates​

No major cookbook updates.

The Spice Cookbook includes 82+ recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.10.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.10.1 image:

docker pull spiceai/spiceai:1.10.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed​

Changelog​

• Return summarized spicepods from /v1/spicepods by @phillipleblanc in #8404
• DynamoDB tests and fixes by @lukekim in #8491
• Use an isolated Tokio runtime for refresh tasks that is separate from the main query API by @phillipleblanc in #8504
• fix: Avoid double hashing cache key by @peasee in #8511
• fix: Remove unused Cayenne parameters by @peasee in #8500
• feat: Support vortex zstd compressor by @peasee in #8515
• Fix for idle DynamoDB Stream by @krinart in #8506
• fix: Improve Cayenne errors, ID selection for table/partition creation by @peasee in #8523
• Update dependencies by @phillipleblanc in #8513
• Upgrade to gospice v8 by @lukekim in #8524
• fix: Add recursion depth limits to prevent DoS via deeply nested data (DynamoDB + S3 Vectors) by @phillipleblanc in #8544
• fix: Add warning when multiple partitions are defined for the same table by @peasee in #8540
• fix: Eagerly drop cached records for results larger than max by @peasee in #8516
• DDB Streams Integration Test + Memory Acceleration + Improved Warning by @krinart in #8520
• fix(cluster): initialize secrets before object stores in executor by @sgrebnov in #8532
• Show user-friendly error on empty DDB table by @krinart in #8586
• Move 'test_projection_pushdown' to runtime-datafusion by @Jeadie in #8490
• Fix stats for rewritten DistributeFileScanOptimizer plans by @mach-kernel in #8581