Vector Engines

🎓 Learn how it works with the Amazon S3 Vectors with Spice engineering blog post.

Data sourced by Data Connectors, or views built atop them with vector embedding columns can be indexed and efficiently searched using a vector engine.

A vector engine will store all vector embeddings associated with columns in a dataset/view, provide efficient vector search operations and avoid unnecessary recomputation of embeddings.

A vector engine is configured by setting the vectors configuration. E.g.

datasets:
  - name: dataset_with_embeddings
    vectors:
      enabled: true

For the complete reference specification see datasets.

Supported Vector engines:

Name	Description
s3_vectors	AWS S3 vectors

Limitations

• A dataset or view must be accelerated (i.e. .dataset[].accelerated.enabled: true, see docs) for a vector engine to be provided the appropriate data to ingest.

Vector Engine Docs

📄️ Amazon S3 Vectors

Amazon S3 Vectors Engine Documentation