01 / Multimodal vector database

Search everything you can embed.

Drop in any document, image, audio, or video — Veda runs the whole pipeline (OCR, chunking, modality-routed embeddings, indexing, fused search) and makes every modality searchable in one cross-modal space, on a single pure-Rust engine.

$git clone https://github.com/kshitij2605/Veda.git && cd Veda
$docker compose up

The first run builds the image from source, so allow a few minutes.

index algorithms
mandatory services
distance metrics
modalities

02 / Why Veda

Built multimodal-first, end to end

Most vector databases just store vectors. Veda owns the whole path — from raw files to fused, cross-modal search.

01

Multimodal-native pipeline

Upload a PDF, image, audio, or video — Veda parses it, chunks the text, and routes every modality to its embedder; point it at one unified omni model and a text query retrieves audio, video, and images from a single cross-modal space. Vectors, metadata, and blobs stored together — no external services to stitch.

02

Pure-Rust + SIMD

API to disk in one Rust binary, with SimSIMD distance kernels (AVX-512, NEON, SVE) in the hot loop — up to ~10× faster than scalar. No GC pauses, no CGo boundary, zero-copy mmap reads.

03

11 vector index types

HNSW, DiskANN, IVF-PQ, Flat, three RaBitQ variants, Sq4 + Sq8 scalar quantization — pick the recall, latency and memory trade-off per collection instead of working around one index. A suffix array and scalar filter indexes sit alongside them, and cuVS builds add a GPU CAGRA index that builds a 1M-vector index in 86 seconds.

04

Multi-vector fusion

Search text, image, and metadata spaces at once and fuse the results with RRF, weighted sum, or intersection — in a single API call.

05

Built-in document intelligence

Drop in a PDF, an Office file (DOCX / PPTX / XLSX), or a scan — Veda’s OCR pipeline (GLM-OCR + layout analysis) extracts the tables and images, chunks, and embeds it into searchable vectors in one API call. Zero external extraction tooling to stitch.

06

Lightweight by default

Three mandatory services — Veda, MinIO, NATS — with no external coordinator, message broker, or metadata store to provision. One compose file brings the whole thing up, and there is nothing else to stand up before you can index your first document.

07

Observability + built-in AutoOps

Metrics, traces, and logs out of the box — Prometheus, Tempo, Loki, Grafana — plus an Ops Advisor that detects issues across 36 rules (latency, disk, CPU, error & log anomalies, SLO burn-rate, recall drift, capacity forecasts, change-points, LLM cost & quality), explains the root cause, alerts Slack/PagerDuty, and can auto-remediate.

08

An app platform, not just an index

JSON-path filters (meta.user.tier = "gold"), per-record row-level security, record TTL with scheduled sweeps, cron jobs, and OLAP-style aggregations — built into the engine.

09

Agents & MCP server

Connect Claude Desktop, Cursor, or any MCP client to your data — and get cross-modal retrieval no text-only vector DB can offer. Built-in agents, an 8-tool catalog, and A2A, all behind Veda’s JWT auth and row-level security.

10

SLOs & service reliability

Define service-level objectives with error budgets and Google-SRE multi-window burn-rate alerting, run synthetic self-checks against Veda’s own search / embed / health paths, and triage any incident with an AI ops agent that reads your signals and explains the root cause.

11

GenAI tracing + cost observability

Agent and LLM calls emit OpenTelemetry GenAI-semantic-convention spans that drop natively into Datadog, Grafana Tempo, Phoenix, and Langfuse — with per-run token, latency, and USD cost metrics, and W3C context propagation that stitches Veda into your existing distributed trace.

12

Grounded agents with faithfulness scoring

Every agent answer returns citations to the exact retrieved chunks and a 0–1 faithfulness + relevancy score (LLM-judge, embedding fallback) — the #1 RAG quality metric, scorable only because Veda owns the retriever. Plus an eval harness that graduates production traces into regression tests.

13

AIOps: forecasting + incident correlation

Veda forecasts disk-full, memory, and recall-decay ETAs, detects sustained change-points, and semantically clusters related alerts into single incidents — flagging unprecedented ones — while an AI agent drafts the root cause from a cross-signal bundle (logs + queries + traces + resources). Predictive, causal, vector-native AIOps a metrics store without an embedder can’t do.

14

Self-hosted web RAG

Crawl the open web straight into a collection (crawl4ai → clean markdown → chunk → embed) and give agents web_search + fetch_url tools to search → fetch → ground over live pages — all on open-source, self-hosted services. No external API, no data leaving your trust boundary.

15

Semantic LLM cache & guardrails

Veda turns its vector engine into the LLM layer: a near-duplicate prompt is served from a semantic cache with no LLM call, an unsafe prompt is blocked by embedding-similarity guardrails, and every run lands in a trace store you search and cluster by meaning — observe, optimize, and secure in one engine.

16

Answers that span documents

Ask which plant assembles a product and whether that plant is ISO 14001 certified, and no single document holds the answer. Veda’s agent searches, reads the name out of the result, and searches again. On a 2,012-document corpus with Qwen3.5-122B it answered two-document and one-to-many joins correctly in about two searches. No knowledge graph, no query planner, nothing to keep in sync.

17

Guardrails on what you retrieve

The dangerous instruction is not always in the prompt — it can be sitting in a PDF someone uploaded. Veda screens retrieved chunks at ingest, reusing the embedding it already computed, and drops the ones it flags at the tool-dispatch choke point, so every agent surface is covered by one mechanism. Raw search annotates instead of dropping, so recall stays measurable. A defence-in-depth layer, and the docs are explicit about what it does and does not catch.

18

Query your Parquet where it lives

Register a collection over existing Parquet files and search them in place — no ingestion, no second copy, no duplicate storage. Schema and vector dimension are read from the footer and validated up front, filters use the same evaluator as internal collections, and search is an exact scan at recall 1.0.

19

Runs entirely on your infrastructure

The embedder, OCR, reranker, web search, and the LLM endpoint are all self-hostable and swappable, so the whole stack can run air-gapped on your own hardware. No data leaves your trust boundary and nothing phones home — the privacy and sovereignty posture regulated and on-prem deployments need.

20

Self-tuning RAG pipeline

Veda generates a labeled test set from your own corpus, then searches its configuration space — chunker, embedder, index, fusion, rerank — for the best recall / faithfulness / latency trade-off. Apply the winner with one click (reversible), and optionally auto-re-tune every time the collection grows.

03 / Developer experience

The whole pipeline, one API call

No glue code, no external embedding or OCR services. Create a multimodal collection, fuse across embedding spaces, or turn a PDF into searchable vectors — each is a single request.

curl -X POST localhost:6340/v1/collections \
  -H 'content-type: application/json' \
  -d '{
    "name": "products",
    "fields": [
      { "name": "text_embedding",  "type": "vector", "dimension": 1024,
        "metric": "cosine", "model": "BAAI/bge-m3" },
      { "name": "image_embedding", "type": "vector", "dimension": 768 },
      { "name": "title", "type": "string", "bm25": true },
      { "name": "price", "type": "float32", "filterable": true }
    ]
  }'

04 / How it works

From raw files to search, in one pipeline

Veda's document endpoint runs the entire ingestion path natively — no Unstructured.io, no LangChain loaders, no glue code.

01

Upload

PDF, image, audio, video, or text — one endpoint.

02

OCR

GLM-OCR + FlashParse extract text, tables, and figures.

03

Chunk

Seven strategies, from markdown-recursive to semantic.

04

Embed

Each modality auto-routed to the right model.

05

Index

HNSW, DiskANN, IVF-PQ, RaBitQ — your choice.

06

Search

Dense, sparse, hybrid, and multi-vector fusion.

06 / Architecture

Lightweight by default, deep when you need it

Three mandatory services get you running in seconds. Eleven vector index types let you tune recall, latency, and memory per collection.

Three services. That's it.

No external coordination service, message broker, or metadata store to provision — the whole engine runs on three:

  • Veda — API, indexing, search — one Rust binary
  • MinIO — S3-compatible blobs + metrics storage
  • NATS — Sub-millisecond message queue, ~20MB

One compose file brings all three up together — see the quick start.

IndexBest forAdvantage
HNSWGeneral-purpose, low-latencyTunable M / ef
DiskANNBillion-scale beyond RAMVamana graph on SSD
IVF-PQMemory-constrained throughputPQ compresses 8–32×
FlatSmall sets, 100% recallBrute-force baseline
RaBitQUltra-fast screeningBinary + Hamming
Sq44× compression, strong recall4-bit scalar quant
Sq8Highest-recall quantized8-bit scalar quant
Suffix arrayExact text patternsHybrid text + vector
ScalarMetadata filteringRange / inverted / bitmap

06 / Get started

Run it on your own machine

Clone the repo, bring up three services, and ingest your first multimodal document. The quick start walks through it end to end.