[Projects]() / Vector Search Portability # Vector Search Portability Real hnswlib and sqlite-vec comparison with identical vectors, filters and updates. Recall against exact search, concurrent read load and postfilter result loss. Author: [Lucio Durán]() · Python 3.11.13 · License: [MIT]() Page published: 2026-10-21 · Page updated: 2026-10-21 [Repository]() · [Original evidence]() · [Release v0.1.0]() ## Problem and relevance to agents An engine migration can preserve result counts while changing IDs, filter costs or behavior after updates. This project defines a shared contract and measures those differences. An agent retrieving filtered context depends on filter semantics as well as ranking. The comparison exposes IDs, distances, predicates and candidate loss before a failure is attributed to the model. ## Architecture 1. **Frozen input.** 5,000 float32 vectors with 64 dimensions, 40 queries and 500 vector-plus-metadata updates. The published NPZ fixes the input bytes. 2. **Native adapters.** hnswlib 0.8.0 and sqlite-vec 0.1.6 share squared L2 distance and AND filters. Exhaustive NumPy float64 search provides an independent reference. 3. **Processes and traces.** Three isolated repetitions per engine alternate execution order. Records cover native search and postfiltering, one- and four-client load, CPU, RSS and snapshots. - [Design and methodology]() ## What this demonstrates - Comparable engine contracts with explicit metrics and filter semantics. - Recall evaluation before and after vector and metadata changes. - Adapter-cost analysis with preserved original and confirmation measurements. ## Published results Reference results on Intel Xeon W-3223, macOS and Python 3.11.13. Load tables show medians of three trials, including medians of trial percentiles rather than pooled percentiles. Each load trial performs 400 mixed-filter searches after updates. Recall@10 compares IDs with the exact top results. Result loss measures missing items relative to min(10, eligible candidates). Postfiltering starts from 40 global neighbors. ### Original read-load measurements | Engine / clients | Requests/s | p50 (ms) | p95 (ms) | | --- | --- | --- | --- | | hnswlib / 1 | 1503.6 | 0.363 | 2.295 | | hnswlib / 4 | 100.3 | 9.402 | 149.332 | | sqlite-vec / 1 | 1412.7 | 0.666 | 0.801 | | sqlite-vec / 4 | 2637.5 | 1.476 | 1.660 | ### Filters after updates, both engines | Mode | Recall@10 | Result loss | | --- | --- | --- | | Native, nonempty groups | 1.0000 | 0.0000 | | Tenant postfilter | 0.4725 | 0.5275 | | Selective postfilter | 0.0275 | 0.9725 | Both engines reached native recall 1.0 on this fixture. Selective postfiltering lost 97.25% of available results despite 14 eligible candidates. Four clients degraded the HNSW adapter with its Python predicate; the experiment does not isolate callback, GIL and O(N) mask contributions. The original run retains 9,600 searches. A further run adds 9,600 after a CPU accounting fix, with the same native recall and selective loss. Table values remain the original observations. [Results and analysis](). ## Reproduce Python 3.11, a C++ compiler for hnswlib and SQLite with loadable extensions. setup.py prepares local dependencies. No model, server or API is required. Commands use POSIX paths. Some macOS Python builds disable SQLite extensions; the README documents a compatible installation. Keep the fixed NPZ, choose a new output directory and avoid concurrent heavy measurements. ### Replay saved evidence ``` git clone https://github.com/lucioduran/vector-search-portability.git cd vector-search-portability git checkout 691a60ceecd4cb5eaaaf8570a073b80d96b9563e python3 scripts/setup.py .venv/bin/python -m vector_portability verify --output results/reference .venv/bin/python -m vector_portability verify --output results/confirmation ``` ### Repeat the local measurement ``` mkdir -p build .venv/bin/python -m vector_portability run --output build/reproduced .venv/bin/python -m vector_portability verify --output build/reproduced ``` ## Scope and limitations - Vectors are synthetic: this evaluates geometry and adapters, not linguistic relevance or production scale. - Engines have different durability. SQLite ingestion includes commit; HNSW does not guarantee crash persistence. - Local closed-loop load without networking. RSS includes the interpreter and traces; three repetitions on one host do not establish a general winner. - [Vector databases: engines, filters and architecture decisions]() ## Sources and revision Tables summarize the October 21, 2026 reference run. Manifests link six original and six confirmation reports with their hashes. The executed revision shown here is the reference revision; confirmation evidence identifies a416742cb7df20476b67c75dc4c13fe67c2d7ead. - Reviewed revision: [` 691a60ceecd4cb5eaaaf8570a073b80d96b9563e `]() - Revision recorded by the experiment: [` 7ee57a5af5e0f5bf4b46caf7eb55f0e1abd9fcf1 `]() - Report timestamp (UTC): 2026-10-21 - Release published: [v0.1.0]() · 2026-10-21T22:16:31Z - Corpus SHA-256: ` 53fd9ca277d581075d2eb7df63348fbd52e92d4a0797b1e40454367ed3d409e0 ` - [README and usage]() - [Original evidence]() - [Confirmation evidence]() - [Results and analysis]() - [Design and methodology]() - [Corpus and provenance]() - [Measured environment dependencies]() - [MIT license]()