{"schemaVersion":1,"id":"https://lucioduran.com/projects/bilingual-retrieval-evals","url":"https://lucioduran.com/projects/bilingual-retrieval-evals","title":"Bilingual Retrieval Evals","description":"EN/ES retrieval evaluation with BM25 and multilingual MiniLM. Separates languages, directions, accents and cultural expressions, with replayable rankings and evidence.","language":"en","datePublished":"2026-10-22","dateModified":"2026-10-22","contentFormat":"text/markdown","content":"[Projects](<https://lucioduran.com/projects>) / Bilingual Retrieval Evals\n\n# Bilingual Retrieval Evals\n\nEN/ES retrieval evaluation with BM25 and multilingual MiniLM. Separates languages, directions, accents and cultural expressions, with replayable rankings and evidence.\n\nAuthor: [Lucio Durán](<https://lucioduran.com/>) · Python 3.11.13 · License: [MIT](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE>)\n\nPage published: 2026-10-22 · Page updated: 2026-10-22\n\n[Repository](<https://github.com/lucioduran/bilingual-retrieval-evals>) · [Original evidence](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/report.json>) · [Release v0.1.0](<https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0>)\n\n## Problem and relevance to agents\n\nAn aggregate retrieval score can hide a search engine that works in English and fails across languages. Translating queries does not remove cultural ambiguity or lexical distractors.\n\nThe project exposes which documents each method retrieves for parallel English and Spanish queries. It makes ranking errors visible even when Recall@3 is perfect.\n\n## Architecture\n\n1. **Corpus and explicit judgments.** 32 fictional documents, 68 query variants and candidate labels with supporting spans. Development and evaluation families remain separate.\n2. **Directional retrieval.** Each query searches 16 target-language documents. Methods are BM25, accent-folded BM25 and a revision-pinned multilingual MiniLM encoder.\n3. **Breakdowns and replay.** Saved vectors, complete rankings, group metrics, paired contrasts and family intervals permit metric reconstruction without downloading the model.\n\n- [Design and methodology](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/methodology.md>)\n\n## What this demonstrates\n\n- Multilingual evaluation with explicit denominators and four separate retrieval directions.\n- Cultural queries, distractors and paired accent contrasts with traceable relevance judgments.\n- Ranking-error and uncertainty analysis that does not treat correlated variants as independent samples.\n\n## Published results\n\nLocal measurement on Intel Xeon W-3223 with two threads. Headline tables use 32 original queries from eight evaluation families, excluding accent-omission variants to avoid overweighting Spanish.\n\nCutoff 3, BM25 k1=1.2 and b=0.75, 128-token embedding limit. The full report includes 46 evaluation variants. Intervals use 2,000 family bootstrap resamples and a different estimator from the original-query pooled average.\n\n### Retrieval over 32 original queries\n\n| Method | Recall@3 | NDCG@3 | MRR@3 |\n| --- | --- | --- | --- |\n| BM25 | 0.719 | 0.719 | 0.719 |\n| Accent-folded BM25 | 0.719 | 0.719 | 0.719 |\n| Multilingual MiniLM | 1.000 | 0.977 | 0.969 |\n\n### NDCG@3 by direction\n\n| Direction | Queries | BM25 | MiniLM |\n| --- | --- | --- | --- |\n| EN → EN | 8 | 1.000 | 1.000 |\n| EN → ES | 8 | 0.625 | 0.954 |\n| ES → EN | 8 | 0.375 | 0.954 |\n| ES → ES | 8 | 0.875 | 1.000 |\n\nMiniLM reached 1.000 Recall@3 but ranked a distractor above the relevant document for a cultural query and a parcel query. Finding a document in the top three does not imply correct ordering.\n\nFamily intervals, entity breakdowns and seven accent pairs per target language are published with their denominators. [Results summary](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/results.md>).\n\n## Reproduce\n\nPython 3.11 or later. Validation and replay use the standard library. Neural measurement requires pinned dependencies and an explicit public encoder download.\n\nCommands use a POSIX shell and fresh output directories. Run one measurement at a time. The lock targets the measured Intel macOS environment; other systems may need compatible PyTorch distributions.\n\n### Replay saved evidence\n\n```\ngit clone https://github.com/lucioduran/bilingual-retrieval-evals.git\ncd bilingual-retrieval-evals\ngit checkout 8cc64e903e89ef399db5c4648b23cee52038b83b\npython -m bilingual_retrieval validate\npython -m bilingual_retrieval verify --report results/dev/report.json\npython -m bilingual_retrieval verify --report results/eval/report.json\n```\n\n### Repeat the local measurement\n\n```\npython3.11 -m venv .venv\n. .venv/bin/activate\npython -m pip install -r requirements-local.lock\npython -m bilingual_retrieval download\nOMP_NUM_THREADS=2 MKL_NUM_THREADS=2 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 TOKENIZERS_PARALLELISM=false python -m bilingual_retrieval run --split dev --output build/dev\nOMP_NUM_THREADS=2 MKL_NUM_THREADS=2 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 TOKENIZERS_PARALLELISM=false python -m bilingual_retrieval run --split eval --output build/eval\npython -m bilingual_retrieval verify --report build/dev/report.json\npython -m bilingual_retrieval verify --report build/eval/report.json\n```\n\n## Scope and limitations\n\n- Synthetic author-known corpus. Labels still lack independent bilingual human review and do not represent dialect coverage or real traffic.\n- Eight families provide weak uncertainty estimates. Entity and language groups are not causal treatments.\n- Timings are one batched pass, not latency percentiles. Replay checks internal consistency, not the authenticity of embeddings or measurements.\n\n- [Vector search and multilingual evaluation](<https://lucioduran.com/blog/vector-databases-pgvector-qdrant-pinecone>)\n\n## Sources and revision\n\nValues on this page come from the linked report. They are not live measurements. The report records the executed revision; the reviewed revision includes the subsequent publication of that report.\n\n- Reviewed revision: [` 8cc64e903e89ef399db5c4648b23cee52038b83b `](<https://github.com/lucioduran/bilingual-retrieval-evals/commit/8cc64e903e89ef399db5c4648b23cee52038b83b>)\n- Revision recorded by the experiment: [` d89585a8e53aa43a0fa8bc6fe50d43635172c5de `](<https://github.com/lucioduran/bilingual-retrieval-evals/commit/d89585a8e53aa43a0fa8bc6fe50d43635172c5de>)\n- Report timestamp (UTC): 2026-10-21T22:59:22.243350+00:00\n- Release published: [v0.1.0](<https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0>) · 2026-10-21T23:04:15Z\n\n- [README and usage](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/README.md>)\n- [Original evidence](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/report.json>)\n- [Results summary](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/results.md>)\n- [Design and methodology](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/methodology.md>)\n- [Data and provenance](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/data/README.md>)\n- [Development evidence](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/dev/report.json>)\n- [Error analysis](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/errors.json>)\n- [Models, revisions and licenses](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/model-notices.md>)\n- [Measured environment dependencies](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/requirements-local.lock>)\n- [MIT license](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE>)","headings":[{"level":1,"text":"Bilingual Retrieval Evals","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#title"},{"level":2,"text":"Problem and relevance to agents","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#problem"},{"level":2,"text":"Architecture","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#architecture"},{"level":2,"text":"What this demonstrates","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#demonstrates"},{"level":2,"text":"Published results","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#results"},{"level":3,"text":"Retrieval over 32 original queries","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#results-retrieval"},{"level":3,"text":"NDCG@3 by direction","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#results-directions"},{"level":2,"text":"Reproduce","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#reproduce"},{"level":3,"text":"Replay saved evidence","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#reproduce-replay"},{"level":3,"text":"Repeat the local measurement","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#reproduce-inference"},{"level":2,"text":"Scope and limitations","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#scope"},{"level":2,"text":"Sources and revision","url":"https://lucioduran.com/projects/bilingual-retrieval-evals#sources"}],"sections":[{"heading":"Bilingual Retrieval Evals","level":1,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals","content":"[Projects](<https://lucioduran.com/projects>) / Bilingual Retrieval Evals"},{"heading":"Bilingual Retrieval Evals","level":1,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#title","content":"EN/ES retrieval evaluation with BM25 and multilingual MiniLM. Separates languages, directions, accents and cultural expressions, with replayable rankings and evidence.\n\nAuthor: [Lucio Durán](<https://lucioduran.com/>) · Python 3.11.13 · License: [MIT](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE>)\n\nPage published: 2026-10-22 · Page updated: 2026-10-22\n\n[Repository](<https://github.com/lucioduran/bilingual-retrieval-evals>) · [Original evidence](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/report.json>) · [Release v0.1.0](<https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0>)"},{"heading":"Problem and relevance to agents","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#problem","content":"An aggregate retrieval score can hide a search engine that works in English and fails across languages. Translating queries does not remove cultural ambiguity or lexical distractors.\n\nThe project exposes which documents each method retrieves for parallel English and Spanish queries. It makes ranking errors visible even when Recall@3 is perfect."},{"heading":"Architecture","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#architecture","content":"1. **Corpus and explicit judgments.** 32 fictional documents, 68 query variants and candidate labels with supporting spans. Development and evaluation families remain separate.\n2. **Directional retrieval.** Each query searches 16 target-language documents. Methods are BM25, accent-folded BM25 and a revision-pinned multilingual MiniLM encoder.\n3. **Breakdowns and replay.** Saved vectors, complete rankings, group metrics, paired contrasts and family intervals permit metric reconstruction without downloading the model.\n\n- [Design and methodology](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/methodology.md>)"},{"heading":"What this demonstrates","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#demonstrates","content":"- Multilingual evaluation with explicit denominators and four separate retrieval directions.\n- Cultural queries, distractors and paired accent contrasts with traceable relevance judgments.\n- Ranking-error and uncertainty analysis that does not treat correlated variants as independent samples."},{"heading":"Published results","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#results","content":"Local measurement on Intel Xeon W-3223 with two threads. Headline tables use 32 original queries from eight evaluation families, excluding accent-omission variants to avoid overweighting Spanish.\n\nCutoff 3, BM25 k1=1.2 and b=0.75, 128-token embedding limit. The full report includes 46 evaluation variants. Intervals use 2,000 family bootstrap resamples and a different estimator from the original-query pooled average."},{"heading":"Retrieval over 32 original queries","level":3,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#results-retrieval","content":"| Method | Recall@3 | NDCG@3 | MRR@3 |\n| --- | --- | --- | --- |\n| BM25 | 0.719 | 0.719 | 0.719 |\n| Accent-folded BM25 | 0.719 | 0.719 | 0.719 |\n| Multilingual MiniLM | 1.000 | 0.977 | 0.969 |"},{"heading":"NDCG@3 by direction","level":3,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#results-directions","content":"| Direction | Queries | BM25 | MiniLM |\n| --- | --- | --- | --- |\n| EN → EN | 8 | 1.000 | 1.000 |\n| EN → ES | 8 | 0.625 | 0.954 |\n| ES → EN | 8 | 0.375 | 0.954 |\n| ES → ES | 8 | 0.875 | 1.000 |\n\nMiniLM reached 1.000 Recall@3 but ranked a distractor above the relevant document for a cultural query and a parcel query. Finding a document in the top three does not imply correct ordering.\n\nFamily intervals, entity breakdowns and seven accent pairs per target language are published with their denominators. [Results summary](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/results.md>)."},{"heading":"Reproduce","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#reproduce","content":"Python 3.11 or later. Validation and replay use the standard library. Neural measurement requires pinned dependencies and an explicit public encoder download.\n\nCommands use a POSIX shell and fresh output directories. Run one measurement at a time. The lock targets the measured Intel macOS environment; other systems may need compatible PyTorch distributions."},{"heading":"Replay saved evidence","level":3,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#reproduce-replay","content":"```\ngit clone https://github.com/lucioduran/bilingual-retrieval-evals.git\ncd bilingual-retrieval-evals\ngit checkout 8cc64e903e89ef399db5c4648b23cee52038b83b\npython -m bilingual_retrieval validate\npython -m bilingual_retrieval verify --report results/dev/report.json\npython -m bilingual_retrieval verify --report results/eval/report.json\n```"},{"heading":"Repeat the local measurement","level":3,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#reproduce-inference","content":"```\npython3.11 -m venv .venv\n. .venv/bin/activate\npython -m pip install -r requirements-local.lock\npython -m bilingual_retrieval download\nOMP_NUM_THREADS=2 MKL_NUM_THREADS=2 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 TOKENIZERS_PARALLELISM=false python -m bilingual_retrieval run --split dev --output build/dev\nOMP_NUM_THREADS=2 MKL_NUM_THREADS=2 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 TOKENIZERS_PARALLELISM=false python -m bilingual_retrieval run --split eval --output build/eval\npython -m bilingual_retrieval verify --report build/dev/report.json\npython -m bilingual_retrieval verify --report build/eval/report.json\n```"},{"heading":"Scope and limitations","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#scope","content":"- Synthetic author-known corpus. Labels still lack independent bilingual human review and do not represent dialect coverage or real traffic.\n- Eight families provide weak uncertainty estimates. Entity and language groups are not causal treatments.\n- Timings are one batched pass, not latency percentiles. Replay checks internal consistency, not the authenticity of embeddings or measurements.\n\n- [Vector search and multilingual evaluation](<https://lucioduran.com/blog/vector-databases-pgvector-qdrant-pinecone>)"},{"heading":"Sources and revision","level":2,"url":"https://lucioduran.com/projects/bilingual-retrieval-evals#sources","content":"Values on this page come from the linked report. They are not live measurements. The report records the executed revision; the reviewed revision includes the subsequent publication of that report.\n\n- Reviewed revision: [` 8cc64e903e89ef399db5c4648b23cee52038b83b `](<https://github.com/lucioduran/bilingual-retrieval-evals/commit/8cc64e903e89ef399db5c4648b23cee52038b83b>)\n- Revision recorded by the experiment: [` d89585a8e53aa43a0fa8bc6fe50d43635172c5de `](<https://github.com/lucioduran/bilingual-retrieval-evals/commit/d89585a8e53aa43a0fa8bc6fe50d43635172c5de>)\n- Report timestamp (UTC): 2026-10-21T22:59:22.243350+00:00\n- Release published: [v0.1.0](<https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0>) · 2026-10-21T23:04:15Z\n\n- [README and usage](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/README.md>)\n- [Original evidence](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/report.json>)\n- [Results summary](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/results.md>)\n- [Design and methodology](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/methodology.md>)\n- [Data and provenance](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/data/README.md>)\n- [Development evidence](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/dev/report.json>)\n- [Error analysis](<https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/errors.json>)\n- [Models, revisions and licenses](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/model-notices.md>)\n- [Measured environment dependencies](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/requirements-local.lock>)\n- [MIT license](<https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE>)"}],"links":[{"text":"Projects","url":"https://lucioduran.com/projects"},{"text":"Lucio Durán","url":"https://lucioduran.com/"},{"text":"MIT","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE"},{"text":"Repository","url":"https://github.com/lucioduran/bilingual-retrieval-evals"},{"text":"Original evidence","url":"https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/report.json"},{"text":"Release v0.1.0","url":"https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0"},{"text":"Design and methodology","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/methodology.md"},{"text":"Results summary","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/results.md"},{"text":"Vector search and multilingual evaluation","url":"https://lucioduran.com/blog/vector-databases-pgvector-qdrant-pinecone"},{"text":"8cc64e903e89ef399db5c4648b23cee52038b83b","url":"https://github.com/lucioduran/bilingual-retrieval-evals/commit/8cc64e903e89ef399db5c4648b23cee52038b83b"},{"text":"d89585a8e53aa43a0fa8bc6fe50d43635172c5de","url":"https://github.com/lucioduran/bilingual-retrieval-evals/commit/d89585a8e53aa43a0fa8bc6fe50d43635172c5de"},{"text":"v0.1.0","url":"https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0"},{"text":"README and usage","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/README.md"},{"text":"Data and provenance","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/data/README.md"},{"text":"Development evidence","url":"https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/dev/report.json"},{"text":"Error analysis","url":"https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/errors.json"},{"text":"Models, revisions and licenses","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/model-notices.md"},{"text":"Measured environment dependencies","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/requirements-local.lock"},{"text":"MIT license","url":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com/"},{"@type":"WebSite","@id":"https://lucioduran.com/#website","url":"https://lucioduran.com/","name":"Lucio Durán","inLanguage":["en","es"],"publisher":{"@id":"https://lucioduran.com/#person"}},{"@type":"WebPage","@id":"https://lucioduran.com/projects/bilingual-retrieval-evals#page","url":"https://lucioduran.com/projects/bilingual-retrieval-evals","name":"Bilingual Retrieval Evals","description":"EN/ES retrieval evaluation with BM25 and multilingual MiniLM. Separates languages, directions, accents and cultural expressions, with replayable rankings and evidence.","inLanguage":"en","author":{"@id":"https://lucioduran.com/#person"},"publisher":{"@id":"https://lucioduran.com/#person"},"isPartOf":{"@id":"https://lucioduran.com/#website"},"isAccessibleForFree":true,"mainEntity":{"@id":"https://lucioduran.com/projects/bilingual-retrieval-evals#software"},"datePublished":"2026-10-22","dateModified":"2026-10-22","breadcrumb":{"@id":"https://lucioduran.com/projects/bilingual-retrieval-evals#breadcrumb"}},{"@type":"SoftwareSourceCode","@id":"https://lucioduran.com/projects/bilingual-retrieval-evals#software","name":"Bilingual Retrieval Evals","description":"EN/ES retrieval evaluation with BM25 and multilingual MiniLM. Separates languages, directions, accents and cultural expressions, with replayable rankings and evidence.","url":"https://lucioduran.com/projects/bilingual-retrieval-evals","codeRepository":"https://github.com/lucioduran/bilingual-retrieval-evals","programmingLanguage":"Python","runtimePlatform":"Python 3.11.13","license":"https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/LICENSE","version":"v0.1.0","identifier":{"@type":"PropertyValue","propertyID":"git-commit","value":"8cc64e903e89ef399db5c4648b23cee52038b83b"},"author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"isAccessibleForFree":true,"citation":["https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/README.md","https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/report.json","https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/results.md","https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/methodology.md","https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/data/README.md","https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/dev/report.json","https://raw.githubusercontent.com/lucioduran/bilingual-retrieval-evals/8cc64e903e89ef399db5c4648b23cee52038b83b/results/eval/errors.json","https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/docs/model-notices.md","https://github.com/lucioduran/bilingual-retrieval-evals/blob/8cc64e903e89ef399db5c4648b23cee52038b83b/requirements-local.lock","https://github.com/lucioduran/bilingual-retrieval-evals/releases/tag/v0.1.0"],"publisher":{"@id":"https://lucioduran.com/#person"},"mainEntityOfPage":{"@id":"https://lucioduran.com/projects/bilingual-retrieval-evals#page"}},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Projects","item":"https://lucioduran.com/projects"},{"@type":"ListItem","position":2,"name":"Bilingual Retrieval Evals","item":"https://lucioduran.com/projects/bilingual-retrieval-evals"}],"@id":"https://lucioduran.com/projects/bilingual-retrieval-evals#breadcrumb"}]}],"contentHash":"fa14626a1feac11c8addc972b5b722d299b9b6510677149c8be4d8680709277f","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/projects/bilingual-retrieval-evals","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":47130,"contentBytes":7859}}