"Beyond English: Navigating the Challenges of Building a Ukrainian-language RAG System", Roman Romanov
0 likes122 views
We'll explore the key challenges of developing RAG-systems for the Ukrainian language. We'll discuss evaluation methods for such systems in specific domain areas, like consulting citizens on public services, and uncover practical insights and common "gotchas" that arise during implementation.
![Retrieval-Augmented Generation
arXiv.org
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Large pre-trained language models have been shown to store factual knowledge
in their parameters, and achieve state-of-the-art results when fine-tuned on
Definition: models which combine pre-trained parametric and non-parametric memory for language generation
CONTEXT:
doc id=1 Fwdays+DevRain AI 5G89 6 Mercure Congress Centre /doc
doc id=2 ' Mercure Congress Centre T G:@8FFO /doc
doc id=2 @864T 75V@ =4 :648@:?F9@ 8;O 229-7 :@97 54F4;L9=G 127-W 5@87488 @. /doc
QUERY: - @58F8, O:M =4 fwdays 5G89 46V4F@8674?
Based on the context provided, please answer the query.
If the context doesn't contain relevant information, say
I don't have enough information about this topic.
Always cite your sources from the context.
5 ?5@5664=F5, 6 Mercure Congress Centre T G@FFO [1]
4VFL FG4, F4 4=4FLF5 =4 644@?F5@ [2}](https://image.slidesharecdn.com/copy-of-retrievalaugmented-generation-rag-250707205240-f451d3cb/85/Beyond-English-Navigating-the-Challenges-of-Building-a-Ukrainian-language-RAG-System-Roman-Romanov-3-320.jpg)
"Beyond English: Navigating the Challenges of Building a Ukrainian-language RAG System", Roman Romanov
- 1. 4 @564@ 4A7?V=ELBW: 6? CD CB5G4B6V GD4WAB@B6ABW RAG EEF5@ $4=6 $4= AI Center of Excellence
- 2. 5D54G@B68 68A8A5AAO RAG %F4F8GAV 7A4AAO B89?L @4T ?H9 FV ;A4AAO, OV 5G? ;4?489AV CV8 G4E A46G4AAO 4?NF8A4FVW G:9 CD468BCB8V5AV, 4?9 A9CD46?LAV 6V8CB6V8V 5@B6?86B 6V4EF568F8 C5DLB465D5?B A9 @B:9@B GVFB 6V8EF9:F, O@ GAB@ @B89?L ;79A9DG64?4 6V8CB6V8L 4DFVEFL fine-tuning VA4, OG F C?4FH B:9A D4;, B? F EBG9H ;4?4EF AB6V ;A4AAO 6 @B89?L
- 3. Retrieval-Augmented Generation arXiv.org Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on Definition: models which combine pre-trained parametric and non-parametric memory for language generation CONTEXT: doc id=1 Fwdays+DevRain AI 5G89 6 Mercure Congress Centre /doc doc id=2 ' Mercure Congress Centre T G:@8FFO /doc doc id=2 @864T 75V@ =4 :648@:?F9@ 8;O 229-7 :@97 54F4;L9=G 127-W 5@87488 @. /doc QUERY: - @58F8, O:M =4 fwdays 5G89 46V4F@8674? Based on the context provided, please answer the query. If the context doesn't contain relevant information, say I don't have enough information about this topic. Always cite your sources from the context. 5 ?5@5664=F5, 6 Mercure Congress Centre T G@FFO [1] 4VFL FG4, F4 4=4FLF5 =4 644@?F5@ [2}
- 5. High-Level Pipeline 1. Indexing Chunk docs, embed, store vectors. 2. Retrieval Search over Index for most relevant information 3. Augmentation Inject knowledge base information into prompt. 4. Generation LLM produces answer 5. Post-process Formatting/verification/citations process
- 6. 1. Indexing 1 Ingestion normalization Convert heterogeneous sources (PDF, HTML, Markdown, databases) into clean text; strip boilerplate, remove duplicates, and standardize line- breaks and punctuation. 2 Chunking strategy Split content into some context windows, that are sized enough, to be granular in search, and be complete for the context 3 Embeddings: Dense and/or Sparse Encode each chunk with some vector value (dense or sparse) for further search 4 Vector databse Store embeddings and metadata in a vector database such as PGVector/Pinecone/FAISS/Chroma, using HNSW or IVFFLAT for ANN search
- 7. Stage 2: Retrieval 1 Query Pre-processing Clean spelling, expand acronyms, decompose multi-intent queries. 2 Query Embedding Convert the normalized query into dense vectors and/or sparse BM25 features for hybrid search. 3 Candidate Retrieval Run ANN vector lookup and/or BM25 to collect the top k (typically 3-10) text chunks that roughly match the query intent. 4 Hybrid Score Fusion Merge dense and sparse results with Reciprocal-Rank Fusion (RRF) or learned weights to maximize recall while keeping latency low. 5 Re-ranking Pass the top candidates to a cross-encoder that re-scores passages for fine-grained relevance. 6 Filtering Trust Gates Drop near-duplicates, enforce domain/recency constraints, and run PII/safety checks before handing evidence to the generator. 7 Iterative Retrieval Frameworks like Self-RAG let the LM detect weak evidence, reformulate the query, and fetch again, cutting hallucinations by ~30%.
- 8. Stage 3/4: Augmentation + Generation LLM choice Different LLMs can perform Prompt Structure Prompt = System + Context + Query. Context Management Concatenate chunks; if context window ³ summarize with map-reduce chain. Token Budgeting (#chunks × chunk_tokens) + query f model_limit. Citation Preservation Create some citation markers, so that LLM can create some citation references, like doc_1 /doc_1
- 9. Stage 5: Post-processing Safety Policy Guardrails Funnel the draft through red-team filters (Guardrails, Llama Guard) that strip PII, hate speech, and other policy-violating content before it can reach users. Citation Verification Formatting Cross-check every inline reference ID, drop broken or orphaned links, renumber sequentially, and convert to Markdown/HTML anchors to maintain traceability. Stylistic Refinement Localization Apply a lightweight LLM pass for tone, length, summarization, or translation into the user's locale while preserving citation anchors. Human-in-the-loop Escalation Automatically route low-confidence or policy-flagged answers to human reviewers.
- 10. RAG tutorials python.langchain.com Build a Retrieval Augmented Generation (RAG) App: Part 1 | _ Lang One of the most powerful applications enabled by LLMs is sophisticated question-answering (QA) chatbots. These are applications that can answer colab.research.google.com Google Colab
- 11. Evaluation of RAG Layer What to measure Core metrics 1. Retrieval quality Can the retriever surface the right chunks4ranked sensibly4before we ever hit the LLM? Recall@k (did we find all relevant passages?) Precision@k (how much junk is in the top k?) MRR / mAP (reward early hits) nDCG (position-aware) 2. Generation quality Given the retrieved context, does the answer stay grounded, complete, helpful? Answer relevance Faithfulness / hallucination rate Factual corectness Embedding-similarity scores (cos » between answer reference)
- 12. %B?, B4 4 4 G7 BCB '@4W=AL4?
- 13. Massive Text Embedding Benchmark 4649F5 ECDB5GT@B 67444F8, EV?L8 55AG@4DV6 D54?LAB VEAGT 4?O GD4WAELBW @B68 6 RAG 74644AAV?
- 14. 4= - BelebeleRetrieval 6V= 4EF4F=L EG=V6=W OEFV
- 15. Expectation/Reality D47: 4AV CDB OpenAI 9@5988VA7 ; 59AG@4D4: BEL 6:9 A4HV ;4@VD CB 6AGFDVHAV@ 59AG@4D4@ D9FDV64?4:
- 16. Expectation/Reality 464: Llama 4 Scout: =4 @9;V7V 6884TBLAO, M 74@=89 :=:C@9=B GPT-4o AL O: 6=4 A?V;:CTBLAO C:@4W=AL:N:
- 17. E5 CB74AB? V, A4ECD468V 7V8AE 64DV4AFV6 5V?LH AV: 8BEF4FALB Google/OpenAI @B45?V B89?V 6V8 Google/OpenAI 8B6B?V 8B5D9 CD4FNNFL ; GD4WAELBN: Gemma/Gemini 8B6B?V 8B5D9 CD4FNT ; GD4WAB@B6A@ BAF9EFB@ B@@5DFV9AV @5544VA78 9: E4@9 V CB 9@5988VA74@: gemini-embedding-001 F4 text-embedding-3-large A9CB74AB CD4FNNFL ; GD4WAELBN. Open-source @5544VA78 open-source DVH9AL, F4B: 64DFB CB86FEO A4 LaBSE 45B MiniLML12 'D4WAB@B6AV EC5FV4?V7B64AV @B45?V -9 EBG9FLEO BD9@B 6V8;A4GF GD4W=6=V 95988V=7, F4 6 @4=5GFALB@G - GD4WAB@B6AG LLM
- 18. 4 ?B@V5= 5V;LH5 O:VA=8E 55=G4@:V6 4 ?B@V5=, I5 6=8 5C;8 open- source E =4O6=VABL 44ABL 6;86VABL 4;O @768B:C B4 ?:@4I5==N V 7 5:C ?@64945@V6, 4465 6=8 486;OBLAO =4 55=G4@:8 ?5@54C =5?4E4=5 ?;5, B4 4C65 :@CBV 6;86ABV À
- 19. @575=F4FVO 0 V= LinkedIn 4 5V@ =4 ?V4?