RAG in Practice: What We Covered in Our Advanced Training

On September 14 and 15, 2026, we ran a two-day advanced training on RAG (retrieval-augmented generation), fully virtual. We developed the concept, around 95 slides and the labs ourselves: eight blocks, each with its own Jupyter lab. The dataset was a fictional mechanical engineering company with a test set of 64 questions.

The guiding principle: “Day 1: Find the right thing. Day 2: Prove that it’s right.”

Day 1: Retrieval quality

• Error analysis: is the problem in retrieval or in generation?
• Hybrid search with BM25 and dense retrieval, reciprocal rank fusion, query rewriting and HyDE
• Reranking: cross-encoder compared with ColBERT
• Parsing and chunking with Docling

Day 2: Proving and running quality

• Compressing and reordering context
• Evaluation: test set, Ragas, LLM-as-a-judge, regression tests in the CI/CD pipeline
• Security: prompt injection, RAG poisoning, permission-aware retrieval, audit trails, GDPR and the EU AI Act
• Operations: tracing, semantic cache, cost model, re-indexing without downtime
• Architecture review to close

We worked with Python, LangChain, Qdrant, Docling, Ragas, DeepEval and Arize Phoenix.

What the labs showed

The following figures are measurements from the labs on the course dataset. They don’t transfer one-to-one to other data.

• Reranking: Hit@1 rose from 0.917 to 0.950, MRR from 0.946 to 0.975. The trade-off: latency grew from 9 to 222 milliseconds.
• Chunks with a heading path: on 13 difficult questions, Hit@1 rose from 0.692 to 0.846.
• Contextual retrieval: no measurable effect on this dataset.

That was exactly the key message: measure, don’t assume. A technique that looks convincing in an article can turn out to have no effect on your own data. Without a test set and clean ground truth data, every optimization remains a gut feeling.

Our IntranetGPT project shows what RAG looks like in a real company setting. We also build RAG systems for clients; find out more on our services page.