RAG Refactoring: Better Results Start With Better Context

Next task was to improve the RAG files for better responses and potential scalability. The whole thing now has the following structure, which Greg Nudelman proposed in the training:

In the structure, it looks like this:

The two initial RAG documents became six:

• The EU AI Act
• GDNG got company:
• MDR for medical device qualification
• DSGVO for data protection
• the DiGA guidelines from BfArM
• and responsible AI design principles collected by myself

The registry came back properly, with keyword-based trigger matching and a “Sources consulted” block in the UI so you can see what fired for each query.

The output switched from JSON to XML with a defined schema: a contract between the model and the interface. Feature summary, risk signal, red flags, what doesn’t apply, tasks grouped by timing, open questions tagged by expert type.

I also caught the model under-classifying a real case. A preventive appointment feature built on insurer health records came back as minimal risk. The fix was one paragraph in the RAG document: administrative means no individual health data goes in. The purpose of the output doesn’t change the classification of the input.

I see now from my own experience how much testing of the knowledge base is needed in real projects, especially in one like mine connected to legal topics.

This is the current flow — user fills a form, then gets their check to work with:

Honestly, right now I feel like a one-woman team arguing with her own personalities — switching between product owner, UX designer, and developer mid-sentence. Something always has to give. For now I’m keeping the design clean and iterating on UX as I go. Getting it to my actual standards? That’s a longer project.