{"id":165,"date":"2026-11-18T18:44:03","date_gmt":"2026-11-18T14:44:03","guid":{"rendered":"https:\/\/www.kerloys.com\/?p=165"},"modified":"2026-11-18T18:47:19","modified_gmt":"2026-11-18T14:47:19","slug":"what-is-rag","status":"publish","type":"post","link":"https:\/\/www.kerloys.com\/index.php\/2026\/11\/18\/what-is-rag\/","title":{"rendered":"What Is RAG? How AI Answers From Your Documents Instead of Its Memory"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Ask a chatbot about your company&rsquo;s leave policy and one of two things happens. It politely says it doesn&rsquo;t know &mdash; or, worse, it confidently invents a policy that sounds completely plausible. Neither is much use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is part 3 of <strong>AI Explained<\/strong>. In <a href=\"https:\/\/www.kerloys.com\/index.php\/2026\/11\/18\/what-is-an-llm\/\">part 1<\/a> we saw that an LLM answers from &ldquo;memory&rdquo; &mdash; whatever it absorbed during training. In <a href=\"https:\/\/www.kerloys.com\/index.php\/2026\/11\/18\/prompting-101\/\">part 2<\/a> I suggested pasting a document and saying &ldquo;use only the text below&rdquo;. RAG is that same idea, built into a system and done automatically.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The short answer<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>RAG &mdash; Retrieval-Augmented Generation &mdash;<\/strong> means that before the AI answers your question, it first <strong>looks up<\/strong> the most relevant information from a trusted collection of documents, and then writes its answer <strong>based on what it found<\/strong>.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Without RAG, the AI is sitting a <strong>closed-book<\/strong> exam: it can only use what it remembers. With RAG, it&rsquo;s an <strong>open-book<\/strong> exam &mdash; and you get to choose which books are on the desk.<\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">Breaking down the name<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Retrieval<\/strong> &mdash; find the relevant pieces of information.<\/li>\n\n\n<li><strong>Augmented<\/strong> &mdash; add those pieces to the question that goes to the AI.<\/li>\n\n\n<li><strong>Generation<\/strong> &mdash; the AI writes (&ldquo;generates&rdquo;) the answer using them.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A clumsy name for a simple idea: <em>look it up first, then answer<\/em>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How it works, step by step<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Say your company builds an internal assistant on top of its HR handbook, IT guides and policies. You ask: <em>&ldquo;How many days of annual leave can I carry over?&rdquo;<\/em><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>You ask your question<\/strong> in plain language.<\/li>\n\n\n<li><strong>The system searches the library<\/strong> &mdash; the HR handbook, policy documents, intranet pages.<\/li>\n\n\n<li><strong>It picks the few most relevant passages<\/strong>, for example the paragraph on leave carry-over from the 2026 HR policy.<\/li>\n\n\n<li><strong>It hands those passages to the LLM along with your question<\/strong>, with an instruction like &ldquo;answer using only this information&rdquo;.<\/li>\n\n\n<li><strong>The LLM writes a clear answer<\/strong> &mdash; ideally with a link to the source, so you can check it yourself.<\/li>\n\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Think of it as a <strong>librarian and a writer working together<\/strong>. The librarian finds the right pages; the writer reads them and explains them to you in plain English.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How does it find the right pages?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Ordinary search matches words. If you search for &ldquo;vacation&rdquo; and the handbook says &ldquo;annual leave&rdquo;, old-fashioned search might miss it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">RAG systems usually <strong>search by meaning<\/strong> instead. Every passage in the library is turned into a kind of numeric &ldquo;fingerprint of meaning&rdquo; (techies call these <em>embeddings<\/em>, stored in a <em>vector database<\/em>). Your question gets a fingerprint too, and the system pulls out the passages whose fingerprints are closest. That&rsquo;s how &ldquo;vacation&rdquo; finds &ldquo;annual leave&rdquo;, and &ldquo;my laptop won&rsquo;t connect&rdquo; finds the Wi-Fi troubleshooting guide.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">You&rsquo;ve probably used it already<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>&ldquo;Chat with your PDF&rdquo;<\/strong> features &mdash; upload a contract or a manual and ask questions about it.<\/li>\n\n\n<li><strong>AI search engines<\/strong> that show sources next to their answers &mdash; they search the web first, then summarise what they found.<\/li>\n\n\n<li><strong>Customer-service chatbots<\/strong> on bank, airline and telecom websites that answer from the company&rsquo;s help pages.<\/li>\n\n\n<li><strong>Workplace AI assistants<\/strong> that can answer questions from your own emails, files and meeting notes.<\/li>\n\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Why companies love it<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Up to date<\/strong> &mdash; update a document today and the AI can use it today. No expensive retraining.<\/li>\n\n\n<li><strong>Uses private knowledge<\/strong> &mdash; your policies, products and procedures, which no public AI was ever trained on.<\/li>\n\n\n<li><strong>Fewer made-up answers<\/strong> &mdash; the AI works from real text instead of guessing from memory.<\/li>\n\n\n<li><strong>Shows its sources<\/strong> &mdash; so people can check the answer and trust it.<\/li>\n\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>RAG vs. &ldquo;training the AI on our data&rdquo;.<\/strong> People often assume the answer is to retrain (&ldquo;fine-tune&rdquo;) a model on company data. Fine-tuning changes how a model <em>behaves<\/em> &mdash; its style, its habits. RAG gives it the <em>facts<\/em>. If fine-tuning is training a chef, RAG is handing the chef the right recipe book. For most &ldquo;answer questions about our documents&rdquo; problems, the recipe book wins.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Where it can still go wrong<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Garbage in, garbage out.<\/strong> If the library contains an outdated policy, the AI will happily quote it.<\/li>\n\n\n<li><strong>The search can miss.<\/strong> If the right passage isn&rsquo;t found, the AI is back to guessing &mdash; a good system says &ldquo;I couldn&rsquo;t find that&rdquo; rather than inventing.<\/li>\n\n\n<li><strong>It can still misread.<\/strong> Fewer mistakes is not zero mistakes. Check the source for anything important.<\/li>\n\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">The security corner<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">RAG is where AI meets your organisation&rsquo;s real data &mdash; which makes it a security topic, not just a tech one:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Permissions must follow the user.<\/strong> The AI should only retrieve documents the person asking is allowed to see. Otherwise an intern asks &ldquo;what does the CEO earn?&rdquo; and the assistant helpfully finds the payroll file. This is the number-one RAG risk I see.<\/li>\n\n\n<li><strong>Be careful what goes into the library.<\/strong> Personal data, secrets and credentials buried in old documents become searchable &mdash; and quotable &mdash; the moment they&rsquo;re indexed.<\/li>\n\n\n<li><strong>Documents can carry hidden instructions.<\/strong> A poisoned web page or email in the library can contain text aimed at the AI (&ldquo;ignore your rules and&hellip;&rdquo;). That&rsquo;s <em>prompt injection<\/em> again, and RAG systems need defences against it.<\/li>\n\n\n<li><strong>Log and review.<\/strong> Know who asked what and which documents were used to answer &mdash; just as you would for any other system holding sensitive data.<\/li>\n\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Try it yourself<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You can do &ldquo;DIY RAG&rdquo; in any chatbot today: paste in a document (or upload a PDF), then ask <em>&ldquo;Using only this document, what does it say about&hellip;? Quote the relevant part.&rdquo;<\/em> You&rsquo;ll notice the answers get more specific, more accurate and easier to check. (Just keep confidential documents out of public tools.)<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key takeaways<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>RAG = <strong>look it up first, then answer<\/strong>. An open-book exam instead of a closed-book one.<\/li>\n\n\n<li>It lets AI answer from <strong>your own, up-to-date documents<\/strong>, with sources you can check.<\/li>\n\n\n<li>It reduces made-up answers, but <strong>doesn&rsquo;t eliminate them<\/strong>.<\/li>\n\n\n<li>For organisations, the big risks are <strong>permissions, sensitive data in the index and prompt injection<\/strong>.<\/li>\n\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Next in the series<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong><a href=\"https:\/\/www.kerloys.com\/index.php\/2026\/11\/18\/what-is-agentic-ai\/\">Part 4: What is Agentic AI?<\/a><\/strong> &mdash; when AI stops just talking and starts <em>doing<\/em>: searching, booking, clicking and acting on your behalf. Exciting, powerful &mdash; and a whole new set of risks.<\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>RAG turns AI from a closed-book exam into an open-book one: look it up first, then answer. What it is, how it works, where you&rsquo;ve already seen it &mdash; and the security risks to watch. Part 3 of AI Explained.<\/p>\n","protected":false},"author":1,"featured_media":188,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[36],"tags":[40,46,43,45],"class_list":["post-165","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-ai-explained","tag-ai-security","tag-generative-ai","tag-rag"],"_links":{"self":[{"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/posts\/165","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/comments?post=165"}],"version-history":[{"count":2,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/posts\/165\/revisions"}],"predecessor-version":[{"id":170,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/posts\/165\/revisions\/170"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/media\/188"}],"wp:attachment":[{"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/media?parent=165"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/categories?post=165"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kerloys.com\/index.php\/wp-json\/wp\/v2\/tags?post=165"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}