• Welcome to Digital Expressway Consulting
  • +91 98180 30135
  • info@digitalExpressway.net

RAG for Enterprise AI Search: How One Bank’s AI Search Cut Wrong Answers From 20–30% to Under 10%

← Industry Foresight & Innovation Hotbeds · Generative AI search

RAG for Enterprise AI Search: How One Bank’s AI Search Cut Wrong Answers From 20–30% to Under 10%

At Axis Bank, around 40% of its nearly 100,000 employees now use an internal AI search tool regularly, according to Inc42. The same report says wrong or fallback answers fell from 20–30% in early versions to below 10%. At Morgan Stanley, more than 98% of financial adviser teams use an AI assistant that answers from the firm’s own research library, according to OpenAI, which supplies the model. Both answer from the organisation’s own content, although Inc42 does not say which method Axis uses. The best-known way to do this is called RAG: let the AI look things up in your documents before it answers.

Now the other side. In a 2024 Stanford study, specialist legal research tools built on that same idea still gave incorrect or poorly supported answers more than 17% of the time, according to Stanford HAI. One tool did so more than 34% of the time. Same approach, very different reliability. The difference lies less in the AI model and more in the documents, the testing and the controls around it.

If your teams spend hours hunting through policies, contracts, manuals or past client work, this matters to you. Enterprise AI search can turn that hunt into a short, cited answer. Done carelessly, it can also hand people a confident answer that the cited source does not support.

What you will take away from this article

  • The moves the leaders make: four concrete practices from Morgan Stanley, DoorDash, LinkedIn and Axis Bank that you can borrow.
  • Why most efforts stall: the five common failure points, from stale documents to leaky permissions.
  • What the successful few do differently: a one-page table of practices and the evidence behind them.
  • How it works: the moving parts of retrieval augmented generation, explained without jargon.
  • How to start: a 30-day checklist and the habits that separate trained teams from the rest.

About a 12-minute read.

First, a quick definition. RAG stands for retrieval augmented generation. In plain words, it is a two-step process. First, the system retrieves: it searches your own documents and picks out the passages most likely to answer the question. Second, it generates: a large language model (the kind of AI behind tools such as ChatGPT or Claude) writes an answer using those passages, ideally citing which document each point came from. The model is told to answer from the passages, not from memory. That is why companies use it: answers can be grounded in their own policies, contracts and manuals, not in whatever the model picked up from the public internet. (A note to avoid confusion: in project management, “RAG” also means a red, amber and green status report. This article is only about the AI meaning.)

Four Moves the AI Search Leaders Make

  • Test before launch, then test every day. Morgan Stanley tested every AI use case against an evaluation framework before deployment, according to OpenAI, which supplies the technology. OpenAI also reports that the firm ran daily regression tests (repeat checks against a fixed set of sample questions) for compliance and quality. The result it reports: more than 98% of adviser teams use the assistant, and access to documents rose from 20% to 80%. What to borrow: write down 50–100 real questions with agreed correct answers before you go live, and rerun them after every change.

  • Put a checker between the AI and the user. DoorDash’s support chatbot for delivery drivers retrieves knowledge-base articles and then passes each draft reply through a “guardrail”, according to DoorDash’s engineering blog. The guardrail checks whether the reply is grounded in the retrieved articles and complies with policy before the driver sees it. DoorDash says this cut hallucinations (confident but false statements) by 90% and potentially severe compliance issues by 99%. What to borrow: treat “is this answer supported by the source?” as a separate, automated check, not an assumption.

  • Keep the structure of your knowledge, not just the text. LinkedIn’s customer service team used a RAG system that organised past support tickets as a connected map of issues and their relationships, according to a 2024 research paper by the system’s developers. The paper reports that this improved retrieval accuracy by 77.6% over a standard approach and cut median time to resolve an issue by 28.6% after about six months in use. What to borrow: if your documents have natural structure (sections, ticket fields, clause numbers), preserve it so the system can find the right piece.

  • Measure wrong answers openly and drive them down. Axis Bank tracked the share of fallback or incorrect answers from its employee search tool, according to Inc42. Inc42 reports the rate fell from 20–30% in early versions to below 10%, while usage grew to around 40% of staff. What to borrow: publish one simple accuracy number to sponsors from day one, and expect the first version to be rough.

Why Most Enterprise AI Search Efforts Stall

1. The answer is not in the documents, or the documents are wrong. A 2024 study of three real RAG projects listed “missing content” as the first failure point: users ask questions the documents cannot answer, according to Barnett and colleagues. Stale or contradictory documents cause the same problem in a quieter way. The AI faithfully repeats an out-of-date policy. Gartner predicts that through 2026 organisations will abandon 60% of AI projects unsupported by “AI-ready” data, according to a Gartner press release based on a July 2024 survey of 1,203 data management leaders.

2. The search step misses the right passage. The same study found cases where the answer existed but did not rank highly enough to be retrieved, or was retrieved but never reached the AI, according to Barnett and colleagues. When retrieval misses, even a strong model writes a weak or wrong answer. The study’s authors concluded that a RAG system can only be properly validated once it is in real use, and that its robustness “evolves rather than designed in at the start”.

3. Citations look reassuring but do not support the answer. The Stanford legal study counted an answer as a hallucination if it was wrong, or if it cited a source that did not back up its claim (“misgrounded”), according to Stanford HAI. A separate Stanford study of public AI search engines found only 74.5% of citations supported the sentence attached to them, according to Liu, Zhang and Liang. A March 2025 test of eight AI search tools by Columbia University’s Tow Center found they failed to retrieve correct source information more than 60% of the time, according to Nieman Lab.

4. Access permissions were never cleaned up. An AI search tool can surface any file a user technically has access to, including files that were shared too widely years ago. In a Gartner survey of 132 IT leaders, 40% said “oversharing” concerns delayed Microsoft 365 Copilot rollouts by three months or more, according to Computerworld. A Gartner analyst summed it up, according to the same Computerworld report: “The better the information retrieval tool, the better your information governance has to be.”

5. Teams assume RAG removes hallucination. It reduces it; it does not remove it. Stanford researchers found that RAG-based legal tools made fewer errors than general chatbots but concluded that “legal hallucinations have not been solved”, according to Stanford HAI. The peer-reviewed version appeared in the Journal of Empirical Legal Studies in 2025, according to Stanford Law School.

Case in point: legal research tools marketed as “hallucination-free”

What happened: Leading legal research vendors added RAG to their products and described their citations as reliable, some using phrases like “hallucination-free”, according to Stanford HAI.

What went wrong: Tested on more than 200 pre-registered legal questions, the tools still produced incorrect or misgrounded answers more than 17% of the time; one exceeded 34%. Causes included hard retrieval, retrieved documents that did not apply, and systems accepting users’ false premises.

What changed: The study, later peer-reviewed, gave buyers an independent benchmark and showed why vendors’ claims need evidence.

The lesson: A citation is a pointer, not a proof. Someone must still check that the source says what the answer claims.

What the Successful Few Do Differently

What they do What it means in practice The evidence
Build an evaluation set before launch A fixed list of real questions with agreed answers, rerun after every change Morgan Stanley ran pre-deployment evaluations and daily regression tests, according to OpenAI (supplier-reported)
Check grounding automatically A second step tests whether each answer is supported by the retrieved text DoorDash reports 90% fewer hallucinations after adding a guardrail (DoorDash, company-reported)
Curate content and assign owners Remove duplicates, retire outdated versions, name an owner per document set 71% of 382 surveyed executives and managers cite data quality as a top issue (DBTA/Unisphere, January 2025)
Fix access rights before switching on Review who can see what; close old open links 66% of 279 Microsoft 365 decision-makers paused or stopped Copilot plans over data exposure fears (IT Brief, July 2026, CoreView survey)
Keep a person in the loop People review high-stakes answers; the tool drafts, the person decides 89% of respondents with AI in production say a human in the loop matters to some degree (DBTA/Unisphere)

Survey figures show what organisations report and worry about; they show correlation, not proof that one practice causes success.

Where RAG Already Works

Adviser knowledge at a wealth manager (supplier-reported). Morgan Stanley’s assistant grew from answering a set of 7,000 questions to answering from a corpus of 100,000 documents, according to OpenAI. Over 98% of adviser teams use it, OpenAI reports.

Customer service at a professional network (paper by the system’s developers). The ticket-based RAG system used by LinkedIn’s customer service team cut median issue resolution time by 28.6%, according to the 2024 research paper describing it.

Firm-wide AI at scale in banking (trade press report). JPMorgan Chase’s LLM Suite platform reached 200,000 users within eight months of its summer 2024 launch, according to AI News. The same report describes it as connecting AI models to firm-wide data and workflows. It does not break out results for document search alone.

India view. Axis Bank describes its employee tool as “a Google-style search engine for employees”, built on policies, documents, videos and internal data, according to Inc42. The bank also admits its “semantic layer” (the shared glossary that tells systems what business terms mean) still needs work. Inc42 does not say which retrieval method the tool uses.

How It Works

A RAG system has five parts. None of them is exotic, but each can fail.

  1. The content library. The documents the system is allowed to use: policies, contracts, manuals, past tickets. Quality here sets the ceiling for everything else.
  2. Preparation. Documents are split into short passages and indexed, so the system can search them by meaning rather than exact keywords.
  3. Retrieval. When someone asks a question, the system finds the passages most likely to contain the answer. It should only search what that person is allowed to see.
  4. Generation. The AI model receives the question plus the retrieved passages, and is instructed to answer only from them and cite each source.
  5. Checking and feedback. Automated checks test whether the answer is supported. Users flag bad answers, and those flags feed back into fixing content or retrieval.

Illustrative example (a composite, not a real organisation): a relationship manager at a mid-sized bank asks, “Can we waive the processing fee for a senior citizen top-up loan?” The system searches only the documents her role can access. It finds two passages: the current fee policy and a circular from last quarter. The AI drafts an answer citing both, and flags that the circular overrides part of the policy. A grounding check confirms each sentence matches a passage. Because fee waivers are a credit decision, the answer goes to a credit officer, who approves it before the customer is told. Her thumbs-down on an earlier, outdated answer had already prompted the policy owner to retire an old version.

[DIAGRAM: Employee question → Permission-filtered search of approved documents → Top passages retrieved → AI drafts cited answer → Automated grounding check → Credit officer approves (human approval point) → Answer shared, feedback logged]

Value It Unlocks

The measured gains cluster in three areas. First, finding information faster: Morgan Stanley’s document access rose from 20% to 80%, according to OpenAI (supplier-reported). Second, resolving work faster: LinkedIn’s team cut median resolution time by 28.6%, according to its developers’ paper. Third, fewer errors with controls in place: DoorDash reports 90% fewer hallucinations once its guardrail was added (DoorDash). Anthropic reports that one customer, Endex, cut source hallucinations and formatting issues from 10% to 0% using its Citations feature, according to Anthropic, which sells the feature.

The gains are uneven. Most published figures come from the companies or their suppliers, not independent audits. Independent tests, such as the Stanford legal study, show much higher error rates in hard, specialised domains. Adoption is also patchy: Axis reached 40% regular use, not 100%. Expect strong gains on frequent, well-documented questions, and weaker results on rare, ambiguous or judgement-heavy ones.

Where Claude Fits, and Where It Doesn’t

Claude, Anthropic’s AI assistant, already uses RAG in everyday work. In Claude’s Projects feature, retrieval switches on automatically when uploaded knowledge approaches the model’s working memory limit, expanding capacity “by up to 10x”, according to Anthropic’s help centre. This is available on paid plans. For developers, Anthropic’s Citations feature links each claim to “the exact sentences and passages” it came from, according to Anthropic. Claude is strong at reading long documents, comparing versions, summarising and drafting a cited answer quickly. A person must stay in charge in four places: deciding which documents are authoritative; setting who may see what; checking that citations actually support high-stakes answers; and making any decision with legal, financial or customer impact. Claude can be wrong, and so can the documents it reads.

The principle: let the AI find and draft; let accountable people verify and decide.

How to Start: A 30-Day Checklist

  • [ ] Week 1: Pick one narrow use case with frequent, repetitive questions and clear written answers, such as HR policy or product manuals.
  • [ ] Week 1: Name a content owner who can say which documents are current and authoritative.
  • [ ] Week 1: Write 50–100 test questions from real queries, each with an agreed correct answer and source.
  • [ ] Week 2: Clean the content library. Remove duplicates and outdated versions; add effective dates.
  • [ ] Week 2: Review access rights for every folder you plan to connect, and close overly open links.
  • [ ] Week 3: Run a small pilot with 10–20 users, requiring citations in every answer.
  • [ ] Week 3: Score the test set for correct answers, correct citations and “I don’t know” when the answer is missing.
  • [ ] Week 4: Add a feedback button and a weekly review of flagged answers with the content owner.
  • [ ] Week 4: Decide go or no-go using one published accuracy number and the user feedback.

Why Trained Teams Succeed

An “anti-pattern” is a habit that looks sensible in the short term but quietly causes failure later. RAG projects have their own set.

Short-sighted habit What a trained team does instead
Connecting the whole shared drive “so nothing is missed” Starts with a curated, owned set of authoritative documents
Trusting an answer because it has a citation Opens the cited passage and checks it supports the claim
Testing with a few friendly demo questions Builds a test set of real, hard and unanswerable questions
Switching on search before fixing permissions Audits who can see what before connecting any source
Treating launch as the finish line Reviews flagged answers weekly and keeps content fresh

Training changes behaviour. In BCG’s AI at Work 2025 survey of 10,635 employees across 11 countries, 79% of people with more than five hours of training were regular AI users, against 67% of those with less, according to UNLEASH. Yet in BCG’s June 2026 edition, with 11,749 respondents, only 36% felt they had received sufficient training, according to BCG. For RAG, the habits that matter most (curating sources, testing and checking citations) are learned, not intuitive.

Build the skills to run AI that answers from your own documents, safely. The CCAO-F prep course from Digital Expressway Academy prepares professionals who run AI work without writing code (consultants, managers, analysts, operations and delivery leads) for Anthropic’s Claude Certified Associate exam, with modules on project knowledge, connectors and checking AI output. See the course

Bringing AI to your whole team? Talk to us about our Industry Advisory, Innovation and Pilot Build, and Mentoring and Skill Development services.

Sources

All sources were opened and checked in October 2026. Figures described as company-reported come from the organisation itself.

Digital Expressway Insights. Published November 2026; last reviewed October 2026. Not affiliated with or endorsed by Anthropic. Claude is a product of Anthropic. Examples marked “illustrative” are composites created to explain an idea and do not describe a real organisation.