RAG: Build or Buy? A Cost Framework for B2B Leaders, Featuring Morgan Stanley and Bloomberg

Most B2B leaders pricing their first RAG (Retrieval-Augmented Generation) system anchor on the wrong number. They estimate the foundation model API cost, get a figure under $40K a year, and assume the rest is engineering effort. Then production hits — vector storage scales 5×, evaluation tooling costs more than the model itself, and the in-house build comes in at 3–5× the original estimate.

The structural problem is that "RAG" describes both a $50K vendor product and a $10M custom build, and the decision between them is rarely framed clearly before the budget is set. Menlo Ventures' 2025 State of Generative AI in the Enterprise report, based on a survey of 495 U.S. enterprise AI decision-makers in November 2025, put enterprise AI spend at $37 billion for 2025 — a 3.2× year-over-year increase from $11.5 billion in 2024 — with $18 billion of that going to infrastructure including foundation model APIs ($12.5 billion). The same report found 76% of enterprise AI use cases are now purchased rather than built internally, up from 53% the year before. Buy has won at the margin. But the cases where build still wins matter, and getting the decision right is the most expensive call a B2B leader will make this cycle.

This guide documents the six cost categories every RAG project must price, the decision criteria that separate buy from build, and two public reference cases that anchor the extremes — Morgan Stanley (bought) and Bloomberg (built). Revenue Experts uses this framework with mid-market and enterprise B2B clients to set the budget before the architecture is locked. Foundational RAG concepts are covered in Revenue Experts' earlier piece What is Retrieval-Augmented Generation? A 2026 Guide for Decision-Makers. This piece picks up where that one ends — with the build-or-buy decision.

What does "buying" a RAG system actually mean in 2026?

Buying RAG in 2026 means assembling a vendor stack: a foundation model API (OpenAI, Anthropic, Google), a managed vector database (Pinecone, Weaviate Cloud, Qdrant Cloud), and a retrieval orchestration layer (a packaged tool, an internal wrapper, or a no-code workflow). The custom code you write sits at the orchestration layer and the prompt layer. Everything underneath is rented.

The economics of buying are dominated by per-token API fees and per-operation vector DB charges. As of early 2026, the major foundation model and vector DB pricing breaks down like this:

• OpenAI GPT-5 family: $1.25–$2.50 input / $10–$15 output per million tokens. Mini and nano variants run 5–10× cheaper.
• Anthropic Claude Opus 4.6: $5.00 input / $25.00 output per million tokens.
• Anthropic Claude Sonnet 4.6: $3.00 input / $15.00 output per million tokens.
• Pinecone Serverless (managed vector DB): $0.33 per GB/month for storage, with a $50/month minimum on Standard plans and $500/month on Enterprise. Read and write units are billed separately.

Sources: IntuitionLabs for foundation model pricing; Pinecone for vector DB pricing.

Per-million read unit rates vary by plan and region. Third-party benchmarks place Standard plan reads in the $8–$16 per million range and Enterprise reads higher, so workload modeling against Pinecone's calculator before committing is the safer path than relying on a single quoted rate.

For a typical mid-market B2B RAG system — 50,000 to 500,000 documents, 10,000 to 100,000 queries per month — the buy stack runs $2,000 to $15,000 per month all-in, before integration and engineering time. The Menlo Ventures 2025 report tracked enterprise LLM API spend at $8.4 billion by mid-2025, more than double the $3.5 billion of late 2024 (HPCwire), with Anthropic now leading the enterprise LLM API market at 40% share against OpenAI's 27% and Google's 21% (Menlo Ventures). Most of this is buy-side spend.

What does "building" a RAG system actually mean in 2026?

Building means owning more of the stack. It exists on a spectrum. The lightest version is self-hosted retrieval infrastructure (pgvector on PostgreSQL, self-hosted Qdrant, Milvus on Kubernetes) paired with API-based foundation models. The heaviest version is what Bloomberg did in 2023 — train a custom foundation model from scratch on proprietary data.

In 2026, the heaviest version of "build" is rarely justified. The base models from OpenAI, Anthropic, and Google have improved enough that custom-trained domain LLMs lose the comparison on most tasks, and the team that trained BloombergGPT has publicly acknowledged the lesson learned: with stronger base models available now, continued pretraining or fine-tuning from a strong open-source base like Llama is usually the right call instead of training from scratch (Rajiv Shah, LinkedIn).

The realistic build path in 2026 is: foundation model via API (sometimes fine-tuned), self-hosted vector database, custom retrieval and reranking layer, custom evaluation harness, and a dedicated ML engineering team to run it. Build economics are dominated by engineering hours and infrastructure capacity, not per-token fees. A typical mid-market B2B build engagement runs three to five engineers for three to six months at the front end, plus 20–40% of that team's time on ongoing operations. The vector DB cost can drop sharply at scale: at 10 million vectors, Pinecone Serverless runs roughly $70/month against $45/month for self-hosted pgvector on RDS. At 100 million vectors, the gap widens to $700+/month for Pinecone against under $100/month for self-hosted (LeanOps 2026 benchmark).

For deeper technical comparison between fine-tuning and RAG as architecture choices, see RAG vs Fine-Tuning for B2B Knowledge Systems: The Decision Framework.

Case study — Morgan Stanley: when buying is the right call

Morgan Stanley Wealth Management announced its strategic partnership with OpenAI in March 2023, becoming OpenAI's only strategic client in the wealth management sector and one of a handful of GPT-4 launch organizations. The bet was a buy decision at scale: GPT-4 plus Microsoft Azure plus a custom RAG layer over Morgan Stanley's proprietary research corpus, with a zero data retention agreement so client and proprietary data would not be used to train public models.

The architecture is a textbook B2B RAG buy stack — the foundation model is rented, the retrieval and orchestration layer is custom, the data stays internal. According to OpenAI's published customer story, the system grew from answering 7,000 questions in its first deployment to handling queries across a corpus of 100,000 internal documents, with one industry analysis pointing to a growing index of 350,000+ proprietary research documents by 2024. AI @ Morgan Stanley Assistant rolled out to financial advisors in September 2023. AI @ Morgan Stanley Debrief, a meeting summarization tool built on GPT-4 plus Whisper, launched firm-wide in June 2024. AskResearchGPT followed in October 2024 for the institutional securities side.

Two outcomes are publicly documented. First, over 98% of Morgan Stanley's advisor teams actively use the AI Assistant — a near-universal adoption rate that is rare even for well-built enterprise software. Second, the firm has expanded the platform from wealth management into investment banking, sales and trading, and research, indicating the buy decision scaled rather than capping out. The build was led by an internal team of fewer than ten people working with OpenAI directly, against a problem that would have taken a custom-model build team at least 18 months to ship at lower quality.

Why buy was the right call:

• Standard B2B RAG use case. Retrieval over institutional research, summary generation, meeting note synthesis. None of this required a custom-trained model — GPT-4 with retrieval grounding handled it.
• Speed to value. First production deployment six months from announcement. A custom-build path would have meant 18+ months minimum.
• Compliance via vendor contract, not in-house build. The zero data retention agreement and Microsoft Azure deployment satisfied financial regulatory requirements without Morgan Stanley needing to train models in-house.
• Concentration of effort. A small internal team plus OpenAI's research-side support produced a result that 50+ engineers at peer banks have not matched on adoption rate.

The Morgan Stanley case is the clearest enterprise-scale public reference for buy-side RAG done well. It also dispels a common assumption — that highly regulated industries need to build because vendors are off-limits. They don't. They need vendor contracts that meet the regulatory bar, which is a procurement problem, not an engineering one.

Case study — Bloomberg: when building is the right call (and when it stops being)

Bloomberg announced BloombergGPT in March 2023 — a 50.6 billion parameter language model trained from scratch on 709 billion tokens, with 363 billion tokens drawn from Bloomberg's proprietary FinPile dataset (40 years of financial documents) and 345 billion tokens from public sources. Training required 1.3 million GPU hours on 512 NVIDIA A100 40GB GPUs over roughly 53 days, with estimated training cost between $3 million and $10 million depending on infrastructure pricing (HPCwire). The team was nine people: four hands-on engineers and five advisors, plus four months of full-time work.

The bet was a build decision at the heaviest end of the spectrum: train a domain-specific LLM from scratch on proprietary financial data, deploy it as a competitive moat, and serve it through Bloomberg Terminal. BloombergGPT outperformed existing models on financial tasks by significant margins without sacrificing general-purpose performance, per the original paper.

Why build was the right call in 2023:

• Proprietary data was the moat. 40 years of structured financial documents that no public model had access to. The training data itself was the differentiator.
• Domain-specific accuracy was business-critical. Financial NLP has near-zero tolerance for hallucination. A general-purpose model could not be trusted on Bloomberg's regulatory and compliance use cases.
• Existing ML infrastructure. Bloomberg had the AWS budget, the GPU access, and the ML research team in place. The build cost was incremental, not foundational.
• Long-term moat at terminal scale. Bloomberg Terminal serves 350,000+ subscribers paying $30,000+ per year. A 1% improvement in research efficiency justifies an eight-figure model build.

The retrospective lesson, however, is the more interesting part of the case. Rajiv Shah, who interviewed the BloombergGPT team in 2023, summarized the team's own takeaway: "Don't train from scratch: Times have changed, and we now have stronger base models, so start experimenting with continued pretraining/fine-tuning from a strong base model." By 2026, with GPT-5 and Claude Opus 4.6 outperforming most domain-specific models on benchmarks Bloomberg measured against, the from-scratch path that made sense in 2023 would be much harder to justify today.

The lesson is not that build never wins. It's that the heavy version of build — train your own foundation model — has narrowed dramatically as base models have improved. The buy-side stack with selective fine-tuning now covers most of the use cases that the heaviest build path covered three years ago.

What are the six cost categories every RAG project must price?

Regardless of whether the decision lands on buy or build, every production RAG system has six cost categories. Underestimating any one of them is the most common cause of budget blowouts.

1. Foundation model (LLM) inference. The per-token API fee, or for a build, the GPU compute cost of running the model. This is the line item most teams price first, and it is usually the smallest of the six at moderate scale. Pricing varies by 50× across model tiers — GPT-5.4 Nano at $0.20/$1.25 per million tokens against GPT-5.4 Pro at $30/$180 (CloudZero) — so model selection is the single biggest cost lever within this category.

2. Embedding model. The cost of converting documents and queries into vectors. OpenAI's text-embedding-3-large costs $0.13 per million tokens (CloudZero), and Pinecone's hosted embeddings run around $0.08 per million. This category is small in absolute dollars but compounds rapidly when you re-embed a large document corpus or run multilingual retrieval.

3. Vector database. Storage, read units, write units, and (often missed) cross-region egress. Pinecone Serverless, Weaviate Cloud, Qdrant Cloud, and self-hosted pgvector each have different cost curves. Per the LeanOps 2026 analysis, the gap between vendor pricing-page estimates and actual production bills averages 2.5× to 4×, driven by underestimated read units, dimension count overhead, and namespace operations.

4. Retrieval and ranking infrastructure. The orchestration layer that runs hybrid retrieval (dense + BM25), reranking (cross-encoder models like Cohere Rerank or BGE-reranker), and chunk preparation. This category is mostly engineering hours plus a small inference fee. It is the layer most teams underestimate because it looks like glue code, but in production it is where most retrieval quality is won or lost.

5. Evaluation and observability. The harness for measuring retrieval quality, hallucination rate, and answer accuracy over time, plus the logging and monitoring stack. This is the category most B2B teams forget to budget. In regulated industries it is non-negotiable. The Morgan Stanley deployment built a rigorous evaluation framework with summarization evals, translation evals, and human-in-the-loop grading (OpenAI customer story) — that framework is what enables the 98% advisor adoption number, not the model itself.

6. Integration and ongoing maintenance. Connectors to source systems (CRM, document management, ticketing), security review, ongoing prompt iteration, model updates as engines reshuffle. This is dominated by engineering time and tends to scale with the number of source systems, not the volume of data. A 10-source integration is roughly 5× the maintenance cost of a 2-source integration.

The buy-vs-build tension shows up differently in each category. Buy reduces categories 1, 3, and 5 to API fees plus vendor product cost. Build trades those API fees for engineering hours and infrastructure capacity. Categories 2, 4, and 6 are similar in absolute cost regardless of buy or build — they are mostly engineering work either way.

When does buy beat build?

Buy beats build when most of the following conditions are met:

• The use case is standard B2B RAG. Internal knowledge retrieval, customer support, sales enablement, document Q&A, meeting summarization. The Morgan Stanley deployment is the canonical example — none of these required custom-model training.
• Data sensitivity is solvable through vendor contracts. Zero data retention agreements, regional processing endpoints, dedicated tenancy, and SOC 2 / HIPAA compliance from the vendor cover most regulated B2B use cases. If a vendor will sign the data agreement your legal team needs, buy is on the table.
• Time-to-value pressure is real. Buy paths ship to production in 8–16 weeks. Build paths ship in 6–18 months. If the business case requires returns inside 12 months, buy is the only viable answer.
• Internal ML engineering capacity is constrained. Build requires a dedicated team of three to five engineers with ML, retrieval, and infrastructure experience. If the team is two generalists splitting time across other projects, build will fail at production quality.
• Volume is below the API-cost crossover. For most mid-market B2B systems, foundation model API costs run $2K–$15K per month. Self-hosting a model becomes economically rational only above $50K–$100K per month in API spend, and even then often loses on quality.
• The use case is not a permanent moat. If the AI capability is operational rather than a core product differentiator, buy is the right answer. Build is justified only when the system itself becomes a competitive moat.

For most B2B mid-market clients Revenue Experts engages with, buy beats build on at least four of these six criteria. The default recommendation is a buy stack with selective fine-tuning where domain accuracy demands it.

When does build beat buy?

Build beats buy when most of these conditions are met:

• Proprietary data is the moat. When the data corpus itself is the competitive differentiator and cannot be exposed to a vendor contract, build becomes structurally necessary. Bloomberg's FinPile is the canonical example.
• Domain-specific accuracy is regulator-grade. Financial services, healthcare, defense, legal — categories where hallucination tolerance is measured in basis points, not percentages. Buy stacks have closed much of this gap, but specific high-stakes use cases still justify build.
• Volume is high enough to cross the cost curve. Self-hosted vector databases and self-hosted models become economically rational at scale — typically above 100M vectors or above $100K/month in foundation model API spend. Below that threshold, the operational overhead of build exceeds the savings.
• Existing ML and infrastructure team is in place. Build is not a project a marketing or product team can execute. It requires dedicated ML engineers, MLOps infrastructure, and a budget for ongoing operations. If the team is not already in place, the ramp cost dwarfs the build cost.
• The system is the product, not the tool. When the RAG system is the product offering itself — Bloomberg Terminal's research assistant, a vertical AI SaaS startup, a regulated AI service — build becomes the right answer because the system is the moat.

In practice, very few mid-market B2B brands meet four of these criteria. The cases that justify build today are large enterprises (Bloomberg-scale) or vertical AI SaaS startups whose product is the AI system itself. Everyone else is a buy candidate.

For the strategic case for RAG as the foundation of enterprise AI, see The Knowledge Layer: Why RAG Became the Foundation of Enterprise AI.

Apply this framework to your company

We run this framework as our $497 AI Visibility Audit, with the build-vs-buy diagnostic included for clients who are pricing their first RAG system or repricing one that came in over budget.

What you get:

• All six cost categories priced for your specific scenario at three usage scales — pilot, year-one production, year-three production
• Side-by-side TCO comparison: buy stack vs build stack, with the crossover point identified
• Decision criteria scorecard against the six buy criteria and five build criteria
• A concrete architecture spec — foundation model selection, vector database, retrieval orchestration, evaluation harness, integration plan
• Citation audit findings if your category overlaps with B2B AI search visibility
• A 30-minute walkthrough call to explain results

Turnaround: 5-7 business days from kickoff.

Pricing

You can also stop after the audit. Plenty of clients run the audit, get the architecture spec, and execute the buy stack themselves. The audit is useful even if you never hire us for follow-on work. 100% of the audit fee credits toward the Foundation sprint if you decide to move forward.

Want to see this framework applied to your scenario?

Book the $497 audit. 5-7 day turnaround.

Book the audit →

Frequently asked questions

How much does a typical B2B RAG system actually cost?

A mid-market B2B buy stack — 50,000 to 500,000 documents, 10,000 to 100,000 queries per month — runs $2,000 to $15,000 per month in vendor and API fees, plus an initial integration cost of $30,000 to $120,000 in engineering time. A comparable build engagement runs $200,000 to $600,000 in year-one engineering and infrastructure cost, with 20–40% of that as ongoing year-over-year operations. Buy wins on year-one TCO for almost all mid-market scenarios; build wins only at scale or when proprietary data demands it.

What is the biggest hidden cost in a RAG system?

Evaluation and observability. Most B2B teams budget for the foundation model and the vector database, then forget the harness needed to measure retrieval quality, hallucination rate, and answer accuracy over time. Without it, the system degrades silently as engines reshuffle and document corpora drift. Morgan Stanley's evaluation framework is the layer that enabled their 98% advisor adoption — without that layer, even a strong vendor stack will produce inconsistent results in production.

How does fine-tuning fit into the build-vs-buy decision?

Fine-tuning sits between buy and build. The foundation model is still rented, but the model has been trained further on proprietary data to improve domain accuracy. Fine-tuning costs are modest — GPT-4.1 fine-tuning runs roughly $3.00 per million training tokens, with inference at $3.00/$12.00 per million (CloudZero) — but the engineering work to assemble and validate the training data is substantial. Fine-tuning makes sense when buy's accuracy is close-but-not-quite and the gap is in domain-specific terminology or output format.

When does pgvector beat Pinecone for B2B RAG?

pgvector wins when the team already runs PostgreSQL in production, the vector workload is below 10 million vectors, and the team has DBA capacity. Above 10–20 million vectors, the operational overhead of self-hosted pgvector starts to exceed the savings against managed Pinecone. The crossover is roughly $300/month — below that, pgvector is cheaper; above that, Pinecone's operational simplicity wins for most teams.

Can a B2B team build a RAG system with no-code tools?

Yes, for the orchestration layer. Tools like n8n, Zapier, and LangFlow can assemble a working RAG pipeline using vendor APIs without writing custom code. This is a buy-stack execution path, not a build path — the foundation model, vector database, and retrieval are still rented. For tactical guidance on setting up the data layer with no-code tools, see How to Build Your First RAG Knowledge Base (No Coding Required).

Should I be worried about vendor lock-in with a buy-stack approach?

Lock-in is real but manageable. The mitigation is to keep your source-of-truth document corpus and your evaluation prompt set in a vendor-neutral format. The foundation model can be swapped (OpenAI to Anthropic, or vice versa) in days if the orchestration layer is well-abstracted. The vector database is harder to swap because reindexing 10M+ vectors is expensive, but storing source embeddings in cold object storage (S3 or GCS) makes a re-hydration possible without paying egress fees twice.

How do I know if my use case justifies build?

Score it against the five build criteria — proprietary data moat, regulator-grade accuracy demand, high volume, existing ML team, system-as-product. If you cannot score yes on at least three, build is not the right answer. The Bloomberg case scored yes on all five in 2023; very few B2B mid-market scenarios do today.

Conclusion

The build-vs-buy decision is the most expensive RAG decision a B2B leader will make, and the structural problem is that it is rarely framed clearly before the budget is set. The two public reference cases that anchor the extremes are Morgan Stanley (bought GPT-4 plus Microsoft Azure, reached 98% advisor adoption across approximately 16,000 financial advisors, scaled into investment banking) and Bloomberg (built BloombergGPT from scratch at $3M–$10M and 1.3M GPU hours, with the build team's own retrospective acknowledging that today's stronger base models would change that decision). For most mid-market B2B brands in 2026, buy is the right answer. The criteria that justify build — proprietary data moat, regulator-grade accuracy, high volume, existing ML team, system-as-product — apply to a small subset of cases.

The Revenue Experts framework runs three phases — cost projection across all six categories, decision criteria scorecard, architecture spec — to set the budget anchor before vendor demos pull the decision toward one path or the other. The output is a side-by-side TCO comparison and a concrete architecture brief.