Most engineers constructing on massive language mannequin APIs function below a flawed assumption: that per-token pricing is flat no matter what number of tokens get despatched. It’s not. LLM context price modeling reveals a nonlinear actuality as soon as prompts exceed sure thresholds. Google applies tiered Gemini pricing throughout a number of context ranges, and Anthropic introduces important shifts in immediate caching economics at 200k tokens. A 300k-token immediate on Google Gemini prices as much as twice as a lot per enter token as a 100k-token immediate; on Anthropic, uncooked enter charges are flat, however caching economics shift materially above 200k tokens. At 1,000 requests per day, a 300k-token workload on Gemini 1.5 Professional overspends by roughly $125/day in comparison with what a flat-rate assumption would predict.
This text builds a purpose-built TypeScript library and interactive CLI device for modeling tiered pricing past 200k tokens. The device calculates precise nonlinear pricing thresholds, immediate cache break-even factors, and retrieval-vs-long-context price trade-offs throughout Anthropic, OpenAI, and Google APIs in a single unified mannequin. Each price perform runs in opposition to actual utilization patterns so you possibly can calculate prices as a substitute of estimating them.
Pricing disclaimer: All costs on this article replicate printed charges as of mid-2025. Confirm at anthropic.com/pricing, openai.com/api/pricing, and ai.google.dev/pricing earlier than manufacturing budgeting.
Desk of Contents
The Non-Linear Value Actuality Throughout Main Suppliers
How Anthropic, OpenAI, and Google Construction Token Pricing
Every main supplier buildings token pricing in a different way, however all share one trait: prices usually are not uniform throughout the total marketed context window as soon as caching and tiering are thought-about.
Anthropic costs Claude fashions with a boundary at 200k tokens that impacts caching economics. For Claude 3.5 Sonnet, Anthropic costs $3.00 per million enter tokens in any respect context lengths. The 200k boundary impacts immediate caching economics solely, not base enter pricing. On cache writes, Anthropic provides a 25% premium, so the primary time tokens enter the cache they price $3.75 per million fairly than the usual $3.00. Claude 3 Opus costs $15.00 per million enter tokens, with output tokens at $75.00 per million (confirm at anthropic.com/pricing earlier than manufacturing budgeting).
OpenAI costs GPT-4o at $2.50 per million enter tokens and $10.00 per million output tokens. OpenAI’s computerized immediate caching offers a 50% low cost on cached enter tokens, with no extra write price. OpenAI applies a flat fee throughout the total 128k window, however factoring in caching habits, efficient enter price drops from $2.50 to $1.25 per million on cache hits, a 50% discount that reshapes budgets at scale.
Google applies essentially the most explicitly tiered construction. Gemini 1.5 Professional costs $1.25 per million enter tokens for prompts as much as 128k tokens, then jumps to $2.50 per million for prompts exceeding 128k. Gemini 2.5 Professional makes use of an analogous tiered construction. Output pricing follows the identical sample: $5.00 per million tokens below 128k, $10.00 per million above it for Gemini 1.5 Professional.
| Supplier | Mannequin | Enter ≤128k (per 1M) | Enter 128k-200k (per 1M) | Enter >200k (per 1M) | Output (per 1M) |
|---|---|---|---|---|---|
| Anthropic | Claude 3.5 Sonnet | $3.00 | $3.00 | $3.00 | $15.00 |
| Anthropic | Claude 3 Opus | $15.00 | $15.00 | $15.00 | $75.00 |
| OpenAI | GPT-4o | $2.50 | $2.50 | N/A (128k max) | $10.00 |
| Gemini 1.5 Professional | $1.25 | $2.50 | $2.50 | ≤128k: $5.00 / >128k: $10.00 | |
| Gemini 2.5 Professional | $1.25 | $2.50 | $2.50 | ≤128k: $10.00 / >128k: $15.00 |
This desk is designed to be referenced and shared. The important thing perception it surfaces: Google’s pricing doubles on the 128k boundary, making it the supplier the place tiered modeling issues most for uncooked enter prices. Anthropic’s enter pricing is flat throughout all context lengths, however its caching economics introduce nonlinearity above 200k tokens.
Google’s pricing doubles on the 128k boundary, making it the supplier the place tiered modeling issues most for uncooked enter prices.
Why Prices Escalate: The Infrastructure Behind Lengthy Context
The pricing tiers replicate actual infrastructure prices. Transformer-based fashions preserve a key-value (KV) cache that grows with sequence size. Consideration computation scales quadratically with context size in naive self-attention implementations, although manufacturing methods use FlashAttention to scale back reminiscence overhead. Whereas suppliers like Google have invested in ring consideration and different parallelism strategies, serving a 500k-token context requires roughly 10x the KV-cache reminiscence of a 50k-token context. Suppliers cost extra for longer contexts to cowl this overhead.
This creates a distinction value internalizing: the marketed context window is the utmost a mannequin accepts, whereas the cost-efficient context window is the vary the place pricing stays economical for a given workload. For Gemini 1.5 Professional, the marketed window is 1 million tokens, however cost-efficient utilization stays below 128k, the place per-token enter price is $1.25 fairly than $2.50. Engineers who conflate these two numbers constantly overshoot budgets.
Designing the Value Modeling Library in TypeScript
Venture Setup and Dependencies
The device makes use of TypeScript on Node.js ≥18 (required for ESM and WASM help), tiktoken for token relying on OpenAI fashions, and commander.js for CLI construction.
Warning:
tiktokenis calibrated to OpenAI’s tokenizer solely. For Anthropic fashions, use the@anthropic-ai/sdk‘scountTokensmethodology. For Google Gemini, use the Gemini API’scountTokensendpoint. Tokenizers differ throughout suppliers, and counts can fluctuate by 10-20% for non-English or code-heavy content material.
Conditions:
- Node.js ≥18 — run
node --versionto confirm earlier than putting in dependencies - npm ≥8 (or pnpm/yarn equal)
{
"title": "llm-cost-modeler",
"model": "1.0.0",
"kind": "module",
"bin": {
"llm-cost": "dist/cli.js"
},
"scripts": {
"construct": "tsc",
"begin": "node dist/cli.js"
},
"dependencies": {
"commander": "^12.1.0",
"tiktoken": "^1.0.15"
},
"devDependencies": {
"typescript": "^5.5.0",
"@sorts/node": "^20.14.0"
}
}
{
"compilerOptions": {
"goal": "ES2022",
"module": "Node16",
"moduleResolution": "Node16",
"outDir": "dist",
"strict": true
},
"embody": ["src"]
}
Word: With module: Node16, all relative imports should use .js extensions even when importing .ts supply recordsdata. For instance: import { calculateContextCost } from './pricing.js'.
Venture construction:
llm-cost-modeler/
├── bundle.json
├── tsconfig.json
└── src/
├── pricing.ts
├── cache.ts
├── rag.ts
└── cli.ts
Defining Supplier Pricing Tiers as Knowledge
Nonlinear pricing guidelines are finest encoded as typed configuration fairly than buried in conditional logic. Every supplier’s pricing turns into a structured array of tier objects, the place every tier defines a token ceiling and its corresponding per-token fee.
export interface PricingTier {
readonly maxTokens: quantity;
readonly inputPricePerMillion: quantity;
readonly outputPricePerMillion: quantity;
readonly cachedInputPricePerMillion: quantity;
readonly cacheWritePricePerMillion: quantity;
}
export interface ProviderConfig {
readonly title: string;
readonly mannequin: string;
readonly tiers: readonly PricingTier[];
}
export const suppliers: ProviderConfig[] = [
{
name: "Google",
model: "Gemini 2.5 Pro",
tiers: [
{
maxTokens: 128_000,
inputPricePerMillion: 1.25,
outputPricePerMillion: 10.0,
cachedInputPricePerMillion: 0.3125,
cacheWritePricePerMillion: 1.25
},
{
maxTokens: Infinity,
inputPricePerMillion: 2.50,
outputPricePerMillion: 15.0,
cachedInputPricePerMillion: 0.625,
cacheWritePricePerMillion: 2.50
},
],
},
{
title: "Anthropic",
mannequin: "Claude 3.5 Sonnet",
tiers: [
{
maxTokens: 200_000,
inputPricePerMillion: 3.0,
outputPricePerMillion: 15.0,
cachedInputPricePerMillion: 0.30,
cacheWritePricePerMillion: 3.75
},
{
maxTokens: Infinity,
inputPricePerMillion: 3.0,
outputPricePerMillion: 15.0,
cachedInputPricePerMillion: 0.30,
cacheWritePricePerMillion: 3.75
},
],
},
{
title: "OpenAI",
mannequin: "GPT-4o",
tiers: [
{
maxTokens: 128_000,
inputPricePerMillion: 2.50,
outputPricePerMillion: 10.0,
cachedInputPricePerMillion: 1.25,
cacheWritePricePerMillion: 2.50
},
],
},
];
Word that Anthropic’s cacheWritePricePerMillion is $3.75, reflecting the 25% premium on the $3.00 base enter fee. OpenAI’s cached fee is precisely 50% of ordinary enter, with no separate write premium (the cacheWritePricePerMillion is about equal to the usual enter fee as a result of OpenAI doesn’t cost a definite write payment).
The Core Value Calculation Operate
The algorithm splits the whole token rely throughout tiers, multiplies every section by its relevant fee, and accumulates. Output tokens are priced individually since they usually don’t observe the identical tiered construction as enter tokens (although some suppliers tier each).
interface CostBreakdown {
supplier: string;
mannequin: string;
inputTokens: quantity;
outputTokens: quantity;
tierBreakdown: { tier: quantity; tokens: quantity; price: quantity }[];
totalInputCost: quantity;
totalOutputCost: quantity;
totalCost: quantity;
}
export perform calculateContextCost(
config: ProviderConfig,
inputTokens: quantity,
outputTokens: quantity
): CostBreakdown {
if (inputTokens < 0 || outputTokens < 0) {
throw new Error(
`Token counts should be non-negative. Acquired inputTokens=${inputTokens}, outputTokens=${outputTokens}`
);
}
const tierBreakdown: { tier: quantity; tokens: quantity; price: quantity }[] = [];
let remainingTokens = inputTokens;
let totalInputCost = 0;
let previousMax = 0;
for (let i = 0; i < config.tiers.size; i++) {
const tier = config.tiers[i];
const tierCapacity = tier.maxTokens === Infinity
? remainingTokens
: tier.maxTokens - previousMax;
const tokensInTier = Math.min(remainingTokens, tierCapacity);
if (tokensInTier <= 0) break;
const price = (tokensInTier / 1_000_000) * tier.inputPricePerMillion;
tierBreakdown.push({ tier: i + 1, tokens: tokensInTier, price });
totalInputCost += price;
remainingTokens -= tokensInTier;
previousMax = tier.maxTokens === Infinity ? previousMax : tier.maxTokens;
}
if (remainingTokens > 0) {
throw new Error(
`${remainingTokens} enter tokens couldn't be assigned to any pricing tier ` +
`for ${config.title} ${config.mannequin}. ` +
`Guarantee the ultimate tier has maxTokens: Infinity or that inputTokens doesn't exceed the supplier's context window.`
);
}
const applicableTier = config.tiers.discover(t => inputTokens <= t.maxTokens);
if (!applicableTier) {
throw new Error(
`Enter token rely ${inputTokens} exceeds all outlined tiers for ${config.title} ${config.mannequin}`
);
}
const totalOutputCost = (outputTokens / 1_000_000) * applicableTier.outputPricePerMillion;
return {
supplier: config.title,
mannequin: config.mannequin,
inputTokens,
outputTokens,
tierBreakdown,
totalInputCost,
totalOutputCost,
totalCost: totalInputCost + totalOutputCost,
};
}
The sting case value noting: batch pricing. Each Anthropic and OpenAI supply 50% reductions on batch API requests submitted by way of devoted batch endpoints; real-time API calls are ineligible for batch pricing. This perform handles real-time pricing; extending it for batch mode requires halving the relevant charges, which may be completed by including a mode parameter to the supplier config.
Immediate Caching Break-Even Evaluation
How Immediate Caching Adjustments the Math
All three suppliers now supply immediate caching, however the mechanisms differ in ways in which have an effect on your break-even math. Anthropic requires express cache management headers and costs 25% above base fee on the primary write, then reductions cache reads by 90%. Anthropic requires cache-eligible content material to seem in the beginning of the immediate and to exceed a minimal block dimension (1,024 tokens for Claude 3; 2,048 tokens for Claude 3.5 and later). Content material that doesn’t meet these necessities is processed as commonplace uncached enter. OpenAI routinely caches prompts longer than 1,024 tokens with a 50% low cost on cached tokens and no write premium. Google’s context caching presents a 75% low cost on cached enter tokens.
The cache hit ratio — the share of requests that reuse cached context — is the one most vital variable. A system immediate reused throughout each request may have a success ratio close to 100%. A per-user doc context that adjustments often would possibly sit at 10-20%.
Calculating Your Cache Break-Even Level
The break-even calculation solutions: at what cache hit ratio does the whole price (cache writes plus cached reads plus uncached reads) drop beneath the price of sending full uncached context each time? The writesCount parameter accounts for cache TTL expiry: if inter-request gaps exceed the supplier’s cache TTL (about 5 minutes for Anthropic), the write price is incurred once more. Set writesCount to the anticipated variety of cache inhabitants occasions throughout the request window.
import { suppliers } from "./pricing.js";
import kind { ProviderConfig } from "./pricing.js";
interface CacheBreakEven {
supplier: string;
mannequin: string;
inputTokens: quantity;
breakEvenHitRatio: quantity;
costAtFullCache: quantity;
costWithoutCache: quantity;
}
export perform calculateCacheBreakEven(
config: ProviderConfig,
inputTokens: quantity,
totalRequests: quantity,
writesCount: quantity = 1
): CacheBreakEven {
const tier = config.tiers.discover(t => inputTokens <= t.maxTokens)
?? config.tiers[config.tiers.length - 1];
const uncachedCostPerReq = (inputTokens / 1_000_000) * tier.inputPricePerMillion;
const costWithoutCache = uncachedCostPerReq * totalRequests;
const writeCostPerEvent = (inputTokens / 1_000_000) * tier.cacheWritePricePerMillion;
const totalWritesCost = writeCostPerEvent * writesCount;
const cachedReadCost = (inputTokens / 1_000_000) * tier.cachedInputPricePerMillion;
const denominator = totalRequests * (uncachedCostPerReq - cachedReadCost);
if (denominator <= 0) {
throw new Error(
"Cache learn price >= uncached price; caching by no means breaks even for this config."
);
}
const breakEvenHitRatio = totalWritesCost / denominator;
const costAtFullCache = totalWritesCost + (totalRequests * cachedReadCost);
return {
supplier: config.title,
mannequin: config.mannequin,
inputTokens,
breakEvenHitRatio: Math.min(Math.max(breakEvenHitRatio, 0), 1),
costAtFullCache,
costWithoutCache,
};
}
For Anthropic’s Claude 3.5 Sonnet with 200k enter tokens and 100 requests, the break-even hit ratio is 1.4% — a single cache write adopted by two cache hits already saves cash, as a result of the 90% cached low cost is steep. Groups delay implementing caching as a result of they assume they want near-100% hit ratios. The mathematics says in any other case. This components makes use of the writesCount parameter to account for TTL-based expiry in sustained workloads. For a single cache inhabitants occasion, the default writesCount = 1 applies. For an hourly workload in opposition to Anthropic’s ~5-minute TTL, set writesCount to 12 per hour to mannequin repeated write prices precisely.
Lengthy Context vs. RAG: Modeling the Value Commerce-Off
When Retrieval Is Cheaper Than Stuffing the Context
The lengthy context vs. RAG price comparability requires accounting for the total retrieval stack: embedding API calls to vectorize queries, vector database internet hosting prices (Pinecone, Weaviate, or pgvector), and the diminished LLM context wanted per question. In opposition to that, long-context approaches ship the complete corpus per request however skip retrieval infrastructure solely.
The trade-off just isn’t purely monetary. RAG introduces retrieval latency and the danger of missed related chunks — a threat you possibly can tune by adjusting top-k and recall thresholds, however by no means eradicate solely. Lengthy context avoids retrieval errors however incurs larger per-request prices. Latency additionally grows with context size: count on response instances to roughly double between 50k and 200k enter tokens on most suppliers (benchmark by yourself deployment, as {hardware} and batching methods fluctuate). The accuracy implications are workload-dependent, however the fee implications are calculable.
The Crossover Level Calculator
The crossover perform compares day by day prices for each approaches given a corpus dimension, common question dimension, and day by day question quantity. The retrievedChunkTokens parameter controls the assumed dimension of retrieved context per RAG question (e.g., top-10 chunks at 200 tokens every = 2,000 tokens).
import { calculateContextCost } from "./pricing.js";
import kind { ProviderConfig } from "./pricing.js";
interface CostComparison {
supplier: string;
longContextDailyCost: quantity;
ragDailyCost: quantity;
advice: string;
}
export perform modelRetrievalVsContext(
config: ProviderConfig,
corpusTokens: quantity,
avgQueryTokens: quantity,
outputTokens: quantity,
queriesPerDay: quantity,
embeddingCostPerMillion: quantity,
vectorDbDailyCost: quantity,
retrievedChunkTokens: quantity = 2_000
): CostComparison {
const longCtx = calculateContextCost(
config,
corpusTokens + avgQueryTokens,
outputTokens
);
const longContextDailyCost = longCtx.totalCost * queriesPerDay;
const embeddingCost =
(avgQueryTokens / 1_000_000) * embeddingCostPerMillion * queriesPerDay;
const ragCtx = calculateContextCost(
config,
retrievedChunkTokens + avgQueryTokens,
outputTokens
);
const ragDailyCost =
(ragCtx.totalCost * queriesPerDay) + embeddingCost + vectorDbDailyCost;
return {
supplier: config.title,
longContextDailyCost,
ragDailyCost,
advice:
longContextDailyCost < ragDailyCost ? "Lengthy Context" : "RAG",
};
}
The crossover level shifts with question quantity. At fewer than 10 queries per day, vector database internet hosting prices dominate, making lengthy context cheaper. At excessive volumes with massive corpora, RAG wins as a result of every request processes only some thousand retrieved tokens fairly than the total corpus. The pattern session beneath reveals the magnitude: at 500k tokens and 100 day by day queries on Gemini, RAG prices $3.45/day vs. $140.00/day for lengthy context.
CLI Instructions and Interactive Prompts
The three library capabilities map instantly to 3 CLI instructions: price, cache-breakeven, and evaluate.
import { Command } from "commander";
import { suppliers, calculateContextCost } from "./pricing.js";
import { calculateCacheBreakEven } from "./cache.js";
import { modelRetrievalVsContext } from "./rag.js";
const program = new Command();
program
.title("llm-cost")
.description("LLM context price modeling device")
.model("1.0.0");
perform parsePositiveInt(worth: string, fieldName: string): quantity {
const parsed = Quantity(worth);
if (!Quantity.isInteger(parsed) || parsed <= 0) {
console.error(
`Error: --${fieldName} should be a constructive integer, received: ${worth}`
);
course of.exit(1);
}
return parsed;
}
perform parseNonNegativeFloat(worth: string, fieldName: string): quantity {
const parsed = parseFloat(worth);
if (isNaN(parsed) || parsed < 0) {
console.error(
`Error: --${fieldName} should be a non-negative quantity, received: ${worth}`
);
course of.exit(1);
}
return parsed;
}
program.command("price")
.description("Calculate tiered context price")
.requiredOption("-p, --provider <title>", "Supplier: Google, Anthropic, OpenAI")
.requiredOption("-i, --input <tokens>", "Enter token rely",
(v: string) => parsePositiveInt(v, "enter"))
.choice("-o, --output <tokens>", "Output token rely",
(v: string) => parsePositiveInt(v, "output"), 1000)
.motion((opts) => {
const config = suppliers.discover(
p => p.title.toLowerCase() === opts.supplier.toLowerCase()
);
if (!config) {
console.error(
`Unknown supplier: ${opts.supplier}. Legitimate choices: ${suppliers.map(p => p.title).be a part of(", ")}`
);
course of.exit(1);
}
const outcome = calculateContextCost(config, opts.enter, opts.output);
console.desk(outcome.tierBreakdown);
console.log(`Complete: $${outcome.totalCost.toFixed(4)}`);
});
program.command("cache-breakeven")
.description("Calculate immediate cache break-even hit ratio")
.requiredOption("-p, --provider <title>", "Supplier title")
.requiredOption("-i, --input <tokens>", "Enter token rely",
(v: string) => parsePositiveInt(v, "enter"))
.choice("-r, --requests <rely>", "Complete requests",
(v: string) => parsePositiveInt(v, "requests"), 100)
.choice("-w, --writes <rely>",
"Variety of cache write occasions (for TTL expiry modeling)",
(v: string) => parsePositiveInt(v, "writes"), 1)
.motion((opts) => {
const config = suppliers.discover(
p => p.title.toLowerCase() === opts.supplier.toLowerCase()
);
if (!config) {
console.error(
`Unknown supplier: ${opts.supplier}. Legitimate choices: ${suppliers.map(p => p.title).be a part of(", ")}`
);
course of.exit(1);
}
const outcome = calculateCacheBreakEven(
config, opts.enter, opts.requests, opts.writes
);
console.log(
`Break-even hit ratio: ${(outcome.breakEvenHitRatio * 100).toFixed(1)}%`
);
console.log(`Value with out cache: $${outcome.costWithoutCache.toFixed(4)}`);
console.log(`Value at 100% cache: $${outcome.costAtFullCache.toFixed(4)}`);
});
program.command("evaluate")
.description("Examine RAG vs long-context prices")
.requiredOption("-p, --provider <title>", "Supplier title")
.requiredOption("-c, --corpus <tokens>", "Corpus dimension in tokens",
(v: string) => parsePositiveInt(v, "corpus"))
.requiredOption("-q, --queries <rely>", "Queries per day",
(v: string) => parsePositiveInt(v, "queries"))
.choice("--query-tokens <n>", "Avg question dimension in tokens",
(v: string) => parsePositiveInt(v, "query-tokens"), 500)
.choice("--output-tokens <n>", "Output tokens per question",
(v: string) => parsePositiveInt(v, "output-tokens"), 1000)
.choice("--vector-db-cost <{dollars}>", "Vector DB day by day price in USD",
(v: string) => parseNonNegativeFloat(v, "vector-db-cost"), 1.0)
.choice("--retrieved-chunks <tokens>",
"Retrieved chunk dimension in tokens",
(v: string) => parsePositiveInt(v, "retrieved-chunks"), 2000)
.choice("--embedding-cost <perMillion>",
"Embedding price per 1M tokens",
(v: string) => parseNonNegativeFloat(v, "embedding-cost"), 0.10)
.motion((opts) => {
const config = suppliers.discover(
p => p.title.toLowerCase() === opts.supplier.toLowerCase()
);
if (!config) {
console.error(
`Unknown supplier: ${opts.supplier}. Legitimate choices: ${suppliers.map(p => p.title).be a part of(", ")}`
);
course of.exit(1);
}
const outcome = modelRetrievalVsContext(
config,
opts.corpus,
opts.queryTokens,
opts.outputTokens,
opts.queries,
opts.embeddingCost,
opts.vectorDbCost,
opts.retrievedChunks,
);
console.log(
`Lengthy Context: $${outcome.longContextDailyCost.toFixed(4)}/day`
);
console.log(
`RAG: $${outcome.ragDailyCost.toFixed(4)}/day`
);
console.log(`Suggestion: ${outcome.advice}`);
});
program.parse();
Pattern CLI Session
$ npx llm-cost price -p Anthropic -i 350000 -o 2000
┌─────────┬──────┬────────┬────────────┐
│ (index) │ tier │ tokens │ price │
├─────────┼──────┼────────┼────────────┤
│ 0 │ 1 │ 200000 │ 0.6000 │
│ 1 │ 2 │ 150000 │ 0.4500 │
└─────────┴──────┴────────┴────────────┘
Complete: $1.0800
The tier breakdown reveals enter prices solely. The whole consists of 2,000 output tokens at $15.00/M = $0.03, along with the enter tier prices proven ($0.60 + $0.45 + $0.03 = $1.08).
$ npx llm-cost cache-breakeven -p Anthropic -i 200000 -r 100
Break-even hit ratio: 1.4%
Value with out cache: $60.0000
Value at 100% cache: $6.7500
$ npx llm-cost evaluate -p Google -c 500000 -q 100
Lengthy Context: $140.0000/day
RAG: $3.4500/day
Suggestion: RAG
The pattern output demonstrates the core perception: at 500k tokens and 100 day by day queries on Google’s Gemini pricing, RAG prices $3.45/day versus $140.00/day for long-context processing. The evaluate command accepts --vector-db-cost, --query-tokens, --output-tokens, --retrieved-chunks, and --embedding-cost flags to mannequin your precise deployment; defaults are $1.00/day for vector DB, 500 question tokens, 1,000 output tokens, 2,000 retrieved chunk tokens, and $0.10/M embedding price.
At 500k tokens and 100 day by day queries on Google’s Gemini pricing, RAG prices $3.45/day versus $140.00/day for long-context processing.
Three Guidelines for Context-Size Budgeting
- Mannequin prices on the precise working context size, not the bottom tier fee. Google’s pricing doubles at 128k tokens. Utilizing the sub-128k fee to finances a 300k-token workload understates prices by almost 50%.
- Immediate caching reaches break-even at decrease hit ratios than most groups count on. At 200k tokens throughout 100 requests, Anthropic’s break-even hit ratio is 1.4% with a single cache write — even sporadic cache reuse pays off in opposition to the write premium. This assumes the cache stays populated; use the
writesCountparameter (or--writesCLI flag) to account for TTL-based expiry in sustained workloads. - Selecting between RAG and lengthy context relies on price as a lot as structure. The crossover level is calculable and shifts with question quantity, corpus dimension, and supplier pricing.
So as to add a brand new mannequin, append a ProviderConfig entry and each calculation updates routinely. Token pricing engineering and immediate caching price optimization usually are not one-time analyses. They save extra as utilization grows. Re-verify supplier pricing constants frequently — LLM pricing adjustments often, and rancid constants will silently produce incorrect estimates.

