LLM Savings: This Week's Cost-Cutting Insights

The LLM Cost‑Cutter

Weekly briefing on advances that shrink the cost of large language models (Feb 19–Feb 25, 2026)

Pricing announcements from AI labs this week

  • OpenAI: No new API per‑token price changes announced this week; current public price card continues to list GPT‑5.1 at $1.25/M input and $10/M output, GPT‑5.2 at $1.75/M input and $14/M output. (platform.openai.com)
  • Anthropic: No new API price cuts; Claude Sonnet 4.6 remains at $3/M input and $15/M output. (anthropic.com)
  • Google: No model price cuts; Vertex AI Agent Engine runtime and add‑on charges remain in effect. (cloud.google.com)
  • Mistral: No new public API price changes posted this week. (mistral.ai)

Bottom line: No headline token‑price cuts; most savings this week come from cheaper GPU hours, denser/cooler facilities, and inference efficiency research.

—

1) Street GPU prices you can use: fresh cuts and anchors

  • What’s new

    • Lambda updated its public price card on Feb 23, listing on‑demand instances at $5.74/GPU‑hr for B200 SXM and $3.44/GPU‑hr for H100 SXM; cluster commitments advertise B200 “as low as” $3.79/GPU‑hr and $3.49 with 1‑year reserve. These list rates are useful negotiation anchors even if your quote varies by region and term. (lambda.ai)
    • Runpod’s pricing pages continue to publish both pods and serverless tiers, with transparent per‑second serverless rates across L40/4090/A100/H100/H200/B200. Even when some on‑demand SKUs say “contact sales,” the visible serverless numbers help benchmark effective $/million tokens. (runpod.io)
    • CoreWeave still posts reference pricing (e.g., HGX H100), which buyers routinely use to frame bids and check reasonableness. (coreweave.com)
  • Why it matters for LLM costs

    • Public rate cards compress the spread between “headline” and “contracted” pricing. If you combine cheaper GPU hours with serving optimizations (see Section 4), you directly lower $/M tokens and raise “intelligence per dollar.” (lambda.ai)

—

2) Hardware and supply: memory prices up, capacity still expanding

  • What’s new

    • HBM4 pricing is trending higher: reporting this week pointed to Samsung planning HBM4 at roughly 20–30% above HBM3E. That keeps accelerator bills elevated even as next‑gen GPUs ramp. (barrons.com)
    • TSMC’s record capex authorization (about $45B) two weeks ago highlights an aggressive plan to grow leading‑edge wafer and advanced packaging capacity across 2026—critical for GPU supply and delivery times. (tomshardware.com)
    • Nvidia reports earnings after the bell today (Tue, Feb 25). Street focus includes data‑center mix and any color on Blackwell throughput per megawatt—signals that translate to $/token and tokens/watt as fleets refresh. (investors.com)
  • Why it matters for LLM costs

    • If HBM stays tight, server prices remain sticky; but sustained foundry/packaging capex should ease bottlenecks through 2026–27. More supply plus more efficient parts pushes tokens/sec per rack up and $/answer down, improving “intelligence per watt.” (tomshardware.com)

—

3) Power and policy: moratorium map shifts; PJM co‑location rules proceed

  • What’s new

    • Local permitting is fluid: Big Rapids Township, MI, enacted a 1‑year data‑center moratorium on Feb 23; meanwhile, Hood County, TX rejected a moratorium (again) on Feb 24, and Sangamon County, IL advanced a major site despite public pushback. These show siting risk varies county‑by‑county. (bigrapidsnews.com)
    • In PJM, FERC’s directive to create co‑location‑friendly services (including new firm and non‑firm “Contract Demand” transmission options) hit a key filing milestone on Feb 16; responses are due Mar 18. Expect clearer paths to put generation next to load—i.e., power plants next to AI campuses. (ferc.gov)
    • The “shadow grid” trend—building off‑grid or semi‑islanded campuses with on‑site gas and some solar—continues to gain coverage, citing faster time‑to‑compute alongside scrutiny on emissions. (washingtonpost.com)
  • Why it matters for LLM costs

    • Faster, more predictable power hookups reduce delays (time is money) and dampen price spikes during grid stress—shrinking effective $/answer. Co‑location options in PJM can cut interconnection uncertainty and improve fleet utilization, while local politics can still swing siting timelines. (ferc.gov)

—

4) Facilities and cooling: toward megawatt‑class racks

  • What’s new

    • Industry leaders say the sector is racing from today’s 100‑200 kW racks toward near‑megawatt densities, accelerating liquid‑cooling adoption. Vendors tied to power/cooling report strong backlogs as AI loads surge. (m.economictimes.com)
  • Why it matters for LLM costs

    • Better cooling lowers overhead power (PUE), holds boost clocks, and enables denser layouts—so more useful tokens per watt and per square foot. As racks densify, the “cooling line” in your cost stack falls per unit of inference delivered. (m.economictimes.com)

—

5) Serving research: fresh methods to cut GPU time per token

  • What’s new

    • WiSparse shows a training‑free, weight‑aware activation sparsity method that keeps ~97% of Llama‑3.1 quality at 50% sparsity and reports ~21% end‑to‑end speedup—useful when you can’t retrain. (arxiv.org)
    • MoE‑Spec introduces expert “budgeting” during speculative decoding for MoE models, yielding 10–30% higher throughput vs strong baselines at similar quality—addressing MoE memory churn during verification. (arxiv.org)
    • Dynamic Delayed Tree Expansion refines multi‑path speculative decoding, reporting ~5% higher average throughput than traversal‑verification baselines by delaying branching based on context. (arxiv.org)
    • LycheeDecode proposes hybrid‑head sparse decoding for long contexts, claiming up to 2.7× speedups at 128K tokens without hurting quality, by letting retrieval heads pick shared key tokens. (arxiv.org)
  • Why it matters for LLM costs

    • These techniques reduce GPU time and memory traffic per generated token—directly lowering $/M tokens and raising “intelligence per watt.” They also stack with cheaper GPU hours and better cooling for compound savings. (arxiv.org)

—

6) Model product moves without price cuts

  • What’s new

    • Anthropic is pushing Sonnet 4.6 as the default in its apps at the same API price as 4.5, with a 1M‑token context in API beta. For many tasks, that’s “near‑Opus” capability at Sonnet prices. (anthropic.com)
  • Why it matters for LLM costs

    • More capability at the same per‑token price means more real work per dollar. Pairing a stronger mid‑tier model with the inference tricks above can beat flagship models on both cost and “intelligence per watt.” (anthropic.com)

—

This week’s impact in plain English

  • No API token price cuts, but total cost per answer is still sliding as: GPU list prices at some clouds moved down or clarified; liquid‑cooling and density plans cut overhead power; and new decoding/sparsity methods cut GPU time per token. Together, these raise tokens per dollar and “intelligence per watt.” (lambda.ai)

—

Action checklist (1–2 hour tasks)

  1. Re‑shop inference capacity: use Lambda’s Feb 23 price card and Runpod serverless rates to anchor a re‑bid on your steady‑state fleet; model $/M tokens using your observed tokens/sec. (lambda.ai)
  2. Pilot a speculative‑decoding upgrade on one MoE route using MoE‑Spec; target 10–20% throughput gain at equal quality. (arxiv.org)
  3. For long‑context routes, A/B LycheeDecode or a similar sparse‑decoding approach at 128K; measure latency and acceptance vs. baseline. (arxiv.org)
  4. Power risk scan in PJM: note Feb 16 filings and Mar 18 response date; map whether co‑location services could accelerate your next campus. (ferc.gov)
  5. Cooling ROI quick pass: estimate PUE delta moving a pilot rack from air to liquid under your $/M‑token targets; factor raised rack density. (m.economictimes.com)

—

Sources and further reading

—
If you want, I can attach a simple calculator that turns your observed tokens/sec and these current GPU rates into estimated $/M tokens for your stack.