LLM Cost Savings: Weekly Update!

The LLM Cost‑Cutter

Weekly briefing on advances that shrink the cost of large language models (Nov 26–Dec 2, 2025)

Pricing announcements from AI labs this week

  • No public per‑token price cuts were announced by OpenAI, Anthropic, Google, Meta, Mistral, or Cohere over the last 7 days. OpenAI’s pricing page and Anthropic’s pricing remain unchanged week‑over‑week; Google’s Vertex AI release notes list no new model price reductions in this window. (platform.openai.com)

—

1) AWS unveils Trainium 3 (and teases Trainium 4): more tokens per dollar and per watt

  • What’s new: At re:Invent, AWS introduced Trainium 3, claiming roughly 4× the performance of the prior generation and positioning it as a lower‑cost alternative to NVIDIA for training large models. Reports also note AWS is already planning Trainium 4. Plain English: Amazon is doubling down on its own AI chips so customers can train models cheaper and use less power per unit of work. (investors.com)
  • Why it matters: If Trainium 3 delivers on performance‑per‑watt, model training costs fall and capacity expands without needing as much electricity or as many GPU‑hours. That improves “intelligence per watt” and gives labs leverage when negotiating cloud prices—particularly for batch training and fine‑tuning.

2) Transparent GPU prices keep pressure on cost floors

  • What’s new: Public price cards continue to show downward pressure at the “good‑enough” tier:
    • CoreWeave lists RTX Pro 6000 Blackwell Server Edition at $20/hr; H100/H200 racks are also posted. Plain English: you can still see — and shop — posted rates for many high‑end nodes. (coreweave.com)
    • Runpod’s pages list a wide spread of consumer and data‑center GPUs (including RTX 5090) and serverless per‑second pricing that makes bursty inference cheaper. (runpod.io)
    • Salad/Vast‑style marketplaces keep RTX 5090 rental prices in the ~$.25/hr ballpark for non‑SLA workloads. Plain English: hobbyist‑class silicon remains a bargain for evals and batch jobs. (techradar.com)
  • Why it matters: Being able to mix‑and‑match cheaper GPUs for the right jobs cuts dollars per request immediately. Pairing low hourly rates with modern serving tricks (see item 5) compounds savings.

3) State rules are settling: “large‑load” tariffs lock in predictable power costs

  • What’s new: Missouri finalized two large‑load tariffs in mid‑to‑late November (Ameren and Evergy), with requirements like long terms, minimum bills, and “pay your share” protections for other customers. Kansas approved a similar plan earlier in November. While approvals are just outside the 7‑day window, commissions posted confirmations and recaps this week. Plain English: big data centers can now plug in under clear rules and known prices. (psc.mo.gov)
  • Why it matters: Predictable $/kWh and faster interconnection processes reduce project risk and financing costs. That lowers the all‑in cost per token over a data center’s life and improves utilization—more tokens per dollar and steadier “intelligence per watt.”

4) Capacity expansion on non‑NVIDIA silicon: AMD MI355X cluster announced

  • What’s new: Vultr will invest $1B in an Ohio AI cluster (50 MW, 24,000 GPUs) using AMD Instinct MI355X, aiming to offer services at roughly half hyperscaler rates. Plain English: cheaper, alternative chips are being deployed at meaningful scale. (reuters.com)
  • Why it matters: More vendor diversity and Ethernet‑fabric clusters push market prices down and broaden availability. For teams willing to tune inference/training stacks on AMD, cost per token can fall materially, particularly for batch and fine‑tune jobs.

5) Serving efficiency keeps improving (software > hardware upgrades)

  • Recent wins to watch (building on last month’s momentum):
    • NVIDIA’s Blackwell stack continues to emphasize low‑precision formats (FP4/FP8) and efficiency; MLPerf v5.1 training results underscore throughput per GPU improvements that generally translate to lower energy per token. Plain English: newer kernels squeeze more speed from each watt. (developer.nvidia.com)
    • On the AMD side, MLPerf v5.1 materials highlight MI350‑series gains vs. prior AMD generations, aided by ROCm optimizations. Translation: tuning the software stack matters as much as chip choice. (amd.com)
  • Why it matters: Compiler, kernel, and precision updates reduce latency and energy without waiting for new racks. If you’re already deployed, these are near‑term, low‑friction unit‑economics wins.

6) Cooling spotlight: reliability events and supply scaling nudge PUE down

  • What’s new:
    • A late‑November cooling‑failure outage put a fresh spotlight on thermal risk, while suppliers scaled up liquid‑cooling portfolios and M&A to meet AI densities. Plain English: better, more available liquid‑cooling gear reduces wasted power. (reuters.com)
    • Analysts keep naming Vertiv a top pick as demand for AI thermal gear grows; more capacity should shorten lead times and reduce costs. (barrons.com)
  • Why it matters: Lower PUE (less facility energy overhead) means more of your electricity becomes useful compute. That directly improves intelligence per watt and reduces $/token—especially at high rack densities.

7) Market/strategy signals that could affect future costs

  • NVIDIA/OpenAI supply alignment is still being negotiated, per NVIDIA’s CFO on Dec 2. Plain English: big allocation deals influence future GPU availability and pricing for everyone else. (reuters.com)
  • OpenAI partnership and ecosystem moves drew headlines, but no API price changes accompanied them. Translation: keep monitoring the strategy noise, but budget to today’s posted price cards. (theverge.com)

—

What this means for your unit economics this week

  1. Pilot Trainium 3 where supported and priceable. If you train or fine‑tune regularly, evaluate Trainium 3 vs. H100/H200 on your actual workloads. If claimed 4× gains hold for you, you’ll see lower training $/run and better energy efficiency. (investors.com)
  2. Use posted GPU price cards as your floor. Start with consumer/Pro‑tier GPUs (e.g., RTX 5090/6000 Blackwell) for evals, batch inference, and non‑frontier tasks; only step up to H100/H200/B200 when memory, latency, or SLA require it. (coreweave.com)
  3. Lock in tariffs early in MO/KS‑style states. If you’re siting in the Midwest, these large‑load plans give predictable $/kWh and protect against cross‑subsidy fights. Model your minimum‑bill exposure and term length. (psc.mo.gov)
  4. Consider AMD clusters for batch jobs. With MI355X capacity coming online at challenger clouds, you may cut training/inference spend if your stack runs well on ROCm. Test before committing. (reuters.com)
  5. Keep upgrading runtimes. Enable low‑precision paths (FP8/FP4), kernel updates, and scheduling that increase tokens/sec per watt on your existing fleet. These are compounding gains with near‑zero capex. (developer.nvidia.com)
  6. Push for liquid‑cooling commitments in contracts. Vendors’ expanded portfolios and service offerings can lower PUE and improve uptime; ask for metrics and SLAs tied to thermal performance. (reuters.com)

—

Quick reference links

  • OpenAI pricing (reference): platform.openai.com/pricing. (platform.openai.com)
  • Anthropic pricing (reference): anthropic.com/pricing. (anthropic.com)
  • Vertex AI release notes: cloud.google.com/vertex‑ai/generative‑ai/docs/release‑notes. (docs.cloud.google.com)
  • CoreWeave public pricing: coreweave.com/pricing. (coreweave.com)
  • Runpod pricing: runpod.io/pricing. (runpod.io)

If you want, next edition can add a live delta tracker for posted GPU rates (esp. RTX 5090 and Pro 6000 Blackwell), cloud Trainium 3 availability by region, and state‑by‑state large‑load tariff adoption timelines.