LLM Savings: Your Weekly Cost-Cutting Update
The LLM Cost‑Cutter
Weekly briefing on advances that shrink the cost of large language models (Jan 21–27, 2026)
Pricing announcements from AI labs this week
- No new per‑token API price cuts from OpenAI, Anthropic, or Google were posted in the past 7 days. OpenAI’s public pricing card still shows GPT‑5.2 at $1.75/M input and $14/M output, with 90% input cache discount. (openai.com)
- Google reminder: starting January 28, 2026, Vertex AI Agent Engine begins separate billing for Code Execution, Sessions, and Memory Bank (runtime per‑vCPU/GB prices unchanged since the December cut). (cloud.google.com)
- Anthropic’s latest top model, Claude Opus 4.5, remains listed at $5/M input and $25/M output (no change this week). DeepSeek’s ultra‑low prices remain $0.28/M input ($0.028 cached) and $0.42/M output. (anthropic.com)
—
1) DOE’s Jan 24 emergency order for PJM: keeping power (and AI) online through the cold snap
- What happened (Jan 24): The U.S. Department of Energy issued a Federal Power Act §202(c) emergency order authorizing PJM to run generators up to their maximum output, even beyond certain permit limits, from Jan 25–31 to maintain reliability during extreme weather. (energy.gov)
- PJM’s Jan 26 update: Grid operators are coordinating with large customers, including data centers, to voluntarily switch to on‑site backup generation if needed to avoid load shed. (insidelines.pjm.com)
- Why this reduces LLM costs and boosts “intelligence per watt”: Avoiding outages prevents costly failovers, SLA penalties, and idling GPU clusters. A steadier grid lets operators run hotter racks efficiently and keep utilization high—more useful tokens per kWh and lower amortized $/token over the week. Extreme weather also highlights the value of sites with firm power or backup‑gen integration, which lowers risk premia in financing new AI capacity. (barrons.com)
2) Nvidia invests $2B in CoreWeave to accelerate 5 GW of AI “factories” by 2030
- What’s new (Jan 26): Nvidia bought ~$2B of CoreWeave stock, deepening a partnership aimed at building more GPU‑dense data centers—CoreWeave guided to 5 GW of capacity by 2030. (businessinsider.com)
- Why this matters for costs: More supply and tighter NVIDIA–cloud alignment should increase availability of the newest accelerators and keep public, posted GPU rates competitive—downward pressure on training and inference $/token as throughput per watt improves on newer platforms. (barrons.com)
3) Real‑time market anchors: posted GPU prices you can use to negotiate
- CoreWeave (checked Jan 26–27): GB200 NVL72 listed at $42/hr; B200 8× at $68.80/hr; H200 8× at $50.44/hr; H100 8× at $49.24/hr; RTX Pro 6000 Blackwell 8× at $20/hr. GB300 NVL72 now appears on the card (contact sales). Plain English: you can right‑size fleets and benchmark quotes against these numbers. (coreweave.com)
- Lambda: on‑demand B200 “Instances” advertised at $4.99 per GPU‑hour; B200 “1‑Click Clusters” from $3.79 per GPU‑hour with short commitments. This provides a second public yardstick for negotiating reserved capacity. (lambda.ai)
- Why it matters: Transparent rate cards compress margins and help you mix premium GPUs for training with cheaper cards for inference. As software stacks mature (see item 4), the same hourly rate buys more tokens/sec—compounding savings.
4) Software wins you can deploy now: up to 2.8× more tokens/GPU on Blackwell
- NVIDIA’s Jan 7 stack (teams continue rolling it out): TensorRT‑LLM updates—multi‑token prediction (MTP), NVFP4 4‑bit formats, and faster all‑to‑all—deliver up to 2.8× higher Mixture‑of‑Experts throughput on Blackwell versus Q4’25 software, at the same power. (developer.nvidia.com)
- Why it matters: These are “free” speedups—no new hardware. Update containers and you reduce cost per answer and raise “intelligence per watt,” especially for MoE models that were previously communication‑bound. (howaiworks.ai)
5) Cooling breakthroughs: toward cheaper, denser racks and heat reuse
- Passive, 3D‑printed thermosiphon cooler (reported Jan 22): A Danish team demonstrated a fan‑less, pump‑less liquid cooler dissipating up to 600 W, with 60–80°C outlet temps that suit heat‑reuse loops. While early‑stage, this points to simpler, lower‑maintenance cooling at the device level. (tomshardware.com)
- Why it matters: Reducing auxiliary power (fans/pumps) and enabling heat reuse lowers facility overhead (PUE), so more of your electricity drives inference. Over time, cheaper, modular cooling supports higher rack density—more tokens per square foot and per watt.
6) Cooling and power gear scale‑out continues to de‑risk build costs
- Schneider Electric’s recent U.S. deals, highlighted this week by industry press, total roughly $2.3B across Switch and Digital Realty for power and cooling platforms, locking in supply and standardizing deployments for AI campuses. Translation: fewer delays and lower “soft costs” in getting capacity online. (datacenterfrontier.com)
- Why it matters: Shorter time‑to‑power reduces carrying costs and brings new, efficient capacity to market faster—conditions that historically precede cloud price competition.
7) Heads‑up: Google’s Agent Engine add‑on billing starts this week
- On January 28, Google begins billing for Agent Engine Sessions, Memory Bank, and Code Execution (runtime was already reduced to $0.0864/vCPU‑hr and $0.009/GB‑hr). If you rely on these services, budget for the new line items or adjust architecture. (cloud.google.com)
- Why it matters: These are runtime charges, not token prices—but they affect your all‑in cost per workflow. Good accounting avoids surprises when token prices alone show savings.
—
Quick takes (this week)
- DeepSeek pricing page reconfirmed (Jan 25): $0.28/M input ($0.028 cached) and $0.42/M output—still the floor for many use cases. Use as a benchmark in vendor talks. (api-docs.deepseek.com)
- Severe winter weather (Jan 24–26) stressed U.S. grids. DOE’s order and PJM’s coordination with data‑center customers reduced outage risk—reminder to quantify the value of firm power and backup‑gen integration in your cost model. (energy.gov)
- Anthropic experienced a brief outage on Jan 22 (impact resolved same day). Not a cost story—but it underscores the operational value of multi‑vendor routing for steady $/token. (techradar.com)
—
Action checklist for the week
- Roll forward your inference stack on Blackwell/Hopper. Enable NVFP4/MTP and measure tokens/sec and GPU watts before/after; expect material throughput gains on MoE. (developer.nvidia.com)
- Re‑bid capacity using public anchors. Compare your quotes to CoreWeave’s GB200/B200/H200/H100 and Lambda’s B200 rates; pick optimal fleets for training vs. inference. (coreweave.com)
- If you’re in PJM, confirm backup‑gen readiness and curtailment plans. The DOE order shows regulators will use every tool to preserve reliability—your project risk (and cost of capital) drops when you can operate through weather events. (energy.gov)
- Budget for Google’s Agent Engine add‑ons starting Jan 28, or refactor to reduce paid Sessions/Memory/Code Execution where they’re not essential. (cloud.google.com)
If helpful, next week’s edition can include a one‑pager converting your current stack (models, runtimes, GPU mix) into an all‑in $/million tokens—including new runtime line items and recent software speedups.