LLM Price Watch: Cost Trends Unveiled!

The LLM Cost‑Cutter

Weekly briefing on advances that shrink the cost of large language models (Jan 7–13, 2026)

Pricing announcements from AI labs this week

  • No new per‑token price cuts from OpenAI, Anthropic, or Google in the past 7 days. OpenAI’s public price card (e.g., GPT‑5.2 at $1.75/M input, $14/M output) is unchanged as of Jan 13, 2026. (openai.com)
  • Google’s Vertex AI continues billing for the Agent Engine runtime and will begin billing for Code Execution, Sessions, and Memory Bank on Jan 28, 2026 (these are runtime add‑ons, not model tokens). (cloud.google.com)
  • Anthropic’s Claude 4.5 pricing (Opus: $5/M input, $25/M output; Sonnet and Haiku tiers unchanged) remains in effect with no changes this week. (website.claude.com)
  • DeepSeek’s ultra‑low pricing (V3.2: $0.28/M cache‑miss input, $0.028/M cache‑hit input, $0.42/M output) is unchanged this week. (api-docs.deepseek.com)

1) Microsoft pledges to absorb data‑center power costs and publish water use

  • What happened (Jan 13): Microsoft announced it will work with local utilities to expand supply, cover its full data‑center electricity costs (rather than shift them to ratepayers), replenish more water than it consumes, and publish region‑level water metrics. The White House praised the move and signaled more actions to curb consumer electricity impacts from AI data centers. Plain English: big AI campuses will shoulder more of their own power bill and be more transparent about water, easing public pushback and permitting delays. (reuters.com)

  • Why it matters for LLM costs: Lower “soft costs” (fewer delays, clearer tariffs, less political risk) bring capacity online faster and more predictably, which pressures API and GPU rental prices down over time and improves “intelligence per watt” by enabling siting near cheaper, cleaner power.


2) Faster hookups for self‑powered AI campuses: FERC’s PJM deadlines near

  • What’s new this week: Legal memos and agency updates clarified compliance milestones from FERC’s Dec 18 order directing PJM (the largest US grid) to create new service options for co‑located data centers with their own generation. Initial filings are due around Jan 19–20, with fuller tariff revisions by Feb 16; a paper hearing on rates runs through April. Plain English: a standardized, faster, more predictable path for connecting AI campuses that bring their own power. (ferc.gov)

  • Why it matters: Clear interconnection rules and firm/non‑firm service options reduce schedule risk and financing costs for new capacity. More timely, lower‑risk power = cheaper tokens and better odds of future API price cuts.


3) Real‑time market signal: public GPU cloud rates (updated within 5 days)

  • Snapshot: CoreWeave lists indicative hourly rates such as B200 8‑GPU at ~$68.80/hr, H200 8‑GPU ~$50.44/hr, H100 8‑GPU ~$49.24/hr, RTX Pro 6000 Blackwell 8‑GPU at $20/hr, and GB200 NVL72 at $42/hr. Plain English: transparent prices show a clear performance/price ladder and keep rental markets competitive. (coreweave.com)

  • Why it matters: These cards anchor negotiations for reserved capacity and let teams mix fleets (premium for training; cheaper for inference). As software/runtime efficiency improves, effective $/million tokens falls even before new silicon ships.


4) Software wins you can use now: NVIDIA reports up to 2.8× higher MoE throughput on existing Blackwell

  • What’s new (Jan 7): NVIDIA detailed TensorRT‑LLM and stack updates that boost mixture‑of‑experts inference throughput per Blackwell GPU by up to 2.8× versus the software from three months ago. Techniques include NVFP4 4‑bit formats, multi‑token prediction, and improved all‑to‑all communications. Plain English: update your serving stack and you get more tokens per second at the same power. (developer.nvidia.com)

  • Why it matters: Free speedups compound with hardware improvements. Even without changing models, you can cut cost per task and energy per token by rolling forward to the latest inference containers.


5) Research: new 4‑bit method keeps accuracy while speeding RTX‑class inference up to 3×

  • Paper (Jan 12): “ARCQuant” proposes an NVFP4‑friendly quantization approach that maintains unified 4‑bit precision while compensating errors via “augmented residual channels,” reporting accuracy near FP8 and up to 3× speedups over FP16 on RTX 5090/Pro 6000. Plain English: smarter 4‑bit math that still works well—and runs much faster—on modern GPUs. (arxiv.org)

  • How it compounds: If production stacks adopt these low‑bit recipes, servers can serve more users per GPU with less memory traffic, directly improving “intelligence per watt” and lowering serving bills.


6) Power deals for stable, greener kWh: Switch signs a geothermal PPA

  • What’s new (Jan 13): Data‑center operator Switch agreed to buy ~13 MW from Ormat’s Salt Wells geothermal plant in Nevada for 20 years starting in early 2030, with an option to add 7 MW of solar. Plain English: long‑term, around‑the‑clock clean power that isn’t tied to fuel price spikes. (investors.com)

  • Why it matters: Long‑dated renewable PPAs reduce energy cost volatility—the largest operating expense in LLM inference—supporting lower, steadier API and hosting prices over time.


7) AMD’s rack‑scale push at CES: more competition for inference fleets

  • What’s new (Jan 5, highlighted this week): AMD previewed “Helios,” a rack‑scale platform built around Instinct MI455X and EPYC “Venice,” and introduced MI440X for enterprise clusters. AMD positioned Helios at up to 3 AI exaflops per rack and previewed the MI500 series roadmap. Plain English: more options to build dense, power‑efficient racks beyond NVIDIA—competition that tends to push price/performance up. (amd.com)

  • Why it matters: A viable second source for high‑throughput inference squeezes rental and capex quotes. More tokens per rack and more supplier choice = downward pressure on $/token.


8) Cooling and facility efficiency: small but real gains this week

  • Cooling hardware (Jan 8): Cooler Master showed a compact coolant‑distribution unit and an in‑rack CDU for higher‑density setups—niche, but indicative of broader liquid‑cooling availability. Plain English: more off‑the‑shelf options to keep hot racks from throttling. (tomshardware.com)

  • Research (Jan 5): A physics‑guided ML framework on ORNL “Frontier” data identified cooling setpoint tweaks that could recover up to 96% of wasted accessory energy in certain regimes, pointing to practical PUE reductions. Plain English: smarter controls = fewer watts spent on chillers and pumps. (arxiv.org)

  • Why it matters: Cooling overhead is a hidden tax on tokens. Less overhead power means more of your electricity budget goes to the model, improving “intelligence per watt.”


Quick takes

  • GPU price transparency remains a buyer’s friend: public CoreWeave cards (e.g., RTX Blackwell 8× at $20/hr) set a reference for spot and reserved quotes across the market. (coreweave.com)
  • Google runtime charges start Jan 28: cap and monitor Agent Engine usage (Code Execution, Sessions, Memory Bank) so non‑token fees don’t eat your savings from cheaper models or better serving efficiency. (cloud.google.com)
  • Keep an eye on PJM filings Jan 19–20 and Feb 16: on‑time tariff updates should shorten project timelines for co‑located power, lowering soft costs that trickle into API and hosting prices. (nixonpeabody.com)

Action checklist for the week

  1. Update inference stacks. Test the latest TensorRT‑LLM image and enable NVFP4/low‑bit paths; record tokens/sec and GPU watts before/after. (developer.nvidia.com)
  2. Re‑bid capacity using public anchors. Compare your private quotes against CoreWeave’s posted rates; consider mixed fleets (e.g., RTX Blackwell for commodity SLMs). (coreweave.com)
  3. Budget for Google’s Agent Engine add‑ons. Add rate limits and caching so new runtime line items don’t dominate task‑level costs after Jan 28. (cloud.google.com)
  4. Track PJM compliance milestones. If you’re siting in PJM territory, align your interconnect plans with the new service options and hearing schedule. (nixonpeabody.com)

If you want, I can add a one‑page “how this hits $/million tokens” calculator next week using your current stack (models, runtime add‑ons, and GPU mix).