LLM Savings: Your Weekly Cost Update!
The LLM Cost‑Cutter
Weekly briefing on advances that shrink the cost of large language models (Dec 30, 2025–Jan 6, 2026)
Pricing announcements from AI labs this week
- No new per‑token price cuts from OpenAI, Anthropic, or Google in the past 7 days. OpenAI’s current price card (incl. GPT‑5.2 and GPT‑5 mini tiers) remains unchanged as of today. (openai.com)
- Google began billing for “Grounding with Google Search/Web Grounding for Enterprise” on January 5, 2026, and will start billing Agent Engine add‑ons (Code Execution, Sessions, Memory Bank) on January 28, 2026. These are not model‑token prices, but they affect your all‑in cost per task. (cloud.google.com)
- Anthropic’s Opus 4.5 price cut (to $5 input / $25 output per million tokens) from late 2025 remains in effect; no new changes this week. (website.claude.com)
- DeepSeek’s ultra‑low pricing continues (e.g., $0.28/M input cache miss; $0.028/M cache hit; $0.42/M output). No change this week. (api-docs.deepseek.com)
Links:
- https://openai.com/api/pricing/
- https://platform.openai.com/docs/pricing
- https://cloud.google.com/vertex-ai/generative-ai/pricing
- https://cloud.google.com/vertex-ai/pricing
- https://website.claude.com/pricing
- https://api-docs.deepseek.com/quick_start/pricing/
1) Headline hardware: Nvidia unveils Rubin platform with 10× cheaper tokens claim
What happened (Jan 5, 2026): At CES, Nvidia launched its next data‑center platform “Rubin” (Rubin GPU + Vera CPU + NVLink‑6 + Spectrum‑6 + ConnectX‑9 + BlueField‑4). Nvidia says extreme co‑design across these six chips will cut inference token generation cost by up to 10× versus Blackwell and reduce GPUs needed to train mixture‑of‑experts models by up to 4×. Plain English: new racks that work as one system, moving data more efficiently, and squeezing far more tokens per watt and per dollar. (nvidianews.nvidia.com)
Why it matters for costs: If Rubin delivers even part of the 10× token‑cost reduction, API providers and GPU clouds will be able to serve more output with the same power and hardware, opening headroom for future price cuts. Expect effects to phase in H2’26 as systems ship. (investor.nvidia.com)
Links:
- https://nvidianews.nvidia.com/news/rubin-platform-ai-supercomputer
- https://blogs.nvidia.com/blog/2026-ces-special-presentation/
- https://investor.nvidia.com/news/press-release-details/2026/NVIDIA-Kicks-Off-the-Next-Generation-of-AI-With-Rubin--Six-New-Chips-One-Incredible-AI-Supercomputer/default.aspx
2) Platform fees: Google starts charging for Search/Web grounding and posts Agent Engine dates
What’s new: As of January 5, 2026, Google bills per grounded search query ($14 per 1,000), and starting January 28 it will bill for Agent Engine Code Execution, Sessions, and Memory Bank. Plain English: even if your model tokens are cheap, new runtime and grounding line items can add up. (cloud.google.com)
How it compounds: Teams that adopt prompt‑caching and route simpler prompts to cheaper models will still need to watch these runtime add‑ons—especially apps that call grounding frequently. Tight control of grounding frequency and cache utilization keeps total cost per task falling alongside hardware gains.
Links:
3) Real‑time market signal: public GPU prices (CoreWeave)
What’s new (pages refreshed within the last 4 days): Selected hourly rates show B200 8‑GPU at ~$68.80/hr; H200 8‑GPU ~$50.44/hr; H100 8‑GPU ~$49.24/hr; Blackwell RTX Pro 6000 Server Edition 8‑GPU at $20/hr; GB200 NVL72 listed at $42/hr; GB300 NVL72 “contact sales.” Plain English: top‑tier boxes remain visible and competitive, with a clear price ladder emerging. (coreweave.com)
Why it matters: Transparent rates help you forecast $/million tokens and negotiate reserved capacity. As Rubin‑era efficiency arrives later in 2026 and serving software improves, these hourly rates have room to drift down, amplifying end‑user savings.
Links:
4) Grid policy: FERC pushes PJM to enable faster, fairer hookups for co‑located data‑center power
What’s new (Jan 2–5, 2026 updates on a Dec 18 order): FERC found PJM’s tariff unjust/unreasonable for generators serving co‑located loads (like data centers next to their own power plants) and ordered tariff revisions on a fast timeline (initial filings due mid‑Jan to mid‑Feb). Plain English: clearer, standardized ways for AI campuses that bring their own power to connect to the grid and pick firm or non‑firm service, faster. (certrec.com)
Why it matters for costs: Cutting interconnection delays and right‑sizing grid fees reduces soft costs and schedule risk—the hidden premiums baked into API and GPU rental prices. Easier co‑location means more predictable, cheaper electricity per token and better “intelligence per watt.”
Links:
- https://www.ferc.gov/news-events/news/fact-sheet-ferc-directs-nations-largest-grid-operator-create-new-rules-embrace
- https://www.certrec.com/news/ferc-issues-anticipated-order-on-data-center-co-location-in-pjm-region/
- https://www.steptoe-johnson.com/news/fercs-pjm-co-location-order-a-turning-point-for-ai-data-center-power-strategy/
- https://www.gibsondunn.com/ferc-orders-pjm-largest-u-s-grid-operator-to-revise-tariff-to-permit-new-transmission-services-for-co-located-loads-such-as-data-centers/
5) Serving stack upgrades: more tokens per watt from software
What’s new (CES week, Jan 5): Nvidia highlighted open‑source tool upgrades (llama.cpp, Ollama, ComfyUI) that deliver sizable throughput gains on RTX PCs and DGX Spark, including quantization paths like NVFP4/FP8 and concurrency improvements—benchmarks cited up to ~35% faster token generation for some SLM setups. Plain English: the same hardware can now push more tokens per second after a simple software update. (developer.nvidia.com)
Also relevant (late Dec): Nvidia’s vLLM container adds NVFP4 support on Blackwell and other optimizations—useful as RTX Blackwell/Blackwell servers proliferate. Plain English: low‑bit formats cut memory traffic and energy use without noticeably hurting quality when applied correctly. (docs.nvidia.com)
Why it matters: Software speedups stack with hardware and power wins—so even before Rubin lands, you can improve “intelligence per watt” now by updating runtimes, enabling 4‑/8‑bit paths, and tuning batch/concurrency.
Links:
- https://developer.nvidia.com/blog/open-source-ai-tool-upgrades-speed-up-llm-and-diffusion-models-on-nvidia-rtx-pcs/
- https://docs.nvidia.com/deeplearning/frameworks/vllm-release-notes/rel-25-09.html
6) Memory is money: KV‑cache research to cut delay and VRAM pressure
What’s new (paper posted Dec 2025; highlighted this week): Microsoft Research’s “EVICPRESS” jointly optimizes KV‑cache compression and eviction across tiers, reporting up to 2.19× faster time‑to‑first‑token at equivalent quality. Plain English: by squeezing and smartly placing the model’s “short‑term memory,” servers respond faster and fit more sessions per GPU. (microsoft.com)
Why it matters: KV‑cache management directly affects GPU memory usage, batch size, and latency—core drivers of $/token. Expect production systems that adopt similar policies to serve more users on the same fleet, pushing effective costs down.
Links:
Quick takes and how they compound
- Rubin’s 10× token‑cost promise is future supply: If shipping begins in H2’26 as stated, plan capacity contracts and migrations now so you can capture the step‑change when clouds light it up. Pair with low‑bit serving to harvest savings earlier. (nvidianews.nvidia.com)
- Watch “non‑token” meters: Google’s grounding and Agent Engine fees have started or are imminent. For many agentic apps, these can dominate spend unless you cap searches, cache aggressively, and route to cheaper models when possible. (cloud.google.com)
- GPU price shopping still pays: Public cards (e.g., B200 at ~$68.80/hr; RTX Blackwell 8× at $20/hr) give you leverage on reserved quotes and a path to mixed fleets (premium GPUs for training; cheaper SKUs for high‑throughput inference). (coreweave.com)
- Policy tailwinds: FERC’s PJM order shortens timelines and clarifies costs for power‑hungry campuses that self‑supply—a structural deflator for $/token as more capacity comes online without multi‑year grid delays. (ferc.gov)
Actionable checklist for this week
- Audit “non‑token” charges. If you use Vertex grounding, set budgets/rate‑limits; measure how often each request triggers search and what it does to task‑level cost. (cloud.google.com)
- Roll forward serving images. Test latest vLLM/NVFP4 or equivalent quantized paths; record tokens/sec and GPU watts before/after. (docs.nvidia.com)
- Re‑bid capacity. Cross‑shop CoreWeave’s public rates against your private quotes; consider inference on RTX Blackwell clusters for commodity workloads. (coreweave.com)
- Track PJM compliance dates (mid‑Jan/mid‑Feb filings). If you’re siting in PJM, co‑location structures and new firm/non‑firm services could materially change your interconnect economics. (ferc.gov)