OpenAI Jalapeño chip benchmarks reveal a pricing-power shift

The OpenAI Jalapeño chip has posted its first public benchmark results, and the numbers are best read as a statement about who controls the cost of serving AI. Across three public models, the inference processor OpenAI co-developed with Broadcom delivered 1.5 to 1.9 times more AI work per watt at peak throughput than Nvidia's GB200 and GB300 rack systems, with 1.7 to 3.6 times lower end-to-end latency. OpenAI presented the data at the Hot Chips conference this week alongside hardware head Richard Ho.

Testing ran on SemiAnalysis' public InferenceX suite across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Moonshot AI's trillion-parameter model. The comparison is anchored to power ratings: Jalapeño's B0 part is rated at 700 watts and held at or below 550 watts during testing, while the Nvidia systems are rated at 1,200 and 1,400 watts. OpenAI normalized the per-watt figures using those published numbers, so the reported edge reflects both the architecture and the far smaller power envelope.

What the first results show

The widest advantage appeared on interactive workloads, the high-turn agentic traffic that is becoming the dominant shape of production AI. On highly interactive scenarios OpenAI reported 2.1 to 4.1 times higher performance than the comparison systems, and the latency gap widened at the low-latency operating points where agent loops and tool-calling chains spend most of their time. This is a deliberate design outcome: the architecture minimizes data movement between compute and memory, a choice aimed at workloads that constantly shuffle state instead of streaming a single long generation. The three test models span a wide size range, from the 120-billion-parameter GPT-OSS to the trillion-parameter Kimi K2.5, and the advantage held across all of them, which suggests the design is not tuned to a single model shape.

The OpenAI Jalapeño chip is an inference part, which is precisely the point. Training runs are a capital expense paid in bursts; serving is a continuous operating cost that compounds with every user, and it is the layer where AI providers actually lose margin. The chip sits inside a full-stack compute strategy in which custom silicon is tuned against OpenAI's own proprietary models, including GPT-5.6 Sol, and the company frames the first-party path as complementary to its existing Nvidia and Microsoft partnerships rather than a replacement for them.

Latency economics sharpen the story for agentic products. Each reasoning step and tool call is a round trip, so time between tokens sets how fast an agent can chain actions. Cutting end-to-end latency by 1.7 to 3.6 times changes which products are feasible at all, because interactive applications fail on perceived delay long before they fail on raw throughput.

The comparison at a glance

	Jalapeño B0	Nvidia GB200	Nvidia GB300
Power rating	700W	1,200W	1,400W
Power during testing	at or below 550W	not disclosed	not disclosed
AI work per watt (OpenAI-reported)	1.5–1.9x vs Nvidia	baseline	baseline
End-to-end latency	1.7–3.6x lower	baseline	baseline
Design to tapeout	roughly 9 months	n/a	n/a
OpenAI deployment	end of 2026	current generation	current generation

Why the OpenAI Jalapeño chip changes inference economics

This is a pricing-power story. OpenAI is the largest buyer of AI compute in the industry, which makes it a price-taker on Nvidia's roadmap: its cost structure is set by another company's silicon, allocation, and price list. A first-party inference chip that holds up in production breaks that dependency at the exact point where AI services spend their daily operating budget, which is serving responses rather than training runs.

OpenAI's deployment plan makes the economics concrete before any external sale exists. The chip goes into OpenAI's own infrastructure starting at the end of 2026, so the first customer is OpenAI itself, and the design does not depend on lining up outside buyers to justify the program.

Tokens per watt and latency per token are the metrics OpenAI is naming as the next battleground. If Jalapeño delivers close to these numbers at fleet scale, the marginal cost of a ChatGPT or API response falls, and that changes what OpenAI can charge, what it can bundle into higher tiers, and how much room it has to undercut rivals that rent the same Nvidia hardware from cloud providers. Because OpenAI's API pricing anchors part of the market, every provider that prices against Nvidia's cost curve would feel the downstream pressure.

The company's full-stack post places the chip inside a strategy spanning data centers, custom silicon, and software optimization, and it names tokens per watt and latency per token as the metrics that will decide the next phase of AI economics. That framing signals where OpenAI expects the competition to move: serving efficiency, rather than raw training throughput, is the axis it intends to own.

The pace of the program is a second strategic signal. The B0 part went from design to tapeout in roughly nine months, and OpenAI says second and third generations of its custom silicon are already in development. A fast iteration loop co-developed with Broadcom gives OpenAI a hardware cadence it can plan around, in contrast to waiting on an external vendor's release schedule, and it does so without building a chip organization from scratch.

The caveats and the scale test

The headline numbers come with qualifications. The results are OpenAI's own claims, reported by the company and not independently reproduced on the silicon by a third party. The InferenceX suite is public, which keeps the methodology checkable, but the runs themselves were not externally audited. The comparison is also against current-generation hardware: GB200 and GB300 are the systems available today, and Nvidia's next-generation platform is already in the pipeline, so the efficiency gap is a snapshot rather than a settled order. Even so, the OpenAI Jalapeño chip's first results are the strongest public evidence yet that custom inference silicon can undercut general-purpose accelerators on serving efficiency.

The real test comes with deployment. Benchmarks describe a B0 part under controlled conditions; OpenAI's rollout inside its own infrastructure starts at the end of 2026. What the published numbers do not capture is how the per-watt and latency advantages hold under a production fleet running mixed traffic, with yield, power management, and reliability constraints in play. The per-watt figures are reported at peak throughput operating points, and real workloads mix short and long requests in ways a fixed benchmark suite does not fully reproduce. Until production data exists, the 1.5 to 1.9x figures are a strong directional signal, not a proven operating cost.

Scope matters for how the numbers are read. Jalapeño is not a training chip, so OpenAI still buys Nvidia and other vendors for training clusters, and the pricing-power story applies to serving rather than the full compute bill. The strategic weight of this release is that serving is where day-to-day costs concentrate, which makes the inference layer the natural place to start bending the cost curve.

The verdict

For decision-makers, the takeaway is about optionality and leverage. If the OpenAI Jalapeño chip works at scale, the company stops being a pure price-taker on Nvidia GPUs, and the serving cost curve gains a new reference point that competitors must either match or explain. For enterprises buying AI services, the near-term question is whether those efficiency gains surface as lower API prices or get reinvested into capability and margin.

The milestones to watch are the end-of-2026 deployment inside OpenAI's data centers and whether the second and third silicon generations hold the roughly nine-month cadence that produced the B0 part. This week's benchmark release establishes the thesis. The production rollout is what tests it.

Why this matters

The first Jalapeño results move the debate from whether OpenAI can build chips to whether its silicon can reset serving economics at the scale of its real workloads. If it does, the largest buyer in AI gains pricing power over its own infrastructure, and every provider that prices against Nvidia's curve has to respond.

Sources

Jalapeño's first results show industry-leading speed and efficiency

The full stack behind abundant intelligence

✔Human Verified

Researched and cross-referenced against primary sources by the Bytevyte editorial team. This article was generated with the assistance of artificial intelligence and reviewed by the Bytevyte editorial team.