Performance and measurement

Run pnpm benchmark to build fresh artifacts and reproduce the default-compiler benchmark scenarios. TS 7 is the recommended performance baseline; both compiler versions are pinned and checked in CI using the commands below. The pinned reference client is openapi-fetch 0.17.0; the client comparison also pins openapi-typescript-fetch, feature-fetch, axios and superagent. TypeScript and tsdown use the versions in package.json. The runtime, size and comparison tables were recorded on 2026-09-29; the type-system and editor tables below remain historical samples from 2026-09-20 and 2026-09-21. All were recorded on macOS arm64 with Node 24.16.0. They are not a cross-platform throughput guarantee or measurements of the current working tree.

See the archived schema semantics report for the baseline used by the original measurements and the path and reference report for the later shared-prefix type-checking scenario. The subsequent media report records another core size. Use pnpm build && pnpm size:check to measure the checkout you are evaluating.

RuntimeCopy Anchor Link

Each client uses the same Fetch replacement returning an empty HTTP 204 response. Each case performs 100 warmup requests, then five samples of 1000 requests; the reported value is the median in microseconds per request. Calls include chain or path selection on each iteration. There is no network, JSON parsing or server work.

Each cell is the median of three script runs. The 2026-09-20 table (core 1.06-1.45 µs) predates the September 24 runtime hardening; it is superseded rather than comparable. The 10000-route metadata compile took about 12 ms and strict indexing about 33 ms. This setup cost is paid at client construction. Runtime figures include different client response wrappers and Request construction choices; they do not establish which library is faster for real applications. test/routes.test.ts guards against per-request route-table enumeration without relying on noisy timing thresholds.

Type systemCopy Anchor Link

pnpm benchmark:types generates 100, 1000 and 5000 routes using modern generator parameter conventions and calls 25 distinct operations. Each call checks the response type; negative assertions retain required-query and path constraints.

The check runs in pnpm check and CI. Each scenario must stay below 3 million instantiations and 1200000 KB compiler memory. Wall-clock times are reported but not gated on shared runners. The 5000-route fixture is intentionally demanding; large applications should evaluate their actual schemas. It does not model all endpoints being used, complex response unions, or incremental IDE latency.

SizeCopy Anchor Link

pnpm benchmark:size bundles one entry module per client with the same tsdown version, browser platform conditions, ES2022 target, minifier, no source map and gzip level 9. Each entry imports only the client factory an application uses (for example createClient and HttpError, or openapi-fetch's default export), so consumer tree shaking applies equally. Dependencies are bundled and each result must be a single file.

Before 2026-09-29 this benchmark bundled complete entry files on the neutral platform (core 4221 / 1850 B on 2026-09-20). The neutral platform cannot bundle the CommonJS competitors, and the core has since grown; do not compare the two tables.

These entries have different feature sets. Strict is an alternative to core; metadata compilation can run at build time, so neither entry must necessarily ship with every application. Consumer tree shaking can also change the result.

The separate pnpm size:check release gate uses a different stable metric: it concatenates reachable emitted core ESM chunks and gzips that text, including their source-map comments, with a 3584-byte limit. Neither metric equals the sum of separately compressed HTTP assets; compare like-for-like when publishing results.

Comparison with other clientsCopy Anchor Link

pnpm benchmark:competitors compares request overhead under one method; pnpm benchmark:competitors:types compares type-checking cost on both pinned compilers. Every runtime case must first reach the Fetch mock or server exactly once and succeed, including libraries that return failures as values. Each suite runs in its own process; cases run in shuffled order, each round a 200 ms warmup plus a one-second timed loop, and the tables report the median round. The default is three rounds; the tables below used --rounds=5. Timing is informational and does not gate CI.

Fetch mockCopy Anchor Link

This suite mirrors openapi-fetch's own test/bench harness: global fetch resolves an empty JSON 200 response on process.nextTick. Calls include chain or path selection on each iteration. Higher is better; the ratio is relative to the fastest case.

The headers scenario sets one client default header and two per-request headers in each library's own input shape; openapi-chain passes undeclared headers through init.headers. Client construction has no request work: core creates about 7.5 million clients per second because its Proxy tree is lazy, 1.8× openapi-fetch. Strict indexes its compiled metadata at construction, so it creates about 276 thousand per second.

The same script on the preceding source measured core at 155433 and 120762 ops/s (1.29× and 1.17×). That build parsed the unchanged base URL on every request and built an intermediate Headers object for each merged header record; the base boundary is now reused and plain header records merge directly.

Loopback HTTPCopy Anchor Link

A child process serves {} over keep-alive HTTP on 127.0.0.1. Node's Fetch is used by the Fetch-based clients; axios's default adapter and superagent use node:http with a keep-alive agent.

With a real socket every Fetch-based client is within 10% of calling fetch directly, and their differences are within run-to-run noise. The axios node:http figure is roughly one millisecond per loopback request on macOS; treat it as an observation of this setup, not of axios's library overhead.

Type checkingCopy Anchor Link

The types suite generates openapi-typescript-shaped paths with a required query and path parameter, then checks 25 used operations through each client's call style, including negative assertions for a missing query and a wrong path type.

openapi-chain performs about four to five times more instantiations for its prefix tree yet checks this fixture faster; with TS 6 at 5000 routes it uses about twice the memory. This synthetic fixture has uniform routes; the real-schema measurements below do not show a uniform advantage. Check time is the median of three compiler runs; instantiations and memory come from that run. A scoped client reduces openapi-chain's work further; the comparison uses the full path set for both.

Real schemasCopy Anchor Link

pnpm benchmark:real-schemas --compiler=ts6 and --compiler=ts7 download two pinned public documents into .cache/real-schemas, verify their SHA-256, generate paths with the pinned openapi-typescript 7.13.0 and type-check the same operations through both clients:

The script selects GET operations that a fluent chain can call with path arguments only: no other required inputs, no reserved segment names and exactly one application/json 200 response (562 GitHub and 239 Stripe operations). It takes an evenly spaced, sorted sample and asserts each call's success type: the 200 body for openapi-fetch's data, and the 200 body plus any default body for openapi-chain, which types undeclared 2xx statuses through default. Each consumer also keeps one negative path-input assertion. The full variant uses the generated paths; the scoped variant uses Pick<paths, …> of the called paths for both clients, as openapi-chain's CLI scope does.

40 called operations; check time is the median of three runs:

200 called operations with TS 7.0.2:

On these documents openapi-chain has a larger fixed cost: with the full generated paths and 40 calls it checks about 1.4× slower on TS 7 (1.2-1.3× on TS 6) and uses more memory. How the cost grows with calls depends on the document. On GitHub, going from 40 to 200 calls raised openapi-fetch's check time fivefold and made it 2.2× slower than openapi-chain. On Stripe, openapi-fetch grew 3.4× against openapi-chain's 2.7× and remained slightly faster. Scoping to the called paths removes most of openapi-chain's fixed cost: with 40 calls, scoped consumers check within about 10% of each other on both compilers, and with 200 GitHub calls the scoped openapi-chain consumer is 1.2× faster. A single process timing is not editor latency, and your operations may differ from this sample; measure your own schema before relying on a ratio.

Relation to openapi-fetch's published tableCopy Anchor Link

openapi-fetch's README lists sizes and GET throughput from its test/bench/index.bench.js. Reproducing that harness on 2026-09-29 with the versions above showed these differences, which is why this suite validates every call and separates the mock and HTTP scenarios:

• axios's default Node adapter and superagent use node:http, not the stubbed global fetch, so the harness attempts real DNS and TLS connections to api.test.local. Here axios uses its fetch adapter in the mock suite, and both use a local server in the HTTP suite.
• The harness's openapi-typescript-codegen fixture is the generator bundle rather than generated client code, so its benchmarked call throws. Generated clients depend on each schema and are omitted here.
• feature-fetch 0.1.x takes baseUrl rather than the harness's prefixUrl, and returns failures as [false, error] tuples instead of rejecting.

The absolute throughput depends on the machine; compare ratios within one run.

Shared schema graphsCopy Anchor Link

pnpm benchmark:metadata measures complete small documents whose allOf branches reuse the same referenced subtree. Per-compilation memoization prevents repeated expansion; regression tests count observed reads instead of gating noisy timings.

These individual samples include warmup/scheduling effects; deeper cases need not be slower. A 1,000,000-step traversal budget and 128-level depth limit remain active, including when subgraphs are cached. Compile large schemas at build time.

Shared prefixes, response unions and incremental editsCopy Anchor Link

The type benchmark also checks a 1000-route project with shared /orgs/{org}/resources/rN/{id} prefixes, 200 used operations, nested response objects and success-response unions. Negative assertions retain required inputs, path argument types and response-union safety. This runs in pnpm check with the same 3 million instantiation / 1200000 KB memory budgets.

A local Node 24.16.0 / TypeScript 6.0.3 run produced:

The edit adds a new checked operation call to the consumer module. Unchanged build caching is effective, but editing this module still requires substantial checking. These are CLI incremental-build measurements, not language-server completion latency or a guarantee for a real application's schema. Large users should measure their generated schemas and can partition clients by service or route subset to limit each type tree. The earlier 5000-route scenario remains a separate scale gate; this project scenario does not replace it.

Current compiler and scoped-consumer measurementsCopy Anchor Link

Recorded 2026-09-21 on macOS arm64 / Node 24.16.0. Raw outputs, including every editor sample, are in the measurement record. The following figures use a new fixed fixture from scripts/lib/type-fixture.mjs, not the earlier historical tables. Both full and scoped consumers call the same 25 operations and check their responses. Scoping keeps the full declaration input but exposes 250 exact paths via Pick.

These recorded figures originally used a separately installed TS 7.0.2. It is now a pinned typescript7 alias: source type checks, installed ESM/CommonJS declaration consumers and all type-complexity scenarios run on both versions in CI. Generation and declaration building continue using the TS 6 toolchain; the isolated TS 7 consumer test verifies the documented dual-version generator setup, not a wholesale build-tool migration. Compiler times here are individual process measurements, not latency guarantees. Instantiations remain almost equal across these two versions on this fixture, but that is not a cross-version invariant for all programs.

pnpm benchmark:scoping also varies the call count on the 1000-route fixture:

One call still incurs substantial schema-wide work. Call count, route shape and response complexity also matter; do not extrapolate a universal linear formula. The scoped fixture is gated on valid positive/negative types, instantiation and memory limits, and lower instantiation work than its full counterpart in pnpm check. Use the single-scope recipe for a runnable setup.

Editor completion latencyCopy Anchor Link

pnpm benchmark:editor --compiler=ts6 drives real TS 6 tsserver requests; pnpm benchmark:editor:ts7 uses the pinned TS 7 native LSP. For each scenario it starts three fresh servers, opens the same consumer, requests root-chain completion, repeats without edits, changes one comment character and requests completion again. The edited interval includes submitting the edit. Cold timing starts at the first completion request after opening the file, not at OS process creation. The same incremental edit is used in both protocols. Returned route names and exclusion of unselected routes are asserted.

These are three-sample medians on a synthetic root completion, not IDE-wide responsiveness, competing-client benchmarks or a promise for deep nodes and semantic edits. The measured improvement does not equal the instantiation ratio. Run these timing benchmarks separately from other CPU-heavy work. Timing is not a shared-runner CI gate; stalled/failed protocol requests do fail the script.

Reproduce and gate both compiler versionsCopy Anchor Link

After pnpm install --frozen-lockfile && pnpm build, use:

pnpm typecheck
pnpm typecheck:ts7
pnpm benchmark:types
pnpm benchmark:scoping --compiler=ts6
pnpm benchmark:types:ts7
pnpm benchmark:editor --compiler=ts6
pnpm benchmark:editor:ts7
pnpm test:package:ts7

benchmark:types:ts7 covers flat routes, shared prefixes/response unions with cold/unchanged/edited builds, and the scope/call-count fixture. The fixture inputs and semantic assertions are shared across versions. TS 7 type benchmarks explicitly use four checkers. Every benchmark reports the actual compiler version and the type-complexity scripts also report platform, worker setting and budget.

pnpm check runs the TS 6 gates and then pnpm check:ts7; the latter requires a fresh build and covers source types, installed consumers (including independently generated declarations) and all three TS 7 type benchmarks. The existing CI Node/OS matrix runs both compiler versions. A Linux/Node 24 job also runs both editor protocols sequentially, validating returned completions and retaining sample timings in its logs. Elapsed time is reported, not used as a regression threshold on shared runners.

Budgets are separate named records in scripts/lib/compiler.mjs. Initially each version uses the established ceiling of fewer than 3,000,000 instantiations and 1,200,000 KB compiler memory. Missing metrics and changed programs reporting zero work fail. Scoping must reduce instantiations for the same calls. There is no cross-compiler equality or speed-ratio assertion. If a compiler update changes metric semantics, review its budget explicitly rather than silently weakening both versions' gates.

Named --compiler=ts6 and --compiler=ts7 runs resolve the package's own launcher, verify its pinned version and ignore executable overrides, avoiding .bin/tsc collisions. For separate experiments only, OPENAPI_CHAIN_TSC remains available when no named compiler is supplied; it must point to a native TS 7 CLI. Such results do not replace the pinned CI baseline.

Metadata and consumer deliveryCopy Anchor Link

Library-entry size omits application metadata. pnpm benchmark:delivery bundles the same synthetic 1000-route document in three ways and executes a deepObject request through each resulting browser-target bundle. It verifies compiler inclusion/exclusion and that the scoped client rejects unselected routes.

The document alone is 6602 B gzip. Consumer figures compress the whole bundle; they are not sums of separately compressed entry and metadata values. These synthetic schemas compress extremely well and do not establish ratios for Stripe or a representative public API corpus. The scoped scenario intentionally changes the exposed API while preserving the exercised operation. Full metadata retains uncalled routes because the client indexes a dynamic operation table.

Build-time delivery eliminates shipping the compiler and document. Server targets can also benefit in artifact size, initialization and memory, even when browser transfer is not a concern. Neither approach removes the runtime route index or makes cost proportional to inferred source-code call sites.

For the historical 2026-09-21 emitted-entry measurement: core 1916 B, strict 6789 B, metadata 4673 B gzip. That historical core transitive release gate reported 2046 / 2048 B; the current safety budget is 3584 B. Historical tables above remain labeled historical; do not mix their method or baseline with these numbers. The serializer-profile experiment records why no new reduced strict entry is published in this iteration.

The September 24 runtime-hardening implementation measures 3020 B transitive gzip with the size gate's emitted-module method. The current gate is 3584 B (3.5 KiB). The increase includes URL boundary validation, shared query and response contracts, guarded JSON serialization, plain-record and cross-realm native-body checks, and stable error codes with operation context. This replaces the intermediate 2560 B safety budget; it does not revise the historical benchmark tables.

The September 26 core structured-value guard adds 107 B over the preceding 3060 B build: 3167 B transitive gzip. The September 29 request-overhead work (base URL reuse and direct header-record merging) measures 3306 B. Adding the openapi-fetch adapter entry changed shared chunk boundaries without changing core code: 3335 B across four emitted modules. The 3584 B budget retains space for maintenance and correctness fixes. Budget changes require an explained measurement and review; strict/compiler code remains outside this entry.