Shader Specialization Prediction

Programs know things about a frame that a shader cannot see: whether any image in the scene carries a tint, which antialiasing path the surface needs, whether a filter is active, what the output format is. Each of those facts could pick a smaller, faster GPU program — but only if something decides, before a retained scheme is recorded, which program the next hundred frames will want.

This note describes how Goldy makes that decision generically, without growing a named feature flag per specialization and without caching the full combinatorial set of variants.

Specialization is an implementation detail of the runtime, not an API. Every program that submits a retained scheme is a potential beneficiary without changing a line, and the only knob is an environment variable that turns the whole thing off. Baking is a full recompile: the specialized program can elide whole code paths, not merely load a constant. How optimistic to be depends on whether the baked word gates work; see What baking actually compiles.

Status: implemented in src/specialization.rs, driven from Scheme::submit and Scheme::set_node_param. Sections below describe the shipped behaviour; the closing Follow-ups list what was deliberately left out.

Why not a flag per feature

The obvious approach is to let the caller name the fact — a has_tint boolean on a worker description, a second retained scheme for the tinted case — and it does not scale. Tint is one of many statically determinable specializations. Antialiasing mode, filters, non-default blend, and output format are others, and a higher-level compiler will emit facts Goldy has never heard of. Each named flag doubles the number of retained schemes a client has to hold, and each retained scheme carries its own command lists, topology edges, and partition bookkeeping.

Goldy already resists shader permutations rather than industrializing them (see What Goldy Sheds). The mechanism here follows that stance: the runtime holds at most one specialized program per dispatch site, plus a universal program that is always correct.

A CPU branch predictor is the precedent. It does not expose a named flag for each if in user code, and it does not ask the program what it expects; it indexes opaque history by instruction identity and outcome.

CPU branch predictorGoldy
Branch PCStable dispatch-site identity (NodeId on a scheme)
Branch outcomeScalar param wire words the site last dispatched with
Hidden history tableSmall per-site predictor state
Generic code pathUniversal shader (reads its params at runtime)
Optimized targetSpecialized PSO plus the re-recorded partition that binds it

Where the facts come from

Goldy does not need to be told what to specialize on, because it already holds the facts: the scalar params of every dispatch site.

Scheme::with_param takes a u32 wire word. Those words are the program's own encoding of whatever it decided — a tint factor, a mode enum, a filter toggle, a count — and Goldy stores them on the dispatch node, hashes them into the emission fingerprint, and writes them into push constants on every record. A word the caller has never changed since recording is a constant of the recorded program, and a word that has held its value for the last ten submits is a fact about the scene. The runtime can see both without being told.

So the specialization key for a site is derived, not supplied: it is the tuple of scalar wire words the site dispatches with. Goldy never interprets a word. It only compares words for equality across frames, which is exactly what makes the mechanism generic — a new specialization axis is a new with_param in the caller, not a new field in a Goldy struct.

Baking a param

The virtual-main transform reads every scalar param through a preprocessor macro that defaults to the push-constant word:

#ifndef _GOLDY_SPEC_CS_MAIN_UW0
#define _GOLDY_SPEC_CS_MAIN_UW0 _uw0
#endif

[shader("compute")]
[numthreads(64, 1, 1)]
void cs_main(uniform uint _bw0, /* ... */ uniform uint _uw0, /* ... */) {
    uint factor = _GOLDY_SPEC_CS_MAIN_UW0;   // was: _uw0
    // ...
}

Defining that macro to a wire-word literal bakes the value. ShaderModule::variant already merges defines onto a retained module, so a specialized program is variant(&[("_GOLDY_SPEC_CS_MAIN_UW0", "10u")]) and nothing more. The macro is named after the author's [goldy_compute] function (cs_main above; _GOLDY_SPEC_TINT_UW0 for void tint(...)) so that sources with several entries specialize independently, and the define always carries the raw u32 wire word, so the runtime needs no knowledge of the param's Slang type — the existing asfloat / asint / != 0u decode applies to the literal exactly as it applied to the push-constant word. The runtime recovers the function name from the retained source (single_compute_entry_name); a source with zero or several compute entries has no unambiguous macro to define and is left alone.

Two properties make this safe to swap under an already-recorded dispatch:

  • The parameter ABI does not move. The wrapper's signature always declares all eight _uw words regardless of use, so baking one leaves the others at their original offsets. A site with a baked factor and a runtime bias still reads bias correctly. The WebGPU and CUDA lowerings keep their uniform field and kernel argument for the same reason.
  • The recorded payload does not change. Baking rewrites the program, not the command list. The same wire words stay baked into the partition; the specialized program simply stops reading them.

What baking must not do is change the pipeline's binding layout, which is a real constraint on some backends and is covered below.

What baking actually compiles

Baking is a full recompile, not a constant patch. The specialized program is a new ShaderModule::variant with the _GOLDY_SPEC_* macros defined to wire-word literals, so Slang, the backend IR (SPIR-V / DXIL / MSL / PTX), and the driver's ISA compiler all see a compile-time constant. That constant can participate in folding anywhere it flows: branch conditions, loop trip counts, array indices, asfloat(...) arithmetic, resource selection. Nothing is rewritten at bind time, and the universal program is left alone.

Elision is split across two compilers, and that split is easy to misread.

Goldy's default OptimizationLevel::Default does not run Slang's dead-code pass, so a branch whose condition has become false still appears in the dumped SPIR-V — as OpConstantFalse plus the unreachable block. The push-constant load is already gone at that point. Folding a constant condition, deleting the dead block, and reallocating registers is the first thing any production driver compiler does (NVIDIA, AMD's LLVM, Apple's, DXC, Mesa lavapipe). Dumping SPIR-V instruction counts therefore understates the specialized program: the variant that looks almost the same as the universal one at the SPIR-V level can be a fraction of the size after the driver.

Raising Slang to OptimizationLevel::Maximal does the DCE in SPIR-V itself. That moves compile time; it does not improve the ISA the GPU runs. Variants inherit the module's optimization level and there is no reason to raise it for baking.

Measured on lavapipe through Goldy's real Vulkan compile path, a compute shader whose has_tint scalar gated a tinting loop:

ProgramSPIR-V instructionsSPIR-V still has the branch+loopDriver NIR instructionsif / fmul
Universal212yes2011 / 55
Baked has_tint = 0210yes, on OpConstantFalse360 / 0
Baked has_tint = 1207yes1940 / 55
Baked has_tint = 0, Slang Maximal91no360 / 0

The specialized-off program dropped the extra buffer loads, the extra push-constant loads, the unrolled loop, and the registers they held. The specialized-on program dropped only the if and the two unused push-constant loads, which is the expected "only what the constant makes dead goes away" behaviour. Numbers will differ on a hardware compiler; the split between Slang and the driver will not.

When this is worth expecting

A uniform branch on a push constant is already non-divergent and cheap. Removing the if itself is not the win. The win is whatever sits behind it.

Kind of with_paramWhat baking doesHow optimistic to be
Feature / mode flag that gates whole code paths (has_tint, AA mode, filter on)The driver deletes the unused path, its memory traffic, and the registers it held — occupancy can riseHigh, if the gated work is real
Loop bound or "which of N resources" indexTrip count or index becomes a literal; unrolling and addressing followHigh when the bound is small and the body is heavy
Arithmetic coefficient (x * factor + bias)One less push-constant load; maybe a strength reductionModest
Value that changes every few framesPredictor raises the bake threshold and leaves the slot dynamicNone — it will not stay promoted
Value that lives in a bound bufferInvisible to the predictorNone — put scene facts in with_param if they should bake

The cost is a full Slang compile plus PSO creation on a worker thread. Words fixed at record time pay it once, at the first submit; a word that has changed must hold for two submits before it is compiled and ten before it is swapped in. A scheme whose many nodes are all recorded at once spends real CPU in the background for a moment; the 16-entry per-scheme cache bounds the steady state.

Shader authors do not opt sites in, but they do choose where facts live. A mode flag in a Scattered buffer cannot bake. A mode flag passed with with_param, with the expensive work behind if (has_tint != 0u), is exactly the shape the predictor was built for. That is also why Goldy still resists permutation systems: the runtime holds at most one specialized program per dispatch site, not the cross product of every flag.

Per-slot, not per-tuple

Stability is tracked per param slot, not over the whole tuple. A site that passes a stable mode flag and a frame counter can still bake the flag. Keying on the whole tuple would give up on any site with a single volatile param, which in practice is most of them.

Why with_param and not a bound buffer

A fact in a buffer is invisible: Goldy sees a parcel read, not a value, and it would have to read back GPU memory to learn anything. A fact in a with_param is already on the CPU, already hashed, and already known to be stable or not.

The apparent objection is that scalar params are part of the partition retention fingerprint, so flipping one re-records. That is not a cost of this mechanism, it is the signal that drives it: a param that changes every frame re-records anyway and will never earn promotion, and a param that is stable is both free to retain and profitable to bake.

Certain words: what recording already decided

"Compile time" has three meanings here, and only one of them is useful for values:

StageKnowsCan decide
rustc (the #[goldy::compute] macro)Kernel structure: which params are scalars, how the body uses themWhat could bake — not values, which do not exist yet
Recording (Scheme::node … dispatch)Every word and layout the node was built withThat those values are constants of the recorded program until the caller says otherwise
Slang and the driver JITWhatever the defines sayFolding, dead-code removal, unrolling

Recording is the compile step that sees values. A scheme is a program, and the words a node was recorded with are its constants. The only way to change one afterwards is Scheme::set_node_param, so the runtime knows exactly which words have ever changed, and that knowledge is not a prediction.

A word that has not changed since the node's first submit is therefore certain. It needs no history, joins every bake target and counts as proven for promotion. A node whose words are all certain warms at its first submit and promotes as soon as the compile lands, usually on the second. Nobody annotates anything: a shader author cannot say "this is fixed" more reliably than the absence of set_node_param calls does. A change before the first submit is still part of recording and leaves the word certain.

Only words that have changed need history (below). That is where the prediction lives, and where time to promotion is a real cost.

Tensor layouts

Tensor layouts are the other certain source. A GoldyTensorLayout in the metadata parcel is exactly the invisible bound-buffer fact described above, and every tensor kernel used to load it before its first data access. The layout is now split by how widely it is shared:

  • The element offset differs per site, because each site binds a different placement of its parcel. It travels as a launch word: the words after the frame table in PushLayout region C on DX12, Vulkan and Metal (13 of them), or trailing kernel arguments on CUDA. WebGPU and the CPU backend keep reading the parcel, as does a kernel with more than 13 tensors. Offsets never bake, so sites that differ only in placement share one variant.
  • The shape facts are rank, element count, four extents, four strides and flags. Each one reads through a macro like _GOLDY_SPEC_RMSNORM_T0_D0, which defaults to the parcel, so the universal program is unchanged. A node's shape facts are certain: nothing can rebind a node's tensor views after recording. A promoted variant has no layout load and indexes with literal extents and strides.

Tensor shape facts are never compared at submit and never reach the wire; they exist only as bake inputs.

What the predictor can see

Baking a value needs three things at once: the host knows the value, the host knows it is constant, and the shader reads it through a name the runtime can substitute (a _GOLDY_SPEC_* macro). Values fall into three tiers:

TierExamplesToday
VisibleScalar push words (with_param, kernel scalars)Baked
Host-known, behind a loadTensor layouts; buffer lengths (buf.len()); fields of a Uniform<T> written once at record time; dispatch grid dimensionsLayouts baked; the rest not yet
InvisibleAnything the GPU writes, or the host writes into a parcel every frameNever

The middle tier is the interesting one. Each entry is a constant the host already holds, read by the shader through a load or a query the compiler cannot fold. Moving one into the visible tier is the same three steps tensor layouts took: give the shader read a macro that defaults to the load, have recording hand the value to the site, and make the single place that can change it (a rebind, a parcel write, a grid change) report the change the way set_node_param does. See Follow-ups.

Two caches, deliberately separate

CacheKeyed byHoldsEvicting it costs
Variant PSO cache(program identity, baked slots and values)Compiled pipelineA recompile
Per-site predictionDispatch-site identityWhich slots are bakedA re-record

Keeping them separate means a site can be demoted without throwing away the compiled pipeline, so re-promotion later is nearly free: a demoted site whose words come back is promoted straight from the cache once the changed slot has held long enough, with no compile.

The PSO cache is scheme-scoped and bounded (16 variants, LRU). A device-scoped cache was the first design and does not work as stated: a ComputePipeline holds a strong Runtime, so a cache inside the device would form a reference cycle and the device would never be dropped. Scoping the cache to the scheme also settles ownership — the scheme holds an Arc to the variant it has promoted, and the cache holds another, so eviction from the cache never destroys a pipeline a node still binds. The price is that two schemes running the same shader with the same stable words each compile their own variant; the Slang disk cache and the driver's PSO cache make the second compile cheap, and device-level sharing is listed as a follow-up.

Compile workers write into the cache directly, so a variant that finishes after its site lost interest still lands there rather than being dropped.

The Slang bytecode disk cache already sits underneath both, keyed on post-transform source plus defines, so a cold process still avoids full recompiles.

The predictor

The hard part is not compiling two pipelines. It is the words that have changed: once a caller has moved one, nothing knows whether the next hundred frames will use the same value. So the predictor never speculates about the current frame — the params for the current frame are already known exactly, and history only decides whether to prepare a specialization for future frames.

Per-slot history

Every dispatch node with at least one with_param or tensor view gets a site record when it is declared (Scheme::node, ComputeNodeRecord::commit_dispatch_scheme, or a caller-side set_node_pipeline, which starts the site over against the new pipeline). The record holds the caller's pipeline (the universal), the shader's provenance, the node's current words, and per slot:

  • whether the word has changed since the node's first submit. An unchanged word is certain (above).
  • for a changed word, the submit from which it has held its current value. Every change arrives through set_node_param, which restarts the count; nothing else can move a word, so a submit that re-records the scheme for unrelated reasons, or a foreign scheme dirtying shared-parcel topology, says nothing about this node and leaves its history alone. Stability is a property of the dispatch, not of the scheme around it.
  • a bake threshold — how long a changed slot must hold before it is baked. It starts at the warm threshold and grows every time the slot invalidates a compile or a promotion (to the promote threshold, then doubling), so a fact that flips every few frames is baked once, disproved once, and thereafter left dynamic while its neighbours specialize.

The certain slots plus the changed slots at or past their threshold are the site's bake target. It is per-slot: a site with a stable mode flag and a moving counter bakes the flag.

Off the hot path

A site is only worth looking at when something happened to it: it was declared, a word changed, a compile is in flight, or a changed slot is about to cross a threshold. The predictor keeps the sites that need a look in an awake set and the next decision point of every other site in a wake queue keyed by submit number. set_node_param wakes its site; each step computes when the site next needs one (the submit a held slot crosses its threshold, or the next submit while a compile is in flight) or nothing at all.

A settled site — promoted, with every word unchanged or already baked, or pinned — schedules nothing and is never touched again until the caller changes one of its words. At steady state a submit does no predictor work beyond an empty-set check, however many nodes the scheme holds.

Sites are keyed by the dispatch the scheme actually submits. When automatic kernel fusion promotes an execution plan, the constituents' sites retire and the fused dispatch gets a fresh site against the universal fused pipeline. Other sites move to their executed index with their history. Returning to the recorded graph reverses this, and the constituents start fresh.

No submit waits for a compile, and on Vulkan, DX12 and CUDA the compile runs without the backend lock, so a submit on another thread is not stuck behind one. The flip side is that a short run measures the recorded dispatches. A benchmark that wants the steady state keeps submitting until Scheme::compiles_pending reports nothing outstanding.

Stages

Compilation is expensive and uncancellable once started, so promotion is staged. Compiling is separated from swapping, with different thresholds:

StageEntered whenBehaviour
ObservingAfter a demotion, or while a changed slot has not yet held long enoughUniversal runs; held slots accumulate
WarmingBake target non-empty and not what is already promotedUniversal still runs; a variant baking the target compiles on a worker thread (or is taken from the cache)
ReadyThe compile landedUniversal still runs until every changed baked slot has held for the promote threshold
PromotedEvery baked slot certain or past the promote thresholdNode rebound to the variant as a params-only re-record; the site keeps observing the slots it did not bake and may warm a wider variant
PinnedThree failed compilesUniversal only; the site is never consulted again

A new site goes straight to Warming at its first submit, because every word it has is certain.

Defaults are warm at 2, promote at 10 (SpecializationPolicy), and they apply only to slots that have changed. The two-submit warm threshold skips the common ping-pong case, where a value alternates every frame and no specialization would ever pay off. The ten-submit promotion threshold buys confidence that the new value is a scene property rather than a coincidence, and it usually gives the compile enough time to finish before the swap is wanted.

The predictor runs at the top of Scheme::submit, before dirtiness is read for recording, so a promotion is recorded by the very submit that decided it and the scheme is clean again afterwards. It never touches a node whose current words differ from the variant's baked words — a swap is only ever to a program that agrees with the frame. Promotions land as their compiles do, so a scheme whose nodes warm together may still promote over two or three submits, each a params-only re-record.

Demotion is mandatory, not optional

A promoted site is only correct while its baked words match what the program is dispatching. Scheme::set_node_param is the single place a param can change, so it is also the place that must un-promote: if the new word differs from the baked one, the site's pipeline reverts to universal in the same call that marks the scheme params-dirty. The partition then re-records with the universal program bound, and no submit ever runs a program whose baked value disagrees with the frame.

This is the one invariant the implementation cannot get wrong, and it is why the swap lives inside the runtime rather than in a caller's hands.

Cancellation is best-effort

Sites that warm the same program with the same baked words at the same time share one compile: a site whose job would duplicate one still in flight takes a hold on that job instead of spawning its own. Certain words make this common, because every site of a kernel over one shape warms at the same first submit.

A baked word changing while a compile is in flight, in set_node_param, drops the site's hold. A job no site holds any more is skipped if it has not started; it cannot interrupt work already running, because Slang compilation runs behind a process-global lock and the driver's pipeline creation is not interruptible, so a cancelled compile may still run to completion.

When it does, the resulting pipeline is still inserted into the variant PSO cache. It is not swapped in, but it is not wasted either: if those words come back and earn promotion later, there is nothing left to compile.

Cost model

Stability alone is not the whole story. Recording a partition costs CPU time and, on backends that require it, a wait for the previous retained command storage to retire; a tiny dispatch cannot repay that. The shipped predictor has no dispatch-size term — the thresholds are the only cost model — so a small, long-lived, stable dispatch pays one re-record it may never recoup. That is bounded (one record per promotion, promotions are rare by construction) and is listed as a follow-up rather than guessed at without profiles.

First frame, oscillation, reset

  • The first frame always runs universal while the variant for its recorded words compiles. That compile is the one optimistic step: a word the caller changes every frame costs one compile that is dropped on the first change, and then the slot's threshold rises out of reach.
  • A miss demotes immediately, as above.
  • Oscillation is absorbed by the two-submit warm gate and the growing per-slot bake threshold, and three failed compiles move the site to Pinned so a pathological shader cannot make Goldy compile forever.
  • Reset happens only on a caller-side set_node_pipeline, which replaces the universal and restarts every held count. Words that never changed stay certain, and the PSO cache is kept.
  • Turning the feature off at runtime (the environment variable is read every submit) demotes every promoted site on the next submit and drops in-flight compiles.

Baking must not change the binding layout

A specialized variant is swapped under bindings that were already resolved when the node was recorded, so it has to agree with the universal program about what those bindings are. Baking can break that agreement, because making a value constant can make a resource unreachable, and a compiler is entitled to stop expecting a binding nothing reads.

This is not hypothetical, and it is not uniform across backends. Vulkan, DX12, and Metal bind through a bindless heap, so a pipeline's expectations follow the shader signature and survive baking. WebGPU does not: wgpu derives the bind group layout from the compiled WGSL, so it reflects usage. Two consequences, both observed:

  • Baking a value that was the only reason a bound buffer was read drops that buffer from the layout, and the recorded dispatch then supplies a binding the pipeline no longer declares.
  • Baking every scalar of an entry point leaves the generated user-params uniform buffer unreferenced, which drops it the same way — so on those backends at least one scalar slot has to stay dynamic.

Neither constraint is enforced by the predictor, because the predictor does not run where they apply (next section). On the backends where it does run, the layout follows the signature and a variant binds exactly like its universal, so the site's bake target can be every stable slot.

A caller-side set_node_pipeline on WebGPU is a different matter: slot_access cannot check it — that table is derived from the shader signature, so it agrees across a variant that a usage-derived layout would reject — and wgpu exposes no entry count for a BindGroupLayout, so an incompatible hand swap surfaces as a bind group mismatch inside the backend rather than as a set_node_pipeline error.

A site that cannot be specialized without moving its bindings must stay universal. That is a missed optimization, not a bug — but it has to be detected rather than assumed.

Gate the backend, do not tier the feature

The awkward part is that the specializations most worth having are the ones that drop a binding. if (has_tint) { read the tint buffer; blend } is the motivating case, and baking has_tint to zero is exactly what makes that buffer unreachable. On a backend with usage-derived layouts, the predictor would decline promotion precisely where it would have paid off most.

That is a reason to gate WebGPU, not to make specialization a property of "better" backends. Usage-derived layouts are not a WebGPU limitation: the backend passes layout: None and takes wgpu's auto layout. An explicit pipeline layout built from the signature-derived WgpuComputeLayout the backend already computes would be stable under baking, because a layout may declare bindings the shader does not reference. Until that lands, the honest position is that Goldy's WebGPU backend cannot support specialization, not that WebGPU cannot.

So promotion is conditional on a backend capability — GpuBackend::compute_pipeline_layout_follows_signature — rather than on a backend allow-list. Vulkan, DX12, Metal, CUDA, and the mock backend report yes; WebGPU reports no while it uses auto layouts, and flips on by itself once it does not; the CPU backend reports no because its host-callable lowering has no bake macros, so a variant would be the same kernel. The predictor stays backend-agnostic either way, and the capability is internal: specialization is an implementation detail, so it does not belong in RuntimeCapabilities where a program would branch on it. A scheme queries it once, on its first submit.

Off switch

Specialization is on by default. GOLDY_SPECIALIZATION=0 (or false / no / off) disables prediction entirely: no history, no warm compiles, no swaps, and every site stays on the program it was recorded with. The gate follows the convention of GOLDY_DISABLE_CB_REUSE — read through validation_env, with a thread-local override (test_support::SpecializationOverride) so tests that count records across many frames can pin it.

This is separate from the backend capability above. The environment variable is a global kill switch for when prediction is suspected of causing a problem; the capability decides where prediction can be correct at all.

What this rests on

The mechanism depends on runtime properties that are already in place:

  • Params-only dirtiness. Swapping a node's pipeline marks a scheme params-dirty rather than structurally dirty, so only partitions whose baked payload changed re-record. The schedule cache keys on bindings, which a pipeline swap does not touch.
  • Scheme::set_node_pipeline. The swap itself, addressed by a stable node identity.
  • ShaderModule::variant. A new module with merged defines, reusing retained source, search paths, optimization level, and layout checks.
  • Overridable scalar reads. The virtual-main macro described above, which is what makes a param bakeable without the shader author participating.
  • Compile off the device lock. ComputePipeline::new runs Slang outside the backend mutex on Vulkan and DX12, which is what makes an off-thread warm compile possible without stalling submits. Pipeline creation itself still holds the lock, and on Vulkan that is the expensive half (see goldy#175), so a warm compile can still contend with unrelated submits until pipeline creation moves out too.

Shader provenance

A dispatch node holds a ComputePipelineHandle and a ComputePipeline holds a backend handle; neither, on its own, says which program it is. So every ShaderModule keeps its compile inputs — source, search paths, defines, optimization level, layout checks — in a shared ShaderProvenance with a process-unique id, and every ComputePipeline built from the module carries an Arc to it. The runtime can therefore compile a variant of the program a site is running after the caller has dropped the module (ShaderModule::from_provenance), and variants are keyed by program identity plus baked words.

The program identity is usually the provenance id, so each module has its own variants. A fused kernel (FusedKernel) also records a stable KernelId on its provenance. The id is derived from its constituent kernels, argument map and workgroup size, and the cache keys its variants by that id instead. Two fused kernels with one id compile the same program and bind alike, so they share variants. The provenance also names each fused scalar slot after the constituent scalar it binds, and the trace events below use those names.

Ownership matters more than it looks: on Vulkan, destroy_compute_pipeline waits for device idle, so dropping a variant is not a background operation. The scheme holds the variant it has promoted; when a variant is unbound (demotion, a wider promotion, a caller swap) it is parked for two further submits before its Arc is released, by which point no retained command list that bound it is still the one being submitted. The bounded cache usually still holds it after that.

Telemetry

ReplayStats gains specialization_warms, specialization_promotions, and specialization_demotions; Scheme::node_is_specialized(NodeId) answers for one site. specialization_warms counts compiles started, so sites that join an in-flight compile do not add to it. Demotions are visible in the stats immediately after the set_node_param that caused them. Each transition also emits a tracing event under the goldy target (debug for warm / promote / demote, warn for a failed compile or a pinned site). Events carry the site's kernel id (- without one) and its baked slots as name=word. Unnamed slots appear as slot0=0x7; fused slots are named by origin, as in 1:damp.enabled=0x1. Tensor shape facts appear as t{tensor}.{field}, as in t0.d0=0x120.

Backend differences

The predictor's inputs are set_node_param events and the submit count, neither of which depends on command-list retention, so the mechanism behaves the same on backends that retain command lists and on Metal and WebGPU, which do not; only the size of the saving differs.

The overridable macro is emitted by every compute lowering — the native push-constant path, the WebGPU uniform-buffer path, and the CUDA kernel-argument path — each defaulting the macro to its own read expression, so baking works the same way everywhere and the launch layout is unchanged in all three. Graphics stages lower scalars without the indirection; specialization is defined for dispatch sites, so that is deliberate.

Follow-ups

  • Host-known values behind loads. The middle tier of What the predictor can see:
    • Buffer lengths. goldy_buf_len is a GetDimensions query on DX12, Vulkan and the CPU backend, and returns 0xFFFFFFFF on CUDA, Metal and WebGPU, where pointer descriptors carry no length. Passing the bound element count as a launch word (like tensor offsets) would fix those backends and give the read a macro to bake through. Rebinding a node's parcel would report the change as set_node_param does.
    • Record-constant Uniform<T> fields. A uniform written once at record time and never again is certain in the same sense as a scalar never passed to set_node_param. The shader-side read would need a per-field macro, and parcel writes would need to report which node's uniform they touched.
    • Grid dimensions. The dispatch size is host-known and is changed only through the scheme, but kernels do not read it as a value today. It would bake loop bounds derived from num_workgroups().
  • Cost model. A dispatch-size (or measured-duration) term so tiny dispatches are never promoted. Needs profiles from a real consumer before the threshold is more than a guess.
  • Runtime-level variant sharing. Many schemes running one shader with the same stable words compile it once each today. Sharing needs an owner that does not cycle with Runtime, e.g. variants that hold a weak device reference and a cache that drains on device drop.
  • WebGPU explicit layouts. Building the pipeline layout from the signature-derived WgpuComputeLayout would let the backend report compute_pipeline_layout_follows_signature and enable prediction there with no predictor changes.
  • Graphics stages. Draw sites with scalar params lower without the macro indirection; specialization is defined for compute dispatch sites only.
  • Pipeline creation off the lock. Warm compiles still take the backend mutex for PSO creation (goldy#175).

Non-goals

  • Per-image or per-command program switching inside a mixed command-stream walk. A single dispatch over a heterogeneous command buffer cannot pick a different program per element; that stays a branch inside the shader.
  • Speculating incorrectly. A miss always runs the universal program. There is no "probably fine" path.
  • Named feature flags as the long-term API. has_tint and its successors do not belong in worker topology.
  • A public specialization API. Callers do not name keys, supply hints, or opt sites in. The only control is the off switch.
  • Caching every combination. The runtime holds one promoted variant per site and a bounded PSO cache, not the cross product of every axis.