4 terms · each defined as a formula over a named population · version 1.0 · published 2026-08-14

The measurement vocabulary

Four terms this ecosystem measures constantly and has no settled words for. Defined here as formulas rather than descriptions, so somebody else can compute them and get a number comparable to ours.

Every definition names its denominator

“9.7% fail” is not a fact. It becomes one when you say 9.7% of what, tested how, on which date, with what left in the denominator. Most published figures about AI tooling omit exactly that, which is why none of them can be argued with — only believed or not. Each term below carries its formula, its population and what it is explicitly not, so a number computed from it can be checked against a number computed from somebody else’s data.

These are free to use, with or without attribution, including by the catalogues this masthead competes with. A vocabulary only earns anything by being used.

Load rate

The share of published artefacts that a runtime could actually register.

Formula: load rate = artefacts that pass every structural check ÷ artefacts found
Population: Everything discovered, including everything that failed and everything that could not be parsed. Nothing is dropped for being unreadable.
Measured by: SkillWorks
Not this: Not an execution result and not a success rate. Nothing is run. A listing that loads can still be wrong, useless or dangerous — load rate is a floor, not a verdict.

The first question about any artefact, and the one no catalogue in this space asks: before it is good, popular or maintained, does the file parse into something an agent runtime could register at all. Load rate is that share, and its complement — the structural failure rate — is the citable figure, because a population count is a number anybody can produce and a share is one somebody had to go and check.

Measured across 445,348 listings on 2026-08-14, the load rate of the published Claude Code ecosystem is 90.3%. The structural failure rate is 9.70%.

A load rate is only comparable if the checks behind it are enumerated, so they are: frontmatter parses, fields the documentation marks Required are present, manifests are valid JSON, and a marketplace lists at least one plugin. A check that is a matter of taste never belongs in this figure.

Drift

A vendor-stated fact that changed without an announcement.

Formula: drift = a diff between two dated captures of the same vendor-controlled page
Population: The set of pages under watch, captured on a fixed schedule. A page that moved is followed and the redirect published rather than silently resolved.
Measured by: ToolDrift
Not this: Not model drift in the machine-learning sense — nothing here is about a model's output distribution changing. This is drift in published facts about a product.

Prices, model line-ups, rate limits and rankings are facts a vendor states about itself and can restate at any time. Drift is the recorded difference between two captures of the same page — not the vendor’s changelog, which is an edited account of what they chose to mention.

The distinction that makes it a measurement rather than a feed: drift is computed against a stored prior capture, so it is reproducible after the fact and the previous value is still there to be inspected. An unannounced price change and an announced one are the same size of drift and different sizes of story.

Shipping status

Whether a project is still being worked on, decided from activity rather than from a claim.

Formula: shipping status ∈ {maintained, slowing, dead}, recomputed nightly from commit, release and issue activity
Population: Every tracked project, every night. A verdict is never recorded once and kept.
Measured by: StillShipping
Not this: Not an abandonment accusation and not a quality signal. A finished tool that needs no commits reads as slowing, and that is a limitation of the measure, stated rather than hidden.

The reliable early signal that a tool is dying is silence, not an announcement. Almost nothing in this ecosystem publishes its own death; a changelog stops, an issue queue goes quiet, and the landing page keeps saying the same confident sentence for a year.

Shipping status turns that silence into a stated verdict on a fixed schedule. Because it is recomputed nightly rather than stamped once, a project that goes quiet changes its own row with nobody editing anything, and a project that comes back changes it back.

Skill decay

The gap that opens between a published artefact and the runtime it was written for, while nobody touches either.

Formula: skill decay = time since the artefact was last changed, weighed against changes in the spec it depends on
Population: Artefacts that currently load. A broken artefact is a load-rate failure and is counted there instead, so the two figures never double-count the same defect.
Measured by: SkillWorks
Not this: Not the same as being unmaintained. A stable artefact against a stable spec is not decaying; decay is about the gap opening, not about the calendar alone.

An agent skill is written against a moving specification — a frontmatter schema, a tool list, a set of runtime conventions. It does not break when it is published; it breaks when the spec moves and the file does not. Skill decay names that interval, and it is the failure mode this ecosystem should expect to dominate as it ages, because publishing is free and maintenance is not.

Two observable inputs, both in the open dataset: how long since the artefact’s repository was last pushed, and whether it names tools or fields the current documentation no longer describes. Of the population measured on 2026-08-14, 108,266 listings had gone 90 days without a push and 16,282 had gone 180.

Stated as a shortfall: this is the least mature of the four. Decay is currently observed through age and stale tool names rather than through a versioned diff against the spec itself, which is what it should eventually be measured against.

Citing a term

Each term has a stable anchor on this page — /methodology/vocabulary#load-rate and so on — which will not move. The figures quoted above come from The State of Agent Tooling, whose full dataset is published under CC BY 4.0 so any of these can be recomputed from source.