Contract: plugin.toml — [[proof]]

Status: Accepted

Part of the plugin.toml contract: the checks a plugin declares, which must hold before lemonfiber installs it. The requirements this page describes are listed with the rest of the contract’s, under Requirements.

[[proof]] — what must hold before it is installed

[[proof]]
id      = "plex.serves"
title   = "Plex answers on its declared health path"
request = { method = "GET", path = "/identity" }
expect  = { status = 200, json_has_keys = ["MediaContainer"] }
fixture = "fixtures/identity.json"
why     = "The path the health probe asks for is one this image serves."

F3-R1 names a plugin’s proofs among what its manifest declares, F3-R3 runs them in the existing verification engine, and [[proof]] is where they are declared — so F3-R4, a plugin whose proofs do not pass is not installed, has something to evaluate.

expect must constrain the body. A status is a claim about the network path, not about the service: Docker publishes a port by putting a proxy in front of it, and that proxy accepts a connection before knowing whether anything inside is listening. A manifest whose every proof asserts only a status MUST be refused, naming the proofs, because it has declared nothing a replaced container would fail (ARCH-R105). An exit status alone is a status: a run proof whose expect checks only exit counts as one that asserts only a status. The core holds a [[claim.probe]] to its probe’s body constraints and holds a [[proof]] to nothing of the kind: a proof whose expect is a status alone is installed and run as written. 0.18.0 locks ARCH-R105.

What an expectation may say

One vocabulary, read in three places — a [[proof]], a [[claim.probe]] and a contributed check — because all three ask a service a question and judge the answer, and three vocabularies for one job would be three things to keep in step.

The set is closed. A key outside it is refused by name rather than ignored, for ARCH-R91’s reason pointed at an expectation: an assertion nothing evaluates is a proof that silently checks less than it says, which is worse than one that fails.

json_is_absent is the one worth reading twice, because its name invites the other reading — these keys are absent — and the two are not close. It says nothing about keys. It says the answer was not JSON.

Where an expectation looks

The four key-wise constraints above take a place rather than a name. The other five are about the answer as a whole and take none.

A key that does not begin with / is the name of a top-level member, which is what every key written before this generation is and is why none of them changed meaning. A key that does begin with / is a JSON Pointer, with one extension this contract defines.

Standard, and ours. Everything but the third row is RFC 6901 unchanged. The third is the extension, and it is one step and one comparison: a reference token written [field=value] means the entry of this list whose field holds value, exactly one entry must, and there are no operators, no wildcards, no nesting and no indices.

An index would be the trap the selector exists to avoid. Plex answers a hundred and fifty-one settings at /:/prefs and the order of them is not a promise anybody made, so the ninety-first is the wrong answer one release later, in a way that keeps passing.

A selector’s value is escaped like any other reference token, which the fourth row is there to show and which the common case needs: a filesystem path is full of slashes, and a slash in a token is written ~1. Unescaped, [path=/data/media/movies] is four steps rather than one, and is refused by name rather than resolved somewhere nobody meant.

Its cost is stated rather than hidden. [ and ] belong to the selector, so a reference token carrying either is refused rather than read as a member name — a member actually called [a=b] is unreachable through a pointer. That is the price of the extension; it also closes the mistake anybody would actually make, which is writing …/Setting[id=X]/value and being told nothing while it looks for a member with brackets in its name.

Why a name was not enough. A flat name says everything there is to say about a flat answer, and the two plugins published before this one both have flat answers. A service that nests its payload — Plex puts every response one level down under MediaContainer — could only be asserted about at the envelope, so json_has_keys = ["MediaContainer"] was the strongest claim available and it says that the service replied. A probe that passes by observing that something replied is worse than one that fails, because a port proxy replies.

A key naming no place is refused when the manifest is read, naming the key and what is wrong with it. It is not evaluated as a missing member: an assertion nothing can evaluate is one that silently checks less than it says, which is ARCH-R91 pointed at an expectation.

[[recipe.step]].capture.from reads an answer with these same keys, so one dialect says where a value is, whether a proof is asserting about it or a recipe is taking it.

Three verdicts, never two (F3-R5, F4-R7): passed, failed, and could not be run. The third is reported as unproven and is never counted as the first. Against a recording a proof or a check declares it fails on, a fourth is reported in place of the second: failing as declared.

A proof that runs the image’s own program

A part with no network surface has no answer to ask for. Quality sync is a program its image runs on a schedule: it publishes no port, listens on nothing and declares no health probe beyond the container running. Its proof runs the image’s own program once, in a container of its own, and judges what it exits with and what it writes first:

[[proof]]
id      = "recyclarr.runs"
title   = "The pinned quality sync starts as the service runs it"
run     = { args = ["--version"], timeout_s = 30 }
expect  = { exit = 0, stdout_starts_with = "v8.7.3 (" }
fixture = "recordings/recyclarr/runs-version.json"
why     = "It has no port: the live question is whether the pinned program starts under its posture and its user."

A [[claim.probe]] takes run only where the vocabulary declares its probe a run, which it does for quality.sync alone, the one capability whose fillers listen on nothing (capability vocabulary, ADR-0050); it takes request for every other probe. A contributed check is a question asked of a running service, so it takes request and nothing else. A run’s expect takes these two keys and no other, and a request’s takes none of them; each is refused by name where the other’s is written (ARCH-R283).

The manifest names no program. args go to the entrypoint the image declares, the program the service itself runs, so the proof asks the pinned program a question rather than starting something of the manifest’s choosing. lemonfiber starts no shell and passes each string as one argument, so nothing in args is read as a command, a variable or a pattern.

It runs apart from the service, and with less. lemonfiber starts a new container from the image at the digest the manifest pins, never a command inside the running service, and removes it when the run ends (ARCH-R284):

What the run writes is the program’s, not lemonfiber’s, so every report shows it with its control characters removed, and none of it reaches a terminal as anything but text. An engine lemonfiber cannot reach, and an image it does not hold at the pinned digest, leave the proof unproven, naming which (F3-R5).

Where it runs. Wherever a proof does: against its recording in the plugin’s CI and in the release gate, live in the plugin’s CI with the image under its posture, at install and at update, and in lemonfiber plugin prove (ARCH-R286). The doctor runs it too, on its own cadence rather than the dashboard’s: only when the doctor is run, never on the gather that feeds the dashboard or the event stream, and only where something it depends on has changed since the verdict it keeps. That verdict is kept across restarts against the image’s digest, the proof’s arguments, the posture and user it ran under, the digest of the starting files and the lemonfiber release that ran it, and reported with when it was reached. The doctor pulls no image for it. A run is the one check that may hold a doctor run past C1-R12’s thirty seconds, once, after its pin moves (ARCH-R287).

What an assertion is declared to fail on

A [[proof]] and a doctor.check row may carry expected: the recordings it fails on, which of its constraints fails there, and why. It is for an assertion whose passing state nobody can record, such as a check that a server has an owner where claiming one needs an account the plugin’s CI does not hold (F10-R12).

[[contribution]]
at        = "doctor.check"
id        = "plex:claimed"
request   = { method = "GET", path = "/identity", accept = "application/json" }
expect    = { status = 200, json = { "/MediaContainer/claimed" = true } }
fixture   = "fixtures/identity-anonymous.json"
expected  = [
    { fixture = "fixtures/identity-anonymous.json", verdict = "fails", constraint = "json", place = "/MediaContainer/claimed", reason = "Recorded from a server nobody has claimed; claiming one needs a plex.tv account." },
]
# … title, category, timeout_s and why as for any check

An entry names one constraint, and it has to be one the assertion makes. An entry naming a key expect does not carry, or a place that key does not constrain, is refused by name, with the constraints the assertion does make (ARCH-R130).

What each entry changes is the report about that recording and nothing else. Every constraint of expect is judged, so the report can say which failed:

Against any other recording, and against the live service, the assertion is held to expect exactly as if expected were absent (F10-R15). A [[claim.probe]] has no expected: a claim is what other services are wired on, and one whose probe fails is not wired to (F4-R6).

What a recording is

A fixture names a file beside the manifest holding one response somebody recorded from the image this manifest pins. It is what lets a claim be shown where no instance exists — an author’s laptop, the catalogue’s CI, a reviewer’s checkout — and it is ordinary reviewable data rather than a cassette a tool wrote and only that tool reads (F10-R4, F10-R5).

{
  "recorded_from": "ghcr.io/example/thing@sha256:6c2a967…",
  "note": "An unauthenticated read of the catalogue. A refusal is the pass: a library server on the household network that answered this would be publishing somebody's collection to every device on it.",
  "request":  { "method": "GET", "path": "/api/v1/series" },
  "response": {
    "status": 401,
    "headers": { "content-type": "application/json" },
    "json": { "status": 401, "error": "Unauthorized" }
  }
}

It names the image by digest, and that digest is the manifest’s own pin. A recording taken from some other build is a claim about software nobody is installing, and the drift is silent: it passes, and the service it describes is not the service that will run. Moving the pin means re-recording in the same change, and a recording naming a different image is refused rather than trusted.

What was asked for is part of which call a recording is of. A service that negotiates answers two different things at one path, so a recording that did not say which it asked for would be evidence for whichever question somebody later pointed at it. A recording taken plainly is not evidence for a request that names an accept, and one that named an accept is not evidence for a request that does not.

Every recording is taken by lemonfiber plugin record <path>, from the manifest’s pin, so moving the pin and recording again are one command (ARCH-R285). What it takes is the manifest’s list, and no option names a recording. A recording under conformance/ that no case asks is removed.

A pull request proves its own adapter with two options. --check takes every recording again and writes nothing: it fails naming each recording that differs from what it took and each one no case answers. An identifier or a moment the service makes afresh on each run, a UUID, 32 hexadecimal digits or an RFC 3339 date and time, is held to its shape alone, as it is inside a string that holds a JSON list or object. --adapter <image> starts the adapter from an image the engine already holds, the one the plugin’s CI built from its own Dockerfile, in place of its pin, and pulls nothing. A pull request that changes what its adapter answers writes its recordings this way, from the adapter it built, and checks them the same way. The pull request that then moves the adapter’s pin to the published image takes every recording again from that pin and holds it to what was written, so no recording ships that its pin does not reproduce.

A request is recorded by asking it. Each service is started once, from its pinned digest, under its posture, user and memory limit, with throwaway copies of its starting files and data, on a network of its own, its listens port published on loopback alone. Once its health passes, its first run is the core’s own: the in-process adapter api names performs it for a part the bundle assembles, and the plugin’s adapter performs it over its contract, as install does, for a service that adapter fronts. A probe whose vocabulary entry says credential = "operator" is asked with the key that first run left lemonfiber holding. A service an adapter fronts has no such probe, because the manifest refuses one there (ARCH-R302). Anything else presents nothing the manifest does not write. Each recording keeps the request as declared, the status, the content-type, the body as parsed JSON or its first 1 KiB, and the note already there.

What a recording holds is scrubbed. Every credential the run made or read, in each encoding an answer could echo it in, each container’s id, hostname and addresses, and the recording machine’s name, kernel version and operating system as the engine reports them are written as <redacted> wherever they appear, and a recording still holding one is not written. A number an answer gives about the machine, such as its count of processors, is kept as it was answered. A service is recorded whole or not at all: where any step of it fails, its recordings stay as they were and the run fails naming the service and the step (ARCH-R297).

A run is recorded by running it, and its recording holds the image it ran, the arguments, the exit status and the first 1 KiB of standard output, keeping the note an author wrote.

{
  "recorded_from": "ghcr.io/recyclarr/recyclarr@sha256:9c6c74d52cfc6f740a5a3624aa83867e171bb70b90453ddb7a4048b8c80445f9",
  "note": "The pinned program, started as the operator's user with no network, says which release it is.",
  "run":    { "args": ["--version"] },
  "result": { "exit": 0, "stdout_starts_with": "v8.7.3 (Branch.master.Sha.237d4ddea46b12c6d1aca45936ddc410c3a13bd8)" }
}

A recording holds request and response, or run and result, never both. Running it again writes the same result, which is how the plugin’s CI proves it live: a run whose output differs from one run to the next is not evidence of anything, and a recording whose result a fresh run does not reproduce is refused (ARCH-R285).

A recording that is absent, unreadable, or records a request or a run the assertion does not ask is unproven — never a pass and never a failure. Nothing about the service has been established either way, and reporting it as a failure would say the service is broken when the recording is. It is the third verdict’s plainest case, and the one an author meets most often.

Related

• plugin-manifest — the rest of the contract, its validation and its requirements
• plugin-manifest-recipe — the ordered calls a plugin may declare