Codex App Server in Production, Part 11: Trust but Verify: Review Mode, Structured Outputs, and the Publish Gate

Nobody told the inspector about the logo. It read the brief, opened the client's own SVG, compared the fill colors against the brand rules, and wrote this into the record:

The brief makes #2F5233 the house color, but the logo asset shown in the hero and footer (assets/logo.svg) uses #1E3A5F for both the mark and the wordmark, and the client-supplied copy in brief/assets/logo.svg matches that blue version. Because the live page ships the blue logo unchanged, the site presents a brand color that conflicts with the brief; the client needs to choose whether to recolor the logo or update the brand rule before publishing.

That contradiction is not an accident, and it is not new. It was planted in this series' brief bank back in Part 4: Harbor & Vine, the dockside wine bar, whose brief demands deep forest green while the logo the client supplied is navy. Seven parts it sat there, waiting for the builder to inherit it, and for this page, where Pagewright stops taking the builder's word for anything. By the end, Publish is a button that has to be earned: a fresh-eyes inspection with no blockers, a schema-validated manifest, and one deterministic check that exists because a real bug demanded it.

Three protocol features carry this part: review/start (the engine's built-in reviewer, which no sibling series has anything like), outputSchema (structured final answers), and ephemeral threads from Part 5, finally cashing the IOU. But the part opens with none of them. It opens with a button that works perfectly and feels wrong.

The button that ships anything

Publishing a static site is not a hard problem. Pagewright's workspaces already hold a finished site/ directory; "publish" means copying it somewhere public and serving it. The whole feature is one function and two mounts:

    target = site_path(slug)
    if target.exists():
        shutil.rmtree(target)
    shutil.copytree(projects.site_dir(project_id), target,
                    ignore=shutil.ignore_patterns("brief"))
    entry = {
        "slug": slug,
        "project_id": project_id,
        "name": name,
        "url": f"/p/{slug}/",
        "published_at_ms": int(time.time() * 1000),
        "manifest": manifest.model_dump() if manifest else None,
        "forced": forced,
    }
    entries = [e for e in entries if e["project_id"] != project_id] + [entry]
    save_registry(entries)
    projects.update_project(project_id, published={
        "slug": slug, "url": entry["url"], "at_ms": entry["published_at_ms"]})
    return entry

Copy the files (minus brief/, the client's paperwork; hold that thought, it returns with teeth), write a line into published/published.json, mount the folder at /p/{slug}/ with the same StaticFiles pattern the preview has used since Part 4. A slug is derived from the project name; republishing reuses it, and a different project claiming a taken slug gets a numbered suffix instead of someone else's URL. In Part 13 this exact /p/ path goes onto a public server, which is why the shape is worth getting right now.

Wire a Publish button to that and you have a working product. Click, and the client's site is live at a URL you can text to a friend. It works on the first try.

And it feels reckless. Name the feeling precisely, because the feeling is the spec for the rest of this part: nothing stands between "the agent stopped typing" and "this is on the internet." Not a second pair of eyes, not a link check, not a glance at whether the thing even matches the brief. You have spent ten parts building trust in this builder, watching it stream, steer, survive restarts, and plan. Trust is not the problem. The problem is that trust is not a release process.

Checkpoint. Right now you have: a fifteen-line copy, a registry, and a /p/{slug}/ mount that serves anything the builder last wrote. The rest of this part is the word earned.

The inspector takes the case

Codex ships a built-in reviewer. Not a prompt template, a protocol method: review/start runs a dedicated review pass as a turn on your thread, with its own item vocabulary to bracket the work. The concepts page calls it the building inspector, a different clipboard from the builder's, and this section wires the inspection into Pagewright as the pre-publish check.

The request shape is {threadId, target, delivery?}. Three of the four target types (uncommittedChanges, baseBranch, commit) assume a git repo, and a client workspace has none. The fourth is the product lane: {type: "custom", instructions}, free-form instructions, no git anywhere:

    thread_id, _ = await ensure_thread(entry, workspace)
    queue = client.queue_for(thread_id)
    started = await client.request("review/start", {
        "threadId": thread_id,
        # the other targets (uncommittedChanges, baseBranch, commit) all
        # assume a repo, and a client workspace has none.
        "target": {"type": "custom", "instructions": review.INSTRUCTIONS},
    })
    turn_id = started["turn"]["id"]
    turns.begin(project_id, thread_id, turn_id)
    eventlog.begin_turn(project_id, turn_id)
    await eventlog.publish(project_id, {
        "type": "session_start", "session_id": thread_id,
        "project_id": project_id, "mode": entry.get("mode", "standard"),
        "turn_id": turn_id, "started_at_ms": int(time.time() * 1000),
        "review": True})
    CONSUMERS[turn_id] = asyncio.create_task(consume_turn(
        project_id, thread_id, turn_id, "", workspace, queue,
        review_turn=True))

Read the bottom half again: it is the /chat endpoint's rhythm, unchanged. The inspection runs through the same consumer, lands in Part 9's event log, and replays into every tab. The session_start carries no message because nobody typed one; the review: True flag is how the UI knows to render this turn as the inspector at work rather than a conversation. Delivery defaults to inline (the review runs as a turn on your thread); detached would spin up a fresh thread and hand back a reviewThreadId. Pagewright stays inline: one project, one transcript, the inspection on the record next to the build it judged.

review.INSTRUCTIONS is the inspection itself, and it is deliberately generic: check the site against brief/brief.md for required sections, voice, and every explicit brand rule, including opening each file in brief/assets/ and reading its real contents (for an SVG, the actual fills and strokes inside it); check accessibility basics; check that every internal reference resolves. Nothing in the instructions names a client, a color, or a file. The inspector has to earn its findings.

What comes back brackets the investigation with two new item types. enteredReviewMode opens the pass (its review field echoes your instructions back); then real work: commandExecution items you have rendered since Part 3, the inspector running find, reading the brief, catting the logo. Then exitedReviewMode closes it, and its review field is the payload: the findings. The consumer translates the bookends at the same seam every other event crosses:

            if (event["type"] in ("item_start", "item_done")
                    and event["kind"] in ("enteredReviewMode",
                                          "exitedReviewMode")):
                item = note["params"].get("item", {})
                if (event["type"] == "item_start"
                        and event["kind"] == "enteredReviewMode"):
                    await eventlog.publish(project_id, {
                        "type": "review_state", "phase": "entered",
                        "instructions": item.get("review", "")})
                if (event["type"] == "item_done"
                        and event["kind"] == "exitedReviewMode"):
                    raw = item.get("review") or ""
                    findings = review.parse_findings(raw)
                    report = review.save(project_id, turn_id, raw, findings)
                    for finding in findings:
                        await eventlog.publish(project_id, {
                            "type": "review_finding", **finding})
                    await eventlog.publish(project_id, {
                        "type": "review_state", "phase": "exited",
                        "counts": report["counts"], "raw": raw,
                        "at_ms": report["at_ms"]})
                continue

Two new event types on the Part 2 envelope, nine parts old and still absorbing: review_state brackets the pass, review_finding carries one parsed finding each. Because they ride the log, the report card renders in every tab, survives a refresh, and replays for a tab opened tomorrow.

Two honesty beats before the payoff, both verified live because the docs would not commit to either.

Latency. When this series first spiked review/start, a one-file review took 3 minutes 45 seconds, and that number went into the concepts page with a warning. Building this part, that never reproduced: inspections of the one-page Harbor & Vine site measured 20 to 90 seconds across every capture (32s and 21s in the run on this page). Both numbers are true; review latency depends on the workspace and, apparently, the weather. So the UI does not guess: the bar shows "the inspector is reading the site, this takes a minute or two" with a live elapsed clock, and the report card explains that the commands streaming past are the inspector working. Set expectations honestly and slow features survive contact with users.

The model. You might reasonably want the reviewer on a different model than the builder. review/start takes no model param; send one and it is silently ignored (verified: the request is accepted, nothing changes). The real lever lives one level up, and it is a config key:

    started = await client.request("thread/start", {
        "cwd": str(workspace),
        "sandbox": "workspace-write",
        "approvalPolicy": "never",
        "model": MODEL,
        "config": {
            "include_plan_tool": True,
            "features.default_mode_request_user_input": True,
            "review_model": MODEL,
        },

How do you prove a config key routes anything when the output looks the same either way? Break it. Set review_model to no-such-model-xyz and the review turn fails with model_not_found naming exactly that string. The key is live. Pagewright pins the inspector to the series model, gpt-5.4-mini: one model to reason about, one bill to read, and the A/B showed no accuracy difference on this workload (both the default review model and mini caught the planted flaw, 30s versus 26 to 28s).

The planted flaw pays off

Time to collect on a seven-part-old setup. Create a Harbor & Vine project, send "Read brief/brief.md and build the site it describes.", and the builder does what it has always done: a faithful, handsome, forest-green page, 224,574 tokens, 67 seconds. It copies the client's logo into assets/ and ships it in the hero and the footer, exactly as the brief instructs. The preview looks great. Every part before this one would call that done.

Click Inspect site. The card opens in its running phase, the inspector's commands stream past (it reads the brief, then the page, then opens the logo file itself), and 32 seconds later the report card seals with the finding this page opened with. Here is the raw text, verbatim from the wire:

Sit with what had to go right. The brief says #2F5233 in prose. The logo's navy lives inside an SVG as stroke="#1E3A5F", in a file the builder copied without comment, because copying it was literally the instruction. No diff shows a conflict; both files are "correct". The only way to catch it is to read the brief and the asset and the page, and hold all three in mind at once, which is precisely what generic instructions ("check brand colors against the client's own assets; open each file") made the inspector do. Fresh eyes, told where to look but not what to find.

One wrinkle the protocol hands you: the findings are tagged prose, not JSON. A summary paragraph, then markdown bullets tagged [P1]/[P2]/[P3] with path:line references, repeated in a final agentMessage. Your product either renders the markdown or parses the tags. Pagewright parses, humbly:

# One finding = a [P*] tag and everything until the next tag (re.S so a
# finding may span lines). The reviewer writes markdown bullets; the
# stray "- " left at a segment's end belongs to the NEXT bullet.
_FINDING = re.compile(r"\[(P[123])\]\s*(.*?)(?=\[P[123]\]|\Z)", re.S)
# A path-looking token, optionally with :line. Anchored to known static
# site extensions so prose like "e.g." never becomes a location.
_LOCATION = re.compile(
    r"\b((?:[\w.-]+/)*[\w.-]+\.(?:html?|css|js|svg|md|png|jpe?g|webp|ico|txt))"
    r"(?::(\d+))?\b")

    for match in _FINDING.finditer(text or ""):
        severity = match.group(1)
        segment = match.group(2).strip().rstrip("-*• \n").strip()
        segment = _ABS_WORKSPACE.sub("", segment)
        if not segment:
            continue
        parts = _TITLE_SPLIT.split(segment, maxsplit=1)
        title = parts[0].strip().lstrip("*").strip()
        body = parts[1].strip() if len(parts) > 1 else ""
        if "\n" in title:  # no dash seam: first line is the title
            title, _, rest = title.partition("\n")
            body = (rest + "\n" + body).strip()
        if len(title) > 90:
            title, body = title[:90].rsplit(" ", 1)[0] + "…", segment
        location_match = _LOCATION.search(segment)
        location = None
        if location_match:
            location = location_match.group(1)
            if location_match.group(2):
                location += f":{location_match.group(2)}"

The design rule in that code: the raw text is the truth; the rows are a convenience. Unparseable segments become findings whose body is the whole segment, so nothing the reviewer said can silently vanish, and the card keeps "the reviewer's words, unparsed" behind a fold. One field-note baked into _ABS_WORKSPACE: reviewers write absolute machine paths no matter how firmly the instructions say workspace-relative. Every project workspace ends in /site/, so the parser relativizes display text past that marker and leaves the raw findings untouched. Plan for the model to miss formatting instructions; never let that cost you the content.

Three drafts of a severity rubric

Here is the section that earns this part's place in Act III, because the hard part was not catching the flaw. The first rubric caught it fine. The hard part was deciding how much it should matter, and that took three drafts against the live reviewer, each failure a real capture.

Draft one said the obvious thing: tag findings P1, P2, P3 by importance. The reviewer caught the contradiction and filed it as [P2], "should fix soon", with this hedge, verbatim: "the brand palette is inconsistent unless the client explicitly wants the blue mark." Reasonable! Maybe the client does want a navy logo on a green site. But a P2 does not block publishing, so the naive gate would have shipped the contradiction while politely noting it. A reviewer that hedges everything below the blocking line is a reviewer your gate cannot use.

Draft two overcorrected: any conflict between the brief and a client asset is [P1], no exceptions. The reviewer obliged, and filed the P1 against brief/assets/logo.svg, the client's own paperwork. Follow that to its end: the builder fixes the site by recoloring its copy of the logo, green page, green logo, done, and the paperwork still disagrees with the brief, because the builder has no business editing the client's source files. Re-inspect: still P1. Forever. The fix loop cannot close, and a gate nothing can pass is not a gate; it is a wall with a doorknob painted on.

Draft three is what ships, and its principle fits in a sentence: judge what the site ships. A contradiction the page inherits (a navy logo actually displayed on the green site) is P1; the client must decide before this goes live, and the rubric explicitly forbids downgrading it on a guess about what the client might have intended. The same contradiction surviving only inside the client's paperwork, after the page has consistently resolved it one way, drops to P2: worth flagging to the client, not worth blocking a page that is internally coherent. The A/B proof, live: the flawed site (ships the navy logo) draws [P1]; the fixed site (green copy in assets/, paperwork untouched and still navy) draws exactly one [P2] and no blockers. The loop closes.

The full wording lives in app/review.py with both failure modes documented above it. The lesson generalizes past Pagewright: with a working reviewer in hand, severity policy is product work. The model supplied judgment in every draft; deciding whose problem each finding is (the builder's, the client's, or nobody's) was never the model's call to make. If your review feature has one sentence of design effort in it, spend it on the severity rubric.

Fix findings, and the gate that goes stale

The report card ends in a button: Fix findings. There is no special protocol for repair; the reviewer wrote prose, the builder reads prose, and the workspace is the medium between them:

  // the reviewer wrote them, the builder reads them, the workspace is
  // the medium. Sending it stales the gate (session_start does), which
  // is exactly right: after a fix, the site needs fresh eyes again.
  function fixFindings(raw: string) {
    send(
      `The pre-publish inspection found these issues:\n\n${raw}\n\n` +
        "Fix every [P1] finding in the site files (and any lower-priority ones you can).",
    );
  }

One click, one ordinary turn (180,916 tokens, 21 seconds in the canonical run), and the builder recolors its copy of the logo to the house green while leaving brief/ alone; the deflated "...repainting" from the comic, on the wire. Note what it does not do: it does not touch the client's paperwork, which is exactly why the rubric needed draft three.

Now the bookkeeping question that makes the gate honest: after that fix turn, is the old inspection still valid? Of course not; it vouched for files that no longer exist. The registry tracks this with one timestamp: finish_turn stamps built_at_ms on every completed non-review turn, and the gate refuses whenever that stamp is newer than the last inspection. Deliberately conservative: a turn that changed nothing still stales the review, because "is the review older than the last turn" is cheap and honest, while "did that turn REALLY change a file" is a diff-parsing rabbit hole that buys excuses. And one turn must never stamp it: the inspection itself. The inspector looked; it did not touch. If a review turn stamped built_at_ms, every inspection would mark its own report stale, and the gate could never open. That is the whole meaning of the review_turn=True flag threaded through the consumer.

The frontend mirrors the same rule from the log, so every tab agrees on whether the button is earned:

  if (!hasSite) return null;

  const blocked = gate !== null && gate.P1 > 0;
  const armed = gate !== null && gate.P1 === 0 && !working && !publishing;

gate holds the latest inspection's counts; review_state {phase: "exited"} sets it, and any non-review session_start clears it back to null. Re-inspect after the fix: 21 seconds, zero findings, the card seals green, and the Publish button arms for the first time in this series honestly.

Checkpoint. Right now you have: an inspection that catches a planted cross-file contradiction, a severity rubric that blocks what ships and only what ships, a repair loop that provably closes, and a staleness rule with one deliberate exception. The button is armed. Before it fires, the site needs paperwork of its own.

The manifest: a form instead of an essay

When Pagewright publishes a site, the /p/ index needs a card: a title, a one-line description, a page list, an accent color. Ask the agent to "describe the site" and you get a lovely paragraph you cannot render. You want a form, not an essay, and the protocol has a first-class way to demand one: pass outputSchema (a JSON Schema) on turn/start, and the final agentMessage.text arrives as the schema-conformant JSON string. The shape Pagewright asks for:

MANIFEST_SCHEMA = {
    "type": "object",
    "additionalProperties": False,
    "required": ["title", "description", "pages", "accent"],
    "properties": {
        "title": {
            "type": "string",
            "description": "The site's name, as its hero or <title> states it.",
        },
        "description": {
            "type": "string",
            "description": "One sentence describing the site, plain text, "
                           "suitable for an index card.",
        },
        "accent": {
            "type": "string",
            "description": "The site's dominant accent color as a CSS hex "
                           "value, e.g. #2F5233.",
        },
    },
}

Everything required, no extra properties: structured output rejects looser schemas, and a form with optional fields invites the model to skip the hard ones. The turn that fills it is deliberately boring, and boring is the point:

    started = await client.request("turn/start", {
        "threadId": thread_id,
        "input": [{"type": "text", "text": text}],
        "outputSchema": schema,
        "sandboxPolicy": {"type": "readOnly"},
        "approvalPolicy": "never",
        "effort": "low",
        "summary": "auto",
    })
    turn_id = started["turn"]["id"]
    turns.begin(project_id, thread_id, turn_id)
    final_text, status, error = None, "failed", None

Read-only wristband from Part 6 because filling a form must not modify the site it describes; effort: "low" from Part 10 because this is reading, not judgment; and no event log, because the manifest is bookkeeping, not conversation (the publish flow narrates progress through publish_state events instead). run_schema_turn drains the queue to turn/completed and hands back the final text.

Which your server then validates as if the schema had never existed:

class Page(BaseModel):
    path: str
    title: str


class Manifest(BaseModel):
    """The pydantic side: the same shape, validated on OUR side of the
    wire. The engine constrained the model's grammar; this is the
    receiving dock checking the crate anyway."""
    title: str
    description: str
    pages: list[Page]
    accent: str

    @field_validator("accent")
    @classmethod
    def accent_is_hex(cls, value: str) -> str:
        if not re.fullmatch(r"#[0-9a-fA-F]{3,8}", value.strip()):
            raise ValueError(f"accent must be a CSS hex color, got {value!r}")
        return value.strip()

Two sharp edges, both cut us live. First, string values are free text inside the schema's grammar, and the model likes tucking markdown links into them ("Beanline" as a title value); a _scrub pass strips the brackets and keeps the words before validation. Second, accent reaching a stylesheet is an injection surface, which is why the validator insists on a hex color and the /p/ index re-checks it before writing any CSS. Never let stored model output write style rules unchecked.

Why so paranoid, when the engine promised schema-conformant output? Because of what happens next.

Break it on purpose: a schema is a request

This part's deliberate break is quiet, and the quiet is what makes it dangerous. Hand the manifest turn a schema no answer can satisfy and watch what the wire does. Two probes, both captured:

Probe A: a required string with "enum": []. The empty enum means no legal value exists; conformance is mathematically impossible. The turn runs for 34 seconds, investigates the workspace in good faith, and then: turn/completed, status: "completed", no error anywhere, final text {"title": "index.html", "impossible": ""}. The empty string is not in the empty enum. The turn completed successfully with JSON that violates the schema it was given.

Probe B: "minLength": 10, "maxLength": 2 on one field, a shape no string can have. Forty-seven seconds, status: "completed" again, and this time the model abandons the shape entirely: free-form JSON with different keys, no trace of the schema you sent.

Rewrite your mental model accordingly: outputSchema shapes the model's output; it does not guarantee it. On the happy path (every real manifest in this part) the conformance is excellent. Off the happy path, nothing on the wire tells you that you left it; the failure signal only exists if you check. Hence the policy, written down where the next maintainer can read the reasoning:

    manifest, failure = None, None
    for attempt in (1, 2):
        status, text, error = await run_schema_turn(
            project_id, thread_id, queue, MANIFEST_PROMPT,
            publish.MANIFEST_SCHEMA)
        if status == "completed" and text:
            try:
                manifest = publish.parse_manifest(text)
                break
            except (ValueError, ValidationError) as exc:
                failure = f"manifest did not validate: {exc}"
        else:
            failure = error or f"manifest turn ended with status {status!r}"
        await eventlog.publish(project_id, {
            "type": "publish_state",
            "phase": "manifest_retry" if attempt == 1 else "manifest_failed",
            "error": str(failure)})

Exactly one retry, then surface the failure with its reason. Zero retries turns every transient hiccup into a support ticket; unlimited retries is a silent loop that hides a real problem behind a spinner. One retry, then the truth, and the human decides. The sibling series arrived at the same policy from the other direction; this is the Codex-shaped spelling of it.

The bug this part's own tooling caught

A story from building this exact part, and the best argument for it. During a live test run, a build shipped a page whose logo tag read src="brief/assets/logo.svg". Look at that path slowly. The workspace contains brief/assets/logo.svg, so the preview served it and the page looked perfect. The LLM reviewer had no complaint in that run either; the reference resolved, after all. Then the publish copy ran, copytree(..., ignore=shutil.ignore_patterns("brief")) stripped the client's paperwork exactly as designed, and the live site, the one at the URL you text to a friend, had a broken logo. Preview fine, review clean, production broken: the exact failure taxonomy every deployment engineer eventually learns to fear, reproduced by an AI website builder at miniature scale.

Three layers had a chance to catch it (the build prompt says to copy assets; the reviewer's instructions flag brief/ references; the human looked at the preview) and all three let it through, because the break did not exist yet in the place any of them looked. The fix is not a smarter model. The fix is a check that looks at the published copy's reality, and it is nine lines of regex, not tokens:

def brief_refs(site: Path) -> list[str]:
    refs = []
    for page in site.rglob("*.html"):
        for target in _REF.findall(page.read_text(errors="replace")):
            if target.lstrip("./").startswith("brief/"):
                refs.append(f"{page.relative_to(site)} -> {target}")
    return refs

Every href and src in every page, grepped for reaches into brief/. Deterministic, instant, zero tokens, and it cannot hedge. This check joins the gate in the next section, and the reviewer's instructions now name brief/ references as broken too, so the LLM layer and the deterministic layer watch the same hole from both sides. Keep this pattern: when a class of bug is mechanically checkable, check it mechanically, and spend the model's judgment on the bugs that are not.

Smoke evals, kept to a page

The gate protects one project at a time. A different question needs answering before you change a prompt or bump a model: does the builder itself still work, across the brief bank? That is an eval suite, and this series keeps it deliberately tiny: scripts/smoke_eval.py, one page, runnable from cron or CI.

For each of the three briefs: build the site from scratch, run the deterministic checks, ask one judged question. The scaffolding is the Part 5 feature this section exists to cash in:

        started = await client.request("thread/start", {
            "cwd": str(workspace),
            "sandbox": "workspace-write",
            "approvalPolicy": "never",
            "model": MODEL,
            # The whole point: this thread is scaffolding. No rollout
            # file, nothing to clean up, nothing to resume.
            "ephemeral": True,
        })
        thread_id = started["thread"]["id"]
        rows.append(("ephemeral thread (no rollout path)",
                     started["thread"].get("path") in (None, ""),
                     str(started["thread"].get("path"))))

ephemeral: true threads write no rollout at all: thread.path comes back null (the first PASS row asserts it), nothing lands in CODEX_HOME/sessions, and a hundred eval runs leave the archive exactly as clean as zero. The checks are the kind a shell script could do, so a script does them: index.html exists; every internal href/src resolves against the file tree as the published copy will see it (references into brief/ count as broken, the bug from the last section now a permanent regression test); the manifest turn returns JSON that validates against the same pydantic model production uses. Then exactly one LLM-judged question, and note whose eyes it uses:

        # The judge gets its own ephemeral thread: the builder grading
        # its own homework is the failure mode, not the check.
        judge = await client.request("thread/start", {
            "cwd": str(workspace), "sandbox": "read-only",
            "approvalPolicy": "never", "model": MODEL, "ephemeral": True,
        })
        status, text = await run_turn(client, judge["thread"]["id"],
                                      JUDGE_PROMPT, schema=JUDGE_SCHEMA,
                                      read_only=True)
        try:
            verdict = json.loads(text or "")
            rows.append(("llm judge: page matches brief",
                         verdict.get("pass") is True,
                         verdict.get("reason", "")))

A second ephemeral thread, read-only, fresh context: the inspector pattern again, at eval scale, answering "does this page honor the brief?" through a two-field outputSchema ({pass, reason}). The same structured-output machinery from the manifest section, reused as an eval judge one section later. Here is the run from this part's final build, three briefs, eighteen checks:

Why stop here? Because the moment this grows a case format, a runner abstraction, and regression discipline, it stops being a smoke test and becomes an evaluation program, and that is a series-capstone topic of its own. The sibling series teaches the full methodology (graded cases, structured verdicts, the driving test, catching regressions on purpose) in Agent SDK Part 13; everything there transfers to this stack through the same two primitives you have watched all part: ephemeral threads and outputSchema. This page keeps the Codex-shaped echo to a page on purpose.

The gate

Everything assembles. POST /projects/{id}/publish runs the gate before anything else, and the gate is deliberately cheap: three registry reads and one grep, no protocol call, no tokens.

    reasons = publish.gate_reasons(entry)
    if reasons and not req.force:
        await eventlog.publish(project_id, {
            "type": "publish_state", "phase": "blocked", "reasons": reasons})
        raise HTTPException(status_code=409, detail={"reasons": reasons})
    if reasons:
        LOG.warning("FORCED publish of project %s past the gate: %s",
                    project_id, "; ".join(reasons))
        await eventlog.publish(project_id, {
            "type": "publish_state", "phase": "forced", "reasons": reasons})

Four reasons can appear in that 409, each captured live against the real server: the site has never been inspected (run the review first); the last inspection found [P1] blockers (fix them and re-inspect); the site changed after the last inspection (the built_at_ms staleness rule, re-run the review); and the site references the brief/ folder, which does not ship (the earned grep from two sections ago). The refusal is honest by construction: a 409 always carries its reasons as data, the UI shows them, and the same event lands on the stream so every tab learns the gate held.

Which leaves force: true, and a design position worth stating out loud. Gates are sometimes wrong: the reviewer hallucinates a blocker at 2am, the client screams for the page now, a human looks and decides to overrule. A product without an escape hatch teaches its users to route around the whole gate. So the hatch exists, and it is loud: a WARNING in the server log naming project and reasons, a publish_state {phase: "forced"} event every tab renders, "forced": true stamped permanently into the registry entry, and the /p/ index card labeled "published with --force" for anyone browsing. Products need escape hatches; escape hatches need witnesses. The forced publish was exercised live too; its registry entry carries the stamp to this day.

Past the gate: the manifest turn (with its one retry), then the same fifteen-line copy from the top of this page, now with every word of earned behind it. The published card lands in the transcript with the manifest's title, description, and accent swatch, and /p/ renders a server-side index card per site, drawn from the manifests. That index is what the manifest was for.

The meter ritual, verification edition

Real numbers from the canonical run and the suite, per-turn deltas as Part 8 taught:

One wire honesty note for the middle rows: in every capture, the inspection turns emitted no thread/tokenUsage/updated tagged with their turn id, and the thread total after the fix turn (405,490) is exactly the two build turns' sum. The reviewer's cost does not ride the thread meter in 0.142.4, so Pagewright's inspection receipts print elapsed time and skip the token column rather than print a zero it cannot vouch for.

Commit it, from the project root:

git add backend frontend scripts
git commit -m "part 11: the inspector, the manifest, and the earned publish gate"

Pagewright is a product now: it builds, it inspects with fresh eyes, it refuses with reasons, and what it publishes carries a validated manifest and a paper trail. But look at what the builder worked with this whole part: the files in the workspace, the brief the client wrote, and nothing else. No stock imagery, no house style guide, no standing rules about how a Pagewright site should be built. The builder works alone with what's in the room. Next: rented tools and the pattern book.

The complete, tested code for this part lives in part-11-review-publish in the companion repo. Code blocks with a GitHub icon link straight to the exact file; "View full file" shows the whole file in place with this section's lines highlighted.