Research Arena

Leading AI products are converging on 'governor' patterns that show the work: an action plan before acting, inline citations, branches, and a gallery of real examples. Slopworks already has the raw material for all four.

✓ Verified Facts

• The Shape of AI pattern library groups 'Governors' (human-in-the-loop) patterns: Action plan, Branches, Citations.
• It lists 'Example gallery' (share sample generations and prompts) and 'Prompt details' (show what is actually happening behind the scenes) as wayfinders.

? Hypothesis Inferences

• Our ideas.json (plan), research.json (citations) and chat threads (branches) map onto these patterns without new data.
• An example gallery of worked challenges answers 'what can this team do' faster than a role list.

→ Recommendation Lead the showcase with an example gallery of worked challenges, each showing plan, citations, and outcome (recommend pursue)

Uncertainty: Gallery could look staged if examples are not real traces

Cheap experiment: Mock one gallery card from the d4 contrast-check history and see if it answers 'what did they do' in one screen

confidence mediumstatus verifiedsource 1

Three platform features give app-like polish with zero scripts: exclusive accordions via details name=, :target step views (already used for filters), and cross-document view transitions opted into with one CSS at-rule. All degrade to plain pages.

✓ Verified Facts

• MDN: details elements sharing a name attribute form a group where only one is open, giving accordions without scripting.
• Chrome docs: cross-document view transitions are opt-in with one CSS at-rule (view-transition, navigation: auto) on same-origin pages; supported in Chrome 126+ and Safari 18.2, not Firefox.
• NN/g: motion should be brief and used for feedback, state change, and spatial navigation, not to entertain.

? Hypothesis Inferences

• A Research Arena can show one role's report at a time with details name=, no JS.
• View transitions are a safe progressive enhancement: Firefox users get a normal page load.

→ Recommendation One interaction kit for the showcase: exclusive accordions, :target steps, page-to-page transitions, all reduced-motion aware (recommend pursue)

Uncertainty: details name= is newer than details itself; older browsers open several at once, which is harmless

Cheap experiment: Prototype an exclusive accordion and the view-transition at-rule on two pages; check Firefox fallback and reduced motion

confidence highstatus verifiedsource 1 · source 2 · source 3

Hypothesis: clients calibrate trust on ability, reliability, and benevolence, and want limits and data sources stated up front. Our strongest trust asset is visible error-catching, not polish.

✓ Verified Facts

• Google PAIR: trust factors are ability, reliability, and benevolence; communicate capabilities and limitations early; articulate data sources.
• PAIR: indicating a prediction could be wrong may lower trust in that output but raise long-term reliance.

? Hypothesis Inferences

• Our runs already show Verify dropping 10 of 30 backfill candidates and Shield dropping 1 for personal data; that is reliability evidence clients rarely see.
• A 'what we can and can't do' block would answer the client's first question honestly.

→ Recommendation Landing block: can / can't / how we catch our own mistakes, each linking a real trace (recommend pursue)

Uncertainty: Demand claim is untested; no client or reader feedback exists

Cheap experiment: Ryno reads a draft landing and answers the three CEO questions in under 30 seconds

confidence mediumstatus hypothesissource 1

A 'follow one decision' explorer: pick a shipped change and step through pitch, research, challenge, score, PR sign-offs, and Trial's check, every step a real link. Dashboards show status; this shows reasoning.

✓ Verified Facts

• Explorable Explanations describes itself as a hub for learning through play, built by people reuniting play and learning.
• PAIR lists 'explanation via interaction' (let users ask 'what if?') as a way to build mental models and trust.

? Hypothesis Inferences

• Most agent dashboards show live status; few let a visitor replay why a decision was made.
• :target steps make a stepper possible with no JS.

→ Recommendation Case files: 2 real decisions (d4 contrast check, d1 filters vs the input ban) as step-through stories (recommend pursue)

Uncertainty: Thin cases feel padded; only use decisions with real friction

Cheap experiment: Write one case file as plain Markdown first and count how many steps have a real link

confidence mediumstatus hypothesissource 1 · source 2

Cut: a pulsing 'live' activity view (we run once a day), simulated agent playback, numeric confidence percentages, and an Impact Ledger with no measurements. Challenge: demo challenges invented by the team prove less than replaying real work.

✓ Verified Facts

• NN/g: motion attracts attention whether meaningful or not; gratuitous animation distracts, and attention hijacking is a dark pattern.
• PAIR: numeric confidence presumes users understand probability and can be misleading; categorical buckets are an alternative.

? Hypothesis Inferences

• A 'live' feed on a once-a-day routine would be misleading on 23 hours of every day.
• Real case files (contrast catch, input-ban workaround) beat invented demos as proof; demos should come second.

→ Recommendation Ship real case files before the first demo challenge; drop live-activity and ledger ideas until data exists (recommend investigate)

Uncertainty: Cutting too much could leave the landing flat; keep one strong visual

Cheap experiment: Score each cut against 'what can it prove it accomplished?'; anything with no real artifact stays cut

confidence highstatus verifiedsource 1 · source 2

Living Team, Research Arena, case files, and demo challenges all fit: build.py renders them from JSON. Live activity, visitor challenges, and analytics need servers or input and stay out. Pages terms on commercial sites need Ryno's read.

✓ Verified Facts

• GitHub Pages limits: published sites up to 1 GB, soft 100 GB/month bandwidth, soft 10 builds/hour.
• GitHub Pages is not intended to run an online business or a site primarily directed at commercial transactions or SaaS.

? Hypothesis Inferences

• A showcase that describes the team's work is not a transaction site, but pitching for clients moves closer to that line.
• One new data file (cases/challenges) plus schema is the only data-model change needed.

→ Recommendation Add data/cases.json (real case files and labeled demos) with a schema linking research ids, chat ids, and PR numbers (recommend pursue)

Uncertainty: Commercial framing vs Pages terms; page weight as pages grow

Cheap experiment: Draft the cases schema and validate one case file in CI

confidence highstatus verifiedsource 1

The Research Arena: this research round already exists as structured data (14 reports, facts vs inferences, Critic's challenge, Coordinator's scores), so a page can render it next PR with no new data model.

✓ Verified Facts

• MDN: details/summary is baseline widely available since January 2020 and needs no script.
• data/research.json already separates facts, inferences, evidence, confidence, and status per report.

? Hypothesis Inferences

• The Arena is the only concept where the content is done and only rendering remains.
• Shipping it first gives later pages (case files, challenges) a reusable card and label system.

→ Recommendation An /arena/ page for this round: question, each role's answer, the challenge, the scores, the decision (recommend pursue)

Uncertainty: Page weight with 14 reports; keep reports collapsed by default

Cheap experiment: Render the Arena from today's data locally and check weight and keyboard use

confidence highstatus verifiedsource 1

Use a fixed, small vocabulary like intelligence analysts do: separate certain knowledge from reasoned judgment, use a few standard likelihood words, and pair them with a confidence level. Our data already has the fields.

✓ Verified Facts

• Sherman Kent's goal (1964) was to make clear what is certain knowledge and what is reasoned judgment, and what degree of certitude lies behind each.
• The US National Intelligence Council pairs a small set of likelihood words with confidence levels (high, moderate, low).

? Hypothesis Inferences

• Three visual tiers fit our data: Verified fact (facts + source), Hypothesis (inferences), Recommendation (decision with scores).
• Words, not percentages: we have no calibrated probabilities.

→ Recommendation A shared label kit: Verified / Hypothesis / Recommendation / Demo, each with icon, text, and color, never color alone (recommend pursue)

Uncertainty: Too many badges become noise; cap at one tier label plus confidence per claim

Cheap experiment: Relabel 3 existing research cards and check each claim lands in exactly one tier

confidence mediumstatus verifiedsource 1

The risk moves from visitors to our own traces: raw chat transcripts, routine prompts, box paths, gh auth, corrections mail, and Ryno's private messages. Render only from already-public, CI-scanned repo data, never raw logs.

✓ Verified Facts

• OWASP LLM02:2025 lists PII leakage, sensitive business data, and exposing system preambles or configuration as disclosure risks.
• OWASP recommends least privilege, restricting data sources, and pattern-based redaction before output.

? Hypothesis Inferences

• 'Real execution traces' could tempt us to paste raw run transcripts; those can hold paths, prompts, or tokens.
• Our existing CI scans cover data/ and site/, so traces must enter as data/ fields to be scanned.

→ Recommendation Traces = links to merged PRs, chat ids, and research ids only; no raw transcripts, prompts, or usage screenshots (recommend pursue)

Uncertainty: A demo challenge quoting a client-like scenario could invent personal details; use fictional, generic orgs

Cheap experiment: Run check.py scans over a draft case file built from PR comments and chat

confidence highstatus verifiedsource 1

Top misleading states: 'live' or 'working now' when the last run was hours ago, demo content read as real work, missing data rendered as zero, stale research shown as current, and broken trace links.

✓ Verified Facts

• NN/g heuristic 1: keep users informed about what is going on through appropriate feedback within reasonable time.
• NN/g: when time changes system state, explain it briefly rather than letting items silently change or vanish.

? Hypothesis Inferences

• Every time-based claim needs an 'as of' timestamp from runs.json.
• research.json fresh_until gives a free stale check.

→ Recommendation Test fixtures: stale runs.json, an expired fresh_until, a demo item, a missing count; each must render honestly (recommend pursue)

Uncertainty: Fixtures can drift from real data; build them from copies of real files

Cheap experiment: Render the home page with a runs.json whose last run is 3 days old and check the wording

confidence highstatus verifiedsource 1

We can't measure impressed visitors without tracking, so measure proof quality instead: share of showcase claims with a linked artifact, Trial-verified acceptance rate, research-to-ship rate, and caught-before-publish counts.

✓ Verified Facts

• GitHub repo traffic (views, referrers, popular content) is visible only to people with push access and covers the past 14 days.
• That traffic graph covers the repository, not the Pages site.

? Hypothesis Inferences

• Repo traffic is the only existing visitor signal and it is owner-only and partial; it should not drive design.
• Existing data already gives: all 10 shipped team ideas (i10-i15, i17-i20) have Trial acceptance checks; Verify dropped 10 of 30 backfill candidates.

→ Recommendation A proof-coverage check in build: count claims with and without artifact links per page (recommend pursue)

Uncertainty: Proxy metrics can be gamed by adding links; Lens spot-checks that links support the claim

Cheap experiment: Count linked vs unlinked claims on the current home page by hand

confidence mediumstatus verifiedsource 1

Hosting is not the constraint: static pages cost nothing extra on Pages and public-repo CI. Slopbot usage is: each demo challenge is a research session. Cap it at one demo challenge per sprint, 4-5 roles, about 15 reads.

✓ Verified Facts

• GitHub Pages: 1 GB site, soft 100 GB/month bandwidth, soft 10 builds/hour.
• GitHub Actions on standard hosted runners: 6-hour job limit and 20 concurrent jobs on Free; our CI finishes in under 2 minutes.

? Hypothesis Inferences

• This round used 15 page reads for 14 roles, our first real per-round cost figure.
• Pre-rendered demos have no runtime failure modes beyond broken links, which the link check covers.

→ Recommendation Budget line for demo challenges on the schedule page; Sweep logs reads per challenge (recommend pursue)

Uncertainty: Usage is Ryno's plan allowance; a big challenge could crowd out the daily run

Cheap experiment: Log reads and session time for the first demo challenge and compare with this round

confidence highstatus verifiedsource 1 · source 2

Public lesson: a company is liable for what its AI says, static page or chatbot alike. Ours: guardrails held when filters hit the input ban, a duplicate PR (#21/#22), X rate limits. We have never reverted a deploy, so don't stage one.

✓ Verified Facts

• BBC on the Air Canada ruling: the tribunal said it makes no difference whether information comes from a static page or a chatbot; the airline paid damages.
• Team history: radio filters hit check.py's input ban and were rebuilt with anchors (c62); #21 was superseded by #22; X search was rate limited on 2026-10-16.

? Hypothesis Inferences

• Demo challenge outcomes must read as worked examples, not advice or quotes.
• Showing our real, small failures is more credible than a staged dramatic one.

→ Recommendation A 'what went wrong and how we caught it' strip using only real incidents, plus a demo disclaimer on every challenge (recommend pursue)

Uncertainty: A short failure list may read as 'nothing ever goes wrong'; say plainly that the history is young

Cheap experiment: List every real failure in git and runs.json and check each has a link

confidence highstatus verifiedsource 1

Consolidated 13 reports into 5 concepts. Merged: Forge's Arena + Verify's labels; Scoutmaster's explorer + Critic's real-cases challenge (case files); Trial + Advisor on honest status. Prior work: home intro (#23), facts/inferences split (#20).

✓ Verified Facts

• ADR practice: record each decision with its rationale, trade-offs, and consequences, kept together as a decision log.
• Duplicate pairs found: Arena/labels, explorer/case files, status honesty (Trial/Advisor), cost cap (Sweep/Atlas).

? Hypothesis Inferences

• No finding conflicts with a prior decision; Critic's 'cases before demos' reorders but doesn't reject the CEO's demo challenges.
• Behind the scenes already serves as the deep technical ops layer; reuse it, don't rebuild.

confidence highstatus verifiedsource 1

Research Arena first (priority 40), then real case files (12.5), then the outcome-led landing and Living Team (10), then one demo challenge after a cost experiment (3.3). Impact Ledger and Build Observatory wait for real data.

✓ Verified Facts

• Intercom's RICE adds a confidence factor to curb enthusiasm for exciting but ill-defined ideas.
• RICE notes dependencies can justify working out of score order, as long as the trade-off is explicit.

? Hypothesis Inferences

• Our Evidence score plays RICE's confidence role; demo challenges score low on it until one is run.
• The landing depends on Arena and case files existing, so it ships third despite high impact.

→ Recommendation Sequenced 8-PR showcase sprint (i22) (recommend pursue)

Uncertainty: Sprint exceeds the 1 feature + 1 design PR/week cap without a CEO override

Cheap experiment: Run demo challenge 1 inside the research cap before building its page

confidence mediumstatus verifiedsource 1

Top agent pitfalls: sign-offs picked by hand (4 PRs, 6 failed runs), values typed by hand (8 wrong values), repeats (4), unchecked properties (3 AARs), and wrong branch type (2 PRs). A, B, and E need one helper; D needs schema-checked loaders.

✓ Verified Facts

• 31 of 247 CI runs failed, all on PRs: review-gate 6, path rule 4, guardrail-gate 6, repo check 1, unit tests 1, Dependabot guarded paths 13.
• The Lessons page lists 8 AARs. Each traces to a step a bot did by hand or that nothing checked at the time.

? Hypothesis Inferences

• Fixing the tool the bots already use removes the shortest wrong path.
• CI stays the backstop; the helper makes the right path the easy one.

→ Recommendation scripts/bot.py (branch, pr, chat, count) plus strict loaders in build.py; then cut the prose those enforce (recommend pursue)

Uncertainty: A helper the bots can skip; mitigated because CI still enforces every gate

Cheap experiment: Tests run the helper's branch choice against CI's path rule, and the strict loaders against today's build

confidence highstatus verifiedsource 1 · source 2

Weakest assumption: that bots will use the helper. Rules that only the helper enforces go back to being prose if it is skipped. Proceed, but cut prose only where CI enforces the rule too, or where the helper is the only documented path.

✓ Verified Facts

• Two of the four review-gate PRs (#5 and #8) came before the draft flow in #9, so a process fix already cut that failure once.
• The closed-PR list shows #21 redone as #22 after the path rule failed.

? Hypothesis Inferences

• Counts from 3 days of history are small, so the plan should be judged again in 2 weeks.
• Repeats (C) stay a judgment call; a helper can't decide what counts as the same feature.

→ Recommendation Keep CI as the source of truth and the helper as the default path (recommend investigate)

Uncertainty: Over-cutting prose that is still judgment

Cheap experiment: Inventory each cut against an enforcing check or a test

confidence highstatus verifiedsource 1 · source 2

Ship in this order: the helper (priority 40), strict loaders and drift tests (24), then the prose trim (25, after the first two). Corrections-diff and entry-cap checks are deferred: no incidents yet.

✓ Verified Facts

• Scores use the standard formula (I×E×F)/(Effort×Risk).
• The three deferred candidates have 0 incidents in the audit.

? Hypothesis Inferences

• Four PRs fit the cap Ryno set.
• Trimming last keeps every cut tied to a merged enforcer.

→ Recommendation Four PRs: audit, helper, loaders and drift tests, trim (recommend pursue)

Uncertainty: Scope creep; capped at about 4 PRs

Cheap experiment: Prototype questions are answered in the PRs' tests

confidence highstatus verifiedsource 1 · source 2