Battle-Tested

During algorithm development, a tracking approach failed completely on real figure skating videos — detecting 6.47s of air time on a jump that lasted 0.7s. The same approach was retried with small tweaks.

Blindspot Interception — after 2 identical failures, force a root cause analysis. After 30 minutes on a dead-end approach, force a strategy switch.

A data pipeline was accidentally executed in the wrong project directory. Cross-project file contamination was a real risk.

Environment Pollution Guard — any cross-project reference triggers an immediate stop-and-confirm before proceeding.

A new rule for automatic case study collection was added to the project-level memory — visible only in the current project. But case studies are mainly generated during development of other products.

No existing rule covered this — it was a blindspot in the SOP creation process itself.

A new protocol was written to project-level memory instead of the global config. The rule explicitly stating 'check if cross-project rules are globally visible' was already in place — but the AI skipped the self-check.

Blindspot Interception — writing new rules must include a scope self-check.

The Challenge Protocol — requiring the AI to question its own output — was just added to the global config. Minutes later, the AI proactively flagged a limitation in a hook it had just built: the keyword detection only covered Chinese terms, missing English edge cases.

Challenge Protocol — during execution, if you notice a potential issue, raise it instead of silently continuing.

During a session wrap-up, the AI reported 'next step: rewrite algorithm v2' — but v3 had already been completed in a previous session. The stale recommendation came from a data copy that wasn't updated.

No rule covered this — the data copy itself was the architectural flaw.

JumpOnion had reached Phase 4 with 166 tests and 9/11 calibration videos passing. But it had never created a unified rules document for AI agent collaboration — critical lessons were scattered across memory files.

New Project Coverage Detection — on session start, check if the project has a unified agent rules document.

During golden file generation, 3 out of 10 calibration videos downloaded 0 frames from the database. The other 7 worked fine. The temptation was to debug network or permissions.

Data Provenance — after cleaning data, all upstream references must be synced. Blindspot Interception — stop chasing symptoms after 30 minutes.

During E2E validation, a metric showed real data at ~1.0 but the threshold was set at max 0.15. The quick fix: just raise the threshold to 1.0 and everything goes green.

Zero Misdiagnosis Principle — fix the definition first, then adjust the numbers. Verification exists to be meaningful, not to be green.

During calibration with 11 real figure skating videos, the system diagnosed a world champion's textbook triple Axel as 'high under-rotation risk.' The confidence was 0.61 — just barely above the 0.60 threshold.

Zero Misdiagnosis Principle — better to say nothing than to say something wrong. Confidence gating — suppress diagnosis below reliability thresholds.

The diagnosis pipeline was connected to the production route. 732 tests passed, 0 failures. Everything looked ready to ship. But my AI partner remembered something.

Challenge Protocol + Verification Before Completion — eval scripts passing does not equal production verification. The AI cited a previous incident where tests passed but production broke.

Three cross-tool session handoffs occurred in a single day: Claude Code to Cursor, Cursor back to Claude Code, then Claude Code to Cursor again. Each switch risked losing context about what was tested, what was blocked, and what came next.

Cross-Tool Handoff Protocol — every session exit must produce a handoff note. Every session start must read the previous handoff note and cross-validate against the source of truth.

While fixing an export feature, a database query used SELECT * but the manual field mapping missed a column. After fixing it, another missing column was found — the exact same bug pattern.

Bug Confession Protocol — when the AI fixes its own bug, it must self-report the pattern, not just the fix.

IvyBloom added a Terms of Service consent checkbox to all Stripe Checkout flows. Later that day, a JumpOnion session started. Ciki asked: 'Should the payment link add this too?'

Startup Protocol — read the HOME.md hub dashboard first. The IvyBloom row said 'Added ToS consent checkbox to all Stripe Checkout flows' — specific enough to act on immediately.

Across 3 projects and 6 months, AI agents appeared to develop autonomous judgment: catching bugs before the developer, refusing to ship when tests passed, switching strategies when stuck, knowing when to wrap up sessions.

Three-Layer Architecture — Hooks (mechanical guarantee, ~100%), Skills (phase-triggered, ~95%), Rules (AI-executed, ~85%). Every 'autonomous' decision traced back to an explicit SOP rule.

SmartLearning was rebranding to IvyBloom. Not a simple rename — it touched 7 system layers: GitHub repo, git remote, Vercel project, Claude Code/Codex/Cursor session configs, project memory, and CikiBrain hub documents.

Cascade Cleanup Protocol — any rename operation triggers automatic Grep across all systems + fix all references. No waiting for the human to point out missed spots. Combined with Cross-Platform Handoff Protocol for multi-tool sync.

During V1 launch sprint, tool-switching frequency jumped from 1-2/day to 10+/day. The cross-tool handoff protocol — designed for low-frequency switches — collapsed. 10 handoff notes per day became noise instead of signal. AGENTS.md fell 11 days behind. Verification debt became invisible across tools.

Self-Healing Framework — the SOP's built-in audit tool (autoresearch) detected three simultaneous failure signals, traced them to a single root cause (missing aggregation mechanism), and generated a structured fix.

In a 10+ hour marketing strategy session, the AI was loaded with all project context at startup. Hours later, it asked 'Do you have a landing page?' — about a site already in production. Then suggested building a 'free trial' — for a product already live with paid subscribers. Then warned about '$500+ API cost risk' — when the actual 10-day bill was $1.73.

Stale Context Prevention — session-start fact loading must be reinforced mid-session. Default AI training narratives ('solo founders need free trials', 'API costs spiral') override loaded facts after enough conversation turns.

No error log. No stack trace. Just a feeling: 'Something is off — sessions are disconnected, wrap-ups are getting skipped, project truth is scattered everywhere.' The investigation had to start from a vague sense of system degradation.

Root-Cause-First + Autoresearch — decompose a vague feeling into 5 falsifiable hypotheses, then test each against physical evidence (file timestamps, directory counts, config diffs).

A beginner skater's double Axel was diagnosed as a 'fall' with severity 4. In reality, the deep knee bend after landing was normal absorption biomechanics for a young skater — not a fall. The AI had never seen a beginner's landing before.

Zero Misdiagnosis Principle + Root-Cause-First — don't patch the symptom (adjust one threshold). Run ablation experiments to find the real cause. Prompt-level fixes require full regression validation before deployment.

A simple question during a session: 'How does the project dashboard get updated when a session ends?' The answer: it doesn't. The wrap-up protocol had 3 steps, none of which included syncing the dashboard that the human actually checks every day.

Challenge Protocol — the question wasn't a complaint, it was a system design challenge. Combined with Minimal Fix First — don't build a new tool, add one step to the existing protocol.

A popular open-source AI memory tool exploded to 53K GitHub stars, promising '90% token savings.' The temptation: install it immediately. Instead, a structured 10-dimension competitive analysis was triggered — lifecycle hooks, storage, retrieval, compression, token efficiency, search, security, dependencies, information decay, and commercial viability.

Search Before You Build + Buy > Build (with ROI) — evaluate before adopting. Don't let star counts substitute for architectural analysis. 53K stars doesn't mean 53K people got the right solution — it means 53K people had the same problem.

While shipping a new feature, the AI added three columns to a database table and updated the read code. A paying user immediately reported: feature still doesn't work. Root cause: a hardcoded SELECT column list silently dropped the new columns. The function returned the user's profile minus the very fields the new feature needed.

Bug Confession + Memory Enforcement Gap — the same anti-pattern (manual column lists silently dropping new fields) had already been graduated to feedback memory weeks earlier. The memory existed. The AI loaded it at session start. It still didn't trigger when the matching code change happened.

During a security audit, the AI deleted a route mount in the backend and labeled it confidently in the commit message: 'dead code, anonymous uploads.' It was not dead code. It was the active upload pipeline. 987 unit tests passed because no test asserted the route's existence. The deploy succeeded. Thirty hours later, a paying customer reported uploads silently failing.

Verification Before Completion + Caller Audit — any deletion commit that uses confident labels ('dead code', 'legacy', 'unused') must include grep-evidence of caller checks. The AI's high-confidence language is a trust signal in human review — and a hidden attack surface when unverified.

Three hours into a session, the AI proposed a design question based on a wrong premise: 'You don't sell the L3 plan yet, so admin = full access?' L3 had been live for weeks. Pricing, Stripe price ID, and quota were all already configured in code — one grep away.

Founder Override + Fact-Echo Gate — when the AI hallucinates project state mid-session, the human founder's correction cost is a sentence. The AI's verification cost is a single grep. The asymmetry makes 'interrupt and correct' the highest-leverage defense layer.

An open-source AI agent framework went viral with 89K GitHub stars, promising 'self-evolution' through a genetic algorithm that mutates skills and prompts automatically. The temptation: install it. The alternative: a structured 5-dimension comparison against the existing system across memory, skills, self-evolution, behavior enforcement, and cross-tool collaboration.

Search Before You Build + Buy > Build (with ROI) — evaluate before adopting. Rather than 'use or ignore,' the question becomes: which specific ideas are worth zero-cost porting?

A new role-based feature shipped with 8 coach-only write endpoints. Internal testing used 'admin ghost login' — viewing the app as another user via an admin shortcut. All 8 endpoints had a typo in their hardcoded SELECT column list referencing fields that didn't exist in the schema. Every real coach session would 500. Internal testing never went through the coach code path.

Role-Based Testing Discipline — admin ghost login carries the user's ID through user-facing endpoints, not through role-restricted endpoints. It can't substitute for a real account in the new role. Role-restricted features need an independent account smoke test before launch.

After a month of iterating the jump detection algorithm from v1 to v3 — tightening thresholds, adding physics constraints, hitting calibration accuracy — a real parent uploaded a video and got a 1.2-second air-time reading on a jump that physically maxes out at 0.85 seconds. The algorithm's calibration set was solid. The fix wasn't in the algorithm at all.

UI as a Contract — the literal text on a button is a promise to the user about what input the system expects. The button said 'Set Takeoff' and 'Set Landing.' The algorithm's hidden contract required the clip to include on-ice frames before takeoff and after landing. When buttons say one thing and algorithms expect another, the user follows the buttons — and the algorithm receives the wrong input.

A paying customer reported that every drill click in their training plan returned 402 Payment Required. The drill access function read a JSON cache key at the root level — but the actual persisted shape, since a recent bilingual migration, nested the data two levels deeper under jump-type and locale keys. The 23 unit tests for this function used the old flat shape that production had stopped writing weeks ago.

Root-Cause-First on Pipelines + Evidence-Backed Verification — three SQL probes against the actual production data confirmed the schema mismatch in 5 minutes, before a single line of code was touched. Tests that mock a different shape than production writes are theater, not verification.

A coach tagged a jump with three labels matching the LLM's diagnosis exactly. The LLM had produced a detailed coach summary with frame-level evidence ('blade contacts ice approximately 90° short at frame 16'), biomechanical reasoning, and three concrete athlete cues. The coach-tag overlay layer detected serious-negative tags and triggered a template rewrite — replacing the LLM's rich text with a generic boilerplate translation of the tag list.

Conflict Detection Before Rewrite — a safety overlay designed to prevent LLM hallucination should not fire when the LLM and the coach are aligned. The overlay's job is to handle conflict, not to flatten everything into a template by default.

After locating the real production blocker — a model file gitignored in deployment, leaving rotation metrics dark for weeks — the AI proceeded to invalidate stale cached results. The mental model was 'reset cache so it can repopulate.' The SQL nullified three independent caches in one statement: the analytics cache (correctly stale), the diagnosis cache (independent of the bug), and the training plan cache (also independent). 20 rows updated. 7 paying customers' personalized diagnosis text and training plans wiped — not recoverable from the call log because LLM output text wasn't persisted there.

Cache Invalidation Minimum-Blast + Destructive SQL Preview — never NULL a derived cache unless that specific cache's upstream input changed. Any UPDATE/DELETE touching multiple production customer rows must first run the equivalent SELECT, surface affected user emails, and get explicit human approval before executing.

Across one afternoon, eight AI sessions ran through a secondary tool, each correctly updating the project's rolling state file. None of them touched the central dashboard — by design. The primary tool's contract was: read the dashboard, sync to it, push to the project tracker. Result: dashboard 5 hours stale. Project tracker 2 days stale. 26 commits and two architecture milestones invisible to the human partner watching the dashboard.

Cross-Tool Handoff Signal Gap — node-level compliance does not guarantee system-level compliance when no signal channel exists between nodes. Both tools were correct in isolation. The handoff between them was the failure mode.

Session opened with the founder's worry: 'I just did Project A's work in Project B by accident, in a different tool. Please scan to see if my SOP protected me.' Subjective alarm. The default response would be either reassurance or a panicked cleanup script — both wrong without evidence.

Four-Line Evidence Cross-Check — user worry triggers a structured scan, not action. (1) Git logs and working trees in both repos; (2) cross-project keyword grep; (3) cross-tool session metadata; (4) session archive frontmatter. Conclusions only after four lines agree.

A major rewrite shipped. 2,082 tests passing. The founder hit 'regenerate' on a real task and the result came back in 3 seconds — way too fast for the new LLM path that should take 50–60 seconds. Default reaction would be: 'maybe the cache hit, try again.' Or: 'maybe the new code is faster than expected, ship it.'

Smoke Tests Trust DB Audit Fields, Not UI Feel — when rewritten code's production behavior doesn't match expectations, the first check is the audit field that records which code path actually ran, not the output content. UI 'feel' (fast, slow, looks right) is ambiguous; an audit row tagged with model name and prompt revision is unambiguous.

First production smoke of a new LLM training plan endpoint. The browser request hung for a minute, then returned 504 Gateway Timeout. A manual page refresh: the new plan was already there, generated and saved. From the backend's view, everything was fine. From the user's view, it failed.

Platform Timeout < App Timeout — every deploy platform has gateway timeouts shorter than what the app's own timeout config admits. App-level retries don't help when the gateway killed the client connection before the server finished responding. The audit log table caught what nothing else would have: the call succeeded in 59.3 seconds, against a hidden 60-second platform ceiling.

A session opened with the working directory set to a product project, but every file it produced lived in the global config tree — a new skill, a new hook, an updated settings file. Zero changes to the product's source. The work was real, useful, and shipped. But the wrap-up would have logged it under the wrong project. Three different hooks watched this session. None flagged the mismatch.

Ownership Follows Output Path, Not Working Directory — when a session's outputs live in shared territory (skills, hooks, global config), the governance owner is the hub project, regardless of where the editor was pointed. The dashboard, milestone log, and project tracker all index by ownership, not by working directory.

During a cognee security spike, a grep over the Obsidian vault hit a 3-month-old IvyBloom session archive. Inside was a full .env dump — pasted by a collaborator during a debug session. The keys were not just sitting in the file. The vault syncs to Google Drive (with version history). The Google Drive feeds NotebookLM (cloud-indexed). The 'private debug log' had silently become a three-layer public attack surface, and redaction was already 3 months late.

Session Archive Is Attack Surface — anything pasted into an AI conversation should be treated as if published. The cure is not 'remember to redact' (graduated memory ≠ enforced rule). The cure is hooks that scan every PostToolUse, a monthly SessionStart sweep across the entire vault, and a pre-upload gate that blocks any bundle from reaching NotebookLM until it's clean.