An AI chief of staff for engineering, rebuilt from scratch after version one started making things up.
SCRUMBOT / V2CONSTRAINTS: DURABLE
REQUEST→AUTHORITY→DOMAIN→MEMORY
OUTPUT GATEPLAIN. SOURCED. CORRECTED.
NO FAKE IDSGAPS NAMEDMETAPHOR OFF
Responsible AI / Delivery systems / Internal tooling
An assistant for the work around the work.
ScrumBot is a private, document-governed AI assistant that supports technical program management: sprint planning, backlog structure, story writing, estimation and capacity planning, risk and dependency tracking, technical documentation, incident communications, and executive governance.
It is explicitly not a coding assistant. It does not write software. It handles the translation layer between strategy and delivery, the part of the job that eats a program manager's week and never shows up in a commit history.
Version one ran on the hosted Gemini instance available to me at work, assembled with help from ChatGPT. Version two runs as a Claude Project. The interesting part is not the model swap. It is what the first version taught me about where these systems actually fail.
Version one
It fell apart at the seams.
ScrumBot worked well enough that I put its output in front of executives. Then it started drifting.
Not dramatic, obvious failure. Confident, plausible, wrong. It invented space-mission metaphor to describe ordinary infrastructure work. Technical debt became "orbital decay." Routine platform maintenance picked up the phrase "quantum-secure" for no reason. It bolted poetry onto things that needed a plain sentence.
The incident that changed the system
I corrected a specific word choice. The model acknowledged the correction, then used the word again a few turns later.
The correction had expired.
A model that hallucinates occasionally is annoying. A model that cannot retain a constraint is unusable, because every output has to be re-audited from zero and the review cost exceeds whatever the drafting saved.
The governing rule
Instructions are architecture, not vibes.
The fix was not a better prompt. It was treating the assistant as a system with a documentation layer, a routing layer, a defined authority order, and durable memory, then holding it to that structure the same way I would hold a platform team to a runbook.
How the system works
Three layers. Explicit authority.
01
A stated instruction hierarchy
When instructions conflict, resolution order is explicit and written down: domain knowledge documents, then voice and tone, then core operating rules, then generic industry best practice, last and only while saying out loud that is what it is doing.
Specific always beats general
No domain document is overridden by generic best practice
Gaps get named, not improvised over
02
Domain routing instead of one giant prompt
Guidance lives in separate documents for platform and DevOps, agile delivery, IT systems, business governance, resourcing, and formatting conventions. Requests route to the right source of truth, and modes have a declared priority order so an incident never gets answered in sprint-planning voice.
One document per domain, one owner per rule
Rules blend only when the task genuinely spans domains
A missing document is reported as missing
03
Memory that outlives the context window
A voice document captures confirmed style corrections, including a banned-term list with the reason each entry landed there. A project archive holds finished initiatives, so institutional memory survives reorgs and performance review season does not depend on recall.
Corrections are recorded, not just acknowledged
Archive entries are written when work wraps
Historical accuracy stays separate from current accuracy
Engineering decisions
The constraints are the product.
Corrections persist by contract.
A correction mid-conversation becomes a standing constraint for the rest of it. If it is unclear whether an earlier correction still applies, the assumption is that it does. The incident that created this rule is documented in the instruction set as the reason.
No fabricated identifiers, ever.
Ticket IDs, usernames, and internal links are never invented. Placeholder values are flagged as placeholders. In a delivery role, a plausible fake ticket number costs more time than a blank one.
Metaphor has to earn its place.
Grandiose framing is banned outright, including for executive audiences. Sounding impressive is a stated non-goal. Plain description of the actual work beats a poetic reframe every time.
Accessibility is a quality signal.
Incident and status communications target plain language and 8th-grade readability. Architecture and roadmap recommendations get an ethical framing pass across human impact, transparency, accessibility, and durable operations. Tradeoffs are surfaced, not smoothed over for speed.
Every blocker gets three answers.
A quick fix, a process fix, and a strategic fix, with the impact scope named. It keeps the tool from optimizing for the fastest patch when the real problem is prioritization.
The access request was framed honestly.
Getting a second model approved could have been argued as the current one being bad. I framed it as cross-model validation and responsible AI practice instead, which is more accurate and more durable because the failure modes are not unique to one vendor.
The stack
Mostly architecture.
Claude Projects
Structured markdown knowledge base
7 domain documents
Declared instruction hierarchy
Mode routing
Confirmed-correction log
Project and initiative archive
Gemini v1
ChatGPT authoring
The takeaway
The tool is not the system. The structure around it is.
Most AI tooling failure I have seen is not the model being insufficiently capable. It is the absence of a system around it: no stated authority order, no durable memory, no mechanism for a correction to survive past the next response. Version one had a good model and no architecture. Version two is mostly architecture.
