GrantTap field guide · · 15 min read
Reconstruct an agent-assisted incident from evidence
A practical method for recovery, timelines, source and artifact identity, minimal reproductions and useful follow-up work.
When an agent-assisted change goes wrong, the first useful question is what actually happened. The conversation may contain a confident explanation, a command may have exited successfully and a deployment system may have accepted an artifact. None of those facts alone establishes which version a user encountered or why a request failed. A useful incident record reconstructs the relevant sequence from identifiable observations, keeps gaps visible and separates immediate recovery from the later explanation of the cause.
This guide follows a hypothetical incident in which a newly deployed web application rejects an order after a validation change. It is a practical workshop, not a report of a real GrantTap outage or a claim that the product automatically assembles a complete forensic record. GrantTap can help inspect the Task, available execution history, files and device diagnostics. The investigation still needs the actual application, source and serving evidence. The method works best when the agent assists with bounded collection and checks while a human retains a clear view of scope and recovery.
Stabilize the situation before expanding the investigation
If users are currently affected, decide what immediate recovery is authorized and appropriate. That might be disabling a newly introduced feature, restoring a previously verified artifact or stopping a harmful background job. The exact action depends on the application and its data effects. Do not make a broad source rewrite the first response merely because an agent can produce one quickly. Preserve enough information to identify the affected version, then prioritize a controlled recovery whose outcome can be checked.
Keep recovery and diagnosis as separate work items inside the incident. A rollback can restore service without proving the original cause. Conversely, an explanation can be correct while the serving system remains broken. In the workshop, assume a previous verified artifact is available and no irreversible migration is involved. That assumption makes the exercise tractable; it should not be silently applied to incidents involving changed data or external side effects. Record the actual recovery decision, its target and the observation that confirms the service has returned to the intended state.
Give the incident a stable identity
Use a stable incident reference in the Task and the evidence packet. Record the affected application, environment, repository and known source or artifact identifiers. A title such as checkout broken helps a human recognize the issue, but it cannot distinguish today's production incident from last week's staging test. The identity should remain consistent as the work moves between computers or providers. Avoid building the investigation around whichever chat happens to be open when someone first notices the symptom.
In GrantTap, check the Task and current execution before collecting activity. A Task can span multiple executions, each associated with a provider, native conversation and computer route. The Task and execution guide explains that distinction. If a handoff happened during the relevant period, record both sides instead of assuming all commands belonged to the current destination. A clear incident identity lets another developer connect related evidence without treating similar command names or nearby timestamps as proof that records describe the same operation.
Capture the symptom without embedding a theory
Write what the user observed in plain terms. For the workshop, an order containing a particular optional field receives a rejection after the new deployment. Record the expected behavior, actual response, affected environment and a safe example input. Do not begin with the validation library corrupted the request unless that cause has already been demonstrated. The symptom should remain valid even if the first explanation turns out to be wrong. That makes it a reliable anchor for the rest of the investigation.
Separate scope from frequency. One reproduced failure establishes that the behavior can occur under those conditions; it does not establish how many users encountered it. A support report can describe a user's experience without proving every technical detail of the underlying request. If the available logs cover only a sample, preserve that limitation. The agent can help organize these observations, but it should not invent an incident count, a duration or a user-impact percentage to make the report feel complete. Unknown impact is an actionable gap, not a number waiting to be guessed.
Build a timeline from sources with identities
Start with a small set of important events: the change was prepared, a particular check ran, an artifact was built, a deployment was accepted and a user-visible failure was observed. For each event, name the source and the identifier that ties it to the incident. A build ID, request ID or artifact digest can be more useful than the nearest chat timestamp. The timeline is a working structure for evidence, not an invitation to copy every message into a document.
Keep reported time and observed time distinct where they differ. A phone may receive an event later than the computer recorded it. Display time zones explicitly when combining sources, and avoid claiming a precise causal order from two clocks you have not reconciled. If the sequence is uncertain, say that the events occurred within an interval or that their order remains unconfirmed. An honest partial timeline helps choose the next check; an invented precise timeline can send the agent toward a cause that only appears plausible because the record was overconfident.
Read the transcript as a source of claims
The conversation is valuable because it explains intentions, decisions and the agent's reported understanding. It can show why the change was attempted and which checks the agent says it ran. It is not a substitute for the underlying results. A message saying all tests pass needs the relevant command, environment and outcome before it can support a specific conclusion. A message saying deployed needs evidence from the deployment and serving system. Preserve the claim while checking the part that matters to the incident.
Ask the agent to cite the available source of each important statement. If a result is missing or truncated, record the gap instead of reconstructing a successful run from a later summary. GrantTap's sources-of-activity guide explains that provider coverage and imported history can differ. An empty Agents view does not establish that no delegation occurred, and a missing old message does not establish that an operation never happened. The transcript helps direct the investigation, while stronger conclusions depend on evidence that can be connected to the actual action.
Inspect runtime history within its recorded scope
Where the relevant Engine history is available, inspect the recorded invocation and its reported outcome. Check Task, execution and computer identity. A refusal can explain why a step did not proceed; a gap can explain why the record cannot support a stronger claim. The audit and evidence guide describes these distinctions. The public runtime does not provide a universal independently verified chain of filesystem changes and downstream effects for every possible tool call. Do not claim that the incident has such a chain unless the actual evidence supplies it.
Use a reported successful command carefully. It can establish that the tool source reported a successful exit under stated conditions. It does not prove that the command tested the affected input or that its output was the artifact later deployed. Likewise, a content-free journal may preserve useful invocation identity without containing raw arguments or full output. If more detail is needed, collect the narrow additional artifact from the appropriate source. Avoid turning a limited record into a demand to copy an entire private environment into the incident packet.
Establish what version users actually reached
A Git commit identifies source; a build identifies an artifact; a deployment identifies an activation attempt; a serving check observes what a destination returns. Connect those stages rather than treating them as equivalent. In the workshop, confirm which artifact the affected environment was serving at the time of the reproduced failure. If a load balancer or multiple instances are involved, a single response may not describe every instance. State the sampled scope and inspect additional routes when the architecture requires it.
Do not rely solely on a version label embedded in a page if that label can be stale or copied. Use the release system's available artifact identity and a meaningful response check together. If the environment has already rolled back, preserve the previous deployment record and reproduce against the candidate in an isolated environment where possible. A healthy current response demonstrates recovery under the checked conditions; it does not erase the earlier incident or prove that the original candidate never failed. Keep both observations in the timeline with their actual context.
Preserve a useful snapshot before cleaning up
Save the relevant source revision, configuration names, dependency lockfile and narrowly scoped logs before clearing caches or deleting build output. Cleanup can destroy the conditions needed to explain a failure. At the same time, preserving evidence does not require an unrestricted copy of every directory. Define the packet around the incident question and include an inventory of what it contains. Avoid retaining credentials or unrelated customer data simply because collecting everything seems easier than selecting relevant evidence.
For a local reproduction, prepare a dedicated checkout with safe fixtures. If the original workspace contains uncommitted changes, identify them before comparing it with a committed revision. A clean Git status can be useful evidence about tracked files, but it does not describe every external setting or service. The agent should report environment dependencies explicitly. This prevents an investigation from blaming a source change when the meaningful difference was a runtime setting, private registry response or database condition outside the commit being examined.
Create the smallest reproduction that preserves the failure
Reduce the failing input while keeping the relevant behavior. For the order example, remove fields unrelated to validation until the failure depends on a small, understandable case. Keep the expected and actual outcomes beside the fixture. A minimal reproduction is useful only if it still demonstrates the original symptom. If simplifying the input makes the failure disappear, retain the last failing version and investigate the removed condition rather than declaring that the application is now fixed.
Run the reproduction through the boundary that failed in practice. A direct library call can help isolate behavior, but it may omit application mapping, middleware or persistence. Record which layer each test exercises. The agent can propose reductions and run controlled checks, while the reviewer verifies that the final fixture still represents the user-visible issue. The result should be small enough for another developer to understand and safe enough to share through an approved channel without exposing the original customer's payload or the full production environment.
Test explanations one at a time
List a few plausible causes and assign a discriminating check to each. The validation dependency may have changed its default, the application's error mapper may expect an old shape, or the deployed artifact may differ from the one tested. A useful experiment produces different expected observations for competing explanations. Running the same broad test suite repeatedly without changing the question can consume time while adding little information. Ask the agent to explain what a particular check will establish before interpreting its output as progress.
Keep negative findings. If the candidate dependency behaves correctly in an isolated call, that narrows the investigation toward integration rather than ending it. If the same reproduction fails on the previous version, the new upgrade may not be the trigger you assumed. Update the working explanation openly. Do not let the incident report become a story written to defend the first guess. A good record shows how evidence changed the conclusion and makes it possible to revisit a specific assumption when a later observation does not fit.
Use history search only with a meaningful test
When a reproducible failure exists between a known working revision and a known failing revision, version history can help narrow the change. Git bisect uses a binary-search process over commits based on good and bad observations. It is useful when the test describes the relevant property and can run meaningfully on the selected revisions. A historical checkout that cannot build under the available environment should not be casually labeled bad merely because setup failed before the behavior was exercised.
Perform this work in an isolated checkout and preserve the commands and outcomes. If the failure depends on a changing external service, history search may be inconclusive until that dependency is controlled. The agent should explain such limits before announcing a culprit commit. Even a narrowed commit identifies where the tested behavior changed, not necessarily the entire organizational or architectural cause of the incident. Inspect the actual modification and confirm the proposed explanation with a focused test before writing a remediation plan around a single line in version history.
Correct the cause and verify the recovery path
Once the failure mechanism is supported, prepare the smallest change that addresses it while preserving the relevant contracts. Add a regression check that fails under the demonstrated old behavior. Run the required adjacent checks, then review the final diff independently of the agent's explanation. If a workaround is necessary for immediate recovery, label it as a workaround and identify the remaining engineering task. A temporary measure can be appropriate without becoming a fictional permanent fix in the incident report.
Verify the actual release path when deployment is within the authorized scope. Connect the tested source and artifact to the serving check, and exercise the affected user journey with safe data. Preserve the previous artifact until recovery is established under the normal release process. If the incident was already mitigated, distinguish mitigation verification from verification of the permanent correction. GrantTap helps keep the work and evidence accessible across devices, but the application and release system must still provide the observations that establish what users can do now.
Share a sanitized packet with useful limits
Before sharing the report, inspect logs, screenshots and paths for secrets and private data. Replace customer identifiers with stable fictional values that preserve relationships needed to understand the failure. Remove credentials rather than masking only their most visible appearances. A private path can reveal a user name or project arrangement, while a command can include a token in an argument. Review the actual exported files, not just the summary text. The sanitized reproduction should be rerun if changes to the fixture might alter the behavior.
The packet should contain the symptom, scope, timeline, evidence references, supported mechanism, recovery and remaining work. Keep a clear distinction between confirmed facts and open questions. If a source was unavailable, state that limit. Do not describe the packet as a complete audit trail when it is a selected collection of evidence. That honesty helps a teammate or support engineer use it correctly: they can reproduce the demonstrated failure, understand what was checked and identify the next missing observation without being misled by an overly broad claim of completeness.
Turn the incident into a specific improvement
Choose follow-up work that addresses the demonstrated weakness. The right outcome might be a boundary test, a clearer deployment receipt, a corrected status label, a documented runtime requirement or a better recovery command. Avoid a vague task to improve agent reliability without identifying the failure it should prevent or detect. Give each follow-up a concrete acceptance condition and an owner. The agent can help prepare the change, while the team decides how it fits with ongoing work and the actual impact of the incident.
Close the record when recovery is verified and the remaining work is explicitly tracked, not merely when the chat becomes quiet. Preserve the final state in the Task so another execution can continue from accurate context. A useful incident review is neither a defense of automation nor a reason to abandon it. It is a careful account of an observed failure and a practical change to the workflow. That account makes the next agent-assisted task easier to supervise because the team knows which evidence matters and which boundary now has a meaningful check.