When an AI assistant rewrites a prompt, configuration file, or policy document, the new version can look polished while quietly changing a default, removing a constraint, or broadening access. Compare it with the version you trust before accepting it. A diff shows what changed; the surrounding context tells you whether the change is safe and intended.
Start with two clearly identified versions
Save the trusted original and the proposed revision as separate files or clearly labeled text blocks. Do not paste one over the other before comparing; that destroys the baseline you need. If a system generated the new version from a prompt or patch, preserve that prompt or patch too, because it can explain the intended scope.
Name files so their roles are obvious, for example config-before.json and config-proposed.json. Keep secrets out of files you send to third-party diff services. If the content includes credentials, customer information, private prompts, or unreleased plans, use an approved local tool or a redacted copy.
Normalize only what is safe to normalize. Line endings and whitespace can create noisy diffs, but formatting can also be meaningful in indentation-sensitive formats such as YAML. Before trimming whitespace or reformatting, make sure you are comparing equivalent representations and keep an untouched copy of each input.
Read the diff in layers
First: locate the scope
Check whether the edit is limited to the section you asked to change. If the request was “rename this field,” unexpected edits to authentication, network access, file paths, dependencies, or unrelated paragraphs deserve separate review. A small requested task should usually have a narrow change set; if it does not, ask why.
Second: classify each change
Mark additions, removals, moves, and edits. A diff may show a deleted line and a new line as a replacement even when the wording is only slightly different. Review the old and new context rather than judging by color alone.
For a configuration file, trace a setting to its consumer. A key named readOnly may be interpreted differently by two applications. For a prompt, ask whether a new instruction changes priorities, disclosure, audience, or refusal behavior. For a policy, check whether a qualifier such as “may” became “must,” or an exception disappeared.
Third: look for consequential changes
Pay particular attention to:
- Authentication, authorization, roles, scopes, and default permissions.
- Network destinations, shell commands, file-system paths, and executable hooks.
- Logging, telemetry, retention, and disclosure language.
- Error handling, fallback values, retry behavior, and destructive actions.
- Removed limits, validation, warnings, or user confirmation steps.
- New dependencies, environment variables, secrets, or external services.
This list is a review prompt, not an automatic security audit. Whether a line is dangerous depends on how the consuming application uses it.
A worked example: a config change that looks harmless
Suppose an assistant is asked to add a preview mode to a fictional task runner. The old configuration contains:
{
"mode": "preview",
"allowDelete": false,
"allowedDirectories": ["./drafts"]
}
The proposed version is:
{
"mode": "apply",
"allowDelete": true,
"allowedDirectories": ["."]
}
A formatter may confirm that both snippets are valid JSON. The diff reveals three behavior changes: preview became apply, deletion was enabled, and the path scope widened from one folder to the project root. If the user only asked for a preview feature, none of those changes should be accepted without an explicit explanation and a test against the program that reads these keys.
Do not infer that every broad path is exploitable or that every true flag is unsafe. The point is to identify a concrete question: which code interprets the key, what operations does it permit, and what boundary did the request require?
Confirm meaning with the right evidence
A text comparison cannot tell you that code behaves correctly. For JSON, parse the file and compare types and required fields against the schema or consuming code. For YAML, load it with the application’s parser and check indentation-sensitive values. For a prompt, run a small set of representative inputs through the actual model and review the outputs against the intended policy; a diff alone cannot predict model behavior.
For code and config, use the smallest relevant validation: a parser, schema validator, unit test, or dry-run command. Do not run unknown commands copied from generated instructions. Read them first, especially if they can access credentials, delete data, install packages, or contact external services.
If the change affects permissions or data handling, ask a second reviewer to inspect those specific lines. Preserve the accepted diff or a commit identifier so the approved version is unambiguous.
Reduce noisy comparisons without hiding real changes
Formatting both JSON versions with the same formatter can make structural edits easier to spot. But do not sort arrays or rewrite comments automatically unless the format and application treat ordering as irrelevant. In many systems, arrays preserve precedence or order of execution.
For prose, compare paragraphs rather than only lines. A line-oriented diff can make moved text appear as a deletion plus an addition. Read the section before and after the changed passage, and check whether references, headings, definitions, or exceptions still connect correctly.
Use the Toolsnix Difference Checker for a quick text comparison. For JSON-specific syntax checks, use a JSON Formatter and then validate application-level requirements separately. Do not upload confidential material unless the tool’s processing and retention model is appropriate for your data.
A review record that stays useful
For an important change, record four things: the request, the files compared, the consequential changes you found, and the evidence used to accept or reject them. A concise note could say: “Compared config-before.json with proposed revision; rejected the widened directory scope because the request only added preview mode; parser and dry-run both passed on the corrected version.”
This gives teammates enough context to reproduce the decision. It also helps when the same AI-generated file is revised again: the previous review can explain why a constraint exists instead of making it look like arbitrary boilerplate.
Review changes by risk, not by color
Most diff tools use colors to show additions and removals, but color alone is not a review strategy. Read every changed line in context and sort the changes into low, medium, and high consequence. A formatting-only change may be low consequence if the parser treats it equivalently. A changed permission, endpoint, default, or exception is high consequence even if it occupies one line.
For a prompt, compare instructions that govern priority and boundaries. If “do not send personal data” disappears, ask whether the revised prompt still gives that instruction elsewhere and whether the system has another enforcement layer. If a new instruction tells an assistant to call a tool, check what that tool can access and whether the user must approve the action. A prompt can influence behavior, but code-level authorization should not depend on a sentence alone.
For a configuration file, trace the key to the implementation. Search for every read of the setting, confirm the default when it is absent, and check whether an environment override changes the effective value. A diff may show timeout: 30 changed to timeout: 300; the meaning depends on the unit and consuming code. Do not assume the field is seconds just because the name sounds familiar.
Use a review matrix for consequential changes
| Change type | Question to answer | Evidence to seek | |---|---|---| | New permission | What can this actor or process now do? | Policy, code path, role tests | | Removed validation | Which malformed inputs now pass? | Schema, parser behavior, boundary checks | | Changed endpoint | Who receives the data? | Service configuration and approved destinations | | New default | What happens when a caller omits the field? | Consumer implementation and tests | | Deleted documentation | Was a requirement removed or only moved? | Source of truth and linked policy |
Do not use a matrix as a substitute for judgment. It helps make review questions visible and ensures you gather evidence instead of relying on the AI’s explanation of its own changes.
Separate author intent from observed behavior
The assistant’s summary is a claim about intent. The patch and test results are evidence about the artifact. Compare the two: if the summary says “only rename the field” but the diff also changes a permission, treat that discrepancy as a review blocker until it is explained.
When a reviewer requests a correction, ask for a narrow follow-up and inspect the new diff against the same baseline. Avoid reviewing only the latest patch if the tool rewrote earlier changes. Keep a known-good copy and record which revision passed validation.
Keep provenance and assumptions with the review
A useful comparison begins with a baseline you can trust. Record where the previous version came from: a released commit, approved document, or configuration generated by a known process. A file called final-v2-really-final.json does not establish provenance. If the origin is uncertain, identify that uncertainty before you approve the new version.
Capture the instruction given to the AI and any relevant constraints, but do not treat the prompt as a substitute for the acceptance criteria. It can explain why the output changed; it cannot prove the change is correct. If the model received only part of a file, check whether it accidentally dropped unseen sections or references.
For a document with multiple reviewers, separate proposed edits from accepted edits. Use version control or tracked changes so you can see who approved each consequential change. Avoid copying a reviewed paragraph into a new file and losing the comments that explain the decision.
When the AI rewrites instead of editing
Sometimes a model returns a complete rewritten file even when asked to change one paragraph. A whole-file diff may then show many formatting differences and make important semantic changes harder to locate. Ask for a narrow patch or compare normalized copies while retaining the originals. Never discard the original just because the new output parses.
If reformatting is needed, first compare the parsed structures or run the formatter on both versions with the same settings. Then inspect semantic differences. For prose, use a paragraph-aware comparison and read surrounding sections. For code, review the actual patch in the consuming project, not just a generated explanation.
A large rewrite may still be justified, but it needs a clear reason and enough review time. If the changes exceed the request, split them into smaller commits so formatting, behavior, and documentation changes can be evaluated separately.
FAQs
Does a clean diff mean the AI change is correct?
No. A diff reports textual changes. Validate syntax, schema, behavior, and security properties with the evidence appropriate to the file.
Should I accept a change if the output looks more readable?
Readability is useful, but it does not establish that the behavior stayed the same. Review changed values and removed constraints, then parse or test the file.
Is it safe to paste a prompt into an online diff tool?
Only if the prompt is suitable for that service. Prompts can contain proprietary instructions, personal information, or credentials. Use a local comparison or redact sensitive parts when necessary.
What if the diff is very large?
Ask for a narrower revision or split the work into focused changes. Large unrelated edits make review harder and increase the chance that a consequential change is missed.
Next step
Keep the original, compare a proposed revision, investigate unexpected scope, and validate the result with its real parser or consumer. Treat every highlighted change as a question to answer, not a verdict.