# Redacting personal / confidential data from a .docx

Use when the user asks to strip personal data, names, positions, or company
names from a Word file before sharing ("обезличить", "удалить личные данные",
"убрать названия компаний"). Goal: no residual identifier anywhere in the
package, structure and formatting intact.

## Rule 0 — the body is not the whole document

Visible text is only part of what leaks. Scan **every** part of the package,
not just `word/document.xml`:

| Where | What lives there |
| --- | --- |
| `word/document.xml` | body paragraphs, tables, text boxes |
| `word/headerN.xml`, `word/footerN.xml` | running company/title lines (one footer per section — a doc can have 20+ identical footers, all must be edited) |
| `docProps/core.xml` | `dc:creator`, `cp:lastModifiedBy` — often the real author's **email** and full name |
| `docProps/app.xml` | `<Manager>` — often another email |
| `customXml/*.xml` | metadata items |

The header/footer and docProps leaks are the ones people forget. A footer
that reads "<Doc title> ООО «<Company>»" repeats company branding on every
page; each `footerN.xml` is a separate file carrying its own copy.

## Procedure

1. **Baseline scan.** List every part and grep all `.xml` for the terms to
   remove (company name in all grammatical forms, well-known surnames,
   `Архидинамика`-style unit names). Print each hit with ~120 chars of
   context so you can see what is actually there. Also regex the whole
   package for `[\w.+-]+@[\w.-]+\.\w+` (emails) — emails hide inside
   docProps, not the visible text.
2. **Inspect images separately.** `docx_read.py --images DIR` (or unzip
   `word/media/*`) and check each with `vision_analyze` for logos, faces,
   signatures, company names. Pure technical/diagram images are clean and
   need no edit. EMF/WMF logos will not decode as images — rasterize via
   PowerShell `System.Drawing.Image` → Bitmap.Save(png) before viewing.
3. **Replace in-process with lxml, per paragraph.** Walk every `w:t`,
   group runs by their owning `w:p`, join the run texts into the paragraph
   string, apply exact-match and regex substitutions to that string, then
   write the result into the first run and blank the rest. This is the same
   collapse as `docx_edit.py replace` but lets you batch many rules across
   body, tables, and all headers/footers in one pass.
4. **Scrub docProps** with plain string replacement on `core.xml` / `app.xml`
   (empty the creator/manager elements).
5. **Rewrite the zip** preserving every original part name and order;
   only the edited parts change bytes.
6. **Verify against the original** (see below).

## Replacement style (this user)

- Replace the term with underscores of similar width (`___` / `________`),
  not with a fake name — it makes the redaction visible and keeps table
  columns from collapsing. Confirmed as the preferred shape for
  agreement/approval tables and signature blocks.
- Leave dates, version numbers, and city/year unless the user says to
  remove them; offer to strip those too rather than deciding silently.
- Preserve structure: never delete rows/columns of a table to hide a value.

## Pitfalls

- **Regex needs the article/ООО pattern.** `ООО «<Company>»` contains
  guillemets and a space; a bare-company-name regex will not match it.
  Handle `ООО\s*«\s*Company\s*»` first, then the bare name.
- **Header/footer text is in a separate part per section.** Editing
  `document.xml` alone leaves the company name on every page.
- **Running the validator on the original matters.** Run `docx_validate.py`
  on the ORIGINAL first to capture its pre-existing issues as a baseline;
  then run it on the redacted file — the issue set should be identical.
  A file can be `ok: true` even with unresolved image references present in
  the original; do not "fix" pre-existing noise.
- **Maker won't be caught by a name scan if it's only an email.** Grep for
  emails explicitly; `cp:lastModifiedBy` and `<Manager>` frequently hold
  the person's corporate address instead of a display name.

## Checking a redaction you did not do (completeness pre-flight)

A different job from redacting. The document is already redacted; the question
is **what is still visible** before it goes to an external model or into a
report read outside the team. The deliverable is a line for the report header,
not an edited file. Run `scripts/scan_open_pii.py <file.docx>` — it prints what
is closed and what is open, by category, with the finds themselves.

**Two marker conventions exist, and both must be recognised.**

| Convention | Looks like | Why it is used |
| --- | --- | --- |
| Bracketed tokens | `[АДРЕС-003]`, `[ЗАКАЗЧИК-1]` | token substitution — the value stays recoverable from a lookup table |
| Underscore runs | `«________»`, `______` | in-place overwrite — the structure of forms and tables does not shift |

A checker that knows only one convention is worse than none: on a file redacted
the other way it reports "no markers found", concludes the copy is the original,
and stops. Recognise both, and treat "no markers at all" as the only real stop
condition — not "no tokens".

Rules for the check itself:

- **Scan the whole package** — body, table cells, every `headerN.xml` /
  `footerN.xml`, `docProps`. Contract content lives in table cells, and a
  paragraphs-only pass returns nothing, which reads as "all clean".
- **Fold case in Python** (`.lower()`), never through the shell: `grep -i` does
  not fold Cyrillic in a Windows shell and the count silently comes back zero.
- **Absence of findings is not proof** and the wording must say so: "the patterns
  found N", never "the document is clean". A party named descriptively ("the
  investor for the site near X station") matches no pattern at all.
- **Separate what a human must judge.** Utilities and grid operators appear in
  every municipal brief and identify nothing; the customer and the site do.
- **Never open the token-to-value lookup table** to "clarify" a hit — the check
  runs on the redacted copy alone.
- **The agent reads counts, not values.** The full report prints the finds
  themselves — and the finds ARE the sensitive data. That output going into an
  agent's context sends exactly what is being hidden to the model, defeating the
  rule the check exists to enforce. Run `--summary` (counts, no values) whenever
  an agent will read the result; the full report is for a human looking at the
  document directly. Verified on a sample: the full run printed the company
  name, the signatory, the phone and the postal index — i.e. the very values
  being redacted.

## Verification (do all three)

1. Full-package grep for every term again — expect **0** hits across all
   `.xml` parts.
2. Regex the package for surname+initials (`[А-ЯЁ][а-яё]+\s+[А-ЯЁ]\.\s?[А-ЯЁ]\.`),
   full-name triples, emails, and phone numbers — expect empty sets. Long
   runs of spaces from table layout will false-positive a naive
   three-word regex; anchor the name pattern on initials or a known
   surname list instead.
3. `docx_validate.py out.docx` → `ok: true` with the same issues as the baseline.