Secure Software Delivery for Federal Environments Module 3 · Evidence and Assessment

Audit-Ready Documents, Generated

Last reviewed

Intermediate

What you'll learn

~15 min
  • Generate a per-control documentation site from structured sources with a reusable page template
  • Build deterministic DOCX and PDF pipelines - real Word TOCs, CSS-driven classification marks
  • Recognize toolchain rot early and apply the same engineering discipline to doc generators as to product code

The audience that still reads paper

Everything so far is data and machinery — but Lesson 2.3 told you the truth about the destination: AOs read documents, assessors annotate documents, systems of record ingest documents. The last mile of the package is document generation, and it deserves real engineering, because it’s where a year of catalog discipline either shows or dissolves. The reference material gives us both halves of the lesson: a generation architecture worth copying, and a cautionary tale worth fearing.

The per-control site: one template, hundreds of pages

The pattern (from a production control-documentation site spanning twenty control families, one page per control and per enhancement): everything flows from structured sources through one reusable page template —

PER-CONTROL PAGE TEMPLATE
1. overview table id, family, baseline, status, origination
2. the requirement control text + guidance, from the catalog release
(pinned - Lesson 1.2)
3. implementation the human-authored statement (Lesson 2.2), with
statement the CUSTOMER RESPONSIBILITY split rendered as its
own subsection - customer/shared/inherited lanes;
per control, these are the rows that roll up into
the customer responsibility matrix promised in 2.2
4. adaptation delta moderate-to-high before/after table: what
changes at the higher baseline (the evolution-
matrix idea, applied across baselines)
5. related controls links, not repetition
6. references evidence contract entries, line-range cites

Build order: spreadsheet-or-catalog source extracted to JSON, generators emit pages plus navigation, static site builds in CI. The template is the asset — hundreds of consistent, diffable pages from one reviewed structure, and a new control is a data change. (The reference site also carries honest fix-list warts: rendered output committed to the repo instead of CI-built, empty placeholder pages, no git provenance on generated files. Recognize them as the same failures Lesson 3.1 taught — collected-but-not-wired, shipped-but-not-finished.)

Two render pipelines, both deterministic

For deliverable documents, the reference DocGenerator runs two pipelines worth stealing:

Markdown to DOCX via a programmatic docx library: a reference document seeds the styles (body stripped, branding intact — cover page, version table), content renders into it, and the table of contents is a real Word TOC field, not generated text — so it updates inside Word like a native document. The AO’s reviewers get a file that behaves like the Word documents their workflow expects, generated from the same sources as everything else.

Markdown to HTML to PDF via a print engine (WeasyPrint-class): all layout in CSS @page rules — running headers, page counters, and classification/handling marks positioned by stylesheet. That last one matters in this world: banner lines and page-level marks become a style concern, reviewable in one CSS file, instead of a per-document manual chore someone forgets on page 40. (Portion marks — the per-paragraph kind — need per-portion metadata in the source content; a document-level field cannot produce them, so structure your markdown to carry them where they apply.)

Both pipelines are deterministic by construction — no timestamps, no randomness, no network at render time — for exactly Lesson 2.3’s reason: regenerate-and-diff is the review process, and the byte-equality gate works on DOCX and PDF the same as on OSCAL.

The cautionary half

Same codebase, honest inventory: around thirty generator classes plus some forty one-off fix_* scripts, two pipelines with duplicated brand definitions drifting apart, config hardcoded and stale, no CI and no tests, line-by-line string parsing where an AST parser belonged, and committed binaries. Every one accumulated reasonably — a deadline here, a quick patch there — in tooling nobody classed as real software because “it’s just docs.”

That’s the trap named: document generators are load-bearing delivery infrastructure. They render the package the AO signs. The fortieth fix_ script is how a deterministic pipeline stops being deterministic without anyone deciding it should. House rule: doc tooling gets the product bar — CI, tests on golden outputs, one source for brand and config, refactor-or-delete pressure on one-off scripts.

Prompt first: the rot audit

Audit our document-generation tooling (tools/docgen/) as if it were
production code, because it is:
1) inventory: generator modules vs one-off scripts (fix_*, patch_*,
temp_*) - count both, date each one-off by last meaningful change;
2) duplication: find config/branding/style definitions that exist in
more than one place and diff them - report where they have drifted;
3) determinism: find every timestamp call, random value, and network
access in a render path;
4) safety net: which generators have tests against golden outputs? which
have none? emit the CI job that regenerates docs and fails on
byte-inequality;
5) triage: the five changes that most reduce rot, cheapest first.
💬'It's just docs' is how packages fail

No assessor will file a finding titled toolchain rot - they will file findings about the SYMPTOMS: a customer-responsibility section missing from one control page because a one-off script skipped it, classification marks absent on three pages rendered by the drifted second pipeline, a version table contradicting the catalog because config was hardcoded in 2024. Every symptom traces to tooling that got a lower bar than the product. The package IS the product in this world; its renderer is production code.

KNOWLEDGE CHECK

Meridian needs classification and handling marks on every page of its PDF deliverables. Two proposals: (A) each document template includes the marks in its own header blocks; (B) marks are applied by shared CSS at page-render time from a document-classification field in front matter. Which is right, and what is the failure mode of the other?

Key takeaway

Documents are the package’s last mile: generate the control-docs site from one reviewed template (requirement, statement with responsibility lanes, adaptation delta), render DOCX with real Word TOCs and PDF with CSS-driven classification marks, keep both pipelines deterministic — and hold doc tooling to the product bar, because the fortieth fix_ script is how determinism dies unannounced. Module 3 complete: evidence collected, stored defensibly, rendered honestly. Next: the delivery paths — starting with FedRAMP after its rewrite, from the seat you’ll actually occupy.

Search lessons