Testing¶

Every other page in this section explains how to use this template to write your report. This page is different: it documents the test suite in test/, which checks that this template's own prodockit-specific features (References, Acronyms, Glossary, Appendix numbering, word-count exclusions, and so on) actually work and haven't regressed - a contributor/maintainer tool, not something you need while writing your assignment.

Running the test suite¶

The tests check the built output - the website in public/ and the PDF at docs/site_documentation.pdf - not the build process itself, so build both first:

pip install -r requirements.txt -r testrequirements.txt
prodockit pdf
zensical build --clean --strict
python test/run_tests.py

testrequirements.txt is separate from requirements.txt - it's only needed to run the tests (locally or in CI), not to build your own report from this template.

Tests are organised into test batches, one per test/test_*.py file, each reporting its own pass/fail (see Test batches below for what each one covers):

python test/run_tests.py --list            # list available batches
python test/run_tests.py --batch links     # run just one batch
python test/run_tests.py -- -k caption     # extra args passed through to pytest

Running just one batch is the fast path while you're actively working on a specific capability - you don't have to wait on the rest of the suite (including a couple of batches that parse the whole PDF) to check whether your change works.

Adding a new test¶

Most new tests belong in an existing batch - add a test_* function to the matching test/test_<batch>.py file. Start a new batch (a new test/test_<name>.py file) only for a genuinely new category of concern; test/run_tests.py --list picks up any test_*.py file automatically, no registration needed.

Shared fixtures live in test/conftest.py and cover both the failure-with-a-clear-message case (missing build artifacts) and the common lookups most tests need:

Two different styles of test both have a place here, and most batches mix both:

• Black-box, against the built output - most of test_build.py, test_links.py, test_numbering.py, test_word_count.py, test_content.py, and test_pdf_structure.py work this way: build the site/PDF, then check something true about the result (no LAUNCH-type links, chapter numbers sequential, a flagged page's words don't count, and so on).
• Direct unit tests against a helper function, with a hand-written markdown snippet, no build required - test_fences.py works this way throughout, calling functions like macros._count_top_level_headings() directly. Reach for this style when you're testing a specific function's behaviour on a deliberately constructed edge case (e.g. a heading shown literally inside a fenced code example) rather than something you'd have to go hunting for in the real docs to exercise.

Where a check needs "the real thing this feature is supposed to do" as a source of truth (e.g. which pages are flagged is_appendix: true), reuse the actual function from macros.py (via the macros fixture) or from prodockit itself rather than re-implementing the same TOML/front-matter parsing a second time in the test - that would just test the test's own parser, not catch a real regression in the production code.

Test batches¶

Where to go next¶

This is the last of the "Start Here" reference pages. Continue to Finalising your document in Customise document content once your report itself is ready to submit - removing starthere/ (this page included) is part of that step.