living-documentation-publisher

Overview

Per the Serenity BDD book (serenity-bdd/the-serenity-book, living-documentation.adoc):

"Living documentation is generated by the automated test suite, and is therefore by definition it is always up-to-date."

The same source distinguishes living documentation from plain test reports: living documentation is authored before development starts, targets the whole team (BAs, product owners, stakeholders), and reads as a business-functionality narrative, not a pass/fail table.

This skill covers the full workflow: produce Cucumber JSON from a passing suite, feed it into a report renderer, apply tag-based section routing, and publish the HTML artefact to a Pages endpoint so stakeholders get a URL, not a zip file.

Two primary renderers are covered:

• multiple-cucumber-html-reporter (Node, any Cucumber-JS project) - see references/node-reporter.md.
• Serenity BDD aggregate report (JVM, Maven/Gradle projects using CucumberWithSerenity) - see references/serenity-reporter.md.

When to use

• The team already has Cucumber scenarios (see cucumber-testing) and stakeholders need to read acceptance-criteria status without opening a test runner.
• The living-documentation page must track only passing scenarios (not every scenario authored, including pending or skipped ones).
• CI should gate report publication so stakeholders never see a stale or red-run document.

If only engineers read the results, the JUnit XML feed to junit-xml-analysis (in the qa-test-reporting plugin) is sufficient.

How to use

1. Produce Cucumber JSON from a passing suite - add a json: formatter (Cucumber-JS) or let CucumberWithSerenity capture results (JVM). Step 1.
2. Exclude @wip / @pending scenarios so the document shows only accepted behaviour. Step 4.
3. Render the JSON to HTML - Node path via references/node-reporter.md, JVM path via references/serenity-reporter.md. Steps 2-3.
4. Tag scenarios by capability and sprint so stakeholders can navigate report sections. Step 4.
5. Gate publication on a green exit code so no red or stale run ships. Step 5.
6. Publish the HTML to GitHub/GitLab Pages in CI so stakeholders get a URL. Step 6, via references/ci-publish.md.

Worked example

A Cucumber-JS "Checkout Service" suite has 24 scenarios tagged by capability (@billing, @cart) and sprint (@sprint-12); 2 are still @wip.

1. CI runs npx cucumber-js features/ --tags "not @wip" --format json:reports/cucumber.json, producing JSON for the 22 passing scenarios.
2. node scripts/generate-report.js renders ./docs/living-documentation/ under the report name "Checkout Service - Living Documentation".
3. Because set -e precedes the generate step, a red run aborts before publishing.
4. peaceiris/actions-gh-pages@v4 pushes the folder to GitHub Pages.
5. Result: the product owner opens the Pages URL and sees 22 green acceptance criteria grouped under Billing and Cart, with the two @wip scenarios absent - an always-current, business-readable status page.

Step 1 - Produce Cucumber JSON output

The report renderers both consume Cucumber JSON. Configure the JSON formatter before wiring the renderer.

Cucumber-JS (in package.json):

{
  "scripts": {
    "test": "cucumber-js features/ --format json:reports/cucumber.json"
  }
}

For parallel shards, use a timestamped JSON filename to avoid overwrite - see references/node-reporter.md.

Cucumber-JVM with Serenity (in serenity.properties):

The CucumberWithSerenity runner captures results automatically; no explicit JSON formatter is needed. Serenity writes its own JSON artefacts to target/site/serenity/ as part of mvn verify (serenity-bdd/the-serenity-book, maven.adoc).

Step 2 - Generate the HTML report (Node path)

For any Cucumber-JS project, render the JSON with multiple-cucumber-html-reporter: install it, add a scripts/generate-report.js that points jsonDir at ./reports/ and reportPath at ./docs/living-documentation/, then run npm test && node scripts/generate-report.js. Full install, script, and options table: references/node-reporter.md.

Step 3 - Generate the HTML report (JVM / Serenity path)

For Maven/Gradle projects running CucumberWithSerenity, bind the serenity-maven-plugin aggregate goal to post-integration-test and run mvn verify. The Requirements tab renders living documentation from the feature-directory hierarchy. Full plugin config, hierarchy labels, and readme.md enrichment: references/serenity-reporter.md.

Step 4 - Tag scenarios for report sections

Use Gherkin tags to categorise scenarios in both renderers.

In the feature file:

@billing @sprint-12
Scenario: Apply promo at checkout
  Given ...

Serenity tag filtering - run only tagged tests and limit the aggregate report to those requirements (serenity-bdd/the-serenity-book, filtering-reports.adoc):

# Run and report on one sprint
mvn clean verify -Dcucumber.options="--tags=@sprint-12" \
  -Dtags=sprint-12

# Post-run report filtered to a tag
mvn serenity:aggregate -Dtags=sprint-12

Note: "requirements filtering only happens if you specify the tags option" (serenity-bdd/the-serenity-book, filtering-reports.adoc).

Excluding pending/WIP from published docs:

# Cucumber-JS: exclude anything tagged @wip from the JSON
npx cucumber-js features/ \
  --tags "not @wip" \
  --format json:reports/cucumber.json

This keeps the living-documentation page free of scenarios that are not yet passing.

Step 5 - Only publish passing runs

Gate the report publication step on a green test exit code. In a shell script:

set -e
npm test                         # exits non-zero on any failure
node scripts/generate-report.js  # only reached if all tests pass

For Serenity, use serenity:check after verify (serenity-bdd/the-serenity-book, maven.adoc):

mvn verify serenity:check        # fails the build if any scenario is red

This ensures the published artefact reflects only a fully-green run.

Step 6 - Publish to GitHub Pages (CI)

Publish the generated HTML to GitHub or GitLab Pages so stakeholders get a URL, not a zip file. Full GitHub Actions and GitLab Pages jobs (including the Serenity publish_dir override): references/ci-publish.md.

Anti-patterns

Anti-pattern	Why it fails	Fix
Publishing on any push, including red runs	Stakeholders see failing-scenario counts; erodes trust in the document	Gate publish on exit code 0 (Step 5)
Including @wip / @pending scenarios in the report	Document shows work-in-progress as if it is accepted behaviour	Filter with not @wip before generating JSON
One flat tag for all scenarios	Stakeholders cannot navigate to a capability area	Apply two-level tags: @capability-name and @sprint-N
Publishing stale JSON from a previous run	Report shows old results after source changes	Delete reports/*.json at the start of each CI run before running tests
Embedding full screenshots in every step	HTML artefact becomes hundreds of MB	Use Serenity's evidence API (Serenity.recordReportData()) selectively, or configure take.screenshots=FOR_FAILURES in serenity.properties

Limitations

• Serenity aggregate requires JVM test execution in the same Maven lifecycle. Running serenity:aggregate against JSON files from a Cucumber-JS run is not supported; use multiple-cucumber-html-reporter for Node projects.
• multiple-cucumber-html-reporter merges JSON by filename, not by run order. If parallel shards write JSON with identical names, results are silently overwritten. Use timestamped or shard-indexed filenames.
• Pages publish latency. GitHub Pages can take 60-120 seconds to reflect a push; stakeholders may see the previous version briefly after a CI run.
• Serenity requirements hierarchy is derived from directory layout. Renaming a feature directory breaks historical trend data in the report.

References

• ld - Serenity BDD book, living-documentation.adoc: definition, Requirements tab, hierarchy, readme.md enrichment, evidence API.
• mv - Serenity BDD book, maven.adoc: serenity-maven-plugin coordinates, aggregate goal, post-integration-test phase binding, mvn verify, serenity:check.
• fr - Serenity BDD book, filtering-reports.adoc: -Dtags, -Dcucumber.options, requirements-filtering caveat.
• references/node-reporter.md - Node renderer install, generate-report.js, and the full options table.
• references/serenity-reporter.md - Serenity Maven plugin, aggregate goal, requirements hierarchy, readme.md enrichment.
• references/ci-publish.md - GitHub Actions and GitLab Pages publish jobs.
• cucumber-testing - upstream skill: produce Cucumber JSON with --format json:.
• junit-xml-analysis (in the qa-test-reporting plugin) - engineer-facing test result analysis (separate concern from stakeholder docs).
• github-actions-test-jobs (in the qa-ci-integration plugin) - CI job conventions for the publish step.