prokop/thgtoa
1
0
Fork 0
mirror of https://github.com/Anon-Planet/thgtoa.git synced 2026-06-11 00:02:29 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(changelog): prevent history dump and filter noise commits

Browse Source 
commits_since(): when no prior tag exists, scope to commits not yet on
origin/main via merge-base instead of walking the entire history. This
is what caused the v2.0.1 entry to contain every commit back to project
inception.

categorise(): replace the minimal skip pattern with a compiled NOISE
regex that also drops:
  - numbered series commits (3/8, 7/8, etc.)
  - vague WIP messages (Tweaking, Moving some, Still broken, pt2...)
  - one-word infrastructure fixes (Fix workflow, Fix path, Fix README)
  - oops commits (Forgot to, Revert "...")
  - joke messages (One job to rule them all)

Signed-off-by: nopeitsnothing <no@anonymousplanet.org>
This commit is contained in:
nopeitsnothing
2026-05-23 23:55:34 -04:00
parent 3e28ec19ad
commit f71e5e2a28
2 changed files with 58 additions and 137 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/changelog/index.md
+19 -133
View File
@@ -20,153 +20,39 @@ Notable changes to the guide and its tooling. Follows [Keep a Changelog](https:/
--- ---
## [v2.0.1] ## [v1.2.3] — 2026-05-24
!!! Note "Meta" CI/CD pipeline split into independent stages, dark PDF quality improved, release signing automated, and the changelog now updates itself on every build.
- Released 2026-05-24 from [`c658c35`](https://github.com/Anon-Planet/thgtoa/commit/c658c354ee0b982163167a7ac7106a0bf16465ed)
!!! Note "Added"
- Add tag_release.py — guided signed release tagger
- How to verify the authenticity of our files and check virus scans
!!! Note "Changed"
- Rewrite developer guide for current pipeline
- 8/8 chore(bump): v1.2.3
- 8/8 chore(scripts): minor cleanup to setup_workflow.py
- 7/8 docs(guide): bump version string to v1.2.3
- 6/8 chore: track .b2 hash files in .gitignore
- 5/8 docs(changelog): rewrite for v1.2.3 — consolidate and clean up
- 4/8 ci: add automated changelog update workflow
- 3/8 ci: split monolithic workflow into build, sign, release stages
- 2/8 refactor(pdf): wire dark mode through convert.py
- 1/8 feat(pdf): add pixel-based dark mode PDF converter
- Refactor pipeline into independent build/sign/release/changelog workflows
- Submit actual develop page
- Fix copy information in website footer
- Fix some broken YAML references
- Delete stale information
- Sign local copy
- Tweaking some of the build to function
- Tweaking some of the build to function
- Tweaking some of the build to function
- Tweaking some of the build to function
- Tweaking some of the build to function
- Tweaking some of the build to function
- Tweaking some of the build to function pt6
- Tweaking some of the build to function pt5
- Tweaking some of the build to function pt4
- Tweaking some of the build to function pt3
- Tweaking some of the build to function pt2
- Tweaking some of the build to function
- Moving some things around
- One job to rule them all
- Forgot to add flag
- The GPG bit fails, let's try again pt2
- The GPG bit fails, let's try again
- Move out some scripts
- Combined actions refactor
- Combined actions into one file for less overhead
- Overhaul the Hashing, scanning, release management
- Overhaul the Hashing, scanning, release management
- Refactor build action
- Slightly refactor the workflow task
- Fix README
- Build pipeline WIP
- Refactoring the VT job
- Downgrade to working versions to fix broken jobs
- Appendix A6: comment out deprecated ODT information
- Replace broken internal link with correct rel path
- Add nav in mkdocs.yml config
- Fix broken link to page
- Update VT scan workflow
- Use VT v5.x
- Add VirusTotal scans for submitted PDFs
- Fix path
- Refactor PDF build in CI, add dark mode PDF (pt 4)
- Fix PDF build in CI
- Fix PDF build in CI
- Archive Matrix database and shutdown
- The Tor onion v3 address works
- Fix
- Revert "Fix some metadata"
- File endings
- Missing parenthesis
- Extra parenthesis
- Creating your anonymous online identities
- Traffic anonymization
- OPSEC thoughts
- Watermarking
- Browser and device fingerprinting
- Requirements refs
- Local data leaks, forensics
- Whonix virtual machines
- Some more
- Adversarial considerations (threats)
- Some more
- Some Tails refs
- Some additional measures against forensics refs
- Comparing versions ref
- Persistent plausible deniability
- All refs I have time for at the moment
- Fix some metadata
- Update admin info in Matrix listing
- Fix some metadata
- Fix discussion channels
- Fix nope's Matrix pushed to only one repo
- Fix Das' Matrix pushed to only one repo
- Remove commitizen requirement
- Remove than/nope signed canary
- Upgrade pre-commit hooks
- Wikiless.org --> Wikiless.com
- Minor updated information
- Still broken, try again
- Fix workflow
- Missing README.md
- Fix some references and tidy up language
- Cleanup old remnants
- Sample publishing config
- Move CNAME to docs
- Add CNAME
- Remove old site artifacts
- Fix some absolute links
- Add Blackhat USA 2024 conference on Wi-Fi dangers
- Add Blackhat USA 2024 conference on Wi-Fi dangers
- Formatting and tooling upgrades to improve performance
!!! Note "Fixed"
- Actually save per-page PDFs for qpdf, not PNGs
- Fail fast with helpful message if pdftoppm or qpdf missing
- Resolve Pillow JPEG KeyError and cairosvg missing dep
## [v1.2.3] — 2026-05-22
CI/CD pipeline split into independent stages, dark PDF quality improved, and the changelog is now updated automatically on every release. v1.2.2 was just a placeholder, this is a minor but CI breaking change.
!!! success "Added" !!! success "Added"
- **Dark mode PDF** (`scripts/convert.py`): pixel-level converter replaces the broken `--prefers-color-scheme=dark` Chromium flag. Produces a 200 DPI hacker-themed PDF (`#1f1f31` background, `#e0e0e0` text, `#5e8bde` links) with batched page processing to avoid OOM. - **Dark mode PDF** (`scripts/convert.py`): pixel-level converter replaces the broken `--prefers-color-scheme=dark` Chromium flag. Produces a 200 DPI hacker-themed PDF (`#1f1f31` background, `#e0e0e0` text, `#5e8bde` links) with batched page processing to avoid OOM on large documents.
- **Three independent CI workflows** replacing the old monolithic `build-sign-release.yml`: - **Three independent CI workflows** replacing the old monolithic `build-sign-release.yml`:
- `build.yml` — builds PDFs and uploads them as an artifact; no secrets required. - `build.yml` — builds PDFs and uploads them as an artifact; no secrets required, can be re-run freely.
- `sign.yml` — downloads the PDF artifact, computes SHA-256 and BLAKE2b hashes, GPG-signs all outputs, and uploads a `signatures` artifact. Can be re-run against any historical build. - `sign.yml` — downloads the PDF artifact, computes SHA-256 and BLAKE2b hashes, GPG-signs all outputs, and uploads a `signatures` artifact. Can be re-run against any historical build.
- `release.yml` — downloads both artifacts, uploads to VirusTotal, and publishes a tagged GitHub Release with all 12 assets attached. **Can be triggered manually against any previous sign run**. - `release.yml` — downloads both artifacts, uploads to VirusTotal, and publishes a tagged GitHub Release with all 12 assets attached. Can be triggered manually against any previous sign run.
- **`scripts/update_changelog.py`**: reads `git log` since the last version tag, categorises commits by conventional-commit prefix, and prepends a new entry here automatically after each successful build. - **`scripts/update_changelog.py`**: reads `git log` since the last version tag, categorises commits by conventional-commit prefix, and prepends a new entry to this file automatically after each successful build.
- **`changelog.yml`** workflow: commits the auto-generated changelog entry back to `main` after every build, with `dry_run` and `manual_version` dispatch inputs for testing. - **`changelog.yml`** workflow: commits the auto-generated changelog entry back to `main` after every build, with `dry_run` and `manual_version` dispatch inputs for safe local testing.
- **`scripts/tag_release.py`**: interactive guided helper for maintainers to create GPG-signed annotated tags. Checks clean tree and branch, auto-increments the version, pulls the message from the changelog, resolves the release signing key, creates and verifies the tag, then prints the push command.
- **`docs/code/develop.md`**: full developer reference covering prerequisites, local build instructions, the pipeline flow, all required GitHub Secrets, the release process, verification steps, and a troubleshooting section for every known CI failure mode.
!!! warning "Changed" !!! warning "Changed"
- `build-sign-release.yml` is now deprecated — push triggers removed, manual dispatch only. Will be deleted once in-flight runs complete. - `build-sign-release.yml` deprecated — push triggers removed, manual dispatch only. Will be deleted once in-flight runs complete.
- The full pipeline (build → sign → release) now chains automatically via `workflow_run` on every push to `main`. - The full pipeline (build → sign → release → changelog) now chains automatically via `workflow_run` on every push to `main`.
- GPG signing uses `--pinentry-mode loopback` and `--passphrase-fd 0` to avoid interactive prompts on headless runners. - GPG signing uses `--pinentry-mode loopback` and `--passphrase-fd 0` to avoid interactive prompts on headless runners.
- VirusTotal scans moved to the release stage so they run once per release, not once per build. - VirusTotal scans moved to the release stage so they run once per release, not once per build.
- `.gitignore` updated to track `.b2` per-file hash files alongside existing `.sha256` and `.sig` entries.
- Stale information removed from the guide; deprecated ODT section in Appendix A6 commented out.
- Footer copyright information corrected.
!!! bug "Fixed" !!! bug "Fixed"
- Broken internal links and a mismatched cross-reference in `docs/about/index.md`. - `_save_images_as_pdf` in `convert.py` was passing raw PNG files to `qpdf --pages`, which only accepts PDF inputs. Fixed by quantizing each page to palette mode (256 colours, FASTOCTREE) and saving as a single-page PDF before merging.
- Deprecated ODT section commented out in Appendix A6 of the guide. - `convert.py` now fails immediately with install instructions if `pdftoppm` or `qpdf` are missing, instead of crashing with an unhelpful `FileNotFoundError`.
- Pillow `KeyError: 'JPEG'` on CI resolved by installing `mkdocs-material[imaging]` and using palette-mode PDF encoding instead of RGB+JPEG.
- Orphaned footnote citations `[^536]` and `[^537]` (Australian privacy law and the Identify and Disrupt Act) restored at the key disclosure law paragraph in the guide.
- Broken internal links and mismatched cross-references throughout the guide corrected.
--- ---
scripts/update_changelog.py
+39 -4
View File
@@ -84,11 +84,25 @@ def version_from_changelog() -> str | None:
def commits_since(ref: str | None, until: str) -> list[str]: def commits_since(ref: str | None, until: str) -> list[str]:
"""Return one-line commit messages between ref and until (exclusive/inclusive).""" """Return one-line commit messages between ref and until (exclusive/inclusive).
When no ref is given (no prior tag exists) we fall back to the merge-base
between HEAD and origin/main rather than walking the entire history, which
would otherwise dump every commit ever made into the changelog.
"""
if ref: if ref:
log_range = f"{ref}..{until}" log_range = f"{ref}..{until}"
else: else:
log_range = until # No previous tag — scope to commits not yet on origin/main
merge_base = run(
["git", "merge-base", "HEAD", "origin/main"], check=False
).stdout.strip()
if merge_base:
log_range = f"{merge_base}..{until}"
else:
# Truly brand new repo with no remote — limit to last 50 commits
# to avoid dumping the whole history
log_range = f"-50 {until}"
out = run(["git", "log", "--pretty=format:%s", log_range]) out = run(["git", "log", "--pretty=format:%s", log_range])
return [line.strip() for line in out.splitlines() if line.strip()] return [line.strip() for line in out.splitlines() if line.strip()]
@@ -97,9 +111,30 @@ def categorise(messages: list[str]) -> dict[str, list[str]]:
"""Sort commit messages into Added / Changed / Fixed buckets.""" """Sort commit messages into Added / Changed / Fixed buckets."""
buckets: dict[str, list[str]] = {b: [] for b in BUCKET_ORDER} buckets: dict[str, list[str]] = {b: [] for b in BUCKET_ORDER}
# Patterns that are never useful in a human-readable changelog
NOISE = re.compile(
r"""
\[skip\ ci\] # CI skip marker
| ^Merge\ (pull\ request|branch) # merge commits
| ^chore:\ bump # version bump chores
| update\ changelog # self-referential
| ^\d+/\d+ # numbered commit series (e.g. 3/8)
| ^Tweaking # vague WIP messages
| ^Moving\ some # vague WIP messages
| \ pt\d+$ # "...pt2", "...pt3" suffixes
| ^Fix\ (workflow|path|README)$ # one-word infrastructure fixes
| ^Still\ broken # embarrassing mid-fix notes
| ^WIP\b # work in progress
| ^Forgot\ to # oops commits
| ^Revert\ " # reverts (surface the original instead)
| ^One\ job\ to\ rule # joke commit messages
""",
re.VERBOSE | re.IGNORECASE,
)
for msg in messages: for msg in messages:
# Skip automated / noise commits # Skip noise
if re.search(r"\[skip ci\]|^Merge |^chore: bump|update changelog", msg, re.I): if NOISE.search(msg):
continue continue
# Strip conventional-commit prefix to get the plain description # Strip conventional-commit prefix to get the plain description