preprints.ngenesis.co.uk

A monorepo preprint server for preprints.ngenesis.co.uk: self-contained article directories (Markdown + per-paper BibTeX + citation style) built with pandoc (KaTeX math, APA citations) into HTML and submission-ready PDFs, published per-version by git tags, and deployed to Cloudflare Pages by Forgejo Actions.

The homepage source lives in-repo at homepage/index.html (restored here for now). It can later move to its own repo and be pulled in via the registry (see below) without any pipeline changes.

Repo layout

homepage/                        <- the / route source (index.html; in-repo for now)
papers/<slug>/                   <- ONE SELF-CONTAINED ARTICLE per directory
  ├── paper.md                   <- canonical evolving source (front matter + body)
  ├── refs.bib                   <- paper-local bibliography
  ├── apa.csl                    <- optional citation style (project apa.csl is the default)
  └── preamble.tex               <- optional LaTeX preamble for the PDF build
registry.yml                     <- source registry (monorepo entries + external repos)
refs.bib / apa.csl               <- project-level FALLBACK defaults (used when an
                                    article dir lacks its own at a given tag)
scripts/
  ├── build.sh                   <- assembles the deployable site/ from registry + tags
  ├── check.sh / check.py        <- arXiv-style validation gate (CI; wrapper + module)
  ├── meta.py                    <- front matter -> site/index.json manifest
  └── migrate-article.sh         <- move an article out to its own repo (see Versioning)
site-src/                        <- paper CSS, _headers, latest-stub, KaTeX head-include
.forgejo/workflows/deploy.yaml   <- CI checks + CD deployment
archive/                         <- original PDFs + converted v1 source

The deploy artifact layout differs from the repo (generated, never committed):

site/
  index.html                     <- homepage (reads site/index.json)
  index.json                     <- manifest of all published papers/versions
  _headers                       <- Cloudflare Pages headers (index.json: no-cache)
  papers/<slug>/vN/index.html    <- one HTML per published version
  papers/<slug>/vN/paper.pdf     <- when the PDF pipeline succeeded
  papers/<slug>/latest/          <- redirect stub -> highest published version
  assets/style.css

Article independence

Each papers/<slug>/ directory is a self-contained publication unit: it carries everything needed to produce a publication/submission-ready PDF (source, bibliography, citation style, LaTeX preamble) plus the HTML build inputs. Project-level build and CI scripts live in the repo, not in articles.

• Add an article: create papers/<new-slug>/ (self-contained) and add a registry.yml entry (repo: null) - the registry is the source of truth; an unregistered article dir is a check failure. Remove papers/<slug>/ to take one out.
• A removed article's monorepo tags (<slug>/vN) may linger; the build skips them with a warning (build: WARN: skipping tag ... - slug has no active source) and never fails because of them.
• Moving an article to its own repo (and back) is a supported operation — see scripts/migrate-article.sh under Versioning.

Versioning

Three mechanisms exist; two are used today.

1. Monorepo mode (current): slugged tags

A published version is a git tag <slug>/vN at the monorepo level, pointing at a commit whose papers/<slug>/paper.md is that version's content. The front-matter version: must equal the tag (enforced by the check gate at tag-build time). Pushing ordinary commits to main only runs checks — a new tag is what publishes. You can push work-in-progress mid-version change without publishing anything.

URLs:

• https://preprints.ngenesis.co.uk/papers/<slug>/vN/ — exact version
• https://preprints.ngenesis.co.uk/papers/<slug>/latest/ — CI-generated redirect stub to the highest tagged version (with rel=canonical); the "current version" URL is always computed, never stale content
• .../vN/paper.pdf when the PDF pipeline succeeded

2. Escape hatch: registry-mode articles (external repos)

When an article moves out of the monorepo, the registry entry points at its repo. Imported repos must tag versions with pure v1, v2, … at their repo root — those tags ARE the paper's versions. The article files (paper.md, refs.bib, optional apa.csl/preamble.tex) live at the repo root of the imported repo. The registry carries no version bookkeeping: adding a version to an article = tagging its own repo.

A failed clone or an import with no valid tags produces a warning and that article is skipped — the rest of the site (and the deploy) is never blocked by one bad external source.

3. Manual version overrides (supported, intentionally unused)

A registry entry may carry an explicit versions: map (version -> tag or commit) which wins over tag enumeration. No entry uses it today; it exists to pin or freeze an external article without editing its repo.

Rejected approach — do not drift into it

Separate files per version (paper-v1.md, paper-v2.md, …) were considered and rejected: every version duplicates the source, so fixes (a typo, a citation key) must be re-applied per version or versions silently drift; there is no single canonical evolving source for the check gate to validate; and it does not solve versioning for registry-imported repos at all. Tags give the same frozen-content guarantees without duplication (git show <slug>/v2:papers/<slug>/paper.md is the frozen file).

Moving an article out (and back)

scripts/migrate-article.sh <slug> <new-repo-url>

subtree-splits papers/<slug>/ to a branch, pushes it as the new repo's main, renames the monorepo tags <slug>/vN to plain vN in the new repo, and prints the registry entry to switch to. Follow-up commit: remove papers/<slug>/ and update registry.yml (repo: <url>). Moving back in: set the registry entry's repo: null (do not delete the entry — an article dir without a registry entry fails the check gate), copy the article dir back, re-push tags with the <slug>/ prefix restored.

registry.yml

sources:
  # in-repo article (monorepo mode)
  - slug: evaluating-iit
    kind: paper
    repo: null
  # external article: ITS OWN pure v1/v2 tags define the versions
  - slug: future-paper
    kind: paper
    repo: code.ngenesis.co.uk/<owner>/future-paper
  # homepage: in-repo for now; point repo: at the homepage repo to pull it
  - slug: _homepage
    kind: homepage
    repo: null

Rules (validated by the check gate): every source needs slug, kind in {paper, homepage}, and repo null-or-string; slugs are unique; exactly one homepage source. In-repo sources live under papers/<slug>/; external repos are cloned into BUILD/<slug>/src at build time (ephemeral — there is no centralised staging directory on the server; every deploy assembles a complete tree inside the CI job).

Front matter reference

The paper's YAML front matter (pandoc-compatible, identical in both modes):

field	required	notes
title	yes	quote it if it contains a colon
subtitle	no	optional subtitle
slug	yes	URL segment; MUST equal the article directory name (enforced)
version	yes	integer; must match the publishing tag
date	yes	ISO YYYY-MM-DD
author	yes
tldr	yes	1-2 sentence plain-language summary; shown on the homepage card; may be empty for old versions
abstract	yes	may be a `
keywords	yes	YAML list (may be empty)

Optional fields present-but-empty, to fill in later: orcid, doi, corresponding_email, license, funding, preprint_doi, repository.

Math + citations

• Math: KaTeX. $inline$ / $$display$$ in the Markdown; pandoc emits raw TeX and the pinned KaTeX + auto-render setup in site-src/head-include.html typesets it client-side (CDN assets with integrity attributes).
• Citations: [@tononi2016integrated] style (firstauthorYEARword keys) against the paper-local refs.bib (project-root refs.bib is the fallback when a tag predates per-paper bibliographies — a warning tells you).
• Citation style: paper-local apa.csl, else project apa.csl.
• The References section is generated at build time — never hand-write it.
• Article pages get a clickable table-of-contents sidebar (built client-side from the #/##/### heading IDs by site-src/site.js, styled in site-src/style.css). A floating toggle button shows/hides it and the choice is remembered; it also tracks the section you are reading.
• Optional preamble.tex in the article dir is passed to the PDF build only.

Publishing a new version

1. Edit papers/<slug>/paper.md; update the front-matter version: to match the tag you will create. Keep the article dir self-contained (update its refs.bib when adding citations).
2. Commit and push to main — CI runs the check gate. Fix anything it flags.
3. git tag <slug>/v3 && git push origin main --tags — checks re-run, then the deploy jobs assemble all tagged versions (each from its own tag) and publish.

Deployment (Cloudflare Pages)

• Deploys happen on tags only. Pushes to main run checks and never deploy. Homepage/scripts-only changes therefore go live with the next tagged release (tagged versions are immutable, so a rebuild renders the same papers).
• The deploy unit is one complete directory. Cloudflare Pages deploys are atomic; whatever wrangler pages deploy site uploads replaces the site. build.sh always assembles homepage + manifest + every published version, so every deployment is a full, consistent site — partial-tree deploys are impossible by construction.
• Orphan tags (a slug with no active source) are skipped with a warning.
• /index.json is served with Cache-Control: no-cache (_headers) and is generated from each version's front matter at build time — no hand-maintained listing. PDFs appear in the manifest (pdf_url) only when the PDF file is actually present in the deploy directory.

CI/CD (Forgejo Actions)

push to main ───────────────────────────────► check (hard gate)
                                              │ fail → nothing deploys
                                              │ pass → nothing deploys either
tag <slug>/vN pushed ──► check ──┬──► deploy-html (runs-on: ci)
                                 │      assemble from tags + wrangler pages deploy
                                 └──► build-pdf  (runs-on: pdf-build,
                                       continue-on-error)
                                        │ artifact: all versions' paper.pdf
                                        ▼
                                     publish-pdf (if: always())
                                      PDFs? → re-deploy site with PDFs restored
                                      none? → notice + exit 0

• check is the hard gate on every push (branches and tags); deploys run only on tag pushes and never on failed checks.
• deploy-html does not depend on build-pdf: the HTML release goes out even when the PDF runner is offline or the PDF build fails. publish-pdf re-deploys with all versions' PDFs restored when they exist.
• Orphan tags (<old-slug>/vN after an article moved out) still match the */v* trigger and run check + deploy of the remaining site — they never fail the pipeline, but they are not a no-op either. At build time they are skipped with a warning.

Runner labels

Jobs select runners by label:

• [ci] — normal runner. Container pandoc/minimal:3.11-alpine (small; jobs apk add bash/git/python3/nodejs; first step of each job is sh-compatible since the image has no bash).
• [pdf-build] — heavyweight runner for the PDF job. Container pandoc/latex:3.11; the job apt-installs texlive-xetex (the image ships TeX Live scheme-basic, no xelatex), so keep a few GB of disk free.

A pdf-build job with no matching runner stays queued — it never blocks the HTML release (graceful degradation by design), but a dedicated runner keeps PDF churn off your normal CI.

Secrets (Forgejo repo/org → Settings → Actions → Secrets)

• CLOUDFLARE_API_TOKEN — Cloudflare API token scoped to Account: Cloudflare Pages: Edit for the Pages project preprints.
• CLOUDFLARE_ACCOUNT_ID — your Cloudflare account id.

Local build

apt-get install -y pandoc python3        # pandoc >= 3.x, with citeproc
scripts/check.sh                         # validate all in-repo papers + registry
scripts/build.sh                         # HTML + manifest + homepage into site/
BUILD_PDFS=1 scripts/build.sh            # + PDFs (needs xelatex, e.g. pandoc/latex image)
cd site && python3 -m http.server 8000   # preview (homepage fetches /index.json)

With no tags at all, the build produces only an unpublished draft preview under BUILD/draft/ and an empty manifest. External registry sources are cloned from their configured repo URL.