Skip to main content

CI Quality Gates on GitHub Actions: What to Block and When to Warn

Some CI checks should block a release. Others should only warn. I worked out the difference building a GitHub Actions deploy pipeline.

If you run your own blog or portfolio site on your own, this post is for you. Every deploy pipeline has to decide what blocks a release and what just warns. I worked that out the hard way, on this site.

I rebuilt it with Claude last month, moving the site to Hugo. Once it was live, I checked what the rebuild broke, by hand. A link checker script first, then a pass through the real W3C HTML checker, page by page. It found two dead internal links, five broken image references, and one page shipping invalid HTML. Fixing those was quick. The real work was making sure I never had to go looking for them again.

Here’s what I ended up building:

flowchart TD
    Build[Build the site] --> Links[Check links and images]
    Links --> Redirects[Check legacy redirects]
    Redirects --> Tags[Check tag registry]
    Tags -.-> HTML[Validate HTML]
    HTML --> A11y[Accessibility check]
    A11y --> Deploy[Deploy]

Four checks can stop a release outright. One only warns. A sixth, external to all of this, runs on its own schedule and never touches the gate at all.

Why this had to be CI #

CI, continuous integration, means a fixed set of checks runs automatically on every change, before any of it ships.

A manual pass catches whatever I remember to check, the day I remember to check it. It catches nothing on the next change unless I repeat it, by hand, forever. The real failure mode isn’t missing something once. It’s having nothing in place to stop me from missing it again.

CI runs the same checks, the same way, on every push, whether I’m paying attention or not. It doesn’t get tired at 11pm, and it doesn’t skip a step because the change “looked small.” That consistency is the whole value. A check I only run when I remember to isn’t a check. It’s a to-do item I’m already failing.

None of this is specific to a blog. It’s the same reasoning behind any CI/CD pipeline, whether you’re shipping a Lambda function or a set of Kubernetes manifests. Catch the regression before a human has to, and catch it the same way every time, regardless of who’s pushing or how confident they feel about the change.

The GitHub part #

The pipeline is GitHub Actions, the same one from From Ansible to GitHub Actions + Terraform. A push to main triggers a workflow that runs a single job on one of GitHub’s hosted runners, five steps start to finish:

flowchart TD
    Checkout[Check out the repo] --> Build[Build the site]
    Build --> QA[Run each QA check]
    QA --> Sync[Sync to S3]
    Sync --> Invalidate[Invalidate the CDN cache]

The gating isn’t a feature I turned on. It’s how a job already works. Each step has to succeed before the next one runs. A failing check stops the job right there, and the deploy step never runs. The broken version never reaches S3. I didn’t bolt a gatekeeping system onto the pipeline. I just wrote a deploy.yml and configured the steps I needed, in order. Each check is a small Python script that exits non-zero when something’s wrong. Here’s an example, with the real bucket names and paths swapped out:

name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6

      - name: Build site
        run: hugo --minify

      - name: Check links and images
        run: python3 check_links.py public

      - name: Check redirects
        run: python3 check_redirects.py public

      - name: Check tags
        run: python3 check_tags.py content
        continue-on-error: true

      - name: Validate HTML
        run: html5validator --root public

      - name: Accessibility check
        run: python3 check_accessibility.py public

      - name: Deploy
        run: ./deploy.sh

Every step after Build site is just a command that exits non-zero when something’s wrong. That’s the whole trick.

GitHub shows me that same stop point without any extra tooling, too. A red X on the commit, a failed run in the Actions tab, a job summary that shows exactly which step broke. I don’t have to go looking for why a deploy didn’t happen. GitHub already tells me.

That’s the part worth taking even if you’re not running a static site, or not on GitHub at all. You probably don’t need a dedicated quality gate product. You need your CI job to run the right commands and treat a non-zero exit code as the failure it is. Every CI platform I’ve used already works this way. Gating isn’t a premium feature anyone has to buy.


Verify the check actually works #

The three bugs took minutes to fix. What mattered was turning the two checks that found them into a permanent step, so the next version of the same bug fails a build instead of reaching a reader.

A portfolio site’s whole job is to make a good first impression. Dead links and broken images undercut that immediately, and they’re the kind of thing a visitor notices even if they can’t say why the page feels off. Invalid HTML isn’t just a technicality either. Browsers patch over malformed markup in inconsistent, undocumented ways, so a page can look fine in whatever browser I tested it in and break somewhere else, for someone I’ll never hear from.

One script walks every page the build produces and confirms every internal link and image reference resolves to something real, not just to a page that technically exists but renders nothing. The other runs the same open-source validator behind the public W3C HTML checker, packaged to run locally instead of hitting someone else’s server for every page.

Neither went live until I proved it worked. I broke a link on purpose and confirmed the checker caught it, then reintroduced the exact invalid HTML and confirmed the validator caught that too. That step is easy to skip, since a check that imports cleanly and runs without errors looks finished. It isn’t. A check that’s never been shown to catch the bug it was written for isn’t a check. It’s a guess that happens to run in CI.


When to warn and when to block #

Once those two were in place, I kept asking what else a real QA pass would catch that I’d just gotten lucky on. That turned into several more checks, and the harder question wasn’t what to check. It was what should actually block a deploy.

A broken link or invalid HTML is real breakage. A legacy URL losing its redirect target is real breakage too, just quieter. Someone’s old bookmark goes from a working redirect to a dead page, with nothing to notice. Both block.

An unregistered content tag is a different kind of problem. Nothing breaks. The page ships, and a piece of metadata falls back to a default instead of its intended value. That’s worth a warning, not a blocked deploy, so the check reports and moves on.

Matching the response to the actual blast radius, not just the existence of a problem, is the part that’s easy to skip and worth doing anyway.


A check that can’t run in CI isn’t protecting anything #

Screen readers, keyboard navigation, anyone who isn’t browsing with a mouse and a monitor set up exactly like mine, all depend on markup that actually describes what’s on the page. An image with no accessible name is invisible to that entire category of visitor, and nothing about how the page looks warns me it’s there. That’s the bug class this check exists for.

The accessibility check seemed done the moment it passed on my laptop, right up until it hit CI and failed outright on the first two real deploys. A GitHub-hosted runner isn’t the same environment as a laptop. Different user permissions, no display, a browser launched with none of the defaults my own machine quietly provides. A check that depends on a headless browser can hit that difference in ways that have nothing to do with what it’s actually testing. Getting it running in CI at all took real debugging, separate from whatever it would eventually find once it did.

A check that only works where you happen to be testing isn’t a gate. It’s a demo. CI is the environment that matters, it’s the one standing between a change and production, and it’s the one I’d never actually watched the check run in until I made myself watch it fail there first.


Don’t gate your own release on someone else’s uptime #

The last check walks every external link on the site. It doesn’t run on deploy. It runs on its own weekly schedule and never fails a build, on purpose. A dead link on someone else’s page has nothing to do with anything I shipped, and holding my deploy hostage to a third party’s uptime trades a real cost for no real benefit. Keeping that check from crying wolf on the same handful of flaky domains every week is its own, longer story. That’s for later.


Find it once, fix for good #

Two dead internal links. Five broken image references. One page shipping invalid HTML. An accessibility check that passed on my laptop and failed twice in CI before I found out why. That’s the actual list of bugs this pipeline has caught, and it’s not long. Every item on it would still be live right now if I’d only found it once, by hand, and never checked again.

That’s the real lesson, more than any single check. Find a bug once, then build something that keeps finding it, DRY (don’t repeat yourself) applied to QA instead of code. The specific tools don’t matter if you’re not running a static site. The decision behind each one does: block on real breakage, warn on cosmetic drift, verify a check actually runs where it matters, never let your release depend on someone else’s uptime.

If you maintain anything on your own, start with the one thing that’s already bitten you. Write a check for it, verify the check catches it, then put it in whatever CI you’re already running.

None of this came from a best-practices guide or a framework’s defaults. Every decision here came from a real bug that actually happened.