Open source · stable v1 release

Catch accidental noindex before it ships.

Check the production homepage response, robots meta, X-Robots-Tag, canonical, and robots.txt policy in GitHub Actions. Optionally compare synthetic search-crawler responses when a CDN or WAF may disagree with the public rule.

Copy one workflow

Pin the stable v1 release.

The Action writes a job summary and JSON outputs. Homepage indexability inspection is on by default; the optional edge check sends three synthetic homepage requests. It needs only a public website URL—no ActableSite account, package token, or repository secret.

Review source and use the Action

MIT licensed · source and tests are public

name: Production indexability

on:
  pull_request:
  schedule:
    - cron: "17 6 * * 1"

jobs:
  indexability:
    runs-on: ubuntu-latest
    steps:
      - uses: unitedideas/actablesite-check@v1
        with:
          website: example.com
          check-homepage-indexability: "true"
          fail-on-noindex: "true"
          fail-on-blocked: "true"
          check-edge-responses: "true"
          fail-on-restricted: "false"

Choose which production drift should block a deploy.

Inspect indexabilitycheck-homepage-indexability is on by default. It records HTTP status, robots meta, X-Robots-Tag, canonical, and whether noindex was found.
Block noindexSet fail-on-noindex: "true" to fail when the returned robots meta or X-Robots-Tag contains noindex. A missing canonical is reported but does not fail the job.
Observe policySet fail-on-blocked: "false". The workflow stays green and records the eight-token robots result in the job summary.
EnforceSet fail-on-blocked: "true". Any blocked checked token fails the step, which is useful when your publishing policy requires all eight tokens to remain allowed.
Compare the edgeSet check-edge-responses: "true" to record synthetic OAI-SearchBot, Claude-SearchBot, and PerplexityBot homepage responses. Use fail-on-restricted only when a 401, 403, or 429 should fail the job.
Read outputsUse noindex-found, indexability-result, allowed-count, blocked-count, or the compact result JSON in later steps.

Enforcement is intentionally literal: it fails only for the enabled policy or response condition. Synthetic responses do not authenticate a provider crawler or identify the blocking layer. Run the same policy-versus-edge comparison in the browser before choosing a failure mode.

Choose the operating model

Run the check in GitHub—or have it monitored externally.

The policy question is the same. The operational burden is not. Keep the free Action when the repository is the right control point; use Crawler Watch when you want confirmed email alerts without maintaining a workflow.

Free · you operate it

GitHub Action

Run on pull requests or a schedule. Results live in GitHub, and your team owns the workflow, cadence, failure mode, and notifications.

  • Homepage status, noindex, canonical, and eight robots.txt tokens
  • Stable v1 release
  • No package token or secret
Use the free Action
Managed · $9/month

Crawler Watch

External 15-minute checks for one public site. A changed state must appear twice before the checkout email receives an alert.

  • Homepage indexability, policy, synthetic response, sitemap, and llms.txt state
  • Baseline, confirmed-change, and recovery emails
  • No repository, plugin, dashboard, or site password
Monitor one site — $9/monthRenews monthly until canceled through Stripe.

What the check covers—and what it cannot prove.

01 · Indexability

Returned production state

Reads homepage status, robots meta, X-Robots-Tag, and canonical from the raw HTML and headers returned to the workflow runner.

02 · Policy

Eight named tokens

Applies robots precedence to OAI-SearchBot, GPTBot, OAI-AdsBot, Claude-SearchBot, ClaudeBot, Claude-User, PerplexityBot, and Google-Extended.

03 · Boundaries

No browser rendering

The check does not run browser JavaScript. Synthetic responses do not authenticate a provider crawler or prove indexing, citation, ranking, recommendation, or traffic.

Verify production state before you enforce it.

  1. 01

    Run the free browser checker against the production hostname and inspect each crawler token by purpose.

  2. 02

    Add the workflow in observe mode so the first run cannot surprise a deployment.

  3. 03

    Enable enforcement only if every blocked token represents a real policy failure for your site.

Check the current production rules first