Nectenda/nectenda

Watch

Fork 0

Keep the status page out of search: allow the crawl, send noindex #272

Merged

cruelacid merged 1 commit from worktree-nec-237-status-noindex into main

2026-10-07 14:09:08 +01:00

cruelacid commented

2026-10-07 13:58:53 +01:00

Owner

Task: NEC-237

Search Console reported status.nectenda.com as "Indexed, though blocked by robots.txt". Uptime Kuma answers robots.txt with Disallow: /, and every nectenda.com footer links to the status page. Google indexed the bare URL from those links without fetching the page, so it never saw a noindex.

deploy/ops/Caddyfile, status host only:

Caddy answers /robots.txt itself with Allow: /.
Every response carries X-Robots-Tag: noindex.
Everything else still goes to Kuma, unchanged.

Checked by running the block in a local Caddy against a stub upstream whose own robots.txt is Disallow. / and /api/x reach the stub with the header, and /robots.txt returns Caddy's Allow: /.

Merging does not deploy it. The ops Caddyfile reaches the host only at provisioning, so after merging it has to be copied to /srv/ops/Caddyfile by hand and Caddy reloaded.

Changelog

NONE

🤖 Generated with Claude Code

https://claude.ai/code/session_01StURdiv33xnMfE2XRyg8Lt

Task: NEC-237 Search Console reported status.nectenda.com as "Indexed, though blocked by robots.txt". Uptime Kuma answers `robots.txt` with `Disallow: /`, and every nectenda.com footer links to the status page. Google indexed the bare URL from those links without fetching the page, so it never saw a noindex. `deploy/ops/Caddyfile`, status host only: - Caddy answers `/robots.txt` itself with `Allow: /`. - Every response carries `X-Robots-Tag: noindex`. - Everything else still goes to Kuma, unchanged. Checked by running the block in a local Caddy against a stub upstream whose own robots.txt is `Disallow`. `/` and `/api/x` reach the stub with the header, and `/robots.txt` returns Caddy's `Allow: /`. **Merging does not deploy it.** The ops Caddyfile reaches the host only at provisioning, so after merging it has to be copied to `/srv/ops/Caddyfile` by hand and Caddy reloaded. ## Changelog NONE 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01StURdiv33xnMfE2XRyg8Lt

cruelacid added the

area:ops

label

2026-10-07 13:58:53 +01:00

cruelacid added 1 commit

2026-10-07 13:58:53 +01:00

Keep the status page out of search: allow the crawl, send noindex

Release note / release-note (pull_request) Successful in 20s

Details

CI / build (pull_request) Successful in 5m38s

Details

CI / e2e (pull_request) Successful in 9m31s

Details

CI / promote (pull_request) Has been skipped

Details

CI / build (push) Successful in 5m37s

Details

CI / e2e (push) Successful in 9m19s

Details

CI / promote (push) Successful in 32s

Details

f208e5f49b

Search Console reported status.nectenda.com as "Indexed, though blocked by
robots.txt". Uptime Kuma answers robots.txt with Disallow: /, every
nectenda.com footer links to the page, and Google indexes a disallowed URL
from its links alone, without ever fetching it, so it could never have seen
a noindex. Caddy now answers robots.txt for that host itself, allowing the
crawl, and adds X-Robots-Tag: noindex to every response.

Task: NEC-237

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StURdiv33xnMfE2XRyg8Lt

cruelacid merged commit f208e5f49b into main

2026-10-07 14:09:08 +01:00

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

Nectenda/nectenda!272

No description provided.

Rows
Columns