Changeset 0f48411 in Klonkt

Timestamp:

07/31/2026 10:52:04 AM (6 weeks ago)

Author:

Robin <roboburr@…>

Branches:

main

Children:

cae8ded

Parents:

2a17f0a

Message:

Statistieken tellen lezers, geen servers

Robins vraag (31-7) legde een lek bloot: bij een boost halen alle
fediverse-instances de human-URL op voor hun linkkaart (http.rb/
Mastodon, Pleroma, Misskey, ...), en die UA's stonden niet in de
bot-regex: elke instance telde als post-view, pageview EN unieke
bezoeker. Twee lagen dicht: de fediverse-server-UA's in de regex, en
principieel: elk verzoek met een Signature-header of een
activity/ld+json-Accept is per definitie een server en telt nooit,
wat de UA ook beweert.

Changed files:
src/services/StatsService.js

• BOT_RE + mastodon/pleroma/akkoma/misskey/firefish/friendica/ hubzilla/lemmy/pixelfed/peertube/gotosocial/honk/http.rb/activitypub
• isBot: Signature-header of AP-Accept = bot

New file:
test/stats-bots.test.js

• browser telt; preview-fetchers niet; signed nooit; AP-Accept nooit

-robo
Co-Authored-By: Claude Opus 4.8 <noreply@…>

Files:

• src/services/StatsService.js (modified) (1 diff)
• test/stats-bots.test.js (added)

src/services/StatsService.js

@@ -47 +47 @@
 // Skip known bots/crawlers + link-preview fetchers + scripts so they don't inflate
 // view/visitor-day counts. Empty UA = almost always automated.
-const BOT_RE = /bot|crawl|spider|slurp|mediapartners|bingpreview|facebookexternalhit|whatsapp|telegram|discord|twitter|linkedin|embedly|pinterest|redditbot|applebot|petalbot|yandex|baidu|duckduckbot|semrush|ahrefs|mj12|dotbot|uptimerobot|pingdom|statuscake|headless|lighthouse|gptbot|claude|ccbot|perplexity|bytespider|amazonbot|googleother|google-read-aloud|python-requests|scrapy|curl|wget|axios|node-fetch|go-http|java\/|okhttp|libwww|httpclient/i;
+const BOT_RE = /bot|crawl|spider|slurp|mediapartners|bingpreview|facebookexternalhit|whatsapp|telegram|discord|twitter|linkedin|embedly|pinterest|redditbot|applebot|petalbot|yandex|baidu|duckduckbot|semrush|ahrefs|mj12|dotbot|uptimerobot|pingdom|statuscake|headless|lighthouse|gptbot|claude|ccbot|perplexity|bytespider|amazonbot|googleother|google-read-aloud|python-requests|scrapy|curl|wget|axios|node-fetch|go-http|java\/|okhttp|libwww|httpclient|mastodon|pleroma|akkoma|misskey|calckey|firefish|friendica|hubzilla|lemmy|pixelfed|peertube|gotosocial|honk|http\.rb|activitypub/i;
 function isBot(req) {
   const ua = (req && req.headers && req.headers['user-agent']) || '';
   if (!ua) return true;          // empty UA = script/bot
+  // A signed request or an ActivityPub content-negotiation is BY DEFINITION a
+  // server fetching, not a reader (Robins vraag, 31-7): one boosted post made
+  // every fediverse instance's link-preview fetch count as a unique visitor.
+  const h = (req && req.headers) || {};
+  if (h['signature']) return true;
+  if (/application\/(activity|ld)\+json/i.test(String(h['accept'] || ''))) return true;
   return BOT_RE.test(ua);
 }