Changeset cfe1824 in Klonkt

Timestamp:

07/01/2026 12:57:32 AM (2 months ago)

Author:

Robin Genis <roboburr@…>

Branches:

main

Children:

f0807cc

Parents:

e1c4f3e

Message:

fix(federation): serialize featured-pin resync per site (4fk)

resyncFeaturedPins() runs a long Remove-all -> wait 5s -> Add-in-order
sequence and was fired fire-and-forget from /save. Two concurrent saves on
the same site interleaved their Remove/Add sequences and scrambled the
StatusPin order on Mastodon. Wrap it in a per-site serializer: a resync
already in flight coalesces later requests into one rerun after it finishes
(accumulating their extra unpins), so saves never interleave and rapid saves
don't pile up N full resyncs. Renamed the worker to doResyncFeaturedPins();
the exported resyncFeaturedPins() is now the serializing wrapper.

• src/services/ActivityPubService.js — per-site resync serializer + coalescing

Co-Authored-By: Claude <noreply@…>

Files:

• .beads/issues.jsonl (modified) (1 diff)
• src/services/ActivityPubService.js (modified) (1 diff)

.beads/issues.jsonl

@@ -11 +11 @@
 {"_type":"issue","id":"prutfolio-src-5qp","title":"Premium anti-bypass hardening (demotiveren) — lagen 1+2+4","description":"Doel (Robin 2026-06-21): bypassen van de premium-gate DEMOTIVEREN, niet 100% voorkomen (kan niet bij open self-host-code). Economie = $10 \u003c moeite-om-te-kraken (Duo-logica) is de hoofdlever; prijs laag houden.\n\nAKKOORD — implementeren:\n- LAAG 1 (HEB JE AL): Ed25519-getekend entitlement-token, OFFLINE geverifieerd met publieke sleutel. Niemand kan een geldig token vervalsen (geen private key). Kost 0 calls bij gebruik -\u003e schaalt gratis. Evt. pubkey bundelen i.p.v. /pubkey-fetch = nul server-contact op verify-pad.\n- LAAG 2: verspreide/inline checks i.p.v. één premiumUnlocked()-chokepoint -\u003e één 'return true'-patch unlockt niet meer alles; kraker moet ~10 plekken vinden.\n- LAAG 4: integriteits-zelfcheck (hash van gating-bestanden) + permanente 'niet-gelicenseerd'-banner/zachte degradatie bij geforceerd premium zonder geldig token.\n- (bonus) obfuscatie/minify van de gate + token-verify in de GEPUBLICEERDE build.\n- NETWERK-LEVER (al voor Cirkels): cracked instance = verkeerde proto = ongeldige handtekeningen = doet niet mee aan het echte netwerk. Geldt alleen voor netwerk-features.\n\nGESCHRAPT:\n- LAAG 5 (server-afhankelijke feature / phone-home): schaalt mee met GEBRUIK, niet met koppelingen -\u003e bij 1M users continu verkeer + 24/7-beschikbaarheid verplicht + centraal storingspunt + breekt 'no phone-home'. Niet de moeite voor een $10 massa-product.\n\nWAARSCHUWING: niet overinvesteren in DRM -\u003e raakt eerder eerlijke self-hosters (false positives/support) dan krakers, en ondermijnt de gratis-\u0026-open funnel. Volgorde: na Patreon Fase 0 + GitHub-migratie (prutfolio-src-aaj). Context: memory project_klonkt_monetization.","status":"open","priority":2,"issue_type":"feature","owner":"roboburr@gmail.com","created_at":"2026-06-21T23:01:36Z","created_by":"roboburr","updated_at":"2026-06-21T23:01:36Z","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"_type":"issue","id":"prutfolio-src-7cz","title":"Discovery: how Klonkt users find each other (klonkt.com hub + discoverable + FoF)","description":"Turn klonkt.com from a static salespage into the network's front door WITHOUT\nbecoming a content host. Decided direction: Form B (a map/lens over the network),\nNOT Form A (a multi-user signup server — that takes on the host/moderation/liability\nrole we deliberately avoided; that path is managed hosting, a separate business\ndecision, not this).\n\n## What\n- A `discoverable` per-site toggle (Beheer) that sets `discoverable: true/false` on\n  the site's ActivityPub actor (AS2/Mastodon-standard property, already declared in\n  AP_CONTEXT). Opt-in = clean GDPR consent basis. Honored locally by other instances.\n- Phase 1: opt-in artist directory on klonkt.com — pointers only (avatar, name, genre,\n  link). No accounts, no hosted content.\n- Phase 2: network feed — a Klonkt \"network\" actor that FOLLOWS the opt-in artists;\n  their public posts land in its ap_timeline and render as a browsable \"what's\n  happening on Klonkt\" feed. Reuses existing AP machinery (inbox, follow, timeline\n  cache, Delete/Tombstone handling, offline-instance handling) — little new code.\n- Optional: an AP relay so new instances can plug into the public firehose (cold-start),\n  and/or a single-user showcase Klonkt instance next to the salespage as a live demo.\n\n## Why (discovery)\nSingle-user-per-server (our model) has no local timeline, so discovery is inherently\nharder than Mastodon. Layered approach: (0) plain fediverse, (1) friend-of-a-friend\nover the Cirkel graph, (2) auto-detect fellow Klonkt instances via NodeInfo, (3)\ngenre/hashtag browse, (4) this central opt-in on-ramp for cold-start. This issue = layer 4.\n\n## Invariants\n- No auto-follow / auto-boost — discovery = suggestions the user clicks, never an\n  automatic outward action (the no-auto-fediverse rule).\n- Gracefully degrade — a lonely instance still works; discovery is a plus.\n- Opt-in per site (`discoverable`), honored locally; not-discoverable != private\n  (an AP actor is always followable if you know the handle).\n- Scale-safe: cache NodeInfo, use the SSRF guard (safeFetch), prune.\n\n## Maintenance / GDPR (assessed light)\n- Tech: one more pm2 service in the klonkt-stats pattern; reuses the AP stack. Low.\n- GDPR: only public, self-published, opt-in content; we are a cache/lens, not the\n  origin; federated Delete/Tombstone already drops cached posts (erasure largely\n  automatic). Need: a delist/opt-out path, data minimization + pruning, a\n  privacy-policy paragraph (klonkt.com/privacy exists), a takedown contact\n  (info@robingenis.com). Materially lighter than Form A.\n- Moderation: delist a spammy instance = drop the follow (remove a pointer), reversible.\n\n## Suggested order\n1. `discoverable` toggle (Beheer -\u003e actor). Cheap, standalone.\n2. \"Discover: neighbours of your Cirkel\" strip on /cirkel (layer 1, friend-of-a-friend)\n   that honors `discoverable`. Most \"aha\" per effort, fully decentralized, no klonkt.com.\n3. klonkt.com opt-in directory (pointers-only).\n4. klonkt.com network feed (network actor follows opt-in artists) + optional relay.\n","status":"open","priority":3,"issue_type":"feature","owner":"roboburr@gmail.com","created_at":"2026-07-01T00:34:33Z","created_by":"roboburr","updated_at":"2026-07-01T00:34:33Z","labels":["discovery","fediverse"],"dependency_count":0,"dependent_count":0,"comment_count":0}
-{"_type":"issue","id":"prutfolio-src-4fk","title":"Fediverse: serialize pin-resync per site (parallelle /save race)","description":"Hardening-review: twee gelijktijdige /save-calls die pin-rank wijzigen draaien beide resyncFeaturedPins (Remove-all -\u003e 5s -\u003e Add) interleaved -\u003e niet-deterministische StatusPin-volgorde op Mastodon. Serialiseer per slug (in-process Map\u003cslug,Promise\u003e-chain) of snapshot de pins onder de commit-transactie.","status":"open","priority":3,"issue_type":"task","owner":"roboburr@gmail.com","created_at":"2026-06-25T09:53:21Z","created_by":"roboburr","updated_at":"2026-06-25T09:53:21Z","dependency_count":0,"dependent_count":0,"comment_count":0}
+{"_type":"issue","id":"prutfolio-src-4fk","title":"Fediverse: serialize pin-resync per site (parallelle /save race)","description":"Hardening-review: twee gelijktijdige /save-calls die pin-rank wijzigen draaien beide resyncFeaturedPins (Remove-all -\u003e 5s -\u003e Add) interleaved -\u003e niet-deterministische StatusPin-volgorde op Mastodon. Serialiseer per slug (in-process Map\u003cslug,Promise\u003e-chain) of snapshot de pins onder de commit-transactie.","status":"in_progress","priority":3,"issue_type":"task","assignee":"roboburr","owner":"roboburr@gmail.com","created_at":"2026-06-25T09:53:21Z","created_by":"roboburr","updated_at":"2026-07-01T00:55:28Z","started_at":"2026-07-01T00:55:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"_type":"issue","id":"prutfolio-src-4lq","title":"Fediverse: block-handling completeness follow-up","description":"Hardening follow-up. Specifics are intentionally kept in the maintainer's private notes, not in this public tracker. Priority medium/low; the critical fediverse issues (SSRF, stored-XSS, cross-actor delete) are already fixed.","status":"open","priority":3,"issue_type":"task","owner":"roboburr@gmail.com","created_at":"2026-06-25T09:53:19Z","created_by":"roboburr","updated_at":"2026-06-25T19:29:08Z","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"_type":"issue","id":"prutfolio-src-lqx","title":"Fediverse: HTTP-signature hardening follow-up","description":"Hardening follow-up. Specifics are intentionally kept in the maintainer's private notes, not in this public tracker. Priority medium/low; the critical fediverse issues (SSRF, stored-XSS, cross-actor delete) are already fixed.","status":"open","priority":3,"issue_type":"task","owner":"roboburr@gmail.com","created_at":"2026-06-25T09:53:18Z","created_by":"roboburr","updated_at":"2026-06-25T19:29:07Z","dependency_count":0,"dependent_count":0,"comment_count":0}

src/services/ActivityPubService.js

@@ -1062 +1062 @@
 // then ADD in rank-DESCENDING order (rank 1 added LAST → newest StatusPin → shown first,
 // because Mastodon displays pins newest-first). `alsoRemove` = ids to unpin too.
-export async function resyncFeaturedPins(site, alsoRemove = []) {
+// Serialize pin-resyncs per site: two concurrent /save calls would otherwise interleave
+// their Remove -> wait -> Add sequences and scramble the StatusPin order on Mastodon. A
+// resync already in flight for a site coalesces later requests into ONE rerun after it
+// finishes (accumulating their extra unpins), so rapid saves don't pile up N full resyncs.
+const _pinResync = new Map(); // slug -> { promise, pending, pendingRemove:Set, site }
+export function resyncFeaturedPins(site, alsoRemove = []) {
+  if (!site || !site.slug) return Promise.resolve();
+  const slug = site.slug;
+  const running = _pinResync.get(slug);
+  if (running) {
+    running.pending = true;
+    running.site = site; // use the latest site object on the rerun
+    for (const id of alsoRemove) running.pendingRemove.add(id);
+    return running.promise;
+  }
+  const state = { promise: null, pending: false, pendingRemove: new Set(), site };
+  state.promise = (async () => {
+    let extra = alsoRemove;
+    for (;;) {
+      try { await doResyncFeaturedPins(state.site, extra); }
+      catch (e) { console.warn('[AP] pin resync failed:', e.message); }
+      if (!state.pending) break;
+      state.pending = false;
+      extra = [...state.pendingRemove];
+      state.pendingRemove = new Set();
+    }
+    _pinResync.delete(slug);
+  })();
+  _pinResync.set(slug, state);
+  return state.promise;
+}
+
+// The actual resync work — do NOT call directly; go through resyncFeaturedPins() above so
+// it stays serialized per site.
+async function doResyncFeaturedPins(site, alsoRemove = []) {
   const base = (process.env.PUBLIC_BASE_URL || '').replace(/\/+$/, '');
   if (!base || !site || !site.slug) return;