# RFC-2 — CivicIPFS Orchestrator Core **Status:** Draft **Version:** 0.3 **Depends on:** RFC-1 (CivicIPFS Addon Specification) v0.3 **Changes from 0.2:** Authentication section 8.1 corrected — channel Ed25519 key verified from source. Go verification path specified precisely. **Kubo version referenced:** 0.41.0 **Author:** CivicIPFS Project **Repository:** TBD **Note:** The inter-orchestrator protocol, challenge-proof mechanism, and reputation model are specified in RFC-3 (Civic Network Protocol). --- ## 1. Purpose and Scope This document specifies the CivicIPFS Orchestrator — the independent service that sits between the addon (RFC-1) and the IPFS network. RFC-2 covers: - What the orchestrator is and what it is not - Language and tooling decisions - Architecture and internal components - The Kubo integration — the only path to IPFS - The Subscription API implementation - Signed receipts and the ReceiptIssuer component - Service offers and the ServiceOfferManager component - State storage - Operational requirements - Configuration - Deferred items specific to the core The inter-orchestrator protocol, challenge-proof mechanism, peer discovery, and reputation model are specified in RFC-3. --- ## 2. What the Orchestrator Is The orchestrator is a standalone service, independent of Hubzilla, that: - Accepts subscriptions from CivicIPFS addon instances - Manages a Kubo IPFS node on behalf of its subscribers - Pins content, maintains commitments, issues signed receipts - Publishes service offers with operator-defined pricing - Participates in the Civic Network Protocol (RFC-3) for challenge-proof exchange and reputation The orchestrator is infrastructure. It is operated by a person or organisation who has committed storage and bandwidth to the civic network and chosen to make that commitment available to others as a paid or contributed service. **One orchestrator serves many addon subscribers.** A single operator may serve subscribers from many Hubzilla instances, from one, or run their own addon and orchestrator together. **The orchestrator does not require Hubzilla.** It speaks HTTP/JSON to addons. The Hubzilla integration is the primary use case, not a technical dependency. --- ## 3. What the Orchestrator Is Not - A Hubzilla module or plugin - A payment processor — payment is between subscriber and operator, entirely outside this system, permanently - A certificate authority - A registry of other orchestrators - A public HTTP gateway for arbitrary CID resolution --- ## 4. Language and Tooling ### 4.1 Go — the runtime language The orchestrator is written in Go throughout. This is not arbitrary: - Kubo is written in Go - The Kubo RPC client library (`github.com/ipfs/kubo/client/rpc`) is Go — typed, maintained by the same ecosystem, no reimplementation - The IPFS ecosystem's primary tooling is Go - Go produces a single static binary with no runtime dependencies, which simplifies deployment significantly for small operators Every component of the orchestrator — API servers, KuboClient, PinManager, ReceiptIssuer, ServiceOfferManager, and the components specified in RFC-3 — is Go. ### 4.2 Python — operational tooling only Python (with venv for environment isolation) is used for tooling that sits outside the runtime: - Deployment and provisioning scripts - Configuration generators and validators - Operational health-check scripts - Database migration utilities - Analysis tooling over the SQLite state database The boundary is clean: Go runs the service, Python operates and maintains it. Python code never runs in the request path. ### 4.3 No PHP The orchestrator has no PHP dependency. The addon (RFC-1) is PHP because it lives inside Hubzilla. The interface between addon and orchestrator is HTTP/JSON — language-agnostic by design. --- ## 5. Architecture ``` ┌──────────────────────────────────────────────────────────────┐ │ Orchestrator (Go binary) │ │ │ │ ┌─────────────────────┐ ┌───────────────────────────────┐ │ │ │ Subscription API │ │ Core Components │ │ │ │ HTTP/JSON │──▶│ │ │ │ │ IPv4 + IPv6 │ │ SubscriptionManager │ │ │ │ port 8421 │ │ PinManager │ │ │ │ Auth: Zot6 sig │ │ ReceiptIssuer │ │ │ └─────────────────────┘ │ ServiceOfferManager │ │ │ │ │ │ │ ┌─────────────────────┐ │ [Challenger] ─────────────────── RFC-3 │ │ Civic Network API │ │ [ReputationManager] ────────────── RFC-3 │ │ HTTP/JSON │◀──│ │ │ │ │ IPv4 + IPv6 │ └───────────────┬───────────────┘ │ │ │ port 8422 │ │ │ │ └─────────────────────┘ ┌───────────────▼───────────────┐ │ │ │ KuboClient │ │ │ │ sole IPFS interface │ │ │ │ github.com/ipfs/kubo/ │ │ │ │ client/rpc │ │ │ └───────────────┬───────────────┘ │ └────────────────────────────────────────────┼─────────────────┘ │ localhost:5001 │ Kubo RPC API v0 ┌───────────▼────────────┐ │ Kubo IPFS Node │ │ (separate process) │ └────────────────────────┘ ``` The orchestrator runs as a separate long-running process from Kubo. Both are managed by a process supervisor. The orchestrator manages Kubo; Kubo manages the IPFS network. Components marked RFC-3 are present in the binary but their network behaviour is specified in RFC-3. --- ## 6. IPv6 Both HTTP servers — the Subscription API (port 8421) and the Civic Network API (port 8422) — bind on both IPv4 and IPv6 from the start. This is not a future option; it is the default binding. ```go // Both servers bind dual-stack ln, err := net.Listen("tcp", "[::]:8421") ``` The reason is operational, not academic. Many civic operators — community organisations, cooperatives, small public-interest infrastructure projects — will be behind NAT on residential or small-business connections without static IPv4 addresses. IPv6 eliminates the NAT problem entirely: every orchestrator gets a globally routable address without port forwarding, without dynamic DNS workarounds, without depending on a relay service. Kubo already supports IPv6 natively in its swarm listeners. The orchestrator follows the same model. An operator without IPv6 is not excluded — the TCP dual-stack binding degrades gracefully to IPv4-only if the host has no IPv6 address. --- ## 7. The KuboClient Component This is the only place in the orchestrator that knows Kubo exists. All other components call `KuboClient` methods through a defined Go interface. No other component imports the Kubo RPC package directly. ```go type IPFSClient interface { PinAdd(ctx context.Context, cid string) error PinRemove(ctx context.Context, cid string) error PinList(ctx context.Context) ([]string, error) BlockStat(ctx context.Context, cid string) (int64, error) CatRange(ctx context.Context, cid string, offset, length int64) ([]byte, error) RepoStat(ctx context.Context) (StorageStat, error) NodeID(ctx context.Context) (string, error) Version(ctx context.Context) (string, error) } ``` `KuboClient` is the production implementation. A mock implementation of the same interface is used in tests. This is the isolation boundary that makes the orchestrator testable without a running Kubo node. ### 7.1 Kubo API endpoints used ``` POST /api/v0/pin/add?arg={cid}&recursive=true Pin a CID. Blocks until content is fetched and pinned. Used by: PinManager.CommitPin() POST /api/v0/pin/rm?arg={cid} Remove a pin. Used by: PinManager.ReleasePin() POST /api/v0/pin/ls?type=recursive List all recursive pins. Used by: PinManager.AuditPins() — scheduled integrity check POST /api/v0/block/stat?arg={cid} Check if a block is locally available without fetching. Returns size in bytes. Used by: Challenger.VerifyLocalPossession() [RFC-3] POST /api/v0/cat?arg={cid}&offset={offset}&length={length} Retrieve a byte range of content. Used by: Challenger.RespondToChallenge() [RFC-3] POST /api/v0/repo/stat Returns storage used and available. Used by: ServiceOfferManager.CurrentCapacity() PinManager.QuotaCheck() POST /api/v0/id Returns node PeerID and addresses. Used by: startup health check, orchestrator_id derivation POST /api/v0/version Returns Kubo version string. Used by: KuboClient version gate at startup ``` ### 7.2 Kubo version gate At startup, the orchestrator calls `/api/v0/version` and logs the result. If the version is below the minimum supported version — to be established during first operational deployment — a warning is logged. The orchestrator does not refuse to start; the warning is the signal. ### 7.3 Kubo is not exposed The Kubo RPC API must remain on localhost only. The orchestrator is the sole caller. The Subscription API and Civic Network API are the only public-facing surfaces of the orchestrator. --- ## 8. The Subscription API (Implementation Side) RFC-1 section 6 defines this API from the addon's perspective. This section defines the orchestrator's implementation of the same endpoints. ### 8.1 Authentication Every request carries: ``` X-CivicIPFS-Channel: {channel_hash} X-CivicIPFS-Sig: {Ed25519 signature of canonical request body} X-CivicIPFS-Version: 1 ``` **Key type — verified from Hubzilla source:** Hubzilla channels have two keypairs. The primary RSA-4096 key is used for Zot6 protocol signing. The secondary Ed25519 key (`channel_eprvkey` / `channel_epubkey`, generated via `sodium_crypto_sign_keypair()`) is what the addon uses to sign requests. The Zotinfo endpoint (`/.well-known/zot-info`) advertises the Ed25519 public key in the `ed25519_key` field (Multibase-encoded). **Verified:** `include/channel.php` lines 243–245, `Zotlabs/Lib/Libzot.php` lines 2905–2907. The orchestrator verifies each request by: 1. Extracting `channel_hash` from `X-CivicIPFS-Channel` 2. Looking up the channel's hub URL from the local subscriptions table 3. Fetching the channel's Ed25519 public key from the Zotinfo endpoint: `GET {hub_url}/.well-known/zot-info?channel_address={channel_address}` → field `ed25519_key` (Multibase-encoded, decode to 32 raw bytes) 4. Verifying the Ed25519 signature using `golang.org/x/crypto/ed25519.Verify(pubkey, body_bytes, sig_bytes)` where `sig_bytes` is `base64.StdDecoding.DecodeString(header_value)` 5. Caching the decoded public key locally keyed by `channel_hash`, refreshed on signature failure or after 24 hours The orchestrator never issues credentials. The channel's existing Ed25519 identity is the credential. A channel that migrates to a new Hubzilla hub retains its subscription because `channel_hash`, `channel_eprvkey`, and `channel_epubkey` are all preserved through Hubzilla's nomadic migration export/import bundle. ### 8.2 Subscription registration First call from any new addon — establishes the hub mapping: ```json POST /api/v1/register Body: { "channel_hash": "...", "hub_url": "https://hubzilla.example.org", "hub_realm": "CIVICUS_IPFS" } Response: { "registered": true, "orchestrator_pubkey": "...", "accepted_realms": ["CIVICUS_IPFS"] } ``` The `hub_realm` field is validated against `accepted_realms` in the orchestrator config. A registration from `RED_GLOBAL` to an orchestrator configured for `CIVICUS_IPFS` is rejected: ```json { "error": "realm_mismatch", "message": "This orchestrator accepts realm CIVICUS_IPFS. Received RED_GLOBAL.", "accepted_realms": ["CIVICUS_IPFS"] } ``` ### 8.3 Endpoint implementations ``` GET /api/v1/info Response: { "orchestrator_id": "{PeerID from /api/v0/id}", "version": "0.2", "kubo_version": "{from /api/v0/version}", "cid_version": 1, "storage_used_gb": float, "storage_total_gb": float, "storage_avail_gb": float, "reputation_score": float | null, "accepted_realms": ["CIVICUS_IPFS"], "pubkey": "..." } GET /api/v1/offers Response: array of ServiceOffer objects (see section 9) POST /api/v1/pin Body: { "cid": "baf...", "tier": "standard", "duration_days": 365 } Steps: 1. Validate CIDv1 format — CIDv0 rejected with explanation 2. Check per-channel quota for this channel_hash 3. Check storage availability via /api/v0/repo/stat 4. Check total committed quota does not exceed capacity 5. Call /api/v0/pin/add — blocks until pinned 6. Issue signed receipt (section 10) 7. Record pin in state database Response: { "cid": "...", "receipt": {...}, "status": "pinned" } GET /api/v1/pin/{cid} Response: { "cid": "...", "status": "active|expired|challenged|confirmed|failed", "pinned_at": unix_ts, "expires_at": unix_ts, "receipt": {...} } DELETE /api/v1/pin/{cid} Steps: 1. Verify requesting channel_hash owns this pin 2. Call /api/v0/pin/rm 3. Update state database Response: { "unpinned": true } GET /api/v1/pins Response: array of pin state objects for the authenticated channel_hash GET /api/v1/reputation Response: { "score": float | null, "challenges_received": int, "challenges_passed": int, "challenges_failed": int, "last_challenge_at": unix_ts | null } ``` --- ## 9. The ServiceOfferManager Component The operator configures their service offer. The orchestrator publishes it. Nothing is enforced or validated by the orchestrator except structure. ### 9.1 Offer structure ```json { "schema": 1, "orchestrator_id": "...", "orchestrator_url": "https://...", "accepted_realms": ["CIVICUS_IPFS"], "tiers": [ { "name": "standard", "price_per_gb_month": 0.0, "price_per_gb_egress": 0.0, "max_quota_gb": 10.0, "max_duration_days": 365, "sla_description": "Best effort. No uptime guarantee.", "payment_description": "Community contribution. No payment required." }, { "name": "committed", "price_per_gb_month": 0.0, "price_per_gb_egress": 0.0, "max_quota_gb": 100.0, "max_duration_days": 1825, "sla_description": "Monthly challenge participation. 95% uptime target.", "payment_description": "Contact operator to arrange payment." } ], "published_at": 1700000000, "sig": "..." } ``` **Prices are set entirely by the operator.** Zero or non-zero, community or commercial — the orchestrator publishes what it is configured to publish. It never validates, adjusts, or enforces pricing. **Payment description is free text.** The orchestrator does not process payment. This is permanent and non-negotiable. The offer is signed with the orchestrator's Ed25519 key. A tampered offer fails signature verification at the addon. --- ## 10. The ReceiptIssuer Component Every pin commitment is a signed receipt. The receipt is the verifiable record of what was promised, by whom, and when. It is the evidentiary artefact that the CivicIPFS project exists to produce. ### 10.1 Receipt structure ```json { "schema": 1, "cid": "bafybeig...", "channel_hash": "...", "orchestrator_id": "...", "pinned_at": 1700000000, "expires_at": 1731536000, "tier": "standard", "duration_days": 365, "sig": "..." } ``` ### 10.2 Signing `sig` is an Ed25519 signature over the canonical JSON of all other fields — keys sorted alphabetically, no whitespace, UTF-8 encoded. The orchestrator's Ed25519 private key is generated at first run and written to `identity.key_file` (see section 12). The corresponding public key is returned at subscriber registration and in `/api/v1/info`. Ed25519 is chosen because it is compact, fast to verify, widely supported, and consistent with the channel authentication scheme — the orchestrator verifies one algorithm (Ed25519) for both incoming addon requests and outgoing receipt verification. Note: this is the orchestrator's own Ed25519 key, generated independently. It is separate from the Hubzilla channel's Ed25519 key used for request authentication in section 8.1. Both are Ed25519 but they are different keys with different purposes. ### 10.3 Receipt verification by any party 1. Fetch orchestrator public key from `/api/v1/info` 2. Reconstruct canonical JSON of non-sig fields 3. Verify Ed25519 signature A receipt that fails verification is not a valid commitment. The addon stores receipts verbatim in the channel's JSON store and may re-verify at any time without network access. --- ## 11. State Storage The orchestrator uses SQLite with WAL mode. One database file. This is appropriate because the orchestrator is a single process and write concurrency is low — SQLite serialises writes, WAL allows concurrent reads without blocking writes. This differs from the addon's per-channel JSON files deliberately: the orchestrator holds operational state for a multi-subscriber service and needs SQL queries for expiry scanning, quota aggregation, and reputation calculations. JSON files would be awkward for those queries. The orchestrator's database never travels — it is always on one machine. Tables belonging to the Civic Network Protocol (challenges, peers) are defined in RFC-3 but created in the same database file. The schema is managed by versioned migrations run at startup. ```sql -- Core orchestrator tables CREATE TABLE IF NOT EXISTS subscriptions ( channel_hash TEXT PRIMARY KEY, hub_url TEXT NOT NULL, hub_realm TEXT NOT NULL, registered_at INTEGER NOT NULL, last_seen_at INTEGER, active INTEGER NOT NULL DEFAULT 1 ); CREATE TABLE IF NOT EXISTS pins ( id INTEGER PRIMARY KEY AUTOINCREMENT, channel_hash TEXT NOT NULL, cid TEXT NOT NULL, tier TEXT NOT NULL, pinned_at INTEGER NOT NULL, expires_at INTEGER, status TEXT NOT NULL, receipt_json TEXT NOT NULL, UNIQUE(channel_hash, cid) ); CREATE INDEX IF NOT EXISTS pins_expires ON pins(expires_at) WHERE status = 'active'; CREATE INDEX IF NOT EXISTS pins_channel ON pins(channel_hash); -- Schema version tracking for migrations CREATE TABLE IF NOT EXISTS schema_version ( version INTEGER PRIMARY KEY, applied_at INTEGER NOT NULL ); ``` --- ## 12. Configuration File YAML. All operational parameters live here. Nothing is hardcoded in the binary except structural constants and the canonical realm name `CIVICUS_IPFS`. ```yaml # CivicIPFS Orchestrator — configuration # RFC-2 v0.2 orchestrator: data_dir: /var/lib/civicipfs subscription_port: 8421 # Subscription API — addon-facing network_port: 8422 # Civic Network API — inter-orchestrator accepted_realms: - CIVICUS_IPFS kubo: api_url: http://localhost:5001 gateway_url: http://localhost:8080 identity: # Ed25519 key file. Generated on first run if absent. key_file: /var/lib/civicipfs/orchestrator.key service: default_quota_gb: 10 expiry_grace_days: 30 offers: - name: standard price_per_gb_month: 0.0 price_per_gb_egress: 0.0 max_quota_gb: 10.0 max_duration_days: 365 sla_description: "Best effort." payment_description: "No payment required." # Challenge and reputation parameters are in RFC-3. # They appear in this file under the 'network:' key # and are documented there. logging: level: info # debug | info | warn | error format: json output: /var/log/civicipfs/orchestrator.log ``` --- ## 13. Operational Requirements ### 13.1 Hardware minimums The dominant resource consumer is Kubo, not the orchestrator. An operator should plan for: - **RAM:** 8 GB minimum — 6 GB for Kubo, 2 GB headroom for the orchestrator and OS. Larger pinsets require more: approximately 1 GiB per 20 million pinned items for Kubo's DHT reprovider. - **CPU:** 2 cores minimum, 4 recommended. Kubo is highly parallel. - **Storage:** must be at least 20% larger than total committed quota to leave headroom for Kubo's internal overhead and temporary data during fetch operations. - **Network:** reliable uptime. Challenge failures caused by network instability degrade reputation score (RFC-3). An operator who commits to a `committed` service tier is implicitly committing to network reliability. ### 13.2 Kubo configuration The orchestrator requires Kubo to be configured with the RPC API on localhost only: ```json { "Addresses": { "API": "/ip4/127.0.0.1/tcp/5001", "Gateway": "/ip4/0.0.0.0/tcp/8080" }, "API": { "HTTPHeaders": { "Access-Control-Allow-Origin": ["http://localhost"] } } } ``` For operators who want IPv6 on the Kubo gateway: ```json { "Addresses": { "Gateway": [ "/ip4/0.0.0.0/tcp/8080", "/ip6/::/tcp/8080" ] } } ``` The Kubo gateway is optional and operator-controlled. It allows the Kubo node to serve pinned content to the wider IPFS network via HTTP. Whether to expose it is the operator's decision. ### 13.3 Process management Both Kubo and the orchestrator must be managed by a process supervisor (systemd recommended; Docker Compose acceptable) that restarts them on failure and starts them in the correct order — Kubo first, orchestrator second. A Python operational script (`scripts/health_check.py`) verifies both processes are responsive and exits non-zero if either is not, suitable for use as a systemd `ExecStartPost` check or a cron-based monitor. ### 13.4 Quota enforcement Before accepting a pin, the orchestrator checks in sequence: 1. CIDv1 format validity 2. Per-channel quota for the requesting `channel_hash` 3. Total storage available via `/api/v0/repo/stat` 4. Total quota committed across all active contracts Any check failure rejects the request with a specific error code and human-readable message. The orchestrator never accepts a commitment it cannot immediately honour. ### 13.5 Pin expiry The orchestrator runs a daily scheduled task that: 1. Identifies all pins with `expires_at < now` 2. Sets their status to `expired` 3. Notifies the subscriber via the addon (mechanism TBD — out of scope for this RFC, likely a status field the addon polls) 4. After `expiry_grace_days` (default 30), calls `/api/v0/pin/rm` and sets status to `removed` Subscribers may renew before the grace period ends. Renewal is a new `/api/v1/pin` call for the same CID — it extends the expiry and issues a new receipt. --- ## 14. Deferred Items **D1 — Attestation (Duniter or alternative).** Anchoring CIDs to an external manipulation-resistant clock. Specified in a future RFC. **D2 — WireGuard transport for peer connections.** Optional encrypted transport for established peer relationships. Specified in the same future RFC as D1. **D3 — Multi-orchestrator redundancy contracts.** Formal N-of-M co-pinning contracts between orchestrators. Currently a subscriber can subscribe to multiple orchestrators independently. A formal co-pinning protocol is deferred. **D4 — Automated quota renewal.** An automated renewal protocol between subscriber and orchestrator. Deferred. **D5 — Kubo minimum version.** To be established during first operational deployment. **D6 — TLS configuration.** Serving the API ports over TLS in production. Deferred to an operational deployment guide, which will document integration with Caddy as the recommended reverse proxy. The Python tooling directory will include a Caddyfile template. **D7 — Subscriber notification on expiry.** The mechanism by which the orchestrator notifies a subscriber that a pin is approaching or has passed expiry. Deferred — likely a field the addon polls on `/api/v1/pin/{cid}` rather than a push mechanism. --- ## 15. What RFC-3 Specifies The Civic Network Protocol — everything that happens between orchestrators: - The inter-orchestrator API endpoints (`/civic/v1/...`) - The challenge-proof message format and exchange - Byte-range selection strategy for challenges - Peer discovery (manual, static — consistent with RFC-1 design) - The ReputationManager component and scoring model - Reputation propagation rules and anti-spam constraints - The SQLite tables for challenges, reputation, and peers - The `network:` configuration section --- *End of RFC-2 draft 0.2*