Civic IPFS — a federated storage service layer for Hubzilla
This post describes a proposed Hubzilla addon called **Civic IPFS**. It is shared at design stage, before any code exists, because the right place to start is with the community that would be most affected by it — and because the Hubzilla Association's approval is the gate through which any proposal like this should pass before it becomes a real project. If the Association identifies rational objections to the design, the design will be revised or withdrawn. This is not a project looking for permission to ignore the community; it is a project looking for the community's considered judgment.
View article
View summary
## What problem this addresses
Projects that need persistent, censorship-resistant storage currently face three options:
1. Trust a commercial pinning service (Pinata, web3.storage, and others). These are centralized, payment-dependent, and have a track record of shutting down — Infura IPFS closed in June 2023, Estuary followed. A business decision at one company can make content inaccessible.
2. Run their own IPFS node in isolation. Technically correct, but requires infrastructure expertise and provides no redundancy from the community.
3. Rely on the public IPFS network with no pinning guarantees. Content that nobody pins disappears.
There is no fourth option that is federated, community-governed, and accountable. That is the gap this addon is intended to fill.
The design is general-purpose. It does not assume any particular application domain. Community archives, cooperative platforms, local government transparency tools, mutual aid networks, and anything else that needs durable storage could use it. The name "Civic IPFS" reflects an orientation toward community benefit, not a restriction to any specific use case.
---
## What it is, technically
Civic IPFS is a Hubzilla addon that lets node operators offer IPFS pinning as a paid service to other channels and external users. The technical foundation is verified from the Hubzilla core source:
**Realm isolation.** Hubzilla's directory realm system allows a hub to operate in a named federation space separate from the default RED_GLOBAL realm. This is not a workaround — it is a supported feature. The constant `DIRECTORY_REALM` is defined as `'RED_GLOBAL'` in `boot.php`, and the function `get_directory_realm()` reads `Config::Get('system', 'directory_realm')`, falling back to that constant if the key is not set. Setting `system.directory_realm` to a different string, such as `CIVICUS`, causes the hub to federate only with other hubs carrying the same realm value. The `site` table has a `site_realm` column, and `Libzotdir::sync_directories()` uses it to filter which directory servers to sync with. Hubs in a different realm are excluded.
The civic addon would run on a separate realm from RED_GLOBAL. Nodes that do not install the addon are completely unaffected. No packets reach them, no noise enters the main social federation.
**Zot6 extension.** Hubzilla currently uses Zot protocol revision 6.0 (defined as `ZOT_REVISION = '6.0'` in `boot.php`). The Zot6 `Receiver` class dispatches incoming packets by `type` field. Known types — `purge`, `refresh`, `force_refresh`, `rekey` — go to dedicated handlers. All other types, including unknown ones, fall through to `default:` which routes to `Notify()` → `Libzot::fetch()`. This means custom packet types can be introduced by addons hooking into `Libzot::fetch()` without modifying core. Unupgraded nodes receive unknown types, ignore them gracefully, and continue operating normally.
Before any packet reaches the addon's hook, the `Receiver` has already validated an HTTP signature and verified the sender's identity against the hub registry via `Libzot::valid_hub()`. The addon receives a pre-authenticated sender. It does not need to build its own authentication layer.
**The fallback master directory** for RED_GLOBAL is defined in `boot.php` as `#^https://hub.netzgemeinde.eu`. The civic realm would need its own directory server. How that directory is bootstrapped and who operates it needs more research.
**A `realm_token`** mechanism exists in `Libzotdir::sync_directories()` — it reads `Config::Get('system', 'realm_token')` and appends it as a `&t=token` parameter to directory sync requests. This provides a lightweight shared-secret admission control on top of the realm name string alone. Whether to use this mechanism, and how to distribute the token to new members, needs more research.
**The IPFS backend** is a standard Kubo node (the reference go-ipfs implementation) running alongside the Hubzilla instance. The addon connects to it through the local IPFS HTTP API at `localhost:5001`. No exotic infrastructure is required. Kubo is widely deployed and well-documented.
---
## The packet vocabulary
The addon would introduce a small set of new Zot6 packet types. All of them fall through the existing `default:` path in `Receiver::Dispatch()` and are handled entirely by the addon hook:
- `ipfs_service_offer` — operator broadcasts their price list, storage quotas, quality tier, and accepted payment methods to connected nodes
- `ipfs_service_accept` — a customer accepts an offer and initiates a pin contract
- `ipfs_pin_receipt` — operator confirms a pin with a signed, timestamped commitment
- `ipfs_pin` — assertion: I am pinning this CID
- `ipfs_unpin` — retraction: I am no longer pinning this CID; dependents are notified
- `ipfs_announce` — I have seen this CID and can serve it from these gateways; not a storage commitment
- `ipfs_query` — request: who in the civic realm is pinning CID X?
- `ipfs_challenge` — spot-check: prove you are holding this CID
- `ipfs_proof` — response to a challenge with a proof of possession
- `ipfs_reputation` — signed reputation update, propagated to connected nodes
These are proposals, not a final specification. The exact structure of each packet needs community review.
---
## The income model
Node operators set their own prices, quotas, and quality commitments. The addon provides the contractual and communication infrastructure; it does not touch payment.
An operator advertises via `ipfs_service_offer`:
- price per GB of storage per month
- price per GB of retrieval bandwidth
- quality tier (uptime commitment, replication factor, gateway speed)
- maximum quota per customer
- accepted payment methods, described in free text
Payment itself is entirely the operator's responsibility. Fiat, cryptocurrency, community credit, barter, reciprocal storage — any arrangement the operator and customer agree on is valid. The addon records contracts and signed receipts, but does not move money. That boundary is intentional and permanent.
An operator who runs a reliable node can generate recurring income from storage and bandwidth. The civic realm's shared directory gives them customer discovery without needing to market themselves outside the network.
---
## The accountability layer
This is the part of the design that is most novel and most worth scrutinising.
IPFS content addresses are cryptographic hashes (CIDs). An operator cannot claim to store a CID and serve different content — the hash would not match. This property exists at the protocol level with no addon required. What the civic layer adds is a systematic process for testing whether claimed pins are real.
**Spot-checking.** Any node in the civic realm can send an `ipfs_challenge` packet to any other node that has published an `ipfs_pin` assertion. The challenge asks the recipient to return a specific byte range of the pinned content. This proves possession without requiring full retrieval of potentially large files. The response is signed with the operator's channel key.
**Reputation propagation.** Failures — a signed commitment followed by a failed or absent challenge response — are recorded locally and propagated to connected nodes as signed `ipfs_reputation` packets. Over time, a node that consistently fails challenges accumulates a degraded reputation score visible to other operators and customers.
**What this does not prevent.** An operator who accepts payment off-network and then stops pinning cannot be caught until a spot-check fires. The accountability layer is probabilistic, not cryptographic in the way Filecoin's proof-of-replication circuit is. It raises the cost and visibility of fraud; it does not make fraud impossible. The economic deterrent is that getting caught consistently destroys an operator's standing in the civic realm, cutting off future income.
The exact design of the challenge-proof exchange, and the propagation rules for reputation packets (including how to prevent reputation packets from becoming a spam or attack vector), needs more research.
---
## What is genuinely unknown and needs research
The following are not design choices that have been deferred — they are open questions where the right answer is not yet clear:
- **Realm bootstrapping.** How does a new operator join the civic realm? Who holds the `realm_token`? What is the admission process, and who governs it?
- **Directory server.** The civic realm needs at least one directory server. Who runs it, and on what terms?
- **Proof-of-possession strength.** A byte-range challenge is practical but weaker than a zero-knowledge proof. Whether the practical approach is sufficient for the trust requirements of this network needs evaluation by people with experience in distributed storage security.
- **Reputation propagation safety.** A reputation system that propagates signed scores between nodes can be abused — coordinated defamation, selective propagation, Sybil attacks on scoring. The design needs review by people who have built or studied federated reputation systems.
- **Nomadic pin-state sync.** Hubzilla's nomadic identity allows a channel to clone across instances. If a channel's pins are stored on one instance and the channel migrates, what happens to their pin contracts and receipts? Zot6's channel export/import mechanism would need to include pin state, and the design for that is not complete.
- **Streams compatibility.** Streams uses Zot protocol revision 12 (Zot12), which extends Zot6. The custom packet types described here use only Zot6 vocabulary and should be receivable by Streams nodes, which silently drop unknown types. Whether Streams operators would want to participate in the civic realm, and whether any Zot12-specific features would improve the design, needs input from the Streams community.
---
## The opportunity
This is an invitation, not a recruitment drive. People who find any of the following interesting are welcome to engage:
- IPFS infrastructure operations at any scale — where does running a Kubo node actually hurt, and what does the addon need to account for?
- Federated reputation system design — what propagation models are safe, and what are the known attack surfaces?
- Cooperative economics — what pricing models and quality tiers make sense for small operators who are not running data centers?
- Hubzilla addon development — what does the hook architecture actually support, and where are the gaps in this design?
Engagement can mean criticism, questions, counterproposals, or silence. All of those are valid responses.
---
## On the Hubzilla Association and the question of forks
This proposal is submitted to the Hubzilla Association for their consideration before any implementation begins. The Association represents the project's stewardship and its brand, and a federated service layer that carries the Hubzilla name in any meaningful sense ought to carry the Association's endorsement.
If the Association has objections, those objections will be taken seriously — provided they are rational objections: concerns about technical compatibility, about risks to the core project's reputation, about governance of the realm, about specific design flaws. Those kinds of objections are valuable and will change the design.
What would not be a sufficient basis for rejection is objection by inertia, by preference for the status quo, or by discomfort with the idea of node operators generating income from their infrastructure. The federated web was always supposed to create sustainable models for the people who run it.
If the Association declines to engage, or declines to provide reasoned objections, the proposal would naturally be considered by the Streams project, which uses a compatible protocol (Zot12), and potentially by other forks of the Zot ecosystem. This is not a threat — it is the ordinary logic of open source. The design will find a home somewhere. The preference is for that home to be Hubzilla, with the Association's blessing, because Hubzilla is where the most relevant infrastructure and community expertise lives.
---
## Summary
Civic IPFS is a proposed Hubzilla addon that:
1. Operates on a separate realm from RED_GLOBAL, causing zero interference with existing federation
2. Lets node operators offer paid IPFS pinning with operator-set prices and quality tiers
3. Uses Hubzilla's existing Zot6 authentication — no new trust layer required
4. Creates a federated accountability system where claimed pins can be spot-checked by any peer node
5. Leaves payment entirely to the operator and customer — the addon is infrastructure, not a payment processor
6. Is general-purpose — not tied to any specific application domain
Several aspects of the design are incomplete and labeled as such. The proposal is shared now because the questions that remain open are better answered with community input than in isolation.
Comments, objections, and questions are welcome.
Projects that need persistent, censorship-resistant storage currently face three options:
1. Trust a commercial pinning service (Pinata, web3.storage, and others). These are centralized, payment-dependent, and have a track record of shutting down — Infura IPFS closed in June 2023, Estuary followed. A business decision at one company can make content inaccessible.
2. Run their own IPFS node in isolation. Technically correct, but requires infrastructure expertise and provides no redundancy from the community.
3. Rely on the public IPFS network with no pinning guarantees. Content that nobody pins disappears.
There is no fourth option that is federated, community-governed, and accountable. That is the gap this addon is intended to fill.
The design is general-purpose. It does not assume any particular application domain. Community archives, cooperative platforms, local government transparency tools, mutual aid networks, and anything else that needs durable storage could use it. The name "Civic IPFS" reflects an orientation toward community benefit, not a restriction to any specific use case.
---
## What it is, technically
Civic IPFS is a Hubzilla addon that lets node operators offer IPFS pinning as a paid service to other channels and external users. The technical foundation is verified from the Hubzilla core source:
**Realm isolation.** Hubzilla's directory realm system allows a hub to operate in a named federation space separate from the default RED_GLOBAL realm. This is not a workaround — it is a supported feature. The constant `DIRECTORY_REALM` is defined as `'RED_GLOBAL'` in `boot.php`, and the function `get_directory_realm()` reads `Config::Get('system', 'directory_realm')`, falling back to that constant if the key is not set. Setting `system.directory_realm` to a different string, such as `CIVICUS`, causes the hub to federate only with other hubs carrying the same realm value. The `site` table has a `site_realm` column, and `Libzotdir::sync_directories()` uses it to filter which directory servers to sync with. Hubs in a different realm are excluded.
The civic addon would run on a separate realm from RED_GLOBAL. Nodes that do not install the addon are completely unaffected. No packets reach them, no noise enters the main social federation.
**Zot6 extension.** Hubzilla currently uses Zot protocol revision 6.0 (defined as `ZOT_REVISION = '6.0'` in `boot.php`). The Zot6 `Receiver` class dispatches incoming packets by `type` field. Known types — `purge`, `refresh`, `force_refresh`, `rekey` — go to dedicated handlers. All other types, including unknown ones, fall through to `default:` which routes to `Notify()` → `Libzot::fetch()`. This means custom packet types can be introduced by addons hooking into `Libzot::fetch()` without modifying core. Unupgraded nodes receive unknown types, ignore them gracefully, and continue operating normally.
Before any packet reaches the addon's hook, the `Receiver` has already validated an HTTP signature and verified the sender's identity against the hub registry via `Libzot::valid_hub()`. The addon receives a pre-authenticated sender. It does not need to build its own authentication layer.
**The fallback master directory** for RED_GLOBAL is defined in `boot.php` as `#^https://hub.netzgemeinde.eu`. The civic realm would need its own directory server. How that directory is bootstrapped and who operates it needs more research.
**A `realm_token`** mechanism exists in `Libzotdir::sync_directories()` — it reads `Config::Get('system', 'realm_token')` and appends it as a `&t=token` parameter to directory sync requests. This provides a lightweight shared-secret admission control on top of the realm name string alone. Whether to use this mechanism, and how to distribute the token to new members, needs more research.
**The IPFS backend** is a standard Kubo node (the reference go-ipfs implementation) running alongside the Hubzilla instance. The addon connects to it through the local IPFS HTTP API at `localhost:5001`. No exotic infrastructure is required. Kubo is widely deployed and well-documented.
---
## The packet vocabulary
The addon would introduce a small set of new Zot6 packet types. All of them fall through the existing `default:` path in `Receiver::Dispatch()` and are handled entirely by the addon hook:
- `ipfs_service_offer` — operator broadcasts their price list, storage quotas, quality tier, and accepted payment methods to connected nodes
- `ipfs_service_accept` — a customer accepts an offer and initiates a pin contract
- `ipfs_pin_receipt` — operator confirms a pin with a signed, timestamped commitment
- `ipfs_pin` — assertion: I am pinning this CID
- `ipfs_unpin` — retraction: I am no longer pinning this CID; dependents are notified
- `ipfs_announce` — I have seen this CID and can serve it from these gateways; not a storage commitment
- `ipfs_query` — request: who in the civic realm is pinning CID X?
- `ipfs_challenge` — spot-check: prove you are holding this CID
- `ipfs_proof` — response to a challenge with a proof of possession
- `ipfs_reputation` — signed reputation update, propagated to connected nodes
These are proposals, not a final specification. The exact structure of each packet needs community review.
---
## The income model
Node operators set their own prices, quotas, and quality commitments. The addon provides the contractual and communication infrastructure; it does not touch payment.
An operator advertises via `ipfs_service_offer`:
- price per GB of storage per month
- price per GB of retrieval bandwidth
- quality tier (uptime commitment, replication factor, gateway speed)
- maximum quota per customer
- accepted payment methods, described in free text
Payment itself is entirely the operator's responsibility. Fiat, cryptocurrency, community credit, barter, reciprocal storage — any arrangement the operator and customer agree on is valid. The addon records contracts and signed receipts, but does not move money. That boundary is intentional and permanent.
An operator who runs a reliable node can generate recurring income from storage and bandwidth. The civic realm's shared directory gives them customer discovery without needing to market themselves outside the network.
---
## The accountability layer
This is the part of the design that is most novel and most worth scrutinising.
IPFS content addresses are cryptographic hashes (CIDs). An operator cannot claim to store a CID and serve different content — the hash would not match. This property exists at the protocol level with no addon required. What the civic layer adds is a systematic process for testing whether claimed pins are real.
**Spot-checking.** Any node in the civic realm can send an `ipfs_challenge` packet to any other node that has published an `ipfs_pin` assertion. The challenge asks the recipient to return a specific byte range of the pinned content. This proves possession without requiring full retrieval of potentially large files. The response is signed with the operator's channel key.
**Reputation propagation.** Failures — a signed commitment followed by a failed or absent challenge response — are recorded locally and propagated to connected nodes as signed `ipfs_reputation` packets. Over time, a node that consistently fails challenges accumulates a degraded reputation score visible to other operators and customers.
**What this does not prevent.** An operator who accepts payment off-network and then stops pinning cannot be caught until a spot-check fires. The accountability layer is probabilistic, not cryptographic in the way Filecoin's proof-of-replication circuit is. It raises the cost and visibility of fraud; it does not make fraud impossible. The economic deterrent is that getting caught consistently destroys an operator's standing in the civic realm, cutting off future income.
The exact design of the challenge-proof exchange, and the propagation rules for reputation packets (including how to prevent reputation packets from becoming a spam or attack vector), needs more research.
---
## What is genuinely unknown and needs research
The following are not design choices that have been deferred — they are open questions where the right answer is not yet clear:
- **Realm bootstrapping.** How does a new operator join the civic realm? Who holds the `realm_token`? What is the admission process, and who governs it?
- **Directory server.** The civic realm needs at least one directory server. Who runs it, and on what terms?
- **Proof-of-possession strength.** A byte-range challenge is practical but weaker than a zero-knowledge proof. Whether the practical approach is sufficient for the trust requirements of this network needs evaluation by people with experience in distributed storage security.
- **Reputation propagation safety.** A reputation system that propagates signed scores between nodes can be abused — coordinated defamation, selective propagation, Sybil attacks on scoring. The design needs review by people who have built or studied federated reputation systems.
- **Nomadic pin-state sync.** Hubzilla's nomadic identity allows a channel to clone across instances. If a channel's pins are stored on one instance and the channel migrates, what happens to their pin contracts and receipts? Zot6's channel export/import mechanism would need to include pin state, and the design for that is not complete.
- **Streams compatibility.** Streams uses Zot protocol revision 12 (Zot12), which extends Zot6. The custom packet types described here use only Zot6 vocabulary and should be receivable by Streams nodes, which silently drop unknown types. Whether Streams operators would want to participate in the civic realm, and whether any Zot12-specific features would improve the design, needs input from the Streams community.
---
## The opportunity
This is an invitation, not a recruitment drive. People who find any of the following interesting are welcome to engage:
- IPFS infrastructure operations at any scale — where does running a Kubo node actually hurt, and what does the addon need to account for?
- Federated reputation system design — what propagation models are safe, and what are the known attack surfaces?
- Cooperative economics — what pricing models and quality tiers make sense for small operators who are not running data centers?
- Hubzilla addon development — what does the hook architecture actually support, and where are the gaps in this design?
Engagement can mean criticism, questions, counterproposals, or silence. All of those are valid responses.
---
## On the Hubzilla Association and the question of forks
This proposal is submitted to the Hubzilla Association for their consideration before any implementation begins. The Association represents the project's stewardship and its brand, and a federated service layer that carries the Hubzilla name in any meaningful sense ought to carry the Association's endorsement.
If the Association has objections, those objections will be taken seriously — provided they are rational objections: concerns about technical compatibility, about risks to the core project's reputation, about governance of the realm, about specific design flaws. Those kinds of objections are valuable and will change the design.
What would not be a sufficient basis for rejection is objection by inertia, by preference for the status quo, or by discomfort with the idea of node operators generating income from their infrastructure. The federated web was always supposed to create sustainable models for the people who run it.
If the Association declines to engage, or declines to provide reasoned objections, the proposal would naturally be considered by the Streams project, which uses a compatible protocol (Zot12), and potentially by other forks of the Zot ecosystem. This is not a threat — it is the ordinary logic of open source. The design will find a home somewhere. The preference is for that home to be Hubzilla, with the Association's blessing, because Hubzilla is where the most relevant infrastructure and community expertise lives.
---
## Summary
Civic IPFS is a proposed Hubzilla addon that:
1. Operates on a separate realm from RED_GLOBAL, causing zero interference with existing federation
2. Lets node operators offer paid IPFS pinning with operator-set prices and quality tiers
3. Uses Hubzilla's existing Zot6 authentication — no new trust layer required
4. Creates a federated accountability system where claimed pins can be spot-checked by any peer node
5. Leaves payment entirely to the operator and customer — the addon is infrastructure, not a payment processor
6. Is general-purpose — not tied to any specific application domain
Several aspects of the design are incomplete and labeled as such. The proposal is shared now because the questions that remain open are better answered with community input than in isolation.
Comments, objections, and questions are welcome.
Conversation Features
Loading...
Loading...