[🏆 Golden path scenario] Browser-authored content retrievable by another machine through the ipfs.io gateway directly

[BigLep]

Done Criteria

A user can author content in their browser via Helia and have it retrievable by another machine through the ipfs.io gateway without relying on pinning services or preload nodes.

Why Important

This is a common usecase that users hit. Failure here feeds the narrative that "IPFS doesn't just work".

Notes

1. This builds on [🏆 Golden path scenario] Browser-authored content retrievable through the ipfs.io gateway via a pinning service  #256, which used pinning services. We’re assuming it was completed first before taking this on.
2. Even though this usecase request has come in (e.g., HackFS 2023), it’s a lower priority because it has the fundamental flaw of relying on a browser tab to stay open/active. This may make sense for a demo, but has serious usability flaws. In practice, we expect most browser apps will want to be more resilient, and for that, they need a way to get data off the browser (e.g., use a pinning service which [🏆 Golden path scenario] Browser-authored content retrievable through the ipfs.io gateway via a pinning service  #256 satisfies).
3. "retrievability from the ipfs.io gateway" is used as a popular "stand in" for other nodes on the network.
4. We need to enable discoverability of the content created in the browser so that the ipfs.io gateway can discover it. This requires one or more of:
   • Direct reliable libp2p publishing to the "IPFS public DHT" from the browser
   • Direct HTTP writing to an IPNI like cid.contact (/ingest/announce)
   • Delegated HTTP /routing/v1 PUT support and endpoints that can use like routing.delegate.ipfs.tbd, cid.contact, etc. (This is the path we're prioritizing per the tasks below.)
5. DCUtR in js-libp2p is needed so the Kubo gateway can run the protocol with the browser node via the relay and instruct the browser to dial one of its public multiaddrs supported from the browser (e.g., WSS, WebTransport). This will have been handled in [🏆 Golden path scenario] Browser-authored content retrievable through the ipfs.io gateway via a pinning service  #256.
6. The next step here is to allow a private Kubo node (e.g., Kubo running in one's Brave browser) to fetch the content authored in a browser on a separate host. We ultimately need Kubo to support WebRTC since WebRTC is required for browser/private-node connectivity per here. (Kubo tracking issue: Enable WebRTC Transport kubo#9724 ). This should have been completed though by [🏆 Golden path scenario] Browser-authored content retrievable through the ipfs.io gateway via a pinning service  #256.
7. Per above, this isn't a pure Helia issue. Tracking the usecase needs to go somewhere though, so I'm putting it Helia for now so we can link against it.

### Tasks
- [ ] IPIP/spec for HTTP /routing/v1 PUT support (at least writing delegated content records to the DHT): <https://github.com/ipfs/specs/pull/378>
- [ ] Boxo having /routing/v1 PUT support (IPIP-378)
- [ ] Exposing /routing/v1 PUT support in Kubo
- [ ] routing.delegate.ipfs.tbd having /routing/v1 PUT support
- [ ] JS updates to use IPIP-378 (write side of delegated content routing)
- [ ] js-libp2p: Reconnect with relays so that previously published multiaddrs with relays have a higher chance of still working - https://github.com/libp2p/js-libp2p/issues/1955

[aschmahmann]

This is more of a libp2p issue, and more of a go-libp2p issue

While support for WebRTC will certainly help in some scenarios (i.e. the browser does not support WebTransport and the node fetching the data doesn't have a WSS address with a CA cert) IIUC the main difficulty in getting data from browser helia nodes discoverable by gateways, etc. is data not being advertised to the DHT, IPNI, etc.

Am I mistaken and it turns out advertising small amounts of data to the DHT from a helia browser node is working well enough at the moment (at least for browsers that support WebTransport)?

[SgtPooki]

@aschmahmann even in a browser that supports WebTransport, i've been having difficulty getting any successful webtransport connections. I just pushed up a repo where I was playing around: https://github.com/SgtPooki/helia-playground -- it was essentially copied from https://codesandbox.io/p/sandbox/helia-script-tag-forked-3q8y35 to a local workspace so i could modify things more easily.

One thing I started seeing was that activeStreams.length never breaches 0 for me, no matter how many peers or how many connections I have. I suspect a bug in libp2p/webtransport but I haven't been able to fully track it down.

I want to create a simple test where a browser helia node can successfully talk to a backend helia node, but that will have to wait for a bit.

---

ninja-edit:

Also, there seems to be a non-stop spamming of webtransport dial attempts.. and i'm not sure how best to control that with libp2p-connection-manager.

[BigLep]

@aschmahmann : good callouts - thanks.

---

Let's assume:

1. the ipfs.io Kubo node has maximum connectivity possible today with WebTransport and WSS address with a CA cert
2. the browser authoring the data supports WebTransport and WebSockets
3. the ipfs.io Kubo node discovered the multiaddr of the browser that authored the data

How does this ipfs.io Kubo node retrieve the data from the browser node? My understanding is that it still can't initiate a connection to the browser in this scenario and this scenario would only work if there was a preexisting connection between the browser node and the ipfs.io Kubo node.

---

Also, I expanded the "Notes" section in the top description to further expand on the underlying issues:

1. Underlying issue 1: discoverability of the content created in the browser so that the ipfs.io gateway can discover it. This requires one or more of:
   • Direct reliable libp2p publishing to the "IPFS public DHT" from the browser
   • Direct HTTP writing to an IPNI like cid.contact (/ingest/announce)
   • Delegated HTTP /routing/v1 PUT support and endpoints that can use like routing.delegate.ipfs.tbd, cid.contact, etc.
2. Underlying issue 2: libp2p connectivity, especially go-libp2p connectivity, since we ultimately need Kubo to support WebRTC since WebRTC is required for server nodes to dial browsers. (Kubo tracking issue: Enable WebRTC Transport kubo#9724 ).

Please go ahead and fix/correct any mistakes here.

---

Thank you!

[aschmahmann]

How does this ipfs.io Kubo node retrieve the data from the browser node? My understanding is that it still can't initiate a connection to the browser in this scenario and this scenario would only work if there was a preexisting connection between the browser node and the ipfs.io Kubo node.

Yeah, that's right good callout. I had assumed there was some level of support for DCuTR in js-libp2p that came along the relay-v2 support. With the simplest DCuTR support (dialbacks) what would happen is that the helia node would connect to a (limited) relay-v2 node that speaks some protocol the helia node can speak (e.g. WSS, WebTransport, etc.) and they would then have as their address /the/multiaddr/of/the/relay/circuit-relay/p2p/helia-node-peerID which when a publicly reachable node (e.g. the ipfs.io kubo nodes) wanted to contact the helia it would ask the relay to have the helia node dial it back (using WSS, WebTransport, etc.).

This doesn't require any holepunching kinds of magic, just a simple relay + the dialback portion of the DCUtR protocol.

Seems like it might be worth scoping this as a smaller and more important set of work in libp2p/js-libp2p#1460.

[SgtPooki]

Notes from Helia WG 2023-07-27

• We need to make sure Gateways support webrtc
  • either go-libp2p + kubo/boxo need to support webrtc
  • OR js-libp2p on Node needs to support webrtc + we need to stand up a JS backed gateway?
    • webrtc in NodeJS is not supported currently.
• We need DCUtR in js-libp2p so the gateway can run the protocol with the browser node via the relay and instruct the browser to dial it
  • the gateway and the relay need a common transport (e.g. tcp, quic, etc)
  • the browser needs to be able to dial the gateway (e.g. wss or webtransport)
  • the gateway does not need to be able to dial the browser (e.g. WebRTC)

[achingbrain]

DCUtR for js-libp2p is in progress here: libp2p/js-libp2p#1928

[SgtPooki]

Note that the libp2p hole-punching vision table also illustrates the problem here fairly well: https://github.com/libp2p/specs/blob/d2106f43e878ae4c3a1c6465a7c329835290fe22/connections/hole-punching.md#vision

[BigLep]

It's great that progress is happening here.

Folks have correctly pointed out that for the stated usecase of Kubo ipfs.io gateway retrieving content from the browser that go-libp2p WebRTC isn't needed. We only need js-libp2p DCUTR. That's great, and I agree that should be the first usecase.

That said, I don't want to let up there since the ultimate is "universal connectivity". The next step here is to allow a private Kubo node (e.g., Kubo running in one's Brave browser) to fetch the content authored in the browser. For this we ultimately need Kubo to support WebRTC since WebRTC is required for browser/private-node connectivity per here. This can come after, but I have updated the issue notes to be accurate and to discuss this followup step.