Decentralized protocols fall into one of three camps:

1. “double spend solvers”: blockchains or similar where mutable data must not fork
2. “always reachable”: networks that overly assume it’s easy to reach nodes
3. “offline first”: durable, okay with not being online at the moment

Camp one is full of things like NFTs, Bitcoin, Ethereum, etc. - And it’s not a problem that needs solving for many inexhaustable digital copies. Resolution to a single object only needs to happen when transitioning to the real world. Digital objects can simply be copied or forked. (more on that in a future post).

Camp two is full of protocols don’t handle being behind a Nat, or a remote being offline for a while. This includes some DHTs (but usually doesn’t include tor onions, as they can handle NATs).

Camp three is filled with secure-scuttlebutt, local git copies, and similar.

Initially I thought it would be fun to build something that implemented all of the gossip or kademlia distributed-hash-table lookups.

There’s quite a few torrent projects, but not many support seeding from the cli. I couldn’t get webtorrent-cli or webtorrent-hybrid to work (doesn’t support deep directories), or merorafael/torrent-cli or pigeonburger/torrent-dl.

I think intermodal is awesome.

Since it doesn’t do everything needed, I’ll use github.com/anacrolix/torrent to round off a simple bash script.

I present: /bin/rightclick.sh (wget it, porkbun is weird…).

commands:
  crawl      [domain] # wget -> ./domain
  create     [domain] # ./domain -> domain.torrent & magnet link
  seed       [domain] # serves domain.torrent
  leach      [domain] # downloads torrent to ./domain
  serve      [domain] # serves local folder ./domain
  publish    [domain] # crawl, create, seed
  leachserve [domain] # leach then serve
  manual     [domain] # starts transmission-gtk instead
  magnet     [domain] # returns the magnet.$domain txt record

Publishing

/rightclick.sh takes a domain (yours, preferably), crawls its root to a local directory (transforming links… glad I didn’t need to use golang’s html token parser), and creates a torrent & magnet link of it. ./rightclick.sh crawl yourdomain; ./rightclick.sh create yourdomain

You’ll need to manually update your DNS TXT records. For example:

magnet.nonexist.whiting.dev IN TXT "magnet:?xt=urn:btih:34b7db39065417839aa8f1a281e4fde89f14dfa1&dn=whiting.dev&tr=udp://open.tracker.cl:1337/announce"

to publish it, you’ll need to run a seeding peer. You can either do that manually (./rightclick.sh manual yourdomain), or with ./rightclick.sh seed yourdomain.

Downloading

to download a page that’s always offline, we lookup the dns record. Internally this uses ./rightclick.sh magnet yourdomain. Next, it begins leaching it.

The command to download a site is ./rightclick.sh leach yourdomain.

If that fails, you could manually do it (./rightclick.sh manual yourdomain).

To browse a local site, use ./rightclick.sh serve yourdomain.

Summary

Getting a version of files saved onto your disk is a good task for torrents.

Linking to a new version is a good task for DNS.

If I could get webtorrent to work, Nat traversal would be so much damn easier.

in depth

# installers.
hash torrent || go install github.com/anacrolix/torrent/cmd/...@latest
hash imdl || curl --proto '=https' --tlsv1.2 -sSf https://imdl.io/install.sh | bash

# the recursive download command for domain -> ./domain
wget --recursive --convert-links "https://$domain"
# use intermodal to create a torrent from the folder:
imdl torrent create --input "$domain" --output "$domain.torrent" \
 --announce udp://open.tracker.cl:1337/announce
# tell users what TXT record we want
echo "magnet.$domain IN TXT \"$(imdl torrent link --input "$domain.torrent")\""

# seed to peers (no idea which options are best)
torrent download "./$domain.torrent"  --seed --addr ":0" --pex \
 --utp-peers --ipv4 --dht --tcp-peers --webtorrent # more testing is needed
# download from peers (no idea again)
torrent download "$($0 magnet "$domain")" --addr ":0" --pex \
 --utp-peers --ipv4 --dht --tcp-peers --webtorrent --test-peer 127.0.0.1:42185
# verify before serving the local directory (opens browser and starts listener)
imdl torrent verify --input "./$domain.torrent" --content "$domain" && \
xdg-open http://localhost:8000/; python3 -m http.server 8000 -d "$domain"

# our magnet command returns blank if grep fails:
dig +short txt "magnet.$domain" | grep "magnet:?xt=urn:btih"
# manual prefers the file, if it exists,
# or the magnet, otherwise opens just transmission.
r="$domain.torrent"
[ ! -f "$r" ] && r="$($0 magnet "$domain")"
transmission-gtk "$r"