Decentralized protocols fall into one of three camps:
- “double spend solvers”: blockchains or similar where mutable data must not fork
- “always reachable”: networks that overly assume it’s easy to reach nodes
- “offline first”: durable, okay with not being online at the moment
Camp one is full of things like NFTs, Bitcoin, Ethereum, etc. - And it’s not a problem that needs solving for many inexhaustable digital copies. Resolution to a single object only needs to happen when transitioning to the real world. Digital objects can simply be copied or forked. (more on that in a future post).
Camp two is full of protocols don’t handle being behind a Nat, or a remote being offline for a while. This includes some DHTs (but usually doesn’t include tor onions, as they can handle NATs).
Camp three is filled with secure-scuttlebutt, local git copies, and similar.
Initially I thought it would be fun to build something that implemented all of the gossip or kademlia distributed-hash-table lookups.
There’s quite a few torrent projects, but not many support seeding from the cli.
I couldn’t get
webtorrent-hybrid to work (doesn’t support deep
I think intermodal is awesome.
Since it doesn’t do everything needed, I’ll use github.com/anacrolix/torrent to round off a simple bash script.
I present: /bin/rightclick.sh (wget it, porkbun is weird…).
commands: crawl [domain] # wget -> ./domain create [domain] # ./domain -> domain.torrent & magnet link seed [domain] # serves domain.torrent leach [domain] # downloads torrent to ./domain serve [domain] # serves local folder ./domain publish [domain] # crawl, create, seed leachserve [domain] # leach then serve manual [domain] # starts transmission-gtk instead magnet [domain] # returns the magnet.$domain txt record
/rightclick.sh takes a domain (yours, preferably), crawls its root to a
local directory (transforming links… glad I didn’t need to use golang’s
html token parser), and creates a torrent & magnet link of it.
./rightclick.sh crawl yourdomain; ./rightclick.sh create yourdomain
You’ll need to manually update your DNS TXT records. For example:
magnet.nonexist.whiting.dev IN TXT "magnet:?xt=urn:btih:34b7db39065417839aa8f1a281e4fde89f14dfa1&dn=whiting.dev&tr=udp://open.tracker.cl:1337/announce"
to publish it, you’ll need to run a seeding peer. You can either do that manually (
./rightclick.sh manual yourdomain),
./rightclick.sh seed yourdomain.
to download a page that’s always offline, we lookup the dns record. Internally
./rightclick.sh magnet yourdomain. Next, it begins leaching it.
The command to download a site is
./rightclick.sh leach yourdomain.
If that fails, you could manually do it (
./rightclick.sh manual yourdomain).
To browse a local site, use
./rightclick.sh serve yourdomain.
Getting a version of files saved onto your disk is a good task for torrents.
Linking to a new version is a good task for DNS.
If I could get webtorrent to work, Nat traversal would be so much damn easier.
# installers. hash torrent || go install github.com/anacrolix/torrent/cmd/[email protected] hash imdl || curl --proto '=https' --tlsv1.2 -sSf https://imdl.io/install.sh | bash # the recursive download command for domain -> ./domain wget --recursive --convert-links "https://$domain" # use intermodal to create a torrent from the folder: imdl torrent create --input "$domain" --output "$domain.torrent" \ --announce udp://open.tracker.cl:1337/announce # tell users what TXT record we want echo "magnet.$domain IN TXT \"$(imdl torrent link --input "$domain.torrent")\"" # seed to peers (no idea which options are best) torrent download "./$domain.torrent" --seed --addr ":0" --pex \ --utp-peers --ipv4 --dht --tcp-peers --webtorrent # more testing is needed # download from peers (no idea again) torrent download "$($0 magnet "$domain")" --addr ":0" --pex \ --utp-peers --ipv4 --dht --tcp-peers --webtorrent --test-peer 127.0.0.1:42185 # verify before serving the local directory (opens browser and starts listener) imdl torrent verify --input "./$domain.torrent" --content "$domain" && \ xdg-open http://localhost:8000/; python3 -m http.server 8000 -d "$domain" # our magnet command returns blank if grep fails: dig +short txt "magnet.$domain" | grep "magnet:?xt=urn:btih" # manual prefers the file, if it exists, # or the magnet, otherwise opens just transmission. r="$domain.torrent" [ ! -f "$r" ] && r="$($0 magnet "$domain")" transmission-gtk "$r"