RE: Archiving the smallnet & fantasizing about a free mesh based internet

by Ploum on 2022-10-06

Smokey is trying to figure out how big Gemini is and if it made sense to share it through a mesh network.

• Smokey’s post

It turns out that I’ve some very interesting data to share. I’ve been using Offpunk to browse both the web and Gemini for a bit less than one year. During those months, every single resource I’ve read has been saved in my Offpunk cache. I didn’t clean the cache except for one very big video file on Gopher (on the web, Offpunk is smart enough not to download files that are too big if not explicitly requested).

Let’s have a look:

$ du -sh ~/.cache/offpunk
1,1G	gemini
369M	gopher
630M	http
7,6G	https
8,4M	spartan

That includes everything, including pdf, images, etc. (I discovered that Offpunk encountered other unsupported protocols including fb-messenger://, git://, ipfs://, irc://, vscode:// and whatsapp://. I removed those from the results for the sake of clarity).

Of course, the total of bytes transferred is bigger than that as some resources may have been updated and their cache has been overwritten. But this gives a very good estimation.

Offpunk started as an experiment to see one could mostly be offline and use a fictional protocol that I called Offmini.

• Offmini, part 1
• Offmini, part 2

What living with Offpunk told me is that when you don’t consume videos, when you get your news through RSS and are not using a webmail, you can live with a very small bandwidth. I believe I’ve never read as much on the web than with Offpunk. I’m following 50 https RSS feeds that are updated every day. But I read without caring about the design, without loading most of the picture, without loading any CSS or JavaScript. I don’t miss any of those things (in fact, I’m better without them). And yes, I still see all the important pictures. And yet, this is less than 1 Go per month on the average.

One key moment using Offpunk was when travelling: I can spend days without synchronising my computer, answering mails that are already in my inbox, reading my toread list in Offpunk, browsing through old posts, discovering unread gems. Instead of connecting to the wifi in the train or in the hotel (the first thing I always did previously), I sometimes fire my phone access point, make a synchronisation that costs me something like 30 or 40 Mo, at worst, then I’m good for several days.

So yeah, it makes an awful lot of sense to imagine a sneakernet or a mesh network. That’s exactly what Offmini is about.

I invite you to try to browse gemini/web/gopher with Offpunk for a week, with one or two daily synchronisation.

• Offpunk’s repository

But I understand that a terminal interface might not be for everyone. That’s why I plan to abstract the whole caching mechanism and makes it a separate tool (I call that "netcache"). Netcache would be a simple Unix tool that would maintain a cache of networked resources with a list of resources requested but not there yet. If I manage to do this right, this means any client would be able to become offline like Offpunk. This also means that you could easily merge two caches, something that could be done when meeting a friend offline or with a simple USB disk.

I even started to learn Rust to write that tool in Rust but I may write a Python version first. Will be easier for my aging brain…