Permanent Reference Media

See, I told you I’d get tuckered out writing about vehicles and be back to protocols soon enough.

“The internet is forever” is a great warning against the inability to unshare content, but completely irrelevant in regard to linkrot and long term preservation. Links I save or share here often are offline or significantly altered on a scale of months or years, much less over the course of decades. Http is efficient; “go get that file from that server” is fast, but does a rather poor job when said file or server no longer exist. There are options, such as the Wayback Machine or keeping a local copy - two things I try to remember to do when sourcing links¹ - but they have limitations. If it’s content I’ve linked to here, then sharing the entire archive is iffy in the realm of copyright. Beyond that, Archive.org has famously been known to get themselves into copyright trouble, and one too many unforgiving copyright holders could very well end up shutting the service down.

Immutable, content based identification protocols resolve these issues. In the realm of identifying content, all protocols discussed today identify what the user is seeking instead of where to look. This solves all three issues laid out above. On the availability side of things, that means any copy of the content on the network can be served to the user requesting it. It doesn’t matter if servers go down or domain names expire; as long as a copy still exists elsewhere, the content will remain. And, as a bonus, they unify archives. If I, Project Gutenberg, and Archive.org all had a copy of a book available over http or torrent, and one or more of those links went down or torrents stopped being seeded, that source would forever be gone; a user would need to locate and trust another archive from another source. If those three sources shared the same file on a decentralized, immutable system they’d be the same link. As long as one of the three remained, users of all three archives could access it.

Beyond availability, immutability also allows a link to a particular snapshot in time. Content gets updated, modified, and stealth edited all the time. With these protocols, it’d be possible to provide both a live link and a hashed snapshot to a version frozen in time. Finally, being on decentralized systems solves the copyright risks of sharing archived content: putting content onto a decentralized system built around copying is pretty explicit permission to copy within that system². Grabbing a copy of content I’d like to preserve and putting it on my node/server allows the content to be accessed regardless of the state of the original server, without the risk of copyright infringement that’d come along with sharing an entire webpage or video that I’d archived.

I figured I’d start by going over the three protocols that fit this bill: IPFS, Blossom, and Nostr (NIP-23). All three use content based identification, with IPFS and Blossom being immutable while Nostr supports versioned editing. IPFS is the ideal choice if you’re - like me - looking to have archived content alongside regular web content. Still, Blossom may fit use cases such as embedded content or anything needed to be accessed over http, and Nostr offers some CMS style content management, albeit only being plain text or Markdown. After each protocol I figured I’d go over different ways to encapsulate and share content, as well as what I’ve been doing using these protocols.

IPFS

IPFS is by far the best protocol for general purpose, long term file storage and sharing. Short for InterPlanetary File System, I’ve actually written about it before back in 2023. In short, however, it identifies content by its hash - locating all nodes on the network with the file for P2P file sharing. It differs from torrents in that IPFS is network wide, where torrents are often a mini network built around a particular file or folder³. It’s also more decentralized by default, relying only on DHT and bootstrap nodes instead of the centralized trackers Torrents often rely on.

IPFS is also already built into parts of the web. Most notably, services like Cloudflare can serve IPFS content as if it were any other website. Some browsers like Brave can handle IPFS urls, gateways allow rendering and downloading content via http, and JavaScript libraries like Helia allow any browser to natively fetch IPFS content for in browser rendering or downloading. There is a decent chance that many people have accessed content on IPFS while being completely unaware.

IPFS Quick Start

The fastest way to host IPFS content would be a pinning service. Pinata is my favorite; register a free account, upload a file or folder, and you’re instantly given a hash to share. The free account tier is generous enough that it’d be a serious challenge to exceed storage and bandwidth with mostly text based content. Once you’re hosting the content, sharing the hash (plus perhaps a gateway link) is enough for people to start accessing the content.

Accessing content on IPFS is also rather easy. The simplest way of doing so would be through gateways - simply going to ipfs.io/[hash] will fetch the content in a browser. However, relays are centralized gateways, so the quickest way to access IPFS content in a decentralized manner would likely be to use my own vibe coded dweb tool. Using Helia, it fetches a hash on the network directly (no need for centralized gateways), all within the browser.

IPFS Advanced Setup

If you’d like to delve deeper into the IPFS ecosystem, and ensure long term access to files of your choosing, the next step would be to install the IPFS client. On desktop it provides a simple GUI, allowing you to fetch a file by hash. The file will remain cached in the client for a little while; serveing the file to others until the client automatically removes it. If you’d like to assist in preserving a file you can ‘pin’ it, ensuring the client permanently keeps a copy. While you won’t be serving the file unless your client is running, if the initial host goes offline, the file will instantly become available to the network whenever your client is running.

Self hosting content on IPFS is a similar process. Installing the CLI client on a server allows you to pin any hash(s) of your choosing, and as long as the server remains online, the files will be available to the entire network. Because of IPFS’s decentralized nature, the server doesn’t require a domain name or static IP, and could very easily be hosted on an SBC behind a NAT and dynamic IP.

IPFS Final thoughts

The largest drawback to IPFS is its speed. Going straight to a server’s IP is considerably faster than contacting a bootstrap node, downloading a hash table, and querying all nodes known to hold a copy of the file. This makes it a poor choice for non-cached⁴ web content and embedded media that the user expects to load near instantly.

For the purposes of this post, it’s much better suited for archival content and web or document based content in longer form. While a 3-10 second wait to see a photo or tweet like content might dissuade a user, waiting the same amount of time to access a longer blog post, ebook/document, or an archived piece of content no longer available from other sources will feel much more reasonable.

However, speed is simply the cost of having a truly P2P system. Unlike the next two protocols - which use relays - IPFS content will always remain available as long as a single client is running and has a copy of said content. For somebody looking to have permanently referenceable content available alongside copies available over http, then IPFS should be their first go-to.

Blossom

Blossom, developed as a file sharing companion protocol for the text based social media protocol Nostr, is something of IPFS-lite. Like IPFS, Blossom identifies files via hashes - ensuring the user can get and cryptographically verify the authenticity of files regardless of which device served the file. Unlike IPFS, however, Blossom is not P2P. While decentralized, it relies on relays (centralized nodes) to serve content. While some Nostr clients can query multiple Blossom relays for redundancy, content on Blossom is often retrieved over plain http(s).

Blossom’s relay structure solves the load time issue of IPFS. While a file uploaded to Blossom may exist across multiple Blossom relays, often a single relay is chosen, and an http link to that relay is used to access the file. This speed makes Blossom ideal for embedded content and smaller media where a longer wait would be undesirable. However, this comes at the cost of said linked/embedded content being broken if the specific relay being linked to goes down. Since Blossom uploads are generally to multiple Blossom servers, and since Blossom files can be re-synced to new servers if some existing hosts go down, the content can still be recovered if the linked server goes down.

However, recovering content with a broken link requires human intervention. This would generally entail copying the hash contained in the broken link, then querying other Blossom servers to check if they have the file. This could be done manually, or with a tool like dweb tool. Alternatively, one could simply share the hash - like I do with the Untitled Chronicles - on the assumption the user will use a client or tool to query known Blossom relays for a copy of the hashed file.

Blossom Quick Start

The quickest way to share content on the Blossom protocol would be through a supported Nostr client. A client like Ditto allows the uploading of most file types, and comes preconfigured with several Blossom relays (which can be changed in Settings->Network). Since posting a file to Blossom is separate from making a post on Nostr, once a file has been uploaded to a draft post, it’s possible to grab the Blossom hash/link while deleting the post draft. Once the hash and link have been acquired, the content can then be embedded or shared.

Blossom Advanced Setup

For anybody regularly relying on Blossom, the most beneficial deep dive into the protocol would likely be to host their own Blossom server. This would allow hosted media with little more overhead than a static server, while preventing links from breaking in the event a third party server went down or domain expired. At the same time, however, keeping an active mirror on multiple Blossom relays ensures that files could ultimately be located even in the event that your Blossom relay is down.

There are, however, more advanced tools for interacting with Blossom. Dedicated tools like Boutique and more feature heavy Nostr clients like Amethyst can manage existing files (such as deleting previous uploads, or managing which files are stored on which server). There are also Nostr platforms built around sharing and consuming specific content on Blossom, a few examples including DiVine (shortform videos), PressStr (e-books/e-magazines, where I share the makeshift Untitled Chronicles), YourSpace (static websites), and NGIT-cli (git repos).

Blossom Final Thoughts

Again, Blossom is more of an IPFS/http hybrid. It works well in situations where embedded media and web content need to be accessed instantly and/or over native HTTP - such as embedded media, static web pages, or streamed video. Its relay structure, however, prevents it from being as seamlessly redundant as IPFS. While it would be possible to take the hash in an offline link to Blossom content and use it to query other servers, it requires active intervention on the part of the person accessing the content. For archival content - such as blog posts mirrored to a decentralized protocol for archiving - however, Blossom may be an unideal first choice.

Nostr NIP-23

Nostr is one of the big three non-centralized social media protocols - if you’re interested in a breakdown, you can check out my comparison post. While each protocol has its own pros and cons, and permanence is less important for general microblogging, Nostr is relevant here for two reasons. First is its NIP-23, a specification for the formatting of long form markdown enabled content - and second - that it’s the only social media protocol with true decentralization and permanently referenceable events⁵ that are distributed across multiple nodes.

Nostr native content is by far the most restrictive of the bunch, limited only to plain text and markdown based content. Nostr and Blossom both share a very similar structure: content is published to relays as opposed to being P2P like IPFS. However, because Nostr is accessed through Nostr specific clients (as opposed to http) - and since text based content is easier to mirror and fetch using multiple servers - Nostr is, in practice, considerably more distributed than Blossom. Nostr profiles also list which relays they publish to, offering a cheat sheet for clients to check several relays if a particular one is down. Since posts are plain text and small, many clients also automatically rebroadcast events to their own relays when replying or sharing posts, creating additional redundancy and automatically pushing older but often revisited posts onto fresh new relays.

Finally, being more social media oriented, sharing content is much simpler on the protocol. Publishing content is a similar process to something like WordPress, requiring much fewer steps than something like IPFS. The simplicity of an event allows their .json file to be downloaded and re-uploaded at will, in addition to allowing easy JS based rendering of content on standard websites and growing RSS reader support such as in Narr and Feeder.

NIP-23 quick start

The fastest way to begin publishing NIP-23 longform content would be to return to a client like Ditto. Choose ‘sign up’ to generate a keypair, go to articles on the sidebar, and share one. If you already have a markdown file you can paste its contents into the field in code view; otherwise, you can write from scratch in the WYSIWYG editor.

NIP-23 advanced setup

Delving deeper into storing longform writing on Nostr would likely first entail modifying which relays are used. Researching relay providers and adding additional ones to the client would ensure content is spread more widely across the network. Adding a self hosted relay gives a further degree of user controlled redundancy, and additional moderation options regarding who can reply to the posts.

A change to client setups may also provide security benefits and additional options. Bunkers like Amber allow the user’s key to be isolated to one hardened program instead of being added to every client - sort of similar to using PGP locally instead of uploading a private key to Proton or Tuta. Switching to a dedicated longform client like Habla may also provide more options for publishing and managing content as opposed to a general purpose client like Ditto.

Finally, it may be beneficial to keep a local copy of posts. Since Nostr events contain everything in a single .json file (content, signature, timestamp, etc) it’s possible to download an event for later re-upload to relays. There are several ways to download Nostr events; most clients allow you to view the raw .json data, and my vibe coded Nexport allows for profile archiving.

Final Nostr NIP-23 thoughts

Like Blossom, Nostr isn’t ideal if you’d like to have a secondary, standalone, long term archive. If you just want that, again, IPFS is likely your best bet. It does, however, shine in two different circumstances: One, if you want to have a permanently referenceable archive, it is by far the easiest. In a standard POSSE/PESSO fashion, it’d be as simple as pasting a markdown document and hitting share. Second, while I’m yet to see it widely implemented, I wouldn’t be surprised if it’s eventually used as a decentralized CMS for some static sites. Using a static site and some JavaScript, it’s possible to render content posted on the protocol without rebuilding the site for each post, sort of like I’m doing here. Especially with growing support within feed readers, it may become a fairly distributed and permanently referenceable core of some websites in the future.

Protocol Honorable Mentions

Torrents are another option to archive blocks of content indefinitely. They’re older and more widespread than IPFS, and it’d be very possible to archive a document or folder of documents within a torrent. I’d personally prefer IPFS, however, for 4 reasons. First, IPFS acts as one big network. If my IPFS node has a file you’ll be able to get it without question. With torrents, I may have the file, but you’ll be unable to download it if I’m in a different swarm.

Secondly, torrents often rely on centralized trackers. Some clients do support DHT, but they’re more limited. Third, IPFS works better for web content. It’s much easier for a user to download or view a file in browser using a JS library or gateway than it is for them to install a torrent client and use that. Finally, torrents have a reputation for being illegal - and despite that not being the case - you may unintentionally turn people away if they see the word ‘Torrent.’

Hive and other blockchain based social media services may also be a consideration. Alternatively, sticking text based content directly on a blockchain that’s more reputable, like Bitcoin or Ethereum, may be a long term archival method. Although putting content on those systems would cost money, and probably make easy retrieval of your content difficult. Besides, blockchains and hosted web content generally fail to mesh well.

There’s also some protocols I know less about. DAT is, from what I know, similar to IPFS except the user can update files. It’s not immutable, but still likely a way to have content exist beyond the lifetime of the server/node it originated on. There’s also projects like Radical.dev, a decentralized git tool that could likely host a collection of PDFs just as easily as it could host source code. There’s likely a ton more options out there that’d also be great, which aren’t here only because I’m unfamiliar with them.

Content Encapsulation Methods

Lol, that sounds like a bit of a pretentious header, but I’m leaving it there.

IPFS and Blossom will take any type of file format (in addition to directories in the case of IPFS), so you can share content in just about any form. But from the perspective of ‘I have some content on the web I’d like to archive in a permanently referenceable way’ there’d be a few different ways I’d go about it. For a simple dump of source code, unsorted individual documents or html files, or sets of photos - a quick compressed folder would probably be the best way to go about it. Pick the most efficient compression algorithm (prob. xz at lvl 9 compression) and upload to IPFS or Blossom.

HTML based content is another option, and probably the easiest method to use if you had a simple web page you wanted to archive. Tools like the SingleFile plugin can export a page into - as the name implies - a single html file that encapsulates all JS and images. It’s rather easy to use that to download a page, then upload that archive to IPFS or Blossom. As a bonus, since it’s in HTML, said content could be viewed in a web browser via IPFS gateway or Blossom link. For example, my latest social media protocols comparison post saved with SingleFile on IPFS and Blossom

• IPFS hash: bafybeictpsccgrwfpzbyt7loyubxfanzvvxj2d2karbrchlsbht6k7yqhu
• IPFS gateway link: link
• Blossom hash: 596e980a23c404c50b521a5a0c3fa0c62f2d1e918a13ee8e569ffb02c236c7e0
• Blossom link: link

Or, alternatively, it’s possible to use a tool like wget to archive an entire website. The command wget -mpEk4 "https://nate.mecca1.net/" (or wget -mpEk4 --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" --wait=2 --random-wait --limit-rate=200k --waitretry=10 --tries=3 "https://nate.mecca1.net/" if you want to look a little less like a scraper) would scrape an entire site, which could be zipped and uploaded if somebody wanted to have a permanent static archive of their site even if the backend is something like Wordpress. Doing a full site archive annually or quarterly would be a lot less of a commitment than archiving every page or post individually.

Of course the sky’s the limit. Photos and videos can be uploaded directly, optionally compressed with something like Squoosh or HandBrake. Good ‘ol plain text files are still a great way to store information, and quite small as well. You could even just get a directory of content and tell an AI to burn some tokens converting or reformatting it into whatever you’d prefer.

Finally, if you don’t feel like putting in too much work, there’s tools that can assemble content automatically. One tool that comes to mind is Calibre - with its ability to convert most kinds of documents and make ‘magazines’ from RSS feeds. Another is Inkstr, a more recent tool built to make magazines from Nostr content and markdown files.

anything to markdown to anything (my workflow)

If you wanted to go beyond html, however, starting with Markdown can get you pretty much anywhere. A lot of static site generators and notes apps already use markdown, so there’s a decent chance that if you have writing you want shared in a permanently referenceable medium that it’s already in markdown. If not, tools like UrlToMarkdown can convert any web page into markdown, and just about any other type of document can probably be converted using some tool or another.

Once you have a markdown document, you can pretty much do anything. If you wanted to use Nostr’s NIP-23 you could simply paste it there - or if you wanted to upload markdown files directly, you could always just zip and upload to IPFS or Blossom. Otherwise, you can turn those markdown files into just about anything.

For example, in toying around with different options, I’ve found PDFs compiled of multiple related posts to be the most convenient. In this case, I convert my existing markdown files into docx files using my vibed⁶ conversion tool, then copy/paste the contents of each into one big document in LibreOffice Writer. After that, it’s just a matter of some mild stylization before exporting everything as a PDF document. Though it’s worth noting LibreOffice can export documents in many other formats - including epub and html.

From there it’s pretty simple, just upload to your preferred protocols (in my case: IPFS, plus Blossom and Nostr via PressStr). I’m using this method because it’s a bit less of a commitment. I can do or skip whichever posts I’d like, do them in bulk, and do them some time after posting without breaching any unspoken deadline of doing so.

What I’m doing and how this could look

Still, after all that rambling about how this is so great, you might be asking “Hey Nate, if this is so great, why isn’t most of your content mirrored on a protocol like these?” In short: I’m lazy, and not sure much of my content is worth archiving. None of my content is going to be some world changing masterpiece, and a lot of it becomes outdated after months or years. Still, one of the reasons I considered building this site with IPFS is that permanence factor, and I have begun putting some content on these protocols.

First, I’ve been putting a number of my miniblog posts on Nostr (both in NIP-1: plain text shortform content, and NIP-23: longform markdown content). I’ve also started relying less on HTML and Hugo shortcodes in my posts, the idea being I could more easily mirror my content on Nostr if I ever get around to mirroring it. You can view my content posted on the network from within my website here.

I’ve also started what I call the “Untitled Chronicles,” just a collection of PDFs I share here. It doesn’t contain a ton yet, but I’m slowly putting together small collections of posts in e-book/pamphlet form that I’m stashing on IPFS and Blossom (via PressStr). I might also just start adding SingleFile generated archives to IPFS and Blossom as well.

Finally, I made the page /permanent-reference, which is a list of all posts available on these protocols. In the event you wanted to grab a hash of a post on one of these protocols, it can be found on that page by searching the post name. In the event this site goes down and you still wanted to access or reference that post, you’d be able to grab the hash from a personal archive of the page if you made one, or from something like the Wayback Machine.

Finally, while I doubt I’ve got the clout to make this sort of archiving trendy, if I wanted to link to something that had an archive like this I’d include both the standard link as well as the IPFS/Blossom/Nostr/whatever hash either next to it or in the footnotes.

Footnotes: