What is IPFS + basic use

IPFS, an acronym for “InterPlanetary File System”, is a way to deliver files that takes a different approach to HTTP. As I mentioned in my very first post, I originally considered hosting my entire site/blog on it, and while I decided against it for reasons I mentioned there and reasons I’ll get into later here; I am working on something that may involve me sharing files here, which I’ll probably use IPFS for. This is of course in addition to me finding it an interesting method of sharing data and worth speaking about.

What is IPFS?

In short, unlike regular domains and HTTP which focuses on where you can find a file, IPFS focuses on what file you’re looking for. It accomplishes this by getting a hash from any file(s) you intend to share or receive (a short mathematical identifier), which you then request from the network and any device that has a copy of the file provides it to you. When you request a file, your device (or the device that requested it on your behalf) temporarily holds on to a copy of that file for a short period which it can then share with other devices that are requesting the same file. Alternatively, if you wish to permanently hold onto the file you can “pin” the file to share it with any other requester indefinitely.

A good example of how IPFS works would be a theoretical situation where you had separate networks on two different planets, and in this example, it would be very easy to transmit data from one device to another within the same planet but cross-planet communication is slower and more limited. In this case, when somebody on Planet A requests a file from somebody on Planet B the original transfer will be slow, but once it’s on Planet A the original requester can provide it to anybody who requests it on Planet A, and each of those people will provide it to others who request it on Planet A as well, allowing the file to propagate through the network. Obviously in the real world we’re all on the same planet, but you can probably see some parallels when talking about communication across the globe, or with each requester also assisting in the distribution of the file to prevent any one host of a file from being bogged down by many requests (in a very similar fashion to BitTorrent and similar P2P file sharing protocols). At least that’s the ten thousand-foot view of things, though I would encourage you to read up on the documentation if you’re interested in learning more.

Viewing Content on IPFS

If the above description sounds interesting and you wish to give IPFS a try, then the first thing to do would be to try to view a file on IPFS. This can be done running IPFS locally on a device, or through a ‘gateway’ which is a server that acquires the copy of a file on IPFS and then provides it to a browser over HTTP.

Gateways

While they may not be quite as ideal since they don’t leverage all of the benefits of IPFS running a local node would, gateways are the easiest way to access content on IPFS since they don’t require any sort of software, and you may have even used them before without knowing it.

Say you have the hash bafkreie7ohywtosou76tasm7j63yigtzxe7d5zqus4zu3j6oltvgtibeom, a welcome to IPFS image. The simplest way to access it through a gateway would be to add the hash to the gateway URL. The most commonly used gateway would likely be ipfs.io, so in your browser you would type ipfs.io/ipfs/ and then paste in the hash and the file will either be viewed within your browser or downloaded depending on the file type. For example, if you use the hash above you would go to https://ipfs.io/ipfs/bafkreie7ohywtosou76tasm7j63yigtzxe7d5zqus4zu3j6oltvgtibeom and see the image. If you intend to share a file on IPFS but don’t expect the recipient(s) to have the IPFS client installed (or be familiar with IPFS at all) a direct link to the hash in a gateway can be shared. A domain can also point to an IPFS hash in a gateway, which I’ll elaborate on further shortly.

Browsers

Some browsers are also able to open links to IPFS and IPNS hashes natively, loading the file or web page contained within them. Normal browsers like the Brave browser or other browsers with plugins can load content on IPFS without needing to copy a hash into a gateway or desktop client. There are also specialized browsers such as Agregore built specifically for IPFS websites and similar protocols.

Browsers also have the benefit of being able to open IPFS links, such as ipfs://bafkreie7ohywtosou76tasm7j63yigtzxe7d5zqus4zu3j6oltvgtibeom as if they were any other link.

Desktop Client

Installing the desktop client is the best way to fully leverage the capabilities of IPFS, and is available for most desktop operating systems. Viewing files within the desktop client is very straightforward: simply open the client, wait for it to locate other devices on the network, and paste a hash in the bar at the top of the screen. Depending on your preferences, the file can either be directly downloaded to your device’s storage or be loaded in your browser in a ‘local gateway’ hosted by the desktop client.

Quick heads up: if you install the client you may want to close it when it’s not in use to prevent it from eating up resources 24/7. On Windows and Linux, even if you close the window it will usually stay running, but there will likely be an icon on your taskbar that you can click on to close the program when not in use.

Sharing files with IPFS

If you’ve dipped your toes in so far, then sharing files on IPFS isn’t too far of a step from viewing files on IPFS. There are three different ways you can go about this: pinning services, the desktop client, and a server/CLI client.

Pinning Services

Like gateways being easier to experiment with than installing the desktop software, if you wanted to host content on IPFS then pinning services have the least overhead to get started with. There are a lot of them, but Pinata.Cloud is likely the most popular and has a very generous free plan if you want to experiment with hosting content on IPFS. In the case of a pinning service simply register, upload a file or folder -or- pin an existing hash (such as one created on the desktop software), and the content of that hash will be online 24/7.

Desktop Client

If you have the desktop software installed, it’s also possible to host content from within it. Simply go to the files section, choose import, and then like a pinning service you can add a file, folder, or existing hash. Make sure to ‘pin’ the file so it is not automatically removed from the client, and as long as the client is running the content of the hash you created is available to anyone using IPFS.

This does come with the caveat though that it is only being hosted on your computer while your computer is on and the IPFS client is running. Because of the nature of IPFS the content of the hash may be cached on another machine for some time (particularly if your file is popular or was opened in a gateway), but it’s not guaranteed. This would be great if you wanted to share a file with another person but didn’t need it to be online 24/7 (the IPFS client offers a gateway link to share for accessing in a browser if the other person does not have IPFS – but keep in mind the file is publicly accessible), or if it was used in conjunction with a pinning service - but running your personal computer 24/7 and having IPFS eating up resources while you’re trying to do other things is likely not ideal.

CLI Client

Finally, it’s also possible to host content on IPFS using a dedicated server – either a VPS or server running at your home – to host content on IPFS. Being command line it’s the most technical, but if you wanted to self-host content 24/7 this would be the way to go.

You would want to begin by connecting to your server, ideally creating a user account on the server dedicated to IPFS, and then installing the CLI client with these steps. There are also instructions here to get your node up and running, but I’m going to deviate a slight bit from them here. I’m also going to assume that you’re connected with SSH, but as long as you have access to the command line in any way you should be able to do most everything here.

First, use the following command to get the server setup and generate your private keys (they’ll come in handy later) by running the following command:

ipfs init --profile server

If you’re on a VPS or home server (e.g. Raspberry Pi) with minimal resources you can use the following command INSTEAD of the one above, although I think the instructions drastically overestimate the resources required:

ipfs init –profile=lowpower

Once the server is initialized, next you’ll want to start it. Run the following command to get it started:

ipfs daemon &

Pro Unix tip: Assuming you’re on a Unix-style server, the ampersand symbol will make the process run in the background and let you keep typing other commands.

With the IPFS server running, you can now add a file or folder to IPFS. Get the file or folder on the server, then use the following commands for a file or folder respectively:

ipfs add [file name]

ipfs add -r [folder name]

For example, if you created a folder called IPFS and navigated into it, then added a file “cat.txt” and a folder “dog” you wanted to share you would do:

ipfs add ./cat.txt

ipfs add -r ./dog

After adding a file you will be given a hash for the file. If you added a folder you will be given a hash for the folder and a hash for every file in the folder, when sharing the hash make sure to share the one next to the folder name and not the ones you got for individual files in the folder.

Alternatively, you can also use SSH tunneling to access a web interface for the server not too different from the desktop software from within your browser on your personal machine. The web interface is usually on localhost:5001, so using the following command with admin privileges you can then go to localhost:5001 on your local machine and access it there.

ssh -L localhost:5001:localhost:5001 [username]@[serverip]

As long as the server remains online all hashes hosted on it will be available on IPFS. Using a pinning service for redundancy would be optional, but not required, and you’ll be hosting content on IPFS using your own system. Of course, as with most normal websites, people usually use a free or paid host of some sort as opposed to trying to host it on their own server; but the option is still there, and (as I will get to in the pros/cons) IPFS is uniquely suited to trying to host it on a server in your bedroom as opposed to HTTP.

Dynamic Content

As you might have noticed, hashes are well and good for static content, but if a hash points to a file or folder that is exactly as it was when it was uploaded, then anything that isn’t static becomes tricky to host on IPFS. Of course, if you wanted to host a file that’s always the same this isn’t an issue, but if you want to host something like a website you know you will need to change every so often then you will need to do something to point people to a new hash once you update the content. There are a couple ways of going about this, all of which have their pros and cons.

IPNS

The first option, and the one built into IPFS, is IPNS (InterPlanetary Naming System). Remember how I mentioned private keys earlier? With a private key you can get a hash that points to any hash of your choosing, which you can change at any time, and whenever somebody enters the IPNS hash they will be directed to the IPFS hash you set.

On the desktop client you would publish to IPNS by clicking on a hash and then choosing publish to IPNS.

Through the CLI you would use the following command

ipfs name publish [IPFS Hash]

or the following command to specify the non-default key

ipfs name publish --key=[key name] [IPFS Hash]

By default, IPFS will generate one key for you, but that key is only good for one IPNS hash, and you can generate new keys for new IPNS hashes in the settings on the desktop or with the following command on the CLI.

ipfs key gen [keyname]

Once you have published content to an IPNS hash, updating the hash it points to on the desktop client is as simple as getting the new hash, selecting publish to IPNS, and publishing with the same key as you used to publish the hash you wish to update.

On the command line client, if you want to update an existing IPNS hash to point to a new IPFS hash you would run the same command as you did originally but use the new hash instead, and the IPNS hash associated with the key used will now point to the hash you just set it to instead.

You can also use a cloud service to manage IPNS records like Fleek or Web3.Storage, however, both require software written in JavaScript (my arch nemesis for the moment) and I was unable to get Web3’s to run and Fleek’s required me to set an environmental variable that didn’t seem to want to work. That said, they are both in beta and I also certainly cannot rule out user error on my part.

There are some quirks of IPNS to be aware of though. While the IPFS hash an IPNS record points to does not need to be hosted on the same machine (e.g. an IPNS record created on a personal device pointing to an IPFS hash hosted by a pinning service), you cannot ‘pin’ an IPNS hash as only one device is supposed to hold the keys generated on it.

IPNS records are held by the network even if the device that published them goes offline, however, they expire after 24 hours which means that the device you generated them on (either a personal device or server) needs to either be on 24/7 or at least turned on and the IPFS client ran once a day for IPNS to work correctly. I am certain that IPNS will continue to be worked on over time, however, for the moment it is somewhat more difficult to manage than a standard IPFS hash.

DNS Link

Another way to have dynamic content in IPFS is to add a text record to your domain (or subdomain) which points to an IPFS hash, which can then be changed to update what hash your domain points to. In this case, the domain (or subdomain) would NOT be accessible in a normal browser, however, an IPFS client or gateway could use this in place of IPNS to locate the hash that you wish to point to. Official Documentation

This would still provide the benefits of IPFS being distributed, but would of course come at the cost of relying on traditional domains to actually locate the hash you wish to point to instead of relying on a distributed network for that.

Domains

As an opposite side to DNS Links, which work in IPFS clients/gateways but not browsers, you can also simply point a domain to your file or site hosted on IPFS in a gateway. This would ONLY work in traditional browsers and IPFS clients would not be able to get the hash that the domain points to.

This could be accomplished by setting a domain (or subdomain) to automatically redirect to a link to the site or file (preferably 302 forwarding with path forwarding enabled), and then setting the address it forwards to the file or web page open in a gateway. Alternatively, something like Cloudflare’s IPFS gateway service would allow you to truly direct a domain to an IPFS hash, by setting the domain to Cloudflare’s DNS and then setting it to open an IPFS hash within Cloudflare’s gateway which can be changed at any time.

Something like this is how a lot of people host their websites on IPFS, allowing people to access it just like any other website with their users unaware it’s even on IPFS, but at the cost of both cutting out traditional IPFS clients and gateways, relying on the traditional DNS for locating the hash itself, and while the hash would be distributed on the network normally if the gateway the domain points to goes down so does the website.

URLs

Last, of course, you can just manually update IPFS hashes on a website, social media, or wherever else you intend to have people visit to get the IPFS link itself. This is certainly not ideal if you intend to have standalone content (e.g. an entire website), but in the event you have files or portions of a website embedded within an existing website (or shared on social media) you can simply point users to a static hash and/or gateway URL and if it needs to be changed it can be updated by changing the links themselves.

There are likely also a plethora of other creative ways that you could have dynamic content shared, but these are the ones that come to mind and would be available in most normal situations.

IPFS Pros and Cons (as compared to HTTP)

Pros:

Works on non-static IP addresses

When using HTTP your device is told that a particular piece of content is at a certain IP address (or a domain, which is translated to an IP address), and your device then contacts and requests a copy of whatever content it was looking to get. In a situation when IP addresses change, such as with dynamic IPs on home internet, the entire setup falls apart. With IFPS, the content is still available just as it was before since the device is still able to alert the network that the content is hosted on the new IP address.

Distribution

Since a client requesting content on IPFS can get a copy from the original host or new hosts that had previously requested a file it allows files to be available even if the original host goes offline, to instantly scale resources to serve a sudden increase in traffic similar to a CDN, and even be highly resistant to DDos attacks.

Censorship resistance

While censorship is a hot-button issue in certain circumstances, in the case of IPFS anybody with an internet connection can upload a file of any kind that can be instantly replicated and served by anybody anywhere in the world. In this way, information such as a website or video could be uploaded without any overhead or personal information required (except an IP address) and swiftly archived and re-hosted by people in other countries who wish to keep the information public. Similarly, while most modern-day blocking of the internet is done by blocking domains and IP addresses, IPNS with no domains and ever-changing IP addresses can bypass much of that. All of this is also done in a way so that truly reprehensible content illegal most everywhere can easily be identified which IP is hosting it so the hosts of said content can be identified.

Immutability

Because of IPFS’s immutability, you can always be guaranteed that when requesting a particular hash you will get a particular file. While under HTTP the file located at a particular URL can be changed at a moment’s notice, including for malicious purposes, thanks to cryptography you can guarantee that the data on the other side of a particular hash is the same as it has always been.

Cons

Immutability

Despite being a pro in certain circumstances, as was delved into in the section on dynamic content, it can be difficult to host dynamic content on a system built around static content. While it can be done, it often does require extra steps and potentially some trade-offs to see that dynamic content works correctly.

Speed

When compared to HTTP, IPFS is slower to locate content. IPFS relies on DHT (distributed hash tables) to identify which hosts hold what hashes, and to do so it needs to consult the network which takes longer than just pinging a DNS server and asking which IP address a file is located at. Once a file is located, or if it is already cached, the speed should then be similar to transferring over HTTP, but during discovery it will likely take some time to locate the source of the file.

Non-human memorable identifiers

Generally speaking, you can identify content three ways: by consulting a central authority (such as ICANN with normal domains), a decentralized authority such as a blockchain, or math using hashes and public-private key pairs (IPFS being the latter). While this leads to the benefits of IPFS, it also means that there can be no human memorable domains built in.

Lack of adoption

Finally, IPFS lacks full-scale adoption, and most people would likely be entirely unfamiliar with what to do with an IPFS hash. As mentioned above, if set up correctly, a link can easily be shared that works in modern browsers in such a way a user doesn’t even need to know they’re on IPFS – but it does still prevent one from leveraging some of IPFS’s benefits in the process.

Wrap Up

If you made it this far you deserve an imaginary cookie. But seriously, as you can probably guess, I like the concept of IPFS and you’ll almost certainly see content hosted on it here at some point in the form of both hashes and links to content on gateways side by side. While I doubt our PHP and SQL based web is going to switch entirely to IPFS anytime soon, I do believe it has a very useful place alongside it.