LFS Series — The Distributed File System, The Storage Layer & dDrive 5.0
Since the inception of the dWeb, we have been able to distribute and access file systems amongst a group of peers efficiently, which is what you do every single day within dBrowser (http://dbrowser.com) when you type in a dWeb network address and view a website. While this was amazing three years ago when we first started this project, this is something you see with many other projects like IPFS, Hypercore, GunDb and a handful of others like Ethereum’s Swarm. Although, the dWeb’s Storage Layer is still one of the most important layers in the dWeb ecosystem, considering it’s how we serve up the files of our websites and web apps to the people who want to use them. Quite frankly, its how we’ve rid ourselves of the centralized web server. It’s also being used to distribute large media files in applications like dSocial, who are attempting to develop completely server-less and CDN-less experiences on the dWeb. With that said, dDrive 5.0 introduces many new features, many of which may go unnoticed by users of the dWeb but to those who care about performance, they’re crucial when it comes to serving up distributed web applications and large data sets. In fact, it’s giving way to platforms like dVideo (https://dvideo.network), where live streaming will soon be completely distributed and dNames, where we’re rethinking DNS.
==== DDRIVE FOR NODE
dDrive 5.0 is available for Node and can be installed and used in any Node-based application. In fact, creating a dDrive programmatically is quite easy — just check out the following code:
```
var ddrive = require(‘ddrive’)
var drive = ddrive(‘./my-drive’) // drive content is stored in this folder
drive.writeFile(‘/let-freedom.txt’, ‘stream’, (err) => {
if (err) throw err
drive.readdir(‘/’, (err, list) => {
if (err) throw err
console.log(list) // prints the file name “let-freedom.txt”
drive.readFile(‘/let-freedom.txt’, ‘utf-8’, (err, data) => {
if (err) throw err
console.log(data) // prints “stream” since that’s all we inserted into the file
})
})
})
```
NOTE: You may find that the install process for “ddrive” works best with Yarn.
==== COMPLEX FILE SYSTEM REPLICATION, MOUNTING & P2P COLLABORATION
Just like dDatabases, since a dDrive is built on top of dTrie and dDatabase, a dDrive can be replicated between peers easily, like so:
```
var net = require(‘net’)
// on one computer
var server = net.createServer((socket) => {
socket.pipe(drive.replicate()).pipe(socket)
})
server.listen(10000)
// On another computer
var downloadedDrive = ddrive(‘./downloaded-drive’, origKey)
var socket = net.connect(https://remote-ip-address/10000)
socket.pipe(downloadedDrive.replicate()).pipe(socket)
```
dDrive replication occurs through streams, which means you can pipe a dDrive’s replication stream into any stream-based transport that you’re comfortable with. If you have many nested dDrives (will speak on this in a moment) within a parent dDrive, the `replicate` function will end up syncing all of a parent’s nested dDrives in the process. This is powerful, considering these nested dDrives all have their own dWeb network addresses and could be peered by a different set of peers, than the parent. Simply put, 5.0’s replication features simplifies the replication process for complex file systems, which means an entire dDrive, with many nested dDrives, can be shared and replicated amongst peers with a single dWeb network address.
dWeb’s mount system is what allows one dDrive to be nested within another, which makes dDrive’s “composable”. This means that you could create a dDrive for photos, where you nest all of your family’s photo-based dDrives. This way, you could access all of your family photos from a single dDrive. Mounting a dDrive into another, is as easy as the example below:
```
drive.mount(‘photos/my-moms-photos’, <my-moms-photos-dweb-address>)
```
This enables peer-to-peer collaboration, where a single codebase or project could be located within a dDrive, where different portions of the file system are separate dDrives (mounts), that different teams within a project have access to. Once a mounted dDrive is updated, the parent dDrive is updated as well. This form of peer-to-peer collaboration is not as efficient in competing solutions and sets the dDrive solution apart from others. When it comes to using a dDrive for web development, this can be a very useful feature for medium and large-sized web development teams who are used to collaborating in server-based environments.
==== POWERFUL VERSION CONTROL
Every dDrive is versioned, so that any previous version of a dDrive can be accessed, either by version number or an all-new “tagging” system that is a brand-new feature within dDrive 5.0. This is where dDrive gets its GIT-like functionality from and why it’s so powerful when compared to other distributed file systems. It’s the dDatabase beneath dDrive, that allows for old versions of files to be preserved, since a dDatabase is an append-only feed. Using the all-new `checkout` function, you can retrieve a snapshot of a dDrive from any point in time by simply using a version number, by doing something like this:
```
var oldDrive => drive.checkout(5)
```
This is a feature that has been around since dDrive 1.0, but it’s the all-new tagging system that allows a dDrive to act more like a GIT-like repository. Creating and accessing tags programmatically is quite easy and can be accomplished by doing something like this:
```
drive.createTag(tagname, 5, () => {
drive.getTaggedVersion(tagname, () => {
console.log(‘this should return the version associated with tagname’)
})
})
```
You could even get all tags that are associated with a dDrive, by doing something like this:
```
drive.getAllTags()
```
This would return a Map, in the form of:
```
{
name => version
}
```
Tags can also be deleted using the “deleteTag” method.
==== DDRIVE 5.0’s POWERFUL API
dDrive 5.0 utilizes an identical APIs like Node’s FS and Hyperdrive. You can learn more about dDrive’s file system APIs at https://npmjs.org/package/ddrive.
==== BEYOND WEB APPS AND WEBSITES
As mentioned earlier in this post, the primary use-case for dDrives is to distribute the files associated with a website or a web application and thanks to apps like dBrowser, they can be accessed like any other website or web app on the centralized web, over the dweb:// protocol. Although, the use-cases for dDrives continue to grow, where they’re actively being used as alternatives to Name Servers (see https://github.com/distributedweb/whitepaper#ddns) and even as a solution for live streaming large media files. When replicating a dDrive, you can pass the `live` option to the replicate method, and continuously replicate a dDrive’s latest version. This means if one computer is live streaming a video within a dDrive, all of those who are replicating that dDrive will receive any and all updates to that dDrive nearly instantaneously. Since those who are reading a dDrive only download the portions of the files they need, they’re able to do so on demand. This means you can stream media from friends, without all the hassle. As an added plus, seeking happens in an instant and there’s no buffering!
==== THE NEW DDRIVE CLI
In the next few days, we will officially deprecate the dDrive Daemon, which was used to create and manage dDrives via the command line and will replace it with a new dDrive CLI. The new dDrive CLI utilizes the new dHub client-side library and is far more efficient than its predecessor. In the next few days, I will be releasing a post in the LFS series that details all of the new CLI-based developers tools that are a part of the dWeb 3.0 rollout.
NOTE: For a detailed description of how dDrive works, head on over to the official dWeb documentation here: https://developers.dwebx.org/protocols/ddrive
=======
WHAT’S NEXT?
We’ll talk about the release of dSwarm’s all-new DHT, dWeb’s new peer discovery libraries and dWeb’s integration with WebRTC.
=======
