LFS Series: Introducing The dHub Ecosystem

Peeps
5 min readMar 29, 2021

For computers across the world to exchange data on a peer-to-peer basis, there needed to be a software that could operate on personal computers, capable of creating, managing and seeding dDatabase-based abstractions, whether it was raw dDatabases (distributed append-only logs), dTries (distributed trie-based key-value stores), dTrees (distributed binary trees) or dDrives (distributed file systems). In my previous post about the Local Layer, I introduced a new companion library for dDatabase 10.0 known as Basestore, which allowed for the interlinking of dDatabases, even by specific “namespaces.” It does not matter what a dDatabase contains, whether its a dDrive or a dTree, any of these dDatabase-based formats can utilize Basestore underneath. This is the magic that allows for the “mounting” of dDrives within other dDrives and so on.

In the past, we utilized dDrive Daemon (https://npmjs.com/package/ddrive-daemon) for managing dDrives but since dDrives and other dWeb-based data structures are simply dDatabases underneath all the “fluff”, it made sense to have a daemon that simply managed dDatabases so that we could efficiently manipulate all dDatabase-based data structures using a single system. We could easily do this by simply handling raw dDatabases via the Basestore wrapper, instead of worrying about higher-level abstraction layers that live above dDatabase — like dDrives, dTries and dTrees. This sort of design would allow us to develop other command lines tools above this sort of system, which could then handle those higher-level abstractions, and indeed that is what we ended up doing with the all-new dHub project and many of its higher-level command line tools like dWeb CLI.

==== INTRODUCING DHUB & DHUB CLIENT

dHub is a lightweight server that provides remote access to dDatabases (https://npmjs.com/package/ddatabase) and a dSwarm (https://npmjs.com/package/dswarm) instance. dHub exposes a simple, yet powerful, RPC interface (https://npmjs.com/package/@dhub/rpc) that applications can interact with to programmatically create and/or seed interlinked or standalone dDatabase-based structures effortlessly. Applications can easily interact with the “dhub” daemon (https://npmjs.com/package/dhub) using the dHub client module (https://npmjs.com/package/@dhub/client).

First the “dhub” daemon should be installed and started like so:

```

npm install -g dhub

dhub — no-migrate

```

====

WARNING: You must use the “ — no-migrate” command so that dBrowser/dDrive Daemon’s default storage location isn’t migrated. Not doing so will mess up your current dBrowser/dDrive Daemon installation.

====

At this point, dHub will be running and will be binded to a UNIX domain socket (~/.dhub/dhub.sock) or a named pipe on Windows. Now that dHub’s daemon is running, you can use the dHub client to create and manage remote dDatabase structures. Here are a few examples of how dHub’s client library can be used to interact with the dHub daemon locally:

==== Connecting to the dHub daemon

```

const DHubClient = require(‘@dhub/client’)

const client = new DHubClient

await client.ready()

```

==== Make a Basestore and a dDatabase

```

const basestore = client.basestore()

const feed = basestore.get(dWebNetworkAddress) // key of a remote dDatabase

```

NOTE: If you pass a dWeb network address to `.get`, it will retrieve that dDatabase structure remotely (someone else’s published structure), if you don’t pass it a dWeb network address, a new dDatabase is created locally and a new dWeb network address (key) is generated for your newly created dDatabase.

==== Announce a dDatabase to the network (a remote or locally created dDatabase)

```

await client.network.configure(feed, {

lookup: true,

announce: true,

flush: true,

remember: false

})

```

Thanks to the new .replicate method, you could do this much easier like so:

```

await client.replicate(feed)

```

Any personal computer can run dHub’s daemon in the background and applications can interact with the daemon when creating and/or seeding dDatabase structures. All of these dDatabase structures are managed by the daemon and can be manipulated using the client library. An example of this is the upcoming dBrowser release which uses dHub in the background when interacting with dDrives and other dDatabase-based structures. This allows dBrowser to expand beyond dDrives to work with other dWeb-based structures. This will truly take dWeb-based applications to an entirely new level. In other words, when you type in a dWeb address in the address bar, or a dWeb network address is referenced within the code of a webpage (obviously within a dDrive), the dHub daemon is being used in the background to retrieve the dDatabase-based structure the browser is attempting to fetch, whether it’s being fetched from the address bar or a file. This could be a dWeb network address to a dDrive or even a dTree (a database).

====

IMPORTANT: The `.basestore` method creates what are called “RemoteBasestore” instances that mirror the normal Basestore API (https://npmjs.com/package/basestore), the `.feed` method creates what are called “RemoteDDatabase” instances that mirror the normal dDatabase API (https://npmjs.com/package/ddatabase), and the `.network` method creates what are called “RemoteNetworker” instances that mirror the Basestore Networker API (https://npmjs.com/package/@basestore/networker). For a full description of the @dhub/client API, head on over to https://npmjs.com/package/@dhub/client REAQDME, where there is an exhaustive description.

====

====

NOTE: As I just mentioned, dHub daemon can be ran on ANY personal computer and thanks to dWeb’s powerful UDP holepunching technology, most computers that are running the daemon will be able to exchange dDatabase-based structures amongst each other, as long as they are shared on the network, as shown in the above example.

====

dHub’s new ecosystem makes it easy to create new dWeb-based data structures and consume/seed remote structures programmatically, which greatly simplifies our ability to create dWeb (via dBrowser) and desktop-based applications that need to consistently interface with these structures. A great example of this would be a social network, where every post keeps the media associated with it in a separate dDrive. A normal user could be creating/seeding upwards of 8–10 dDrives per day and viewing/seeding hundreds or even thousands of dDrives that belong to the people they follow, just by browsing their feed.

With all of that said, developer tools can be built on top of dHub to create/manipulate specific structures (i.e., create or traverse a dTree), which leads me into my next post, where I will formally introduce the all-powerful dWeb CLI.

====

WHAT’S NEXT?

The rollout of the all-new dWeb CLI, and how dCasts, dTrees, dTries and dDrives can be created/managed via the command line.

====

--

--

Peeps

The creators of the #dweb and the world’s first decentralized and censorship-resistant social network. We’re about to #DecentralizeEverything.