LFS Series — The dSwarm DHT, Peer Discovery and dWeb Networking Tools
Like dDrive, dSwarm (formerly known as dWebSwarm) has been around since the inception of the dWeb but has improved greatly over the last two versions. Initially, dWeb network addresses were stored on a centralized DNS server but in dWeb 2.0, network addresses were moved to dSwarm’s own Kademilia DHT, where today, peer discovery on the dWeb is completely decentralized. With the rollout of dWeb 3.0, there are several new tools and libraries that have greatly improved the processes surrounding dWeb network address announcement, peer discovery and more. There are even tools for you to launch your own dWeb!
==== THE DSWARM DHT
dSwarm’s DHT has been around since dWeb 2.0 and is explained in-depth via dWeb’s documentation (https://developers.dwebx.org/protocols/dwdht). Its functionality is the same, where network addresses in the DHT are “keys” and the value for those keys is simply an object of peers that are swarming (sharing) those keys. dWeb network addresses can be announced and discovered on the DHT programmatically like so:
```
const dht = require(‘@dswarm/dht’)
const crypto = require(‘crypto’)
const node = dht({
ephemeral: true
})
// create a dWeb network address
const topic = crypto.randomBytes(32)
// announce the network address on the DHT
node.announce(topic, { port: 12345 }, function (err) {
if (err) throw err
// now try and find it
node.lookup(topic)
.on(‘data’, console.log)
.on(‘end’, function() {
//unannounce it and shutdown
node.unannounce(topic, { port: 12345 }, function () {
node.destroy()
})
})
})
```
NOTE: Within dSocial’s application, every time a user creates a post that has attached media, that media is packaged within a dDrive and announced programmatically on dSwarm’s DHT, just like we did above. Once announced, the announcer is the initial peer added under the address whom peers can download from. New downloaders are also announced (by design), so that the media is distributed.
You can learn more about the `@dswarm/dht` library at https://github.com/dwebprotocol/dht
==== DSWARM’S CORE API
Announcing network addresses, discovering peers within that network address’s swarm and connecting with those peers is actually much easier when using dSwarm’s high-level API via the “dswarm” Node module. Below is an example of how you can announce an address, join the address, lookup peers under the address and manage incoming connections to other peers within the address’s swarm:
```
const dswarm = require(‘dswarm’)
const crypto = require(‘crypto’)
const swarm = dswarm()
// look for peers listed under a dWeb network address:
const topic = crypto.createHash(‘sha256’)
.update(‘let-freedom-stream’)
.digest()
swarm.join(topic, {
lookup: true, // find and connect with peers
announce: true // announce yourself as a connection target
})
swarm.on(‘connection’, (socket, info) => {
console.log(‘new connection!’, info)
// you can now use the socket as a stream:
process.stdin.pipe(socket).pipe(process.stdout)
})
```
For more information on the dSwarm module, go to https://github.com/dwebprotocol/dswarm
==== DSWARM REPLICATOR
A new feature within the 3.0 release is dSwarm’s replicator library, which allows developers to easily replicate dWeb-based data structures using dSwarm. For example, you could replicate a dDatabase like so:
```
const replicate = require(‘@dswarm/replicator’)
const swarm = replicate(aDDatabase, {
live: true // passed to .replicate on the ddatabase class
})
```
In the above example `swarm` is a dSwarm instance (using the dswarm Node module) that replicates the passed in dDatabase instance. You can use the following options in the replicate instance:
- bootstrap — optionally set the DHT bootstrap servers (default: dht1.dwebx.net, dht2.dwebx.net and dht3.dwebx.net)
- live — passed to .replicate
- upload — passed to .replicate
- download — passed to .replicate
- encrypt — passed to .replicate
- discoveryKey — optionally set your own dWeb network address
- announce — should the dSwarm announce you?
- lookup — should the dSwarm do lookups for you?
- keyPair — NOISE-based keypair for connection encryption
- onauthenticate(remotePublicKey, done) — the onauthenticate hook to verify a remote’s keypairs
For more information on the dSwarm replicator, head on over to https://github.com/dwebprotocol/replicator
==== DWEBSIGN & DSWARM DHT CRYPTOGRAPHIC FUNCTIONS
dWeb 3.0 includes a new library for signing Mutable entries on dSwarm’s DHT or any dWeb-based DHT for that matter (that includes a dMesh DHT). Using the `@dswarm/dwebsign` library, you can:
- generate a keypair
- create a random or hashed salt value,
- create a digital signature based on a specified value
- method for facilitating hardware signing (out-of-band signing)
The `@dswarm/dwebsign` Node module is builtin to the dSwarm DHT module (@dswarm/dht) but can be used directly as well for other purposes.
You can learn more about @dswarm/dwebsign at https://github.com/dwebprotocol/dwebsign
==== DSWARM TUNNELING
Included in the new dSwarm rollout is a new “tunneling service”, which allows two peers to communicate over an encrypted tunnel. Example usage of the tunneling service, is below:
```
const { Remote, Local } = require(‘@dswarm/tunnel’)
// On a remote computer, run the tunneler
const r = new Remote()
r.listen(10000)
// On a client, you can start a server that’s being announced on dSwarm’s DHT
const l = new Local(10000, ‘remoteserver.com’)
const s = l.createServer((socket) => {
// a remote socket
})
s.listen(hash(Buffer.from(‘let-freedom-stream’)))
// Or on the client, you can create a client connection
const s = l.connect(hash(Buffer.from(‘let-freedom-stream’)))
```
The tunneling service also allows ships with a CLI, which you can use to launch your own tunneling server like so:
```
npm install -g @dswarm/tunnel
dswarm-tunnel-server — port 10000
```
To learn more about @dswarm/tunnel, head on over to https://github.com/dwebprotocol/tunnel
==== DWEB CONNECTION TESTING
From time to time, people get paranoid and wonder whether the dWeb is still working. Well, it is and always will! But, if you’re losing your mind and you’re curious, you can test connections on the dWeb under a network address that derives from the string “connect-test”, by using the new `@dswarm/connect-test` CLI. You can also test to see whether your network is hole-punchable.
To do so, simply do the following:
```
npx @dswarm/connect-test
```
You should see something like this:
```
dSwarm Connect-Test
Testing hole-punchability…
Your network is hole-punchable
Joining dSwarm under the sha256(‘connect-test’) topic
Waiting for connections…
New connection!
New connection!
New connection!
```
==== THE DSWARM CLI
One of the cooler features with the new dSwarm release, is the all-new dSwarm CLI. dSwarm’s CLI can be used to interact with the dSwarm DHT or to launch your own dSwarm DHT, all via the command line. In other words, you can launch your own dWeb. dSwarm’s CLI can also be used to perform command line-based DHT address lookups and announcements and even provides a way for developers to see the approximate node count of dSwarm’s public DHT or even their own private dSwarm DHT.
To install dSwarm’s DHT, run the following via the command-line:
```
npm install @dswarm/cli -g
```
To see all of the options included with the CLI, simply run `dswarm`, which will print the help menu.
=====
WHAT’S NEXT?
Learn how dSwarm’s DHT is being used to enable advanced peer-to-peer communication services like voice calling, 1-to-1 encrypted piping, messaging and even SSH
=====
