Categories
trust

A Trust and Moderation System for the Decentralized Web

Public social networks, comment sections, and discussion forums are broken. The internet promised limitless access to knowledge and thoughtful, intelligent conversations with people from all over the world. Instead, we have trolls, manipulators, outrage, and spam. 

It might seem like decentralization would make these problems worse. After all, most decentralized or unmoderated social networks turn into dumpster fires of low quality content over time. Without moderators or the threat of being banned, the worst users feel emboldened to spread their drivel across the entire network. As quality degrades, the best users are turned off and the network becomes less useful and engaging. 

But there is a solution that gives power back to the people, allowing them to moderate their own world, working with friends to keep it free from low thought discourse. It works by mirroring the real world: give everyone a reputation and the ability to ignore those with a poor reputation. 

In this article I’m going to show not only how a decentralized moderation system can be as good as existing systems, but also surpass them, creating a higher quality social network.

But first, let’s take a step back and see what centralized moderation gets wrong.

Centralized Moderation and its flaws

Centralized moderation refers to social networks or forums where one central entity, such as a corporation or the owner of the site, has sole moderation authority over the contents. There is usually one set of rules that all users must follow and if skirted the user’s content is deleted and repeat offenders are banned. 

The flaws center around content that some users enjoy and others do not and how you deal with that content. 

With centralized moderation we all have to play by the same rules, so sites have to do the best job they can. If they are overly harsh in moderating they’ll have a good signal-to-noise ratio, but the content may become overly partisan or boring. If they’re too lenient the content will get noisier which makes it less enjoyable as there are then more low effort posts to sort through. And some of the worst content will turn many people off.  

Content Spectra

Imagine rating every piece of social media content along a content spectra of the following:

  • Low Quality → High Quality
  • Low Offense →High Offense
  • Accurate → Inaccurate

Consider where the content you enjoy most fits on these spectra. It’s likely not a fixed point, but rather a range of what is considered acceptable and a smaller range of what is considered ideal.

The worst content generally lies at the extreme of these spectra. Spam is extremely low quality, Haters are High Offense, and Manipulators are Frequently Inaccurate. 

So, a site has a good signal-to-noise ratio when a good percent of the content falls into one’s ideal zone. 

Users want to see content that falls into their ideal zone, rather than have to hear everyone’s opinion. If too much falls outside their ideal zone they’ll often become bored and leave the site.

Sometimes heavy moderation of content that falls outside of one’s “ideal zone” is a good thing, because it means you’ll enjoy the content you read more, without having to wade through purely acceptable content.

The central flaw of centralized networks is that by having a set of rules that all users must abide by, sites are presetting the allowable content spectra. This content spectra not only decides the kind of users that will be allowed on the site, but also those attracted to the site, because the users whose content spectra more closely match the website will tend to stick around more.

It turns out centralized services have thought about this problem over the past decade and have come up with a solution: algorithmically determined feeds. If everyone has their own feed, an AI algorithm can learn what one’s ideal content spectra is and show the content that most appeals to them. This feed is a filter on the world that helps one view the site in a curated “ideal” light.

Algorithmically determined feeds are the next stage in the evolution of social networks, but these feeds have their own set of problems.

The Algorithmically Chosen Feed Problem

Having a social feed driven by an algorithm can be useful for surfacing all the content you’d like to see most on a social network, but it also has dark sides that most rarely think about. 

The biggest problem with utilizing algorithms to customize social feeds is that the user has no insight into the decisions the algorithm makes. With the rise of neural networks, at times even the developers of these algorithms are unable to explain their internal logic.

When a post is made, even one into which a lot of time and effort has been put, the algorithm may decide that it’s low quality, rank it lowly, and cause it to never appear in friends’ algorithmically chosen feeds. Under this paradigm, neither a user nor their friends’ opinions matter, the algorithm has decided and its decision is final. 

Furthermore, because these algorithms are applied to millions or even billions of people, there is a large financial incentive for manipulative companies or organizations to reverse engineer them. If they can figure out how these algorithms work, companies can tailor the algorithms to prioritize their own content to reach the top of as many feeds as possible. This becomes worse if the social network’s financial incentives are aligned with the company’s. Even if the social media network itself is completely benevolent, this will always be an issue. 

One final flaw of these algorithms is the incentive for social networks to keep you hooked on their product. This means that the algorithms are optimized to display content you like and hide the content you don’t like. Though useful in moderation, this can quickly turn dangerous if optimized for dopamine hits and short term emotional appeal over mental health and well-being. 

Who chose the moderators?

Centralized moderation has a fundamental, rarely discussed, issue: The idea that a third party is needed to moderate user’s content. 

Why can’t we decide that for ourselves? Why do we have to be subject to the whims of covert moderators and opaque algorithms that decide who is allowed and what content is acceptable?

Decentralized worlds do not have to be structured in this way. Decentralized networks have the potential to fulfill the original intent of the internet: worlds open to everyone without global rulers in which everyone is able to participate and moderate their own domain. Users choose who to trust, who to ignore, and how one’s feed is ordered. 

The Solution: A Decentralized Moderation System that scales to everyone

To have the best social experience possible, moderation must be personalized. Individuals must be able to decide who can be a part of their world and who cannot. It’s not possible for any company to make policies that millions approve of and that do not have downsides and unintended consequences.

This doesn’t mean that global moderators are no longer necessary. They are still incredibly useful, just no longer mandatory. Instead, moderators should be opt-in, leaving an individual, or group of individuals, to choose the moderator for one’s “world.”. This allows users to benefit from companies that specialize in detecting and eliminating spam, bot networks, or Astroturfing, without being forced to use them. 

How it works

In real life trust comes from friends and family, the people someone already trusts to help determine who can and cannot be trusted. When a friend begins to associate with bad people one either tries to steer them away or one’s trust in them falls. 

This is exactly how the new decentralized trust system works. A user rates the trustworthiness of some of the people encountered online, and then trust of everyone else is calculated based on these ratings. In this implementation the trust ratings must be public, so that everyone can calculate the trust of friends of friends. In the future a more private trust system could be developed along similar lines. 

The second part is a filter that a user can apply to their social feed to hide everyone with a trust rating below a particular threshold.

Now this isn’t a finalized spec, it’s a rough outline of how this system can work. This essay is about why we should build this and how it could work, more than exact implementation details.  

To bootstrap the system the user would first give a rating to the people they trust/distrust on the network. These might be people known in real life or not. Ratings can be between -100 and 100. This rating should be how much a user trusts them to post content that is similar to their ideal content spectra, because these ratings will shape the content seen on the social network. 

When a user encounters new people online they can give them +/- 1 votes based on individual pieces of content posted. For obvious spammers or trolls one could immediately give them a negative rating. 

So, while individual pieces of content can be rated the trust rating is given to a person. This is because most people post content in a similar content spectra. However, it is important that trust is also linked to content so that when a user asks the question “Hey, why does my friend distrust this guy?”, they could see the exact piece of content that caused the distrust.

After a few trust ratings have been given, the algorithm will go on to calculate the trustworthiness of friends of friends (two degrees of separation away) and even their friends (three degrees away). It does this by taking all the people a user trusts and all the people they trust, then multiplying the trust levels together to come up with one’s own personal trust rating of everyone else. Even if a user only rates ten people, then those ten go on to rate ten others and those rate ten others, this will result in the user having trust ratings for over 1000 people on the network. 

When a trust rating is given it is fixed and not affected by the ratings of others.

The people a user distrusts have no effect on their trust rating of others. This is just like in real life, where when someone encounters someone they don’t like, they don’t care who their friends or enemies are. It also prevents malicious users from gaming the system.

Now instead of having to individually block spammers, trolls and haters from your world, as soon as a trusted friend rates someone poorly, they’ll be hidden automatically, making moderation of obvious trolls or spammers much easier.

The Algorithm

The general algorithm is: the trust rating of anyone who does not already have a set rating is the square root of how much a user trusts a mutual friend multiplied by how much the mutual friend trusts the stranger. In the case of negatives the negative is ignored when calculating the square root (so there’s no imaginary numbers). When a user has multiple mutual friends their scores are summed before the square root and that number is divided by total raters (so that someone who’s popular isn’t naturally more trusted). 

There is a small flaw with this algorithm. If a user has a friend Alice that is rated at a ten, and then they have a best friend Bob that is rated at 100, the score for bob will be (10)(100)= 31.6. 

This is weird because the user doesn’t trust Alice that much, so why would they trust Bob more than her? We can make an improvement to address this: the trust level is capped to how highly the mutual friend has been rated.Similarly to real life, if an acquaintance brings a friend to a party, it doesn’t matter how much they say they like them, they still will not be trusted higher than that acquaintance.

Here’s a demo to showcase this system in action, with randomly generated users that randomly trust other users are random levels:

Total Users
AVG Ratings
Per User
Trust Depth

You can adjust the total users and average ratings per user to see how the system works as more people join in and use it. You can also adjust the depth, or degrees of separation, that the trust ratings are calculated for.

The arrows represent how much each user trusts the user the arrow is pointing to. The color of each user represents how much they are trusted by the selected user (the blue one), calculated via the algorithm described in this post. Bright green represents 100 trust and bright red -100 trust, with a gradient for scores in between.

The source code for this demo is open source on Github. I’ve implemented a simple version of the trust algorithm in JavaScript which you can use for your own social network if you wish and it’s already fast enough for most use cases.

Technical Implementation

This can be implemented on top of Scuttlebutt, or any other open source social network. With Scuttlebutt the only requirement is a trust plugin which allows users to post trust-rating messages that contain a user/feed ID and their rating of that user. With that information, the algorithm can calculate the trust rating of all users based on the trust-rating messages everyone has posted. 

In regards to user experience, this plugin could add upvote/downvote buttons to each piece of content that increases or decreases trust in the user that posted that content. The trust filter can be a simple drop-down in the top right of the feed that, when a trust level is chosen, all posts by users with a trust rating below that level will be hidden. 

In terms of performance, the demo above utilizes a simple algorithm in JavaScript, and even this is able to re-calculate the trust of 100,000+ people in just a few milliseconds. You can see the benchmarks here.

This can scale to hundreds of millions of users because each user only needs to calculate the ratings of a few hundred to tens of thousands of people. Because it’s open source this algorithm can be updated and improved over time if better ways of calculating trust are discovered. A written text example of this algorithm is in Appendix A.

Additional Benefits of Decentralized Trust

Choose your own moderators

It’s still a good idea to have large organizations become moderators, because they have more resources and clout than individuals. The difference is, in a decentralized world, one can choose their own moderators. Imagine a future where the New York Times has their own “Trust Beacon” user on the platform. Whenever they report on fake news or fake content , this trust beacon can mark the content and those who post/create it at say -10 trust. 

If a user trusts the New York Times, they can rate it at say 50 trust, and it will apply it’s trust ratings to  their social feed automatically. It’s the same as trusting any other user. 

These trust beacons could be set up by any media companies, fake news catching websites, or even companies like NetNanny that want to filter out content unsafe for kids. People can trust anyone they like and then those trust ratings will be applied to their feed. This way a big network of friends is not needed to help filter content, these companies can do so if desired.

Trust can be linked to posts or images

This trust system doesn’t need to only apply to users. It can be applied to individual posts or pieces of content. For example, on Scuttlebutt every piece of media is stored as a blob, where the filename is a hash of it’s contents. This hash is essentially a fingerprint of image, video, song, etc. on the network. 

A trust beacon such as Snopes that debunks false information can have an account that flags images that are false. Because that image can be identified across the network by its hash, if a user subscribes to the Snopes trust beacon it can automatically flag and filter this content for them even if that image was reposted 100 times before they saw it. 

This could also be done with pornographic or violent content that’s unsafe for kids. Services such as NetNanny could maintain a list of these adult only images and videos and automatically block them for children, creating a safer space for them to explore than even the major networks can manage. 

You choose what trust means to you

The interesting thing about this system is that a user doesn’t have to trust people based on truthfulness. Trust is an arbitrary number that can be used to represent anything one would like to see more of in your social world. If a user wants trust ratings to be based on shared beliefs, positivity, or even geographical proximity that’s up to them. 

Imagine a world where instead of everyone having one social profile, they have one profile for every topic they’re interested in. One could have a profile for programming, for politics, for games, for decentralization, etc. If this becomes commonplace then one’s trust in someone can be directly related to their area of expertise. Trust is often not in the person but in one’s confidence that they know about whatever they’re talking about at the time. 

Alternatively, it could be implemented in such a way that people have multiple trust ratings, each tagged by topic. However that would add a lot more complexity, especially around identifying what topic each post belongs to. 

Creating Small Communities again

Most of the time friends are chosen through serendipity. It’s been shown that the secret to a friendship blooming is multiple random encounters. One reason people rarely make friends via major social networks (Twitter, Facebook, Reddit, etc.) is because they are so large and spread out people rarely encounter the same person twice if they’re not already friends. 

Imagine if your social feed showed everyone you follow, as well as everyone they follow, with their rank based on trust ratings. You may discover some good friends of friends that you’ve never met posting content that you really like. Over time you could comment on their posts, contribute to the conversation, and, eventually, form a relationship. 

The trust system would allow the best friends of friends to appear in user’s feed more often than a random crazy uncle, posts from groups, or even barely known acquaintances. This could lead to new connections and many amazing new friendships with people that were previously just out of contact. The more interests and people in common, the more users will see each other online and the more chances they’ll have to connect.

This is internet scale moderation

The future will be full of AIs and bots that create hundreds of thousands of accounts in order to take over social networks. With decentralized trust all it takes is one friend or trust beacon to notice these bots and distrust them. It needs to be possible to scale moderation to the level where it can combat thousands or even millions of fake users and bots. It can’t scale by having designated moderators, it can only scale if everyone participates in moderating some portion of the world. 

Downsides of Decentralized Trust

Now there are potential downsides to this system. As with anything, when  people are given freedom some people are going to use it poorly. That’s okay. People shouldn’t be told what to do anyway.

The first obvious issue is the creation of echo chambers or online cults, where instead of choosing truth and seeking the best information, people trust those who confirm their existing beliefs.

This can be somewhat mitigated with the ability to set a trust filter level to a low level, even in the negatives, to see posts and opinions one may not usually be exposed to. This again is up to the end user if they wish to expose themselves to differing opinions or not. 

It is up to individuals to determine what’s best for them. Of course, one would hope they make good choices, but it is not society’s job to save people from themselves. It’s far better to have this freedom and the power to use or misuse it, than to allow companies to build the echo chamber behind closed doors. 

Conclusion

The implementation will be open source and open to anyone to participate. Implementation has already begun, however there is a lot of work to do. If anyone has heard of anyone building something similar it would be wonderful to meet so this idea can be brought to fruition.

It’s exciting thinking about the possibilities this system would bring. There is potential for more stimulating conversation threads without trolls trying to ruin the fun. With the current state of the social internet this is hard, but it’s something worth doing.

Appendix A:
Trust system example

Lets walk through an example, starting with Tom.

  • Tom has a best friend Alice, who he trusts at 100
  • Tom also has a friend Mike, who he trusts at 50
  • Alice knows a guy Dave, who’s a compulsive liar and trusts him at -20
  • Alice has a workmate Jeremy who’s an ok guy but she doesn’t know him well, so she trusts at 10.
  • Alice has a workmate Sophie, who constantly shares false outrage, but is otherwise a good person, who she trusts at -5.
  • Mike is best friends with Jeremy and trusts him 40
  • Mike knows Sophie and likes her, and trusts her 15
  • Dave has a best friend Barry, who he trusts 100
  • Sophie has a best friend Emily, who she trusts 100

While Tom has only rated 3 people, the Trust Software will now calculate his personalized trust ratings for all 7 people here:

First there’s the immediate friends, if you’ve rated someone that is the fixed rating for them, it isn’t influenced by anyone else:

Alice=100

Mike=50

Then the secondary friends:

Dave = \sqrt{(Tom.Alice)(Alice.Dave)} = \sqrt{(100)(-20)} = -45

Jeremy = \frac{\sqrt{(Tom.Alice)(Alice.Jeremy) + (Tom.Mike)(Mike.Jeremey)}}{Total Raters}

= \frac{\sqrt{(100)(10) + (50)(40)}}{2} = 27.5

Sophie = \frac{\sqrt{(Tom.Alice)(Alice.Sophie) + (Tom.Mike)(Mike.Sophie)}}{Total Raters}

= \frac{\sqrt{(100)(-5) + (50)(15)}}{2} = 8

Then the tertiary friends:

Barry is unrated as Tom.Dave < 0

Emily = \sqrt{(Tom.Sophie)(Sophie.Emily)} = \sqrt{(16)(100)} = 40

= \min{(Tom.Sophie, Tom.Emily)} = \min{(16,40)} = 16

There’s a few interesting things happening here. First, Sophie ends up with a slightly positive trust rating despite Tom’s best friend Alice saying she shares too much false outrage. This is because Mike still trusts her. If Tom notices that she does share a lot of false information he can either rate her himself which will mean his friends ratings no longer apply, or he can rate Mike lower because it’s becoming obvious Mike doesn’t care about truthfulness as much.

We also see that Barry’s rating is not calculated at all (so would default to 0). This is because if we don’t trust someone we also don’t trust anything they have to say about the reputation of others, so the system can’t be gamed by malicious actors.  Similarly if Tom rated Sophie at a negative number then her rating of Emily would no longer apply and Emily would have the default rating of 0.

Lastly we see Emily’s rating is capped to the same rating as Tom.Sophie, because we never rate friends of friends higher than the mutual friend.

All the people above are now in a big discussion about politics and Tom can see this discussion happening in his feed. As Tom wants to be well informed without liars being involved he sets his trust filter level to 10, so everyone with less than 10 rating will be hidden. This allows Tom to filter his world without investing much effort in the process. In this feed he would only see posts from Alice, Mike, Jeremy and Emily. Posts from Sophie, Dave or Barry would be hidden.

Categories
scuttlebutt

StudyTime – A Decentralized MOOC Platform

In 2016, Coursera, one of the leading providers of massive open online courses (MOOC), decided to remove several less popular courses from their site because they weren’t profitable enough to maintain. One of my core beliefs is education should be open and free; anyone that has the passion to learn should be able to regardless of their income, background or location. Unfortunately, when education meets capitalism, those with capital decide what courses live and die while those with a passion for learning miss out.

When Coursera announced they were taking these courses offline I started looking around for a MOOC platform that would host them instead. While many of these courses found their way onto academictorrents.com, and other less legitimate places, these are zips of individual PDF’s and videos, not all-encompassing experiences like Coursera’s classes.

Instead of a centralized service that can increase prices or delete courses at any time, what if we built a decentralized app that anyone can use to explore license-free MOOC’s? They’d never have to worry about a course being taken down or censored, because it can be streamed from anyone else on the network.

Let’s explore the design of a hypothetical application – StudyTime – to illustrate this idea. This is a design document for an application that doesn’t yet exist, but is possible to build.

StudyTime Design Document

StudyTime is an application for viewing and interacting with MOOC’s that runs on Secure Scuttlebutt (SSB). It works by streaming all metadata for courses from the SSB network via SSB Feeds. All non-text content is loaded via SSB blobs, though it can also be made available on other distributed systems such as Sia or Torrents. Because all data is stored locally and mirrored to other clients, the network is self-sustaining as long as it has users.

The application creates its own private SSB key and feed upon first load. Anything that needs to be persisted, such as enrolled courses, course progress, personal notes, friends, etc., is stored in this feed. More interaction between students, such as forums or comparing results between a class, could be added over time using the same back-end and user feeds. Implementing these features would require a lot of thought around moderation and handling spam or trolls and, as such, is out of scope for this design.

User Interface

Upon opening StudyTime you are presented with a list of possible subjects to choose from, grouped into major categories with the most popular subjects given higher priority. This could look similar to the homepage for Class Central:

Screenshot from Class Central Homepage – select subject section.

After selecting a category you’re taken to a search page where you can explore all courses tagged with this category. All searching and filtering is done client side.

Screenshot from Class Central – Select Course page

When selecting a course, you’re taken to a course overview page that consists of the course description, estimated duration, a button to start that course, etc.. 

Screenshot from Class Central – Course Overview page

Most of this front-end design can be based off existing MOOC sites. It’s the back-end of this application that is really interesting.

Back-end Implementation

First, there is one or more course list feeds. Each course in the feed has the type studytime_course. Each course item includes a name, description, keywords, and feed ID. One course list feed ID is hard-coded into the application and is downloaded or updated upon opening the application. Because the course list feed is the primary data source and only one person can add items to the feed, having a hard coded feed creates centralization. The application should not be centralized in any way, so to fix this users will be able to change the course list feed to one created by someone else. Users can load multiple course list feeds (perhaps MOOC providers could release their own StudyTime feeds), and the application will load the courses listed in each. If a course exists in multiple feeds it will be de-duplicated by feed_id and the metadata for the first version found will be displayed in the app. 

Here’s what an item in a course list feed looks like:

{
    author: '@masterfeed.ed25519',
    content: {
        type: 'studytime_course',
        name: 'Python Essentials',
        image: '%c7ZAvQRoEP/zXnXgTnkGafP1JSEV29V8G/+Hhk8lSWU=.sha256', 
        description: '',
        keywords: ['python', 'beginners programming', 'computer science'],
        feed_id: 'Pyth0NEsential5.ed25519' // Scuttlebutt feed containing the course's lessons, videos, etc
    }
}

While the user is browsing, StudyTime downloads the course feeds in the background, so there is little delay when opening a course. The courses will be grouped into categories client-side. Users can search, sort, and filter client-side using the name, description and keywords. 

Each course has its own feed, where the author_id is the feed_id in the master feed. Each course feed contains the course content; the lessons and tests it contains, as well as hashes of all the images and videos in the course. When you view an image or video, StudyTime will request it from the SSB network then stream it from other peers who have a local copy. After an image or video is downloaded, the client will automatically share it with other peers to keep the network highly available. Here’s what a course feed looks like:

[{
    author: '@Pyth0NEsential5.ed25519',
    content: {
        type: 'studytime_course_lesson',
        lesson_number: 1,
        name: 'Week 1 - How to install python',
        description: 'This lesson will teach you how to setup Python on your PC',
        lesson_content: 'In this lesson we will learn about installing python...'
    }
},
{
    author: '@Pyth0NEsential5.ed25519',
    content: {
        type: 'studytime_course_lesson_video,
        lesson_number: 1
        name: 'Week 1 Video 1',
        description: 'How to install Python',
        sources: [
            '%c7ZAvQXoRP/zXnXgTnkGatP1JSAV29V8G/+Hhk8lSWU=.sha256',
            'https://siasky.net/AABAC_3Dt0FJsxqsu_J4TodEETCGvtFf1Uys_3EgzO0Tcg',
            'magnet:?xt=urn:btih:a3dced3979cc1a30cc7f648673ced688ce78ce77&dn=studytime-python-essentials'
        ]
    }
},
{
    author: '@Pyth0NEsential5.ed25519',
    content: {
        type: 'studytime_course_lesson',
        lesson_number: 2,
        name: 'Week 2 - Hello World',
        description: 'This lesson will teach you the basics of python programming',
        lesson_content: 'First lets run ipython with the following command:\n`ipython`...'
    }
}]

The front-end uses these feeds to render the course contents and lesson pages, downloading required non-text content as users progress. It pre-fetches all course content in the background after starting a course, so users don’t have to pause at the beginning of each lesson. While browsing through courses, users can navigate quickly as the application is only rendering local data from the feeds.

Updating and Improving Courses

Over time courses may become out-of-date and need to be improved, edited or deprecated. SSB feeds are immutable append-only feeds, so the only way to make changes is via adding messages. A message type studytime_course_update is used to make these updates. Here’s an example of how the Python Essentials course above can be updated:

{
    author: '@masterfeed.ed25519',
    content: {
        type: 'studytime_course_update',
        feed_id: 'Pyth0NEsential5.ed25519'
        name: 'Python 2.7 Essentials',
        keywords: [],
        deprecated: true,
    }
}

This update renames the course to Python 2.7 Essentials, and sets a deprecated flag. StudyTime hides all deprecated courses by default, but could have a “Show Deprecated” option for users that want to see them.

Similarly, course lessons can be updated with a studytime_course_lesson_update message, which contains a new name, description or lesson_content. The lessons will use the latest content when rendering.

Why build this on Scuttlebutt?

Scuttlebutt has a few advantages over building a traditional application with servers:

  • The creator doesn’t need to raise money or take down “unprofitable” content to keep the application running, because the community runs it with their collective computing and networking power. 
  • Provided there are enough users, the application should never go offline, as users are constantly sharing feed data with each other.  Because the content is being mirrored so many times, you will always be able to find a feed. Exceptions include images and video, which are harder to keep constantly mirrored, as they are only downloaded by the people that take a course. As a remedy, these images and videos could be mirrored to a service such as Sia or even S3 as a fallback. 
  • Content cannot be censored or deleted by authorities.

These rules apply to any decentralized application built on Scuttlebutt. I am particularly excited about the first, because there are so many free websites I’ve loved that have closed down over the years simply due to lack of money. If the community supported them with bandwidth and computing power instead, they would last as long as they still have fans.

What are the drawbacks of using Scuttlebutt?

The downsides to using Scuttlebutt rather than centralized servers are mostly constrained to the initial release of the application:

  • If there are very few users online it may take a while to find and stream lesson content.
  • The developers of StudyTime may need to mirror most of the content until at least a few people have taken every course and are actively seeding the content. 
  • SSB feeds cannot be changed, so there is no way to remove content once it’s been added. While it can be marked as deprecated/deleted so the application can hide it client-side, users will always have the information. 

I hope someone either takes this idea and runs with it or becomes inspired by the potential this network brings. There are so many similar problems in many industries that could be saved with a decentralized back-end like Scuttlebutt.

Categories
scuttlebutt

A Decentralized Platform for Innovation

In what is scuttlebutt I discussed why Scuttlebutt can be a better social network than anything else out there. However, it can be so much more. It can be a decentralized platform that any application can be built on top of and immediately become peer-to-peer.

Under the hood, Scuttlebutt messages are just JSON data. Here’s what a normal text post looks like (simplified, additional metadata omitted for clarity):

{
    author: '@Py8stqMfjhdc4Ln92R4Lg+3jITszP94G3P5BNuH+lWY=.ed25519',
    content: {
        type: 'post',
        channel: 'new-people',
        text: 'Hello World!'
    }
}

And here’s what a vote looks like:

{
    author: '@Py8stqMfjhdc4Ln92R4Lg+3jITszP94G3P5BNuH+lWY=.ed25519',
    content: {
        type: 'vote',
        vote: {
            link: '%c7ZAvQXoDP/zXnXgTnkGatP1JSAV29V8G/+Hhk8lSWU=.sha256',
            value: 1,
            expression: 'Like'
        }
    }
}

Notice how the content sections of the post and vote are different? The only thing that is similar is the “type”, meaning the content section can be filled with anything you like. Different applications can send their own messages with their own ‘type’ and information, and clients that understand that type can display the information in a way that makes sense. 

For example, you could build a chess application which sends content like:

{
    author: '@Py8stqMfjhdc4Ln92R4Lg+3jITszP94G3P5BNuH+lWY=.ed25519',
    content: {
        type: 'chess_move',
        game: '%c7ZAvQXoRP/zXnXgTnkGatP1JSAV29V8G/+Hhk8lSWU=.sha256',
        move: 'Qe4'
    }
}

This data is then sent along the Scuttlebutt network where any of your friends who have a chess client that can read the type “chess_move” can see that you’ve made the move “Qe4” which in Portable Game Notation means you moved a queen to e4. You could even play games of chess with people with other chess applications, as long as they can process this “chess_move” type. In fact someone has already built a ssb-chess backend that runs on Scuttlebutt and you can play it today

Because everything is open source and you don’t need permission to join the network, you’re free to innovate as your heart desires. You can build any kind of application using Scuttlebutt as a network layer, and it’ll be decentralized and usable by anyone else on the network right out of the box. 

For instance, someone could build an Instagram-like application that connects to Scuttlebutt, and shows all content of type ‘image’. Someone else could build a Twitter-like application that only displays text posts of less than 280 characters. These applications will talk on the same network as Patchwork, Manyverse, Patchfox, etc., and you can interact with friends using different applications seamlessly. 

This means that you are free to choose the social application you use while still being able to connect to your friends on the same network. What if the developers of Patchwork do something you hate, like add sponsored content, rearrange the feed order, or make it neon orange and green for a real 90’s flair? Instead of being forced to “deal with it” like you are with Facebook or Twitter, you can simply download an alternative application like Manyverse or Patchfox and continue talking to all your friends with the same content. If you’re a programmer, you can even fork Patchwork, or create your own client from scratch just the way you like it. The application you use to view the world may constantly change, but the underlying data always remains the same. 

This would allow Scuttlebutt to live on for decades or even centuries. Social clients may evolve and change over time just as they do now. People will build applications which will grow and die on it. But the underlying data remains the same, and you’ll never need to take your entire social world with you when switching clients again. 

That’s what I love about the potential of Scuttlebutt. It’s not just freedom from corporate control, it’s freedom from anyone telling you what to do and how to interact with your friends. It’s freedom form the invisible hand nudging you to do things others want you to do, like clicking on ads or using their new live streaming product. 

Remember AOL versus the Internet back in the 90’s? Initially everyone loved AOL as it was this beautiful walled garden that gave them news and information and games. But then the internet came along with infinite possibilities; a decentralized world where anyone can create and share and build what they like! It provided so much more utility and freedom that everyone flocked to it. That’s how I see Scuttlebutt today, the free alternative to the social media companies that only seem to be tightening their grip on their user-base, building a bigger wall around their garden every day. 

Categories
scuttlebutt

What is Scuttlebutt?

As mentioned in Why Decentralization, a big focus of this site is the technologies that will make a decentralized world possible. Scuttlebutt is one I’m most excited about because it has a usable social network, and more applications can be built on it for free, without permission, by anyone. 

Scuttlebutt solves multiple problems plaguing the social networking world:

  • One company having control over and access to everything people do and say on the network
  • AI algorithms sorting your timeline to decide what you should read, taking control away from you
  • Ads and the ability for companies to purchase attention, shoving themselves into your personal social timeline
  • That people have to have one profile tied to their real self 

Technically Scuttlebutt isn’t a social network, it’s a system for building any kind of decentralized application, but explaining it in the context of a social network helps. Later I’ll be explaining how it has the potential to be so much more than this.

Scuttlebutt – the social network

Decentralized, open-source social networks have been tried before, the two most well known are Diaspora and Mastadon. With these services there are still servers that people gather around, so if you want an account you have to find a server and register for it. There is still moderation, there is still federation (you have to register an account, and you can get banned) and there are a few central points of failure (the servers themselves). While these networks are a great first step away from centralized corporate-controlled networks (I like them and will be exploring them in more depth soon), Scuttlebutt is how I believe the social web should function in the future.  

Scuttlebutt’s fundamental difference is it is fully decentralized. Scuttlebutt is completely peer to peer, meaning there are no central servers that you connect to, and no one manages or controls the network. You don’t use a website to view Scuttlebutt, instead you download a client to your computer. When you use a client, like Patchwork, it talks directly to your friends over the internet. When you make a post it gets sent to all your friends, so they have a local copy of it. If a friend comes online while you’re offline, their client will automatically download your post/s from your mutual friends. Messages can spread between people constantly without everyone always needing to be online, just like the real world. 

Patchwork Screenshot
Patchwork – A Scuttlebutt client

When you download Patchwork and run it on your PC it’s initially empty, because you don’t know anyone and don’t know how to connect to anyone. You’ll notice you only need to enter a name, picture and description, there is no signup form. Why? Because Scuttlebutt has no central service to register with. You can be anyone you like, and no one can tell you otherwise. Instead of having a registration service Scuttlebutt creates a private key on your computer. This key is used to prove you are who you say you are to your friends, so that no one can impersonate you, and it’s controlled by you alone. 

To begin you should connect to a “pub” with an invite code. A “pub” is a super-friend on the Scuttlebutt network who makes friends with everyone. Pubs are just like normal users except that they’re almost always online, identify as pubs, and will friend you automatically when you redeem an invite code. After you redeem an invite you become friends and can start downloading information about its other friends, their friends, and those people’s friends (3 degrees of separation). These pub servers are mostly run by Scuttlebutt developers. Connecting to a pub doesn’t restrict your network in any way, as they are just super-friends, not central servers. 

While connecting to a pub isn’t required, until you have a few friends it’s worthwhile friending one. For one, pubs are almost always online, so whenever you come online you get all the latest gossip, instead of having to wait for friends to login. Second, pubs can help you discover more people, as you can communicate with anyone up to 6 degrees of separation away (3 degrees in each direction), and pubs are connected to other pubs, so they connect you to the whole Scuttlebutt universe.

Once you’ve downloaded the local gossip from the pub, you’ll see you now have a social feed in Patchwork. This feed is comprised of everything your friends are up to (of which you have 1 right now, the pub) and everything their friends are up to (2 degrees out). Though your client has information about people 3 degrees out for network redundancy purposes, you don’t see them in your feed by default. You can see them under the “extended network” menu option in Patchwork. So now you are connected to all these random people, what do you do? First I’d recommend checking out some of the hashtags/topics which you can see on the left, and subscribe to some that you find interesting. Once you’re subscribed to some topics you’ll see every post from everyone you know about that includes that hashtag.

 You can then follow others who have similar interests to you and post anything you like to the network. Have fun out there! If you want to follow me my ID is: @Py8stqMfjhdc4Ln92R4Lg+3jITszP94G3P5BNuH+lWY=.ed25519 

Categories
vision

Why Decentralization

For the longest time humanity was decentralized. We had small communities of people who knew each other, maybe they had a king far away, but they didn’t really have anyone telling them how to live their lives, they were free to do what they want.

Along came the era of industrialization and some people discovered a thing called “economies of scale”. By building companies to control many people at once you could extract more value than anyone else.

Eventually social media happened and we were all so used to companies ruling our lives that we happily joined a world with all our friends, blissfully unaware of how much power the owner of this world was accruing.

Along comes the 2010’s and people start to realize they’ve been had. The fun happy social place they signed up to is actually ruled by a dictator and he’s started selling their data, changing the product to grow profits instead of individual happiness, and attempting to addict them every step of the way.

Eventually it comes to a breaking point and people are fed up. They join a new social network, run by a new smaller company that promises to do better and be better. This company becomes more profit-centric and exploitative over time, and the cycle continues. Alternatively they drop out of social media completely, keeping their lives almost as private as before the internet.

What if we could have the fun of a globally connected internet, without all the corporate control, exploitation, and addiction that goes along with it?

What if there was a social client that connected you directly to your friends, without any middlemen, so you can chat, share photos, play games and more, and that experience is yours alone? There are no ads, no addictive intrusions, no timeline sorted to be maximally good for the company. 

Furthermore this social network is curated by you. You only see what you want to see, interact with those you care about, and can be anyone you want to be. Why do we have to link our social profile to our real identity anyway? Why can’t we be whoever we want to be in our social world?

What if you weren’t locked into just one social client, you could choose from hundreds? And they all worked on the same network seamlessly, so you and your friends could use different applications and still be together?

That’s the promise of a decentralized world, a beautiful magical place that you can tweak and change to your liking. A place that reflects your unique personality and outlook on life instead of being forced into a box with all the other humans.

This is the world I’d like to explore and build, together, over the next decade. Because we can’t leave the future of the internet (and the world) in the hands of so few. 

This blog isn’t building a product and there’s nothing for sale here. It’s aim is to explore a decentralized world that could (and should!) exist, see the technologies that can make it happen today, and solve the problems that crop up along the way. 

This world I describe has been partially built already! By a project called Scuttlebutt, and the Social Client that runs on it: Patchwork. You can start using them today. I’ll be explaining these in more depth along with other decentralization technologies over the coming months.