How Does Bluesky Work?

https://steveklabnik.com/writing/how-does-bluesky-work

by steveklabnik • 2 years ago

248 91 2 years ago

oDot• 2 years ago

Why not use ActivityPub?

ActivityPub is a federated social networking technology popularized by Mastodon.

Account portability is the major reason why we chose to build a separate protocol. We consider portability to be crucial because it protects users from sudden bans, server shutdowns, and policy disagreements. Our solution for portability requires both signed data repositories and DIDs, neither of which are easy to retrofit into ActivityPub. The migration tools for ActivityPub are comparatively limited; they require the original server to provide a redirect and cannot migrate the user's previous data.

Other smaller differences include: a different viewpoint about how schemas should be handled, a preference for domain usernames over AP’s double-@ email usernames, and the goal of having large scale search and discovery (rather than the hashtag style of discovery that ActivityPub favors).

from https://atproto.com/guides/faq

aatd862• 2 years ago

Domain usernames is a very good point.

Diti• 2 years ago

Until trademark laws come into play, and you find yourself obligated by law to give up your domain username to a big corporation. (A famous example in France is “Milka vs. Kraft Foods”, the court favored the big corporation’s registered trademark over Mrs Milka’s name.)

Granted, my comment doesn’t add much to the discussion, since this domain ownership issue would have been a problem in ActivityPub too.

mariusor4• 2 years ago

Using double-@ user identifiers for actor discovery is not an ActivityPub feature. It's a bastardization of webfinger resource search added and popularized by Mastodon.

In ActivityPub URLs are the identifiers, be it for actors or anything else. And the URL can be just of a domain, therefore it's perfectly possible to have domain named identifiers.

skybrian2• 2 years ago

I believe there’s currently a single relay. When there are multiple relays, each will have a different view of the network, and the one that includes more posts will be better than the others.

It might be quite hard to get relay diversity rather than using a single dominant relay, sort of like attempting to get traction for an alternative DNS system. Then, if your posts don’t get picked up by the dominant relay then it seems like you’re out of luck?

But it’s still very early, so perhaps it won’t work out that way. And even if there is a dominant relay, well, GitHub isn’t so bad, and there are alternatives.

wmf3• 2 years ago

I would expect every relay to have everything (modulo different opinions about illegal content). Why do you think they would have different views of the network?

s1gnp0st• 2 years ago

It's virtually guaranteed that different relays will take different views regarding disinformation, harm, etc.

This is probably good, too. I personally don't want my hand held, but recognize that others justifiably want someone else to filter out gore, or even opinions they find sufficiently off-putting.

I don't want to ram particular content down anyone else's throat, but I'm 100% opposed to anyone making that decision for me.

skybrian2• 2 years ago

Well, they'll try. The relay plays a similar role to a search engine, though without the ranking. Why do different search engines have different content?

As others have said, spam and illegal content are the most likely reasons.

We might also compare to email delivery; maybe ideally every message would be delivered, but in the real world, they aren't, and open relays are a hazard.

layer82• 2 years ago

I would expect clients (App Views) to be able to use multiple relays in parallel if needed. That would be like having a local GitHub UI pulling data from several GitHub instances (“relays”) that may host different but potentially overlapping sets of repositories.

This would be like a Usenet client aggregating newsgroups from several NNTP servers.

e12e• 2 years ago

More like a local git repo having multiple remotes?

evbogue5• 2 years ago

I have previously found Bluesky kind of confusing, but I've recently realized that it was mostly the terminology that was confusing me.

Imagine if you instead referred to the the system as

[microblogs] -> [indexer] -> [client]

There's a lot more to it, but I think this is the basics of how Bluesky works. They just call everything something different than you might think it should be called.

The indexer is only necessary because the user ids do not remain static on the system. Instead there is some weird cryptographic reshuffling going on where DID:PLC starts as a substring of a hash and then because of 'key rotation' the user id of every user actually changes out from under the user themselves over time.

Imagine a system such as Livejournal but your handle is constantly changing so you need an authority to keep track of the handle changing. And of course you are only allowed to post short messages with no titles or markdown support.

I'm personally waiting for a new DID:PLC where you can get a static pubkey where you sign the messages with a private key because I think that'll be way less stuff to keep track of and it'll make self-hosting and local-first development much easier.

steveklabnik12• 2 years ago

Yeah, I hear you about the names. When talking about this on bluesky, someone came up with "data store -> indexer -> view", which I think fits a bit nicer too.

https://bsky.app/profile/aparker.io/post/3km6zvwfen727

str4d3• 2 years ago

The indexer is only necessary because the user ids do not remain static on the system. Instead there is some weird cryptographic reshuffling going on where DID:PLC starts as a substring of a hash and then because of 'key rotation' the user id of every user actually changes out from under the user themselves over time.

The user IDs (e.g. did:plc or did:web) do not change over time. You are correct that a user that uses did:plc has that generated from a hash (of their initial repository state), but subsequent additions to or deletions from the repo don't change the DID. Likewise, if you set up your repository with did:web, the domain you pick is fixed, and you lose access to your repo if you lose control of that domain.

What is changing with key rotation is the repo signing keys that authorize updates to the user repo. If you run an entirely self-hosted PDS then you can choose to never rotate that key, and then you have a "static private key" repository. But key rotation is a necessary part of the design, in part to enable account migration (you wouldn't want an old PDS to still be allowed to make changes to your repository, so you'd rotate those keys to remove the old server's key and add the new server's key).

paholg• 2 years ago

This seems incredibly well thought out, and seems like it solves some of the bigger issues with ActivityPub. I had no interest in Bluesky before; now I think I'll make an account.

city41• 2 years ago

Had a similar response. Dusted off my account and finally posted something.

stavros• 2 years ago

Same, actually. The fact that I can get the @stavros.io handle just by adding a DNS record is a nice touch.

remram5• 2 years ago

My main problem with Bluesky and the AT protocol is not their apparent technical merits so much as the strong "not invented here" approach. Like their "domain as identity" instead of using user@host like everyone else and established protocols like WebFinger. Like using their own mechanism for that instead of looking at how ACME already does it for certificates, leading to someone owning the s3.amazon.com domain [1]. Like using their brand-new RPC protocol, "XRPC". Like whatever Lexicon is, a weird code-generation-based data schema whose only benefit over protobuf and jsonld is that it's newer.

It is hard to trust an "open network" that demonstrates time and time again that they don't respect and don't want to talk to anyone else. It also makes you wonder how many more common mistakes have slipped in the design simply because they reject common knowledge.

[1] https://www.reddit.com/r/programming/comments/138hlf2/s3_dom...

lifthrasiir5• 2 years ago

Your comment could have been more constructive if you have included more "common mistakes" than things you just don't like, the only exception being handle squatting. Like, what is a specific benefit of `user@host` as opposed to a single domain---which can be generated on demand---as an identity? (And who is "everything else" here? Almost all non-Mastodon social networks don't use them.) Also a value of the common schema language is not that high, especially when that language is coupled with the serialization format and prevents any improvement.

miragecraft4• 2 years ago

ATprotocol seems technically superior, but socially set up for failure.

Because there’s no social incentive for anyone to set up and maintain relays and moderation service when divorced from actual communities, what’s left are purely financial incentives - paid service, advertisement insertion, data mining, information manipulation etc.

Also the centralization of data distribution and moderation means they are huge undertaking now, both in terms of hosting cost and human cost, it’s almost forcing these services to be commercialized, or you can say discourage anyone not Bluesky from setting up their own relay or moderation service.

For all the faults of ActivityPub, the extremely “gated community” like approach provides the social incentive for users to feel a sense of belonging to a server, and for the server owner and moderator to feel a sense of pride and ownership over providing service to their users. Being within a valued community also incentivizes people to donate to pay for server cost and moderation effort.

ATprotocol will technically allow multiple relays and moderation services, but in reality how many will actually materialize?

As such I feel judging social networking protocols from a purely technical and adversarial point of view (like ATprotocol does) is a mistake, sure you are now no longer beholden to a server, but at the same time you are divorced from a tight knit community and you no longer have skin in the game.

When you have nothing to lose (data, identity and community wise), that’s a double-edged sword.

inhumantsar• 2 years ago

For all the faults of ActivityPub, the extremely “gated community” like approach provides the social incentive for users to feel a sense of belonging to a server, and for the server owner and moderator to feel a sense of pride and ownership over providing service to their users. Being within a valued community also incentivizes people to donate to pay for server cost and moderation effort.

are social incentives going to be enough long-term though?

I keep looking at this and thinking of three things: trust, security, and Reddit. These are actual questions, not arguments, I'm genuinely curious.

Can you trust an admin to host and police your social network? Moving servers isn't hugely difficult iirc, but it could be very disruptive and drama-ful. It would have to get pretty bad before people start to overcome the network effect inertia and actually move.

What is there to hold hosts accountable for security? The threat of reputational damage doesn't seem like it is going to mean much to pseudonymous volunteers, so what can be done to ensure hosts are properly protecting the system from promise?

Reddit moderators are, by and large, reasonable but there's a non-trivial minority who will do things like stage a coup against the other moderators, start arbitrarily banning or harassing people, or just straight up ghost the site and never follow up on the rules. same as above, what mechanisms exist to prevent or reverse or take over the actions of an rogue/absentee mod?

steveklabnik12• 2 years ago

For all the faults of ActivityPub, the extremely “gated community” like approach provides the social incentive for users to feel a sense of belonging to a server, and for the server owner and moderator to feel a sense of pride and ownership over providing service to their users.

I really misunderstood what you were talking about until you got here.

This mindset is extremely alien to me, so that's probably why it's so hard to make sense of this.

I am glad that you have a system that works for you, and that I have one that works for me.

danabramov2• 2 years ago

I’m still fairly new to the team, and don’t claim to represent their point of view here, but my impression is that services (like moderation services) aren’t intended to be divorced from the communities they’re serving. It’s more that they’re decoupled from the physical data placement. The relationship between users and moderation services would be many-to-many. The incentives to self-organize and to provide a valuable service for your community are still there; it’s just that the notion of belonging to a community is more fluid in itself.

KerrAvon• 2 years ago

I'm skeptical about third-party labelers. How do you prevent that from being abused?

steveklabnik12• 2 years ago

What kind of abuse are you talking about? I'd be happy to give you my take, but it's a bit too broad for me to know how to answer.

edavis2• 2 years ago

Users choose which 3p labelers they subscribe to. If a labeler service starts abusing their position, users can drop it.

https://github.com/bluesky-social/proposals/tree/main/0002-l... is a deep-dive on how Bluesky sees labelers fitting into the ecosystem.

arcalinea3• 2 years ago

We'll be publishing a lot more about moderation soon, but the basics of labelers:

- anyone can label anything in the network — including other labelers

- apps can surface labelers for users to subscribe to, similar to how custom feeds currently work in bluesky

- apps can exercise curation over how labelers get surfaced and ranked, similar to how custom feeds get surfaced (currently ranked by popularity expressed through likes and saves)

taion3• 2 years ago

A fun bit of Bluesky trivia is that it’s also named after a person, apropos that post from a bit ago – the CEO’s first name is literally “blue sky” in Mandarin.

arcalinea3• 2 years ago

Twitter named the project bluesky before I got involved, funnily enough. Coincidence. But my middle name does mean bluesky in Mandarin.

taion3• 2 years ago

One thing I’m curious about here is how BlueSky can credibly commit to only use the open portions of the protocol. BlueSky appears to be de facto quite centralized – given that, it seems like there’s no technical reason why first-party BlueSky clients have to be ATProto clients. Obviously it would be a major betrayal of user trust to do so any time in the near-to-medium-term future, but it seems like the de facto decentralization of ActivityPub gives stronger guardrails here.

charcircuit• 2 years ago

What are first party BlueSky clients? Do you mean that a first party relay and PDS could communicate over a different protocol. I don't really understand what you are worried about. 2 activitypub clients could talk to each other over a protocol that is not activitypub.

steveklabnik12• 2 years ago

I mean, the code for everything is open source. It's credible because they're doing it. If they started changing to not do it, people would notice, very quickly.

taion3• 2 years ago

That makes sense. So first they have to go closed-source before that attack vector is even feasible, and doing so would be sufficient on its own to raise alarms.

jrm4• 2 years ago

While I appreciate them getting to the "speech v. reach" thing; Mastodon still feels long-run smarter out of simplicity (if short-run less fun or useful because Mastodon feed curation is definitely harder)

The main question, to me, is: If a hypothetical (likely wealthy or powerful) bad actor wants to put their thumb the scales here, how can they?

And this article REALLY hand-waved that away very poorly, with the whole "oh, I don't really know how the feeds work" bit. Makes me very suspicious that they know good and darn well not only how the feeds will work, but that they understand that this is where the power and/or money is and will try to keep a hold of that.

steveklabnik12• 2 years ago

Which part implied we don't know how feeds work? They're open source, and exist today, and you can run them.

wants to put their thumb the scales here

Which scales, and in which sense? There's a lot of moving parts here.

vlz• 2 years ago

I don't think "oh, I don't really know how the feeds work" is the best summary. Let me quote the author:

Feeds are a recent addition to atproto, and therefore, while they do exist, they may not be feature complete just yet, and may undergo some change in the future. We’ll see. They’re working just fine from my perspective, but I haven’t been following the lower level technical details.

One paragraph above for some more context:

This to me is one of the killer features of BlueSky over other microblogging tools: total user choice. If I want to make my own [feed] algorithm, I can do so. And I can share them easily with others.

danabramov2• 2 years ago

Here’s some docs and some code that give an overview of how feeds work:

https://docs.bsky.app/docs/starter-templates/custom-feeds

https://github.com/bluesky-social/feed-generator

throwaway290• 2 years ago

Does this mean if bluesky stop supporting did:plc then all preexisting plc identities go kaput? Since plc involves pinging their service...

edavis2• 2 years ago

https://plc.directory can be mirrored and this new mirror would become the authority if something like this happened

BlueTemplar• 2 years ago

For example, if every time I post a new update on BlueSky, if I had to send my post to every single one of my followers’ repositories, that would be extremely inefficent, and make running a popular repository very expensive to run.

Of course it would be inefficient, the Internet is not a broadcast medium !

Which is why AFAIK typically the only thing that is pushed (in large scale applications) is the existence of a message (and maaaybe a short title), your "followers" then pull the message when they want to see it. (Some of course might be big enough "fans" of you that they decide to "subscribe", and set up an "AI" to automatically pull all of your messages as soon as published.)

So it seems to be a weird strawman of a solution nobody would even try to use at large scale, or am I missing something ??

avinassh• 2 years ago

This short comic is also great - https://bsky.social/about/welcome-to-bluesky-comic

mariusor4• 2 years ago

With all due respect to Steve, he only demonstrates how the AT protocol ecosystem has ways of passing the buck from content creator to (uncreative but rich) middlemen, for dissemination, moderation, content curation, etc.

That is antithetic to what the indie web should be in my opinion: allowing corporations, because let's face it who'll be able to scale better, to control who sees what I post and who I see is not what I want from my social media. We have enough of that as it is.

steveklabnik12• 2 years ago

I am going to be honest, I do not see how I demonstrated that. Can you be more specific?

mariusor4• 2 years ago

if I had to send my post to every single one of my followers’ repositories, that would be extremely inefficent, [..] an additional kind of service, called a relay, that aggregates information in the network, and exposes it as a firehose to others

Therefore a company with money/resources to build this relay will be in control if/how the firehose presents my content as a content creator/pds owner. This will create the same imbalance of power you find today with Youtube/Twitch/Instagram/etc.

The solution is not to add intermediaries, but to make this dissemination to all of those followers less inefficient.

Designing a whole protocol to cater to the .1% of users that might get into the million followers is another level of premature optimization, in my opinion. For the others, sending requests to 10k followers should be plenty reasonable to do with off the shelf computing without requiring protocol level intermediaries like relays.

AT protocol, in my opinion, caters to the "scale" crowd and in doing that puts all the power in the hands of the few that can do it better. Instead of democratizing social media, which what a federated protocol would imply, it actually maintains the status quo albeit by changing the companies holding the power.

est• 2 years ago

How does it compare to Nostr?

sph• 2 years ago

They're actually more similar than I expected. Nostr seems to have gotten lost in trying to be a decentralised store of everything [1], while BlueSky is focusing on Twitter-style social media and its challenges—moderation, portability, etc. (though I'm unsure if ATproto can support other types of decentralised content)

Nothing wrong with being a store of everything like Nostr, it's just that a totally open model of contribution with no direction or oversight (well, except fiatjaf) tends to end up bogged down following a million different and often competing ideas. It's very active, but moving in 196 directions: https://github.com/nostr-protocol/nips/pulls

evbogue5• 2 years ago

https://github.com/nostr-protocol/nips/blob/master/26.md NIP-26 (and if there is anything else similar) is one to point to here.

Once you, the developer, are required to implement (thiskey === thatkey) a relay operator has to index the entire network before a post can be rendered in compliance with that NIP. It does bog it way down, as you mentioned.

this public key does not equal that public key either as much as we might wish they could.

grey_earthling• 2 years ago

The Authenticated Transfer Protocol, aka atproto, is a federated protocol for large-scale distributed social applications.

Well, there's the problem: large-scale and social are incompatible. Humans evolved to be sociable with maybe a few hundred people.

No-one has ever made a good restaurant by inventing a system to deliver millions of calories per second to each diner.

SahAssar• 2 years ago

The scale is the whole system, so many different groups of people.

To take your restaurant comparison that'd be like saying McDonalds is an average sized restaurant, which might be true for each individual McDonalds, but not for the entire system of the McDonalds chain of restaurants.

Angostura2• 2 years ago

The idea of users choosing their moderation provider which is independent of their hosting provider so they don’t have to worry too much about choice of instance sounds cool, except that I imagine that instances will still have to moderate to avoid falling foul of hosting locally illegal content.

This feels like it could end up being quite difficult for people running instances and end up as a big of a mess. It may be that I have misunderstood, however.

davidgerard• 2 years ago

you haven't. Bluesky decentralisation is fake. A PDS is a reasonable Ubuntu box, but even this post admits that a relay is a huge expensive central point of control.

What's the Bluesky plan for spam? Block it at the relay, maybe.

The DID server is the other central point of control.

But sure, go wild with PDSes.

steveklabnik12• 2 years ago

It is true that folks who host content will have to deal with the legal repercussions of the content that they host, and that inevitably, they will have to deal with this. That's true no matter what protocols you're using to host, however.

Angostura2• 2 years ago

Yeh, but part of the promise of Bluesky is 'your moderation regimes is picked by you and it's not linked to your host'. Except it is so now your moderation regime is determined by two entities, not one.

xg15• 2 years ago

Moderation tools fall under the “reach” layer: you take all of that speech, but provide a way to limit the reach of stuff you don’t care to see yourself.

Sometimes, people say that BlueSky is “all about free speech” or “doesn’t do moderation.” This is simply inaccurate. Moderation tooling is encoded into the protocol itself, so that it can work with all content on the network, even non-BlueSky applications. Moreover, it gives you the ability to choose your own moderators, so that you aren’t beholden to anyone else’s choice of moderation or lack thereof.

Ah yes, the famous "if you don't want to see this content, just close your eyes" approach to moderation. I know this philosophy is well-liked in Silicon Valley, but I think it's fundamentally flawed: There are legitimate situations in which you want to prevent unrelated other people from talking about a certain thing or acquiring certain content.

Classic examples are cybermobbing, doxxing and revenge porn: Two or more people talking about how to hurt a third person, publishing private, unflattering or false information about the person, etc. Removing this information from the victim's feed is completely useless (in fact it likely won't appear in their feed in the first place) as the harm comes from the fact that other people view the content or engage in the discussion. Nevertheless, the harm is real.

In a system with traditional moderation, a moderator could stop this kind of behaviour by deleting the posts for everyone and/or banning the perpetrators. None of this is possible in a "just hide the content" system.

Shared blocklists or "labels" won't work either as the consumers of the content don't have any motivation to block it - indeed, they want to see the revenge porn. The one who wants to block it is the victim, but it has no power to force everyone to use a particular blocklist. (The whole idea behind this system is to no one can force a blocklist on someone else)

steveklabnik12• 2 years ago

There is more stuff in this space coming; I am focusing on the normal cases in this post, and not the bad cases.

The team has considered stuff like this, “consulted with experts,” even, but it’s ongoing work and so I don’t feel qualified to talk about it. The “labels won’t work because they want to see it rather than block it” is a part of the problem space I’ve seen them acknowledge explicitly.

EDIT: see here too: https://lobste.rs/s/shseqz/how_does_bluesky_work#c_vjvmei

How Does Bluesky Work?

Related Stories