Fosstodon Hub – More Upgrades

https://hub.fosstodon.org/more-upgrades-twitter-storm/

by rbanffy • 4 years ago

121 92 4 years ago

hub.fosstodon.org

kazinator4• 4 years ago

Yay, it's on 4.0.2 now. Tag following can be used.

rwl4• 4 years ago

But not viewed.

https://github.com/mastodon/mastodon/issues/20763

kazinator4• 4 years ago

Right; famous issue. I think by the time all the new people need to do serious spring cleaning with their followed tags, it might be available.

I have a few issues/improvements I'd like to report, but I don't use Github.

danrl• 4 years ago

tough running a popular site w/o a business model

capableweb7• 4 years ago

Indeed, seems there is absolutely no way of surviving unless you think about CASH FIRST! Certainly there is no examples such as Wikipedia, Open Collective, Archive.org or others, as they also clearly have very profitable business models and were started with that in mind.

michaelanckaert6• 4 years ago

The "issue" here is that this is an instance by friends that's growing too big. This isn't a business idea, or a platform, nor is it a venture or a company. Just like minded people who are on a shared instance. Public signups were open since it was doable. Then Twitter started crumbling (or not - I don't care) and user signups exploded.

IMO the only correct way to handle this is to close public signups to your instance or require a fee from (new?) users.

But there was no mistake on the admins part initially - they just ran a fun instance for a group of like minded people who shared interesting content to each other.

lijogdfljk4• 4 years ago

I'd be curious to see Mastodon attempt to add features which help instances pay for hosting. Otherwise i don't see how this can survive.

I'm sitting here waiting for a slot to open up on `mastodon.art`, but the same story will play out there i'm sure too. How is it paid for? How can i help? Most people won't, but i'm happy to pay a small fee to hopefully offset my footprint cost. I just think Mastodon needs features to reduce this friction as best as possible.

.. though maybe it has them already? I've never hosted an instance so what do i know?

simonw• 4 years ago

Plenty of instances are using Patreon for this. Seems to be working well for them, though the huge explosion in users over the past few weeks might change that.

Here's the Mastodon.art one for example (they have two other crowdfunding services as well, linked on their about page): https://www.patreon.com/mastodonart

I think it's going to be OK. People who migrate to Mastodon are the kind of people who understand why it's good to contribute to your instance if you can afford to do so.

michaelanckaert6• 4 years ago

I'm curious why don't join another instance and simply follow people on mastodon.art - Is the local timeline that important for users? I'm genuinely asking since I'm on my own single user instance but follow plenty of people to give me enough content to consume.

samcat116• 4 years ago

I think that the local timeline is the only decent way to have native discovery in Mastodon (especially if you're in a topic-specific instance). I know some people specifically don't want things like algorithmic recommendations, which is fine, but for others this means expanding your following is relatively hard.

lijogdfljk4• 4 years ago

TBH, i don't know entirely, it just "feels" like i want the instance i'm most interested in. Yes, local timeline is neat and a feature i'd like to be able to use, but the Social aspect of Mastodon (vs email) means i tend to view the domain as indicating primary interest. It feels like a badge of community/interest.

Not sure if it makes sense, just trying to explain what's going on in my head. As i have debated that exact question myself, internally.

Matumio3• 4 years ago

Discovery: You'll tend to discover what other local users have already found or subscribed to. (Local timeline, hashtag search.)

Culture and rules: In this case this means how nudity is handled, whether anyone will care if you post spiders or gore without warning, some specific rules about AI-generated art, etc.

I can confirm that mastodon.art feels cozy and well-moderated, been there for years, also a financial supporter. Another sign that they care is that they've closed registrations, instead of trying to instantly moderate 3x the users. They've recently silenced mastodon.social, which again is mostly about calming down the local discovery features (you can still follow/be followed). They are blocking instances that encourage copyright violations, etc.

If you want an .art account, IMO right now you should start on one of the other instances they recommend, and move your account later. You can browse the local .art timeline and their view of hashtags without an account.

lhl2• 4 years ago

I've seen a few Mastodon instances using https://opencollective.com/ which seems like a pretty reasonable way to cover ongoing costs and the like. Here's an example: https://opencollective.com/libreops

GaggiX3• 4 years ago

Looking at mastodon.art, the admins seem to have blocked/muted hundreds of instances, including mastodon.social, mastodon.online, qoto.org, all of which are moderate instances with a clear policy against racism, harassment, etc., I don't see how this is good for artists, who probably need engagement. I think mastodon.art admins are trying to push their agenda by pressuring other instances to adopt stricter restrictions and limit users' freedom, instead of simply protecting their users (which is not achieved by restricting users from interacting with moderated instances).

lijogdfljk4• 4 years ago

Not sure about that, but i'm mostly just interested in the local community.

That said if the cost was a bit lower i was mildly tempted to standup a Blender community. However that seems like a lot of work, in addition to being a bit expensive.

Matumio3• 4 years ago

Here is some background about silencing mastodon.social, and what it means: https://www.patreon.com/posts/silencing-social-74474015

I don't think it's about pushing an agenda. Each instance can only set the rules for their own community. Dozens of art-related instances have popped up like mushrooms overnight, so there is a good chance people will find one where they agree with the rules of the house. And it's easy to move to another instance later.

cldellow• 4 years ago

Presumably the artists are smart folks and can switch instances if they feel the rules are too limiting [1]. mastodon.art's policies are too restrictive for my tastes, and so...I don't use them, and wouldn't encourage others to use them, either, unless they really liked them.

But, I don't see why we have to assume this is the instance admins having an "agenda" and trying to "pressure other instances to limit users' freedom". Maybe they just want to run their instance their way -- the ultimate freedom!

[1]: I think I may have stumbled across the mastodon instance that you use. I can see why you might feel strongly about mastodon.art's policies.

michaelanckaert6• 4 years ago

While I'm glad and support that people donate to their instance, I would encourage more people to start their own instance instead of donating.

The only way the fediverse (and by extension the internet) can remain healthy is by introducing more instances. "We don't need platforms, we need protocols."

The way Mastodon is evolving, thousands of users are concentrating on a couple of instances. What happens when a large corporation starts to "Embrace, Extend and Extinguish" again? What happens when a large instance decides to push out ads? Do we block that instance when it has 50.000 users?

capableweb7• 4 years ago

I agree but also want to note that coordinating 1000 instances over ActivityPub brings it's own set of problems, rather than having one instance with 1000 users. Each new server introduced into the federation puts (minimal) additional load and traffic requirements on the other nodes, so more servers requires better servers.

I do think that's the way forward though, rather than a few big instances, but time will tell what the right percentage of the split is for good performance for everyone.

michaelanckaert6• 4 years ago

Yes, you're right on that point.

I have read the ActivitPub spec but not the Mastodon API - so I'm probably wrong on the following part.

Let's assume that we have 1000 users on a single server, posting a single message to those 1000 followers is writing a 1000 DB records (assuming a table with post - user link).

Let's assume we have one server with 500 users, and 5 other servers with 100 users each. We still write 500 DB records on the first server, and make 5 API calls to the other servers, which each write 100 DB records.

If the above holds true, the overhead is 5 extra API calls? At least, I hope Mastodon doesn't make API calls for every user on a server!

avdempsey• 4 years ago

Does ActivityPub support hierarchical federation? Or could it be grafted on?

kazinator4• 4 years ago

I strongly suspect that connecting a new server to the federation, which has 1000 active users, is better than connecting 1000 more active users to an existing server.

1000 new users on an existing server are 1000 app/browser instances making separate network connections and separate little queries to that server.

Whereas (I would hope) there are aggregation efficiencies in connecting a new 1000-user instance to the network.

Also remember there are certain optimizations in Mastodon. If an instance A generates a toot, but the toot has no followers on instance B, then B doesn't see it. However, an instance always processes a toot from its own directly connected user's browser or Mastodon application.

weinzierl2• 4 years ago

Also the more users are concentrated on a few instances the higher the likelihood they will suspend smaller instances. My fear is that at one point the large instances might switch from a blocklist approach to allow-listing only the instances they federate with. All of this without evil intent, just because they cannot keep up with moderating.

kornhole2• 4 years ago

Only block lists rather than allow lists are supported on most projects. Someone would need to fork Mastodon or other to switch to allow lists. This would be a bad departure but will surely happen in some cases. They could even disable the account move feature to lock people in. Maybe truth.social did that and is why it is basically a closed off island.

divbzero• 4 years ago

Mastodon docs provide a guide to running an instance:

https://docs.joinmastodon.org/admin/prerequisites/

(I don’t run one myself so would be curious if any admins have tips or suggestions.)

8ytecoder• 4 years ago

It was easy enough to set it up on DigitalOcean. My primary objective was my own identity. Turns out the basic $7/mo can’t keep up with the load at all. I’d need to at least double the instance size to be able to serve the single user (me). Mastodon is quite resource intensive to host. I decided to instead donate that to fosstodon and move there.

I have used rails in production for a company with decent traffic. It’s doable but it requires a lot of optimizations and the minimum starting requirement is high.

I think a Masto.host like model (the one Fosstodon uses) but with proper multi-tenant support and custom domain support is required for self-hosted mastodon to become a viable option.

(I have a desktop lying around unused. I’ll give another try with that but I’d probably have to measure the electricity usage and add an UPS)

UncleEntity• 4 years ago

While I'm glad and support that people donate to their instance, I would encourage more people to start their own instance instead of donating.

Sure… but then what?

I probably have the technical chops to setup an instance that won’t end up in a botnet in the first five minutes but that’s about it. No clue how to get people to responsibly (aka moderation) use it or even how to tell people there’s a new instance out there. Federation and all that, assuming there’s coordination between sysops in some way.

And judging by TFA the software is a dog so there’s non-trivial expenses involved. Or take a month to make my own implementation in C++ and have people accuse me of crimes against humanity because I don’t want to learn rust. Maybe there’s better options, dunno?

I assume this is all just backend stuff where people bring their own client?

I actually like figuring out how stuff like this works but “just run your own instance” isn’t helpful.

Admittedly, I haven’t even asked the google any of this but one thing I do know is I can’t afford to spend $1,800/month on anything that doesn’t generate significantly more income than that.

kornhole2• 4 years ago

My little RPI 4B 8GB running Mastodon and Nextcloud on yunohost.org is running well with about 20 users and can accommodate many more. If you have an old laptop or RPI laying around, give it a try.

gigatexal• 4 years ago

The issue is that all content is cached on all nodes so s3 or blob usage goes up a ton

sph• 4 years ago

Any suggestion for decent single-or-few-users ActivityPub servers?

I just want a container or a small binary I can run on my server for myself.

zimpenfish• 4 years ago

Single: ktistec? https://github.com/toddsundsted/ktistec (crystal + sqlite)

Few: Akkoma? https://akkoma.dev/AkkomaGang/akkoma/ (elixir + postgres)

Few (alpha): GoToSocial? https://docs.gotosocial.org/en/latest/ (go + sqlite)

There's also honk - https://humungus.tedunangst.com/r/honk - which tends to do its own thing re: UI and terminology (go + sqlite)

Maybe PleromaPi - https://issueantenna.com/repo/guysoft/PleromaPi - if you've got a Pi lying around doing nothing (Akkoma is a fork of Pleroma)

cpach• 4 years ago

If people dig that kind of stuff, then sure, go right ahead.

As for me, I really don’t enjoy being a sysadmin in my spare time.

YMMV.

cupofjoakim• 4 years ago

Mastodon simply won't be able to scale at the rate this is going. It's simply too expensive to run the instances and too labor intensive to moderate.

TBH I think it might've been a mistake from mastodons side to allow media hosting. Running ffmpeg in the background to transcode video, the image optimization scripts and the hosting of it all gets a bit costly.

I might be wrong on this, I don't run an instance myself, but in my perspective cost cutting should be prio number 1 even if it means you have to cut back on some features. Removing toot edits should allow for a more aggressive caching strategy etc.

This should allow more people to start their own instances without being afraid of potentially hitting it big.

2Gkashmiri2• 4 years ago

https://bunny.net/stream/

how come this isn't being used by activitypub to offload server work?

you dont even have to use ffmpeg, just dump the video directly and get encoded videos back...

if your server is NOT running FFMPEG, that is potentially more resources available for actual server work

6yyyyyy• 4 years ago

Why would uploading the video to a SaSS, paying them to run FFmpeg, and downloading the result be cheaper than running FFmpeg yourself?

capableweb7• 4 years ago

how come this isn't being used by activitypub to offload server work?

Because ActivityPub is the same thing as HTTP, a protocol for exchanging information. Basing a protocol around the idea that you send data to a closed-source platform goes against the whole idea of creating a open protocol in the first place.

Not to mention it goes against the idea and culture behind Mastodon as well, as bunny.net doesn't seem to be FOSS at all.

capableweb7• 4 years ago

Is the actual hardware they are running published somewhere? Didn't seem to be mentioned in this blog post nor the previous ones, unless I missed it.

~2000 USD seems slightly expensive for what they are doing, to be honest. ~900 USD would give you a 32 cores (AMD EPYC 7502P), 1TB RAM and ~15.36TB storage over at Hetzner, just a single instance. Double that and seems they would be covered for quite a while, and it comes just under 2000 USD.

michaelanckaert6• 4 years ago

All costs are detailed here: https://hub.fosstodon.org/about/#current-funding

Yes it could be done cheaper, if they did more work themselves. The masto.host service they're using is basically Mastodon as a service - aka they're paying someone to host (and manage) a Mastodon instance.

All in all fair value for the money IMO.

datalopers3• 4 years ago

All in all fair value

Unless I'm missing something huge, how is $1900/mo for 96GB of database and 700GB of disk storage remotely a fair value? AWS on-demand prices are significantly more affordable than that.

And this is all to support a grand total of 50,000 users? What in the world? Is Mastodon just the most inefficiently written software ever or something?

weinzierl2• 4 years ago

The masto.host "Mastodon as a service" is a one-man show run superbly by Hugo Gameiro. This and comparing it with other Mastodon hosters: absolutely fair value for the money.

capableweb7• 4 years ago

All costs are detailed here: https://hub.fosstodon.org/about/#current-funding

It doesn't actually list where and how the service itself is hosted, it just says "Fosstodon hosting" and the amount. I'm looking to understand what hardware is being used and where it's hosted.

friend_and_foe2• 4 years ago

I don't know why these admins want their servers to grow so big. Just close registrations and encourage people to run servers. It doesn't even have to be all self hosting, running a server capable of supporting 100 or so people is pretty cheap, especially if you use something other than mastodon.

nerdponx2• 4 years ago

Are there any paid Mastodon hosting services out there? They take care of system upgrades, security, downtime, etc., while I give them a few bucks a month for some amount of users up to a given cap.

friend_and_foe2• 4 years ago

There are a ton of them, for different fedi server types. There's masto.host which is widely known.

lhl2• 4 years ago

I signed onto Fosstodon last week to give Mastodon another try. I found the overall experience to be massively improved (last time I touched it was in 2017) and the recent traction it has gotten puts it well ahead of anything else IMO (I'm also testing out Farcaster and a few other services, but those seem pretty niche/immature in comparison).

While Mastodon is a bit rough around the edges, it has that same sort of excitement to me when I signed onto Twitter back in 2006, and most of those same friends/connections from those early days have been migrating. Based on the numbers floating around, I think it has enough traction now/critical mass to be it's own pretty exciting thing. While a lot of instances will fail, I don't think economic sustainability is actually a showstopper. Patreon, OpenCollective, any number of tools can handle recurrent donations enough to sustain larger instances, and there probably will be some alternative approaches as well. Migrating accounts between Mastodon instances isn't perfect, but is a core built-in feature, so while there's a bit of instance roulette, I don't think it's such a big deal. There are tools like https://fediverse.observer/ that might help for picking. It even has a map view, which is pretty neat: https://fediverse.observer/map

Here's a rough view of ActivityPub growth: https://fediverse.observer/stats

FWIW, I decided early on as I started researching into the current state of the Fediverse/ActivityPub that I should probably run my own instance. There are hosting providers that provide Mastodon hosting (although most of them like Spacebear and Mastohost are oversubscribed), but as I was researching, I decided something like Pleroma or Missykey would be a better fit anyway. There are some forks and I ended up using a fork of Pleroma, Akkoma (since, while a bit involved, it has pretty decent docs for setting up w/ Docker Compose and overall seems like the best-performing of the bunch, and has good support for most Mastodon clients). I was able to repoint my account from the Fosstodon Mastodon instance (which included migrating my followers!) to my new personal Akkoma instance pretty easily (the only wrinkle was setting up webfinger since I decided I wanted my canonical account id to not be a subdomain).

For those looking for a bit of an overview of what the current "Fediverse" landscape looks like, some resources that helped me get started:

* https://fediverse.party/

* https://joinfediverse.wiki/Main_Page

* https://www.paritybit.ca/blog/mastodon-is-dead-long-live-mis...

For clients btw, I'm using Whalebird https://whalebird.social/ and Sengi https://nicolasconstant.github.io/sengi/ on desktop, and Tusky https://tusky.app/ on mobile.

aorth• 4 years ago

Database: 96GB Media storage: 682GB Processes: 9 Threads: 300 Monthly hosting cost: $1895.50

Holy cow. This is just one instance in the Fediverse. For me the moral of the story is that we are all incredibly valuable to Twitter if they can do all this and more without charging us anything. Also, it makes me want to quit social media all together.

I'm on ioc.exchange and I'm contributing to their hosting costs, but how much is it worth to me? $5 per month? And I'm also contributing to Tusky, my open source Android Mastodon app of choice, as well as a handful of other projects on GitHub. Is this what it takes to disrupt the big tech companies and their surveillance capitalism?

mg• 4 years ago

If I read the numbers correctly, they have monthly costs of about $0.07 per user.

Interesting, when you compare that to Twitter:

https://masto.ai/@mg/109394485136585413

capableweb7• 4 years ago

I'm not sure it is that interesting, it makes sense. Twitter is a huge organization optimized for reach and earning money by serving its audience ads, with big structure around all of that.

While Fosstodon is a more of loose collective of people spending their free time trying to run a free service for a community around FOSS, with optimizations geared towards making the service itself cheap to run but still resilient and performant.

That they end up at different costs per user at that scale (or even more) makes a lot of sense in my head.

TaylorAlexander2• 4 years ago

I find the actual figures interesting. We tend to imagine that to run a web service you have to run ads, but it sounds like all the work required to build out that infrastructure adds so much cost that offering a donation supported service with no ads may be more viable now.

agilob• 4 years ago

Are users the only metric that counts? Some users post nothing, others post every 20mins average during 24hours. The more activity you have have, the more problems you have, backup costs don't grow linearly. Twitter probably has some SLO that are promised to paying customers, both for displaying ads and giving access to unrestricted APIs, this also grows more than linearly.

That said... CxO of masto.ai probably aren't paid 100s of millions$$$ and don't deal with international regulatory issues in Slovakia?

anderspitman• 4 years ago

Will Mastodon be the answer to centralized social media? Probably not, but we're learning a ton about federated social media by trying and that knowledge is valuable.

yonrg2• 4 years ago

Guys, go and run your own instances!

Don't feel urged to run mastodon, there are other solutions which are not that heavy (or written in a language you might know better). Check fediverse.party and find your match.

Currently, I have an GoToSotial instance for my family. For single user I propose microblog.dev

Also I started implementing my own ActivityPub server, which primarily aims to run on small hardware. But don't wait for that :)

What I already see is that instances get into troubles, not about costs. It's actually the other way round, they get multiple times their server costs in donations (without the proper business to receive donations) They lack people for administration, moderation and now handling of abuse notifications is getting a thing. And pretty much soon I expect first data privicy violations which end up suing the admin who is working hard on her/his free time to keep the service up and running. It would be really sad if capitalism succeeds to take down mastodon. It's the first time after email where a federated protocol got much attention.

nerdponx2• 4 years ago

I'd rather pay someone to host it for me! Where are all the fediverse hosting services?

yonrg2• 4 years ago

Yes!!!

OctopusLupid• 4 years ago

See a list here: https://docs.joinmastodon.org/user/run-your-own/

yborg• 4 years ago

Imagine the Mastodon instance 44 gigadollars could have funded.

dgudkov• 4 years ago

Is it possible to run a paid Mastodon instance? It would make sense, hardware resource bills (and work time of moderators) need to be paid.

dmacvicar• 4 years ago

I continue to see people stumble at mastodon resource requirements when setting up their instance, but I fail to understand why everyone is going for a mastodon instance for their personal use.

I am running https://github.com/superseriousbusiness/gotosocial on a micro instance, probably using 100M ram, with a 30M sqlite database.

True, I don't have all features. It is pure backend, but I can use https://pinafore.social/ as web client and https://f-droid.org/en/packages/su.xash.husky/ as Android app. It implements a good chunk of the Mastodon API. I also use it from Emacs with mastodon.el.

Took me 35 minutes to setup. It runs as a single binary from systemd. Most of the jail features enabled. Not even a system user (DynamicUser).

Mastodon != Fediverse.

cl0ckt0wer• 4 years ago

My dream is to have everyone running off single user instances. Repeal section 230. Stop by the grocery store and buy a matchbox size server that you can plug into your home router. Matrix and activityPub compliant.

Unfortunately raspberry PIs are few and far between.

Monetization would be chaos.

aitchnyu• 4 years ago

Can anybody explain the daily life of a mod and the ROI on paid mods?

Fosstodon Hub – More Upgrades

Related Stories