Moderation at scale

27 December 2023 by Jennifer

On moderation of social media:  principles, workloads, scale.

I was thinking about the overall dynamics of instance-level federation on the Fediverse.

The admins of each instance can choose to stay connected to each other instance as a whole, or can choose to block or limit it.

Blocking instances, blocking individuals

Threads, the Twitter-alikey software from Facebook, has just taken the first small test steps towards connecting with the wider Fediverse.  In that context, it would effectively be a kind of giant Fediverse instance.

It was in one of the conversations about Threads that someone made a comment which I’m bouncing off here.  It was along the lines of, “I’d never block a whole instance, only individuals”.

Principles & practice

I think the writer was saying it as a point of principle:  like “it’s not fair to block one person’s connections for what another person did”.

But suppose you did rule out instance-level blocks, and only blocked people individually.  What would that look like in practice?

Who does the housekeeping

A well-moderated instance does its own housekeeping.

• If one of the people on that instance accidentally crosses a line in the instance’s rules, the mods will have a gentle word.
• If a new person makes an account on that instance, and it turns out they spam people or harass people, they won’t be allowed to keep doing that:  either they’ll stop, or their account will be removed.
• Well-moderated instances often make you write a little intro-application before your account is activated, which cuts down on casual or automated spammers.

A badly-moderated instance neglects that care.

On a badly-moderated instance, new people can join all the time who just want to be annoying!  or worse!  And nobody stops them!

Workload implications

Now let’s think about that difference from the perspective of being an instance admin, with people on your instance that you want to take care of.

If your instance connects to a well-moderated instance, you probably won’t need to do tons to moderate the posts of people from there.  Either the people who join there are considerate of others in the first place, or the mods of that instance take care of it.

Sure, there’ll be situations which still take work.  For example, there might’ve been subtle racism or sexism or ableism in a post, which neither you or the original poster had spotted until the post was reported.  Then, to moderate fairly, you have to raise your own skills, and maybe do some consultation.

But the load is shared.  The obvious problems “at the other end” are taken care of by the mods “at the other end”.  A lot of potential bother never gets as far as your instance.

On the other hand, if your instance connects to a badly-moderated instance, any blocking of badly-behaved people is going to have to be done by you!

You’ve invited an endless, tedious, very boring process of blocking people for doing annoying stuff.

Imagine that situation, and imagine running the numbers on your moderation workload.  If you’re willing to connect with badly-moderated instances, then it’s likely that 90% or more of your moderation workload is going to come from those instances.

Did you really want ten times as much work?

(To keep this post simple, I’m not really talking about “middlingly-moderated instances”, but that’s a thing too.  A common limitation is not really having enough mods for the number of users, so although they do give the boot to annoying accounts, it might take a while.  Also, if you auto-approve without an intro, you’re likely to be hosting more spammers and bots than the instances that ask for intros first.)

“One strike & you’re out” allows a lot of strikes

What’s worse is:  Of all the new accounts being set up on badly-moderated instances every week, you don’t know which ones are going to be spamming or abusing “your people” until after they’ve done it.

If you federate with somewhere badly moderated, then as well as bringing upon yourself more work for you, you’ve opened the door to an endless stream of bother and abuse for the other people on your instance.

Every badly-moderated instance is one more opportunity for spammers and wind-up merchants to make new accounts, again and again each time they’re blocked.  The scale of it is impossibly unwieldy.

When you’ve opened a floodgate, picking out individual droplets from the stream of it will not stop the stream.

The unpleasant posts will be aimed disproportionately at women and at people of colour, because that’s how online harassment stats always go.  If you’re a white bloke (or your account looks like you’re probably a white bloke), you can be pretty sure you aren’t getting the full experience of how bad it is.

So if you’re not content for your instance to be enabling a horrible environment for at least some of the people there, in practice you are going to have to do some wholesale blocking.

(Even blocking whole instances one by one is pretty unwieldy & endless, given how easy it is to start another one – which is why people are working on instance-level opt-in blocklists and allowlists.)

Is it unfair?

Circling back round to the principle:  is it unfair, to “the people who did nothing wrong”?

I think there are some nuances here.

Someone who doesn’t realise how the Fediverse works, and chooses a middlingly-moderated instance as their home base, and finds that there are people they’d like to follow and can’t, due to an instance-level block… I’d agree that’s hard lines on them.

On the other hand, if someone chooses to join an instance whose code of conduct is “no rules” or indeed “we like winding people up”, then they probably weren’t looking to contribute to the overall good vibes of the Fedi, and I don’t feel sorry for them.

But beyond that:  fairness has to include the people on your own instance.  If an instance creator sets out to provide a space for constructive and usually-friendly discussion, and people have signed up on that basis, then there has to be some minimum level of curated space for those people.  That will inevitably sometimes require limiting connections to people who don’t share your aims.

And the way that’s done has to be manageable for the moderation team, or the instance won’t survive.

Selecting your home instance

Part of the beautiful elegance of the Fediverse (notwithstanding its many limitations haha) is that when people choose an instance to make their home base, they can – at least in theory – select for their preference on moderation frameworks, as well as their preference for many other things.

I say “in theory” because of course, actually being able to do that requires (a) realising it’s a possibility, (b) knowing where to look to get a sense of each instance’s moderation policies, and (c) if you’re lucky, finding one that’s able to deliver what you were looking for.

You can intentionally go looking for an instance where the admins prioritise safer space over wider connection.  Or you can intentionally go looking for an instance where the admins are more relaxed about who can connect, at the expense of putting up with more bother.  It’s a kind of spectrum.

Adding further dimensions to the spectrum, you can look for an instance where the admins are skilled in certain types of safer-space moderation, because of their own cultural backgrounds and/or the work they’ve put in to be able to pick up on subtle implications.

(Of course, ideally the mods of all instances would be able to accomplish an adequate level of recognition of, say, racism or antisemitism.  It’s an abdication of responsibility to suggest to Black people or Jewish people that they should go and be on Black-run or Jewish-run instances.  But I’m acknowledging that cultural competence is a thing.  These are skills.)

The option of moving

And if you don’t like the policies or decisions where you ended up, it’s possible (albeit with some tedious admin) to move to a different instance, and take your follow lists with you.  In the year or so that I’ve been following people on the Fediverse, I’ve seen quite a few people do that.

(Mastodon software doesn’t yet let you re-import your old posts at the new place, though I seem to remember hearing that some of the other Fediverse software does enable even that nowadays.  It’s common to leave the old posts up at the old account.)

I’m seeing a bit of an uptick in discussion recently about moving from one instance to another – I think because of the prospect of Threads connecting with the Fediverse.  Some people are about to move to a different instance because they don’t want to federate with Threads, and their instance admins are planning to give it a try – or because they do want to federate with Threads, and their instance admins are planning not to.

A note on Threads

To loop back round to Threads…  I personally don’t see any need to “wait and see how that goes”.  We already know what Facebook is like and what their priorities are:  if in doubt, please read Erin Kissane’s excellent, careful explanation of Facebook’s role in the Myanmar genocide.

I don’t want to federate with their callous, reckless company, because I don’t want to give it any more of my time and energy.  I’m glad that the admins of scicomm.xyz, my main Fediverse base, already came down on the side of no Threads federation.

I do also think that if Threads federation takes off, the admins who decided to “give it a chance” are likely to encounter the dynamics I describe above.  From estimates I’ve seen, I get the impression that numerically, Threads is something like ten times the size of the whole Fediverse;  and I suspect the moderation there is middling at best.  That’d be a hefty addition to the modding workload.