The Reputation Algorithm

Share this post
Twitter's Sensitive Content Algorithm
www.thereputationalgorithm.com

Twitter's Sensitive Content Algorithm

Is it too sensitive?

David Erickson
Aug 21, 2017
Share this post
Twitter's Sensitive Content Algorithm
www.thereputationalgorithm.com
Image featuring the see no evil, speak no evil, hear no evil monkeys

This post was originally published on August 20, 2017.

In March, Twitter’s sensitive content algorithm began its testing phase, a feature that warns users of “potentially sensitive content,” according to Sarah Perez at TechCrunch.

This was the first time I have personally seen such a warning and the algorithm appears to have some failings.

I noticed that a Twitter account called “Raging Bull NO TRUMP,” or @ToDropADime2 had begun following me. When I went to investigate the account by clicking, the account’s profile gave me this warning:

Screenshot of Twitter's sensitive content algorithm

Thank you for reading The Reputation Algorithm. This post is public so feel free to share it.

Share

I was curious as to why an account that appeared to be anti-Trump would be blocked by another account that likely shares its attitude toward the president. Back in March, TechCrunch reported:

Twitter tells us the new feature works similarly to how other sensitive content on Twitter gets flagged, based on users’ settings.

Currently, the company permits content that contains violence or nudity, but it draws the line at “pornography or excessive violence in live video, or in your profile image or header image,” according to its page on sensitive media. It doesn’t mention profanity, racism, bigotry and other types of offenses, however. But sensitive content is not limited to “violence or nudity,” we’re told.

Users can choose to mark themselves as someone who tweets sensitive content through their “Privacy and Safety” settings.

I checked my user settings:

Screenshot of Twitter's sensitive content settings

As you can see, I have the Hide Sensitive Content setting checked because I don’t want to see porn or gratuitous violence on Twitter. Note, however, that the setting claims to prevent potentially sensitive content from displaying in search results.

I did not find this account through search results; I found it within my notifications because it had followed me.

I went ahead and clicked Yes, View Profile to investigate why it had been blocked from me. Here’s what the Raging Bull Twitter profile looks like:

Screenshot of Twitter Raging Bull profile

While I can see why fans of the president wouldn’t like this account, I am not one of those people.

I scrolled through the Raging Bull feed and found what you might expect to find from a political activist opposing the president. But aside from prolific use of  the F-word, nothing I saw appeared to be “sensitive content” to me.

Even after I clicked through Twitter’s warning, I noticed individual tweets with photos or video appeared blocked as well. Here’s an example:

Screenshot of Raging Bull content blocked

I went back to my settings to allow sensitive media to be shown. Here’s what the same tweet looks like with the GIF displayed:

Screenshot of Raging Bull tweet unblocked

Hardly offensive, right?

Is Twitter’s Sensitive Content Algorithm Too Sensitive?

It appears the only “sensitive” content Raging Bull posts is the frequent use of the F-word.

There are two explanations for the warning:

  1. Swear words trigger it, or

  2. Getting reported or blocked by other users triggers it.

If it is only swear words that trigger the warning, that’s an awfully low bar in my book. It seems to me it would be trivial for Twitter to offer a text box setting into which those delicate souls who cant abide by an errant curse can indicate to the service which offending words they wish not to see.

The last explanation is pure speculation on my part. I seriously doubt the Raging Bull account has checked its own settings to declare it tweets potentially sensitive material, so that leaves the likelihood that Trump fans are complaining.

TechCrunch continues:

Twitter users can report tweets to the Twitter team for review. In this case, if the tweet is determined to be potentially sensitive, Twitter will label the content appropriately — or remove it, if it’s a live video. It may also adjust your account setting for you, so your future tweets are marked accordingly, if it deems it necessary.

For repeat violations, Twitter may permanently adjust that setting on your behalf, it says.

The process for marking entire profiles as sensitive follows a similar set of guidelines and processes, including the fact that Twitter can take an active role in identifying these accounts, based on the content of the account’s tweets.

These are the options for reporting a Twitter account:

Screenshot of Twitter's reporting an account window

I imagine Twitter has an algorithm that flags accounts that reach a certain threshold of complaints or of getting blocked.

But that could easily be gamed, with supporters of Trump reporting or blocking this account en mass.

Is Twitter’s Sensitive Content Algorithm Unsophisticated?

Yes, methinks.

Based on what Twitter knows about me (or should know), the platform would have no reason to warn me about the Raging Bull account:

  • I frequently post articles that are critical of the president,

  • I frequently retweet others’ posts that are critical of Trump,

  • I follow many accounts that are critical of Trump, and

  • The media sources I share are likely the polar opposite of those of a Trump supporter.

The only real signal that might throw Twitter’s algorithms for a loop is the fact that I follow many conservatives from my own state and many of them follow me, as well.

Nevertheless, it seems to me Twitter has ignored many of the signals it should be paying attention to when deciding whether or not to warn me about an account like Raging Bull’s.

From that, I would conclude that Twitter is either not collecting the data it should be collecting to understand its users, its algorithms are not very sophisticated, or both.

The Reputation Algorithm is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Share this post
Twitter's Sensitive Content Algorithm
www.thereputationalgorithm.com
Previous
Next
Comments
TopNew

No posts

Ready for more?

© 2023 David Erickson
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing