A few weeks back, Mike wrote a post detailing how absolutely shitty algorithms can be at determining what is “bad” or “offensive” or otherwise “undesirable” content. While his post detailed failings in algorithms judging such weighty content as war-crime investigations versus terrorist propaganda, and Nazi hate-speech versus legitimate news reporting, the central thesis in all of this is that relying on platforms to host our speech and content when those platforms employ very, very imperfect algorithms as gatekeepers is a terrible idea. And it leads to undesirable outcomes at levels far below those of Nazis and terrorism.

Take Supper Mario Broth, for instance. SMB is a site dedicated to fun and interesting information about Nintendo and its history. It’s a place that fans go to learn more weird and wonderful information about the gaming company they love. The site also has a Twitter account, which was recently flagged for posting the following tweet.

For the sin of tweeting that image out, the site’s entire account was flagged as “sensitive”, which means anyone visiting the account was greeted with a warning about how filthy it is. What Twitter’s systems thought was offensive about the image, which comes from another video from a costume company that works with Nintendo, is literally anyone’s guess. Nobody seems to be able to figure it out. My working theory is that the Princess Peach’s lips resemble too closely a more private part of the female anatomy and, when coupled with the flesh-colored face surrounding it sent Twitter’s algorithm screaming “Aaah! Vagina!” leading to the flagging of the account. But this is just a guess, because although the “sensitive” flag was eventually removed, SMB never got any response or explanation from Twitter at all.

SMB went as far as to test through dummy accounts whether the image was the entire problem. It was. After posting the image several times from other accounts, each account was flagged within minutes of the posting. It’s an algorithm doing this, in other words, and one which seems ill-suited to its task.

What we have here is two related problems. We have a company designed to let speakers speak employing an algorithm to flag offensive content, which it is doing very, very badly. We also have a company with a staff insufficiently capable to correct the errors of its incapable algorithm. This would be annoying in any context other than current reality, which sees rising calls for internet sites to automagically block “bad” content and do so with literally inhuman speed.

That means algorithms. But the algorithms can’t do the job. And with sites erring on the side of over-blocking to avoid scrutiny from both the public and governments, that means open communication is the loser in all of this. It’s hard to imagine an outcome more anathema to services like Twitter than that.

Permalink | Comments | Email This Story

Techdirt.