Inbox by Gmail combs through your email looking for frequent correspondents and puts the people who email you the most in a “speed dial” sidebar (that you can’t edit) that puts their names and pictures front-and-center for you every time you go to your email.
Which is great, unless one of the people who emails you the most is a vicious harasser who sends you a constant stream of angry, belittling messages. Remember, you can’t manually edit the speed-dial.
If you purge the harasser from your address book (meaning you can’t find him if he crosses a line and you need to give his details to the police or your employer, etc), then, finally, Gmail will relent.
It’s not that Google wants to do this, it’s that they didn’t anticipate this outcome, and compounded that omission by likewise omitting a way to overrule the algorithm’s judgment. As with other examples of algorithmic cruelty, it’s not so much this specific example as was it presages for a future in which more and more of our external reality is determined by models derived from machine learning systems whose workings we’re not privy to and have no say in.
Later, Inbox decided to switch my mom out for someone else. This someone else had been emailing me a lot, though I think my mom had still been emailing me more. Occasionally, I’d reply when I was obliged to, but I was definitely sending more emails to my mother. Whatever the specifics, the algorithm replaced my mom with him.
What Inbox didn’t know was that this person had been harassing me.
Inbox by Gmail’s accidentally abusive algorithm [Liz Denys]
(via Dan Hon)