Programmer Q&A site Stack Overflow has attracted legions of volunteer moderators (including privileged users possessing various editing capacities) with a “gamified” reputation system, but the quality of this moderation is increasingly coming under attack. A thread on Stack Overflow itself asked in April 2014, Why is Stack Overflow so negative of late?
I’ve been using Stack Overflow for a few years, and initially most questions you asked got a positive reception: happy comments and answers, people that tried to help you out regardless of the way you worded it. Of late the tendency is to simply close or put on hold most questions, or downvote them, or make fun of the writing or the style. […] Pretty much every intelligent question I’ve tried to ask of late gets downvotes, while silly technical issues get well received.
One reply concisely outlines the problem with SO’s moderation system, namely the type of people it attracts. Aside from “caretakers” who actually want to do a good job, you get “vampires” who flood the site with lazy questions and reputation “whores” who happily answer those lazy questions for easy reputation points. Another user made a fine diagram to illustrate the situation:
Reputation whoring is also the central complaint of Michael T. Richter’s now-classic 2013 article, Why I no longer contribute to Stack Overflow. Based on his experience as a top contributor with over 14,000 reputation points, he explains that the gamification incentives are all screwed up:
The Java answer scored me 460 and took me probably under a minute to write. The one about operating systems scored me 60 points and likely took about fifteen minutes to write. If you’re going for points (and that’s the entire raison d’être for gamification!), are you going to waste time like that for 60 points when you could fit in a dozen 460-point answers? Of course not! You’re going to go wherever the points are. And the points are the low-hanging fruit of trivial questions from popular languages. The way Stack Overflow is structured rewards people who put as little work as possible across as many simple questions as possible within only the most popular segments.
Accordingly, Michael got few answers to his own challenging questions, and so rarely bothered asking any. He experienced active hostility as well, of the childish secretive kind often found on the Internet:
The people who hated [my answer] weren’t content, however, with merely voting it down. No, after I posted that answer I had a mysterious downward turn in my reputation as downvotes appeared all over my answers. People got so upset at my mocking one of the Holy of Holies of computing that it wasn’t enough to just downvote the answer, they had to punish me. (Their selected means of punishment was as highly amusing as it was highly ineffective.) This is not the way a community of mature users acts.
The kind of authoritarian moderators who eventually get to dominate SO are those “obsessive twerps” who are most successful at earning points. If they aren’t scavenging for easy questions you might find them at the opposite end of the spectrum: the Soup Nazis of Stack Overflow, as Jason Sachs memorably called them. These extend their war against poor questions to include anything that doesn’t fit their rigid preconception of the ideal question, driving users off the site. Jason gives some examples of bizarrely closed questions, and closes with links to yet more people complaining about SO.
You can count me among them, too. I was never a very active contributor but stopped entirely for all the reasons outlined above. I saw gangs of reputation whores editing obviously lazy questions into a shape they could easily answer, posting needless requests for clarification so as to further simplify their job, then upvote each other for their glorious contributions. Never mind if the “answer” was now barely related to the original question, or outright wrong! Conversely, when I searched for help on some complex topics I found well-stated questions that had been ignored for years. On the occasion that they had been answered I soon realized why that so rarely happened: correct and thorough answers to hard questions would yield only single-digit reputation points, unless given by an already famous user.
2015-07-07: John Slegers’ The decline of Stack Overflow reiterates the points made above, adds even more anecdotes, and concludes that SO is now run by trolls as per Jeff Atwood’s own definition:
I would offer an alternative label for “soup nazis” or “the high school cool kids table”: “overprivileged trolls”. Yes, you read that right. Not only is Stack Overflow plagued by overzealous trolls, but it is a community where the trolls are actually in charge. No wonder hostility and infantile behavior have become the very norm that define one’s experiences at Stack Overflow!
Quantity has a quality all its own
So why is Stack Overflow still so big and successful? Jason Sachs mentions a few obvious points: the site is very fast and highly usable, with great search and notification features, no obnoxious advertising, and no answers hidden behind registration or pay walls. I’ll add another point: there are some truly legendary Stack Overflow users such as Jon Skeet whose many excellent answers to difficult questions contribute a big part of the site’s overall value. However, Jon is also running his own weblog and answering questions on other forums, as do the other great contributors I’m aware of.
Clearly Stack Overflow’s sheer size is a self-perpetuating reason for its ongoing success. That was accomplished by having many volunteer respondents and moderators… and those were attracted by the same gamified system that creates reputation whores and Soup Nazis. Could SO have a moderation system more focused on quality? Probably, but it would be a much smaller site, attracting fewer users and therefore also fewer quality answers in total. The unspoken success model is to pull in as many users as possible, establishing a reasonable chance at getting some good timely answers here and there, then let internal and external search discard the chaff.
If some good contributors burn out and leave due to the pervasive childishness, that’s no big deal because the combination of massive size and seductive gamification ensures they will quickly be replaced. Stack Overflow succeeds not by directly ensuring quality but by ensuring so much quantity that some quality is bound to randomly occur somewhere, despite disincentives and outright hostility. In that sense you could call it a micro-representation of the entire Internet.