Not Everything That Can be Counted Counts

The Shorty Awards are a pretty typical example of the “popularity contest” model so ubiquitous in social media. At their best, they can be used to measure popular opinion, and as we know (perhaps too well) popular opinion does not equal truth. Nor does it indicate quality or value.

“Not everything that counts can be counted, and not everything that can be counted counts.” -Einstein

I’ve written before about the idea that ideas don’t spread because they are good, and the best way to understand this is to remember that there are plenty of good ideas or people that are not very popular and plenty of bad ideas that are, pet rock anyone? If we are trying to engage Twitter users to extract some value either for ourselves or our business, we must remember that sheer popularity, or collecting followers, is often a pretty useless endeavor.

The other problem with popularity contests is that they are vulnerable to all manner of gaming and manipulation (if you don’t believe me, check out Digg or really any other social news voting site) from simple vote begging, which in some cases is accepted or even encouraged, to full-out vote buying.

But could someone buy votes to game the Shorty Awards? It seems to defy not only logic, but the economy of microblogging. But it turns out that Belkin is not alone in its “creative” use of Mechanical Turk for “social media marketing.”

Enter Amazon’s MechanicalTurk.

I am currently the second-place finalist for the Shorty Awards in social media. I got there by encouraging my followers and blog readers to vote for me and making it simple via a shortened link to an automatically filled Twitter page. Twitter does not allow such links to automatically post to someone’s account and a user must click the link, review the text of the Tweet and manually click “update”. I’ve had some wonderful successes recently with TweetBacks and TweetSuite so I’ve been lucky enough to be able to garner over 160 votes (something like 30 more than the current 3rd place finalist). On the Shorty Awards site, they explicitly condone this type of behavior.

Much to my chagrin however, there has been one user, with fewer followers, who has managed to outpace me in vote gathering, no matter how much traffic my site gets, or how many times I ask for votes on Twitter. I had given up on the awards when his vote count reached more than double my own votes. But on Monday night, I was surprised to see a somewhat anonymous email sent via the contact form on my website (if that was you that sent me the email, thank you!).

dhollings is buying Shorty Awards votes using Amazon’s MTurk.

MTurk listing to buy votes for $.48 each:

The URL listed in the HIT ( points to (a domain registered to hollings).


I looked and lo-and-behold, the secret was revealed: he was buying Shorty Votes. A few hours later that MTurk link was removed but a quick search of the site revealed another almost identical posting (with the price changed from $0.48 to $0.33). The most telling part of the listing is the 4th cautionary notice that you will not be paid if you violate:
DO NOT post publically that you are being paid for your work.
Clearly, it is understood that this tactic is gaming the system, and should it become public, the entire ruse would be ruined. The link in the description points to a clever script that rotates the text of the vote to make it look organic.
We contacted Gregory Galant of the Shorty Awards regarding this allegation. He responded promptly:

We’re investigating it now. We take voter fraud, an unfortunate byproduct of any democracy, very seriously. We threw out about 4,000 suspicious nominations during our audit of the first round of the awards, and we’re going to be even more vigilant this round. I’ll let you know the result when we’re done investigating. Since the voting process is completely transparent, we invite anyone in the community to look for suspicious votes and report them to us.

Just as this was about to post, @ShortyAwards tweeted that they had done an audit and discounted votes, and linked to this FAQ regarding vote counts decreasing.

Dan Zarrella is a self-proclaimed social media and viral marketing scientist. Check out his viral marketing blog for more of his posts, research and tools.

Editors’ note: Obviously the TouchBase Blog is not a neutral third party in this situation. Dan Zarrella is a contributor here and is directly involved in the situation he is writing about. However, the evidence he has produced and the response of the Shorty Awards indicates a legitimate basis for this post, and that “something is rotten in the state of Denmark.” My professional opinion is that while perhaps fun, the preponderance of rankings, ratings, lists, metrics of influence, social media awards and other popularity contests are measuring the wrong thing and represent a self-congratulatory drain on our time and energy as professionals.

Leave a comment