Here are some articles that have recently gotten attention amongst media obsessives. They are all fundamentally flawed:

• A list of the Top 25 newspapers on Twitter from Journalistics
• An expanded list of 200 U.S. newspapers, sorted by their Twitter follower counts
• A piece in MediaWeek about Elle UK’s popularity on Twitter

The problem with all of these pieces? The data that underlie the assertions are fundamentally flawed.

Each story uses the advanced research technique of looking at a publication’s Twitter account, then reading the sidebar of their Twitter profile and copying the number of followers listed there. This methodology is useless for determining how many people have chosen to follow a publication, and instead is indicative primarily of whether or not that publication is one of the suggested Twitter accounts that users encounter when signing up for the service. It’s also correlated to how long that publication has been on Twitter accruing those incidental followers.

Big Follower Counts Are Horseshit

I covered much of this topic at the beginning of this year in a post called Nobody Has A Million Twitter Followers. While the literal point of that headline may no longer be true (I’m sure Justin Bieber or Nicki Minaj has actually earned a million organic Twitter followers), the point still stands: Being suggested as an account to follow when users sign up for Twitter so distorts the meaning of follower counts that citing such follower counts without disclaimers is either ignorant or misleading.

In the case of screaming headlines that say “The New York Times has more Twitter followers than subscribers!” we actually veer from misleading to so distorted it’s absurd. Subscribers are people who have, in one way or another, indicated intent. They filled out a form, sent in some money, and established a relationship with The New York Times. The majority of followers of the New York Times on Twitter, however, only established a relationship with Twitter itself, and the Times came along for the ride. MediaWeek actually uses the headline “Elle has a hit with Twitter feed” and this cannot be proven — being on the list myself, I gain users at almost the same rate as Elle UK, and I’m no hit among fashionistas. All we’re getting a measure of is Twitter’s popularity.

If any of these articles included explanation of the fact that the publications with the biggest number of followers were merely those chosen by Twitter to be so, then we could start to have an honest discussion about impact or influence or popularity or whatever the hell it is these writers want to weigh in on. By analogy, if a publisher went and threw its paper on the doorsteps of millions of people without any conscious action on their part, and then crowed about how it had a bigger subscription base than someone else, we’d consider them ridiculous.

So statements like “Maybe The New York Times has such a huge Twitter following because it was the first of the Top 25 to join Twitter, way back in March 2, 2007. ” (from the first article linked above, on Journalistics) show a fundamental misunderstanding of the very numbers they’re trying to report on. If we’re going to make a splash with articles based on numbers, let’s at least pretend to know what the numbers represent.