The Most Influential Terrorist on Twitter Didn't Have the Most Followers

In 2015, researchers estimated that around 125,000 Twitter accounts were run by ISIS sympathizers. If you’re an analyst staring at a haystack that size, you have two bad options. You can train a machine-learning classifier to flag extremist accounts, which is fast but treats every flagged account as equally important, so you trade one giant pile for a slightly smaller giant pile. Or you can read tweets by hand, which is accurate but hopelessly slow at scale. Both leave you with far more than you can actually read.

My group’s information-retrieval project at UVA started from a small reframing of the problem. What if the goal isn’t to find extremist accounts, but to rank them? You don’t need to investigate 125,000 accounts. You need to know which handful to look at first.

The obvious way to rank influence is follower count, and it’s the wrong one.

Follower counts are easy to game (you can buy them) and the people we were trying to rank knew that. The ISIS Twitter Census report out of Brookings documented that ISIS-affiliated accounts manipulate their follower and following numbers specifically to throw off the metrics platforms use to flag them. If your ranking is built on a number your adversary controls, your adversary is the one writing the ranking.

This is the same problem the early web had, and the same problem PageRank solved for it: don’t measure how many links point at a page, measure the structure of the whole link graph. PageRank is harder to fake because no single node controls it. But plain PageRank ignores what accounts are talking about, since all it sees is the link structure. We wanted something that understood topic too, so an account influential among people tweeting about ISIS would rise, rather than an account that’s simply well-connected in general.

That’s what TwitterRank does. It’s an extension of PageRank that folds in topic and homophily, the tendency of people to cluster around shared interests rather than follow each other out of courtesy. Instead of one random walk over the whole network, it does a topic-specific walk, so influence is measured within a community of interest. We didn’t invent it; it comes from a 2010 paper, and we ran an open-source implementation against our data. The algorithm wasn’t the interesting part. Watching what it did with the data was.

The first thing we checked was whether it beat follower count. We took a public Kaggle dataset of about 170 pro-ISIS accounts and ranked them. The account TwitterRank crowned most influential sat only 29th by follower count, with 632 followers, against others in the set with far more. The raw-reach ranking and the structural ranking disagreed, and they disagreed in exactly the direction the Brookings report predicted they should.

Then we tried to break it, which was the fun part. If influence were really just reach in disguise, dropping a few of the most-followed accounts on the planet into the dataset should blow up the rankings. So we did. We added Snoop Dogg, Kanye West, Khloé Kardashian, Leonardo DiCaprio, and the Dalai Lama, each with 18 to 30 million followers, into a pool where the biggest “terrorist” account had around 29,000.

None of them cracked the top five. The list barely moved; three of the original five stayed exactly where they were, and the small shuffles were within the noise of the algorithm’s randomness. Kanye has thirty million followers and zero standing inside a tight, topically-coherent ISIS retweet network, so the algorithm shrugged him off. That is what the method is built to do: score you on where you sit in a community of interest, not on how famous you are to the world at large.

I want to be honest about the limits, because it was an undergrad course project and it shows. The dataset was small and public. We ran an existing implementation rather than building our own. And we never validated the rankings against any real ground truth of “influence,” which is the thing that would actually settle whether follower count is worse. What we showed is narrower: the two rankings disagree, and the structural one doesn’t fall over when you drop five celebrities into it.

The disagreement is the part I kept thinking about. Follower count is the number everyone reaches for, and it was also the one number our adversaries could edit at will. Whatever the right measure of influence turns out to be, it probably has more to do with where an account sits in the graph than with how big the number next to its name is.