The Most Influential Terrorist on Twitter Didn't Have the Most Followers

A college information-retrieval project, a dataset of pro-ISIS accounts, and what happened when we tried to break our own ranking algorithm with Kanye West.

The Most Influential Terrorist on Twitter Didn't Have the Most Followers

In 2015, researchers estimated that around 125,000 Twitter accounts were run by ISIS sympathizers. If you’re an analyst staring at a haystack that size, you have two bad options. You can train a machine-learning classifier to flag extremist accounts — fast, but it treats every flagged account as equally important, so you trade one giant pile for a slightly smaller giant pile. Or you can read tweets by hand — accurate, but hopelessly slow at scale. Either way you drown in noise.

My group’s information-retrieval project at UVA started from a small reframing of the problem. What if the goal isn’t to find extremist accounts, but to rank them? You don’t need to investigate 125,000 accounts. You need to know which handful to look at first.

The obvious way to rank influence is follower count. It’s also wrong.

Follower counts are easy to game — you can buy them — and the people we were trying to rank knew that. The ISIS Twitter Census report out of Brookings documented that ISIS-affiliated accounts actively manipulate their follower and following numbers specifically to throw off the metrics platforms use to flag them. If your ranking is built on a number your adversary controls, you don’t have a ranking. You have a leaderboard they’re writing for you.

This is the same problem the early web had, and the same problem PageRank solved for it: don’t measure how many links point at a page, measure the structure of the whole link graph. PageRank is harder to fake because no single node controls it. But plain PageRank ignores what accounts are actually talking about — it sees the link structure and nothing else. We wanted something that understood topic too, so an account influential among people tweeting about ISIS would rise, rather than an account that’s simply well-connected in general.

That’s what TwitterRank does. It’s an extension of PageRank that folds in topic and homophily — the tendency of people to cluster around shared interests rather than follow each other out of courtesy. Instead of one random walk over the whole network, it does a topic-specific walk, so influence is measured within a community of interest. We didn’t invent it; it comes from a 2010 paper, and we ran an open-source implementation against our data. The interesting part wasn’t the algorithm. It was watching what it did.

Experiment one: does it actually beat follower count? We took a public Kaggle dataset of about 170 pro-ISIS accounts and ranked them. The account TwitterRank crowned most influential sat only 29th by follower count — 632 followers, against others in the set with far more. The raw-reach ranking and the structural ranking disagreed, and they disagreed in exactly the direction the Brookings report predicted they should.

Experiment two: can we break it? This was the fun one. If influence were really just reach in disguise, then dropping a few of the most-followed accounts on the planet into the dataset should blow up the rankings. So we did. We added Snoop Dogg, Kanye West, Khloé Kardashian, Leonardo DiCaprio, and the Dalai Lama — each with 18 to 30 million followers — into a pool where the biggest “terrorist” account had around 29,000.

None of them cracked the top five. The list barely moved; three of the original five stayed exactly where they were, and the small shuffles were within the noise of the algorithm’s randomness. Kanye has thirty million followers and zero standing inside a tight, topically-coherent ISIS retweet network, so the algorithm shrugged him off. That was the whole point made concrete: TwitterRank scores you on where you sit in a community of interest, not on how famous you are to the world at large.

I want to be honest about the limits, because it was an undergrad course project and it shows. The dataset was small and public. We ran an existing implementation rather than building our own. We never validated the rankings against any real ground truth of “influence” — we showed the method behaved sensibly and resisted an obvious attack, not that it was correct. There’s a version of this with real scale and real evaluation, and we didn’t build it. The honest contribution was a clean demonstration that the intuitive metric is the wrong one and that a structural, topic-aware one survives a stress test the intuitive one would fail.

The thing I’m taking from it is simple. The most important item in a pile of data is rarely the biggest or the loudest one; it’s the one sitting in the right place in the graph. Follower count was the obvious signal and the wrong one — the structure of who connects to whom said more than any single number could. That feels like it generalizes well past one dataset of Twitter accounts. The followers were never the point.

Share

The Most Influential Terrorist on Twitter Didn't Have the Most Followers
Older post

Notebooks to Web Apps

Newer post

Instead of Opening the Black Box, I Asked the Model to Explain Itself

The Most Influential Terrorist on Twitter Didn't Have the Most Followers