An analysis of utility versus entropy in social systems, or, the simple solution to fix Twitter


As I wrote yesterday, I’m a huge fan of the beautiful simplicity afforded by Occam’s razor: it posits that, when given several alternative explanations to explain an hypothesis, the simplest solution is often the right one.

And this isn’t just a thing that sounds good to say (well, it does feel good, but that’s not the point). It’s actually almost always true, too; we’ve trusted its elegance countless times for our startup, and it’s never let us down.

It’s time Twitter heeded the same wisdom.

I’ve written time and again (and again) about Twitter’s seemingly always imminent doom, and why it simply cannot — as in, must not — fail: it’s simply too important in the world.

Nevertheless, it is indisputable that Twitter is an incomprehensibly jumbled mess, a veritable wasteland of anarchy and vulgarity, the cacophony of its 300 million users only slightly more intelligible than static on an AM radio frequency.

And this makes me sad. No, not just because we built our startup to help companies get more value out of the utterly chaotic noise of the sprawling Twitterverse that seems to permeate every pore of our society, but because there’s so much potential that’s just being wasted simply because there isn’t any sort of filter.

Imagine if browsing television channels worked like viewing your Twitter stream: it would look like a psychedelic montage of myriad different programs cross fading across the screen, their audio a garbled wreck of dissonant tones. Sure, you’d at times be able to pick out a few gems from the otherwise discordant mess, but it wouldn’t exactly make for a particularly engaging or fulfilling viewing experience.

Yes, I realize this analogy is a bit silly — obviously TV and 140 character tweets are about as similar to one another as a filet mignon and a chainsaw. But while they are indeed vastly different forms of media, they do share one common theme: they are both premised, like any other media, on the notion that you should be able to quickly and easily (a) find what you’re looking for and (b) consume it without difficulty. I mean, just imagine if that filet mignon suddenly start walking away, or if the chainsaw didn’t have a hand grip: holding it by the sharp rotating bits wouldn’t be a very pleasant experience.

So that’s Twitter’s problem: it’s got all the right ingredients, only they’ve been thrown about like confetti at a Super Bowl game rather than methodically measured and delicately assembled.

Fortunately, there is a solution. And it’s beautifully simple:

Twitter needs to limit the number of accounts you can follow.

Dunbar’s number is a commonly accepted limit to the number of useful relationships a person can have in society, typically pegged at 150. The principle of Dunbar’s number is fairly simple: only the first 150 relationships can afford any modicum of intimate knowledge or true friendship; beyond this, such relations are mere superficial acquaintances.

Having learned of this number several years ago, I was astonished to see how well this fit my own world: the number of friends and family I have on Facebook has never ever exceeded 150, and has typically hovered around 130 or so. On Instagram, too, I only follow about a third of that. This isn’t due at all to any sort of aversion to social networking — quite the contrary, I am extremely active on both platforms — but rather because I am highly selective with whom I choose to connect, and limit such connections explicitly to friends, family, and former classmates. For everyone else, there’s LinkedIn.

I don’t mean that distinction offensively, but rather pragmatically: I once met an investor who asked to connect with me on Facebook. Shocked, I just laughed and politely suggested we connect on LinkedIn instead, explaining that Facebook was just for my wife, my parents and extended family, and close friends. A slight exaggeration, but definitely an otherwise accurate statement.

And this is why my LinkedIn account is adorned by the famously, curiously ambiguous “500+” connections counter: LinkedIn fails Dunbar’s number precisely because it is not a network of my “socially meaningful” relationships, but rather because it is a repository of per se networked acquaintances whom I’ve met — yes, even on LinkedIn, I rarely add people whom I’ve never met at least once in real life — at some time or another.

So there’s a big distinction between LinkedIn — essentially an internet-based business card organizer — and, say, Facebook: the former is meant to be an accumulation of as many people as you can, precisely for the purpose of “networking,” i.e., so that you can reach out and easily contact whomever you need with no effort; the latter is for being “social.” One is intermittent pings of contact; the other a steady stream.

You can probably see where I’m going with this: at first blush, Twitter absolutely seems to share more in common with LinkedIn than Facebook: its value is precisely that it allows you to connect with people you do not know personally, and that’s very cool and powerful stuff indeed. The problem, however, is that it also shares qualities of Facebook: while LinkedIn offers a trickling stream of updates to follow, Facebook is a veritable spigot, its News Feed an inexorable torrent of news, friends’ updates, and food photos.

Put another way, if you Venn diagram these three different platforms, Twitter, in a weird and unexpected sort of way — this just occurred to me while typing this right now — sort of falls into the intersection of LinkedIn and Facebook:

It is at once a way to seamlessly connect with people from all walks of life, all over the world, while on the other hand offering an endless stream of updates like Facebook.

The problem, however, is the incessant nature of updates on Twitter pouring out from an overwhelming number of accounts one may follow: and just like that, Twitter’s very raison d’être — its inexorable stream of tweets — becomes its undoing once the number of accounts followed exceeds a practical limit. It’s almost like a car with too much power: eventually, it’s too much to handle. Sure, in the hands of a professional, a Formula 1 car is the fastest thing to lap a racing circuit; but in the hands of an amateur, it’s the fastest way to die.

Although I haven’t researched it, I’m sure there are mountains of research confirming an inverse relationship between utility and entropy: for any given system, utility goes up as orderliness goes up. If LinkedIn represents the lower bound of a system’s entropy (i.e., maximum utility), while Twitter represents the upper bound (lowest utility), then Facebook probably falls somewhere in the bottom third.

The question then becomes, how do you negatively affect a system’s entropy — its chaos — and thus positively affect its utility?

So there are two variables we can play with here: one is the number of elements in a system; the other is the frequency of those elements. In Twitter’s case, there are both (a) too many elements (the typical user follows too many accounts) and (b) the frequency of updates is simply overwhelming (new tweets appear every few seconds).

The solution may seem trivial by now: Twitter needs to either automatically limit the frequency that new tweets appear in one’s timeline, say, to one tweet per minute (or at least let the user set their preferred frequency); or alternatively, Twitter must cap the number of accounts one can follow to something far more manageable, say, 150. Or both.

This isn’t just some random theory I came up with out of the blue: using Twibble for my own Twitter account, I’ve effectively accomplished this very goal: using Twibble as an RSS-to-Twitter scheduling platform, my tweets are almost entirely composed of interesting and relevant content not just for my own followers, but rather for myself.

The effect is that instead of viewing the usual torrent of tweets from all Twitter accounts I follow, I tend instead to view only my own tweets: but because they’re coming from only the most interesting people (or blogs, YouTube channels, etc) that I follow, suddenly I’ve solved both the elements problem and the frequency problem, all at once.

The net result is that, nearly overnight, Twitter’s value — its utility to me — has jumped by orders of magnitude: where once it had been little more than a waterfall of noisy static, a clumsy analogue knob desperately seeking music on an old radio, now it’s been optimized and streamlined with the equivalent of favorite Pandora channels or preset radio stations, showing me precisely what I want, and when I need it; nothing more, nothing less.

This is not a hugely complicated revelation: with all due respect to Jeremy Clarkson, more isn’t always better, and certainly not infinitely so: eventually you hit a wall of diminishing returns. Even he realized this when he claimed, shockingly, that the Ferrari F12 would be even better if it had less power; 700 horsepower was simply stupid. As he explained by way of one of his absurdly hilarious analogies, that car was like practicing tennis with hundreds of tennis ball launchers firing them at you simultaneously: it’s completely and utterly unmanageable.

And that’s Twitter today: it’s like practicing tennis not with a single launcher firing a ball at you every few seconds, but with an entire armada of the things launching hundreds every second. It’s not just uselessly futile, it’s completely stupid.

This isn’t to suggest that, in the right hands, Twitter can’t still offer tremendous value: sure if you’ve got the training, the skills, the patience, and the tools to make it bend to your every whim, then it’s still fantastic. After all, an F-18 fighter jet is an astonishing machine to fly too; but if all you’ve ever flown is a Cessna 152, you won’t get very far with it all. Same as poor Jeremey with his overly-potent Ferrari F12.

What’s the point of a utilitarian tool designed for all to use if nobody can actually get any value of it?

Twitter is a genuinely unique product that’s quite simply suffering — imploding, really — under its own excessive power. What it needs is the software equivalent of traction control and antilock brakes: something to help people harness its power, and temper the otherwise inexorable torrent of tweets that effectively drown out any hope of filtering through anything of true value.

So, the Occam’s razor-approved simple solution to fix Twitter, then:

If Twitter can implement both (a) frequency limits to how many tweets one sees per minute and (b) a cap on the number of accounts one can follow, then its entropy will be reduced, its utility increased by a proportional amount, and all will be well in the world.

Or at least, in the Twitterverse, anyway.