Filter Failure and the Digital Stream

It may be due to the fact that we are still in the Age of Aquarius (1447AD to 3597 AD) but water references certainly seem to dominate our discussions of the Big Content era.

Firehoses, streams, torrents, waves, tides, tsunamis, flows  and oceans are common nouns used to describe the flow (there we go again!) of information, knowledge and data onto our devices and into our consciousness.

The general response is one of resignation and defeat: we are, by turns, overwhelmed, drowning, choking and sinking under the weight of all that knowledge coming our way. The blame tends to be thrown on an unspecified 'it' or 'they:' all those people or algorithms or governments or businesses or machines that generate all that stuff that we then feel compelled to review, primarily because we are afraid we'll miss something.

That we so avidly consume this material, thus stimulating demand for more, rarely seems to enter the discussion. And, as the following article explains, it may actually be our inability or unwillingness to design, install - and then maintain - an effective set of filters that is the real issue.

Whether as individuals or organizations, it is that discipline to manage the information opportunity that will differentiate those who succeed in utilizing it from everyone else. JL

Mathew Ingram comments in GigaOm:

The problem isn’t so much information overload as “filter failure.”

The stream has become the dominant form for content consumption, especially for social media like Twitter, and having a never-ending source of information can be a wonderful thing — but it can also be overwhelming and noisy. We need better ways to manage it.

By now, many of us who live our lives — or at least significant parts of them — online have grown used to the ubiquity of the “stream” metaphor when it comes to consuming content. It probably started with RSS feeds and blogs, but it has become the default for many services, and particularly social ones like Twitter and Facebook and Tumblr. Where once there were individual webpages, now there’s often just a stream that scrolls off into infinity, like a highway that disappears into a distant horizon.
That kind of thing is wonderfully liberating, but it can also be distracting and noisy, and I would argue that Twitter is one of the worst culprits. I’m willing to admit that part of the problem is the way that people like me use it (or possibly over-use it), but part of it is also the lack of filters and other tools that would make the Twitter firehose easier to manage.
Twitter spends a lot of time and resources getting you to follow more people — recommending celebrity accounts, showing you activity in the Discover tab, and using smart algorithmic tools like @MagicRecs to show you who others in your stream are following. But it isn’t so great with features that help you manage your stream. For example, I would love an automated account that did the opposite of @MagicRecs and told me who I should stop following.

Not just a stream but a flood

Like others who adopted Twitter early and use it for their work, I have built the service up over the years to the point where it is like a second brain for me (or at least an extension of the first) and one that I have a love-hate relationship with for a variety of reasons. Among them is the fact that while an infinite river of information is a magical thing to have, the downside is it just keeps flowing, and that makes it difficult to pick things out of it that might be worthwhile or interesting.

That helps explain why some new offerings like Yahoo’s new Tech News app deliberately just give you a finite series of updates, much like you get with a newspaper. One of the psychological benefits of this approach is the feeling of completion you get — you can’t finish Twitter or the internet. That’s simultaneously fantastic and disturbing (I often find myself scrolling backwards through time on Twitter because I am convinced that I have missed something worthwhile).
Alexis Madrigal wrote about this phenomenon recently for The Atlantic, and argued that 2013 might be the year that the obsession with streams would start to wane — a prediction that comes just a few years after TechCrunch declared that everything was becoming a stream. As Madrigal put it, the downside of the stream approach is that everyone is now overwhelmed, but no one wants to admit it:

“Everyone is (over)optimizing for the stream. That makes the media Internet a very fragile place. It’s like a story of ecosystem collapse where once the delicate balances get thrown off, the biome begins to veer off in crazy directions, everything running around like Texas crazy ants.”

You can have too much of a good thing

Part of my Twitter problem is sheer volume, which is why I was interested to read Charlie Warzel’s recent post at BuzzFeed, in which he describes nuking his entire Twitter stream — made up of about 1,800 accounts he had followed over the years — and rebuilding it from scratch. I follow over 3,000 people and media outlets on Twitter, so I can sympathize with Warzel’s pain, and have often felt the desire to delete everyone and start again at the beginning. But it just seems like so much effort that I don’t do it.

I felt much the same way about my Facebook account, where I had deliberately (and, as it turns out, wrongly) accepted almost every friend request I received, even if I didn’t know the person very well. In a binge of unfriending I wrote about last year, I disconnected myself from more than 800 people, or about 80 percent of the people I was “friends” with. And I can say that it improved the experience dramatically for me — although I still get plenty of Facebook ad spam, so my problem isn’t completely solved.
Using Twitter lists is one way of slicing up your stream and bundling it into topics or themes, and in fact I would barely be able to use the service at all if it wasn’t for lists (I have a few public ones and some private ones). Apps like Tweetdeck make it relatively easy to create new lists, add people to them and view those lists as columns — but every time Twitter updates an app, especially its mobile ones, it seems to make lists harder to find and use rather than easier.

Managing the stream is too hard

It’s not that unfollowing people on Twitter is difficult — it’s just a click of a button. But first I would have to decide why I was unfollowing that person, and that would require thinking about why I followed them in the first place. I would have to look at their stream and reconsider their value, and I would have to do that 3,000 times. It’s like cleaning out the garage or indexing your photos; you know that you should do it, but it just seems so daunting that you never get around to it.
That helps explain my interest in tools that help you track who has unfollowed you, and others that show people you follow who aren’t very active. One of my favorite such analytic services is ThinkUp, which was created by former Lifehacker editor Gina Trapani and former Six Apart executive Anil Dash, and allows you to slice and dice your stream in a number of ways, to see which accounts you engage with most actively and who is providing value vs. noise (Salesforce CTO J.P. Rangaswami had some great thoughts about filters in a recent blog post).
For Twitter, one problem is that the company seems focused on adding millions of news users — and oceans of new content through deals with TV networks, etc. — rather than on making things easier for existing users, in part because building up its user base helps justify its multibillion-dollar market value. But if users ultimately just find themselves overwhelmed, that could be a Faustian bargain. The stream can be a harsh mistress.