Twitter, Outlines, Lists, Directories, Y!ou

Humans create the web, but we’ve largely abdicated the act of organizing web content to software. That could change.

  • Twitter this week made its new Lists feature broadly available. As they’ve been described, Lists, allow you to enumerate a collection of some of the Twitter accounts that you follow, and then easily read updates from just those accounts. Others can view your lists, and choose to subscribe to them as well. But Lists are also available for other applications to use, modify and share. Looked at from a slightly different perspective, this means Lists are a way to tag an arbitrary set of realtime web feeds. You could look at the lists that I’ve been added to as a set of tags describing my Twitter feed. tag cloud of Twitter lists for @anildash
  • Much of the precedent for the idea of sharing (non-realtime) feeds comes from the world of outlining, and in particular Dave Winer‘s work here in creating OPML. Though it was designed to generically exchange outlines, OPML is the most popular format today for sharing arbitrary lists of feeds. (The computer science folks balk at some of the technical aspects of OPML but it’s a bit like Churchill’s comments on democracy — it’s the worst format, except for all of the other alternatives.) What’s interesting about having an established format for exchanging feeds is that there doesn’t really need to be any changes in order for the format to accommodate realtime feeds like Twitter accounts. In fact, a few weeks ago, I moved about 150 the noisier, less pressing Twitter accounts I follow into Google Reader, by exporting them as an OPML file. Twitter became more pleasant to use, and I could still keep up with all of those folks by dipping into my feed reader whenever I want to.
  • Lists have a few traits that make them more interesting than they seem; we can think of these as the Laws of Lists. First, you have to be signed in to Twitter with a valid account in order to create them. (This seems obvious, but it’s important.) Second, by adding a Twitter accounts to a list that you create, you follow that user’s updates, at least while viewing that list. This combination of authentication and requirement of relationship is a very good recipe for reducing spam.
  • One of the earliest hopes for organizing web information was the human-edited directory. Efforts like the Open Directory Project still exist, but the model focused a lot on having defined editors for topics and a hierarchy of who could edit the site. That’s a stark contrast to the default-open editing permissions of projects like Wikipedia, and is probably the most significant difference between the “human-edited” and “user-generated” eras of the web — we’ve always had people contributing content, the difference was in how much we trust them. Similarly, more outline-focused directories of content emerged, like Halley Suitt’s Top Ten Sources, which is now defunct, but was based upon the idea of curated lists of feeds by topic. In each case, trying to scale a team of editors to keep up with the rate of growth in new sites on the web has been a losing cause. But we’ve seen sites like Delicious demonstrate the value of tagging individual pages or posts on a site — a new generation of directories could demonstrate the value of tagging entire streams of posts, or as we call them, feeds.
  • Of course, you can’t talk about directories and lists on the web without talking about Yahoo. Yahoo’s original sin was in trying to create a human-edited directory of the web, and before they unfortunately achieved their goal of becoming the only successful web portal, the directory was Yahoo’s signature element. (Until recently, Yahoo had maintained a page with the directory in a format resembling its original state, but even that is basically a blog now.) Instead of embracing authentication and relationships to prevent spam submissions from overwhelming the site, Yahoo leaned heavily towards requiring payment for inclusion of companies in the directory, limiting its utility. Human edited directories became mostly a footnote in both Yahoo’s, and the web’s history.

That fundamental history of being made by humans is some part of Yahoo is trying to evoke with its Y!ou and Yahoo campaign. But of course, it’s a pretty good sign that a campaign isn’t going to hit its mark when a completely unknown brand like HTC can launch virtually the same campaign as a household name like Yahoo, yet both companies think their message is going to resonate.

The truth is, if Yahoo wanted to help people reimagine the web stalwart at its best, they would do well to look to their roots in a human-edited or user-generated directory. Thinking of Yahoo at its peak of influence a decade ago, it becomes clear that instead of trying to insert their ubiquitous exclamation point into you, Yahoo should look at the story of The Matrix. I don’t know if the brothers Warner or Wachowski would be inclined to license the property, but the only way to truly resonate with people in a narrative of Yahoo vs. Google is by adopting this theme: Man vs. Machine.

Just as in the Matrix the humans had originally created the machines that undermined them, to some large degree, Yahoo begat Google. And Yahoo would do well to suggest that the most human way for the web to evolve is if we all work together to organize it ourselves — a mission that happens to fit in well with Yahoo’s largely-mishandled acquisitions of Flickr and Delicious. I’m not sure that the marketing folks at Yahoo are going to embrace that narrative, but an interesting opportunity definitely exists around the larger concept.

We all have the ability to create and exchange curated collections of feeds, using hubs like Twitter’s Lists as connection points. We can extract the descriptions from those collections to form tag clouds about individual feeds. If we want to embrace hierarchy, we can organize the collections into a hierarchy by inheriting the category structure of sites like Wikipedia. If we’re worried about spammers, we can now use widely-available systems of authentication and defined relationships to define who has the authority to create lists in a particular context. And of course, the ability to aggregate all of the distributed content from a defined set of feeds in realtime has now been commoditized, where i would have been exorbitantly expensive a decade ago.

In short, we can learn from Twitter’s Lists to resurrect one of the web’s original ways of organizing itself: Human-curated directories. We’re used to exploring photographs or individual web pages by clicking on tags that were assigned by the creators or their community, and it will be just as valuable and useful to be able to explore entire feeds the same way. Open formats and APIs for exchanging this data already exist, so I can’t wait to see a few enterprising hackers build the tools that let us revisit the idea of web directories. I love computers and robots, but I love humans even more, and I think we can do a pretty good job of guiding each other to the most interesting feeds around.