The Data Deluge

Hilary Mason Breaks Down Bitly’s Social Data APIs for Real-Time Search and Attention Spikes

Float down the stream with Bitly's chief data scientist.
hilary mason Hilary Mason Breaks Down Bitlys Social Data APIs for Real Time Search and Attention Spikes

Ms. Mason

We’ve expected some big (data) developements from Bitly since the New York startup announced a $15 million Series C led by Khosla Ventures July. Today, chief data scientist Hilary Mason is finally ready to show you what they’re working with, empirically speaking.

The company announced the launch of three new data APIs that will radically boost the utility of the service for consumers and business clients. And it’s not even your birthday, data nerds!

There’s a real-time search API, open to any Bitly user. The company is also opening up an API for content analysis and a “bursting phrases” or “attention spikes” API that functions sort of like Twitter’s “trending topics.” Obviously APIs are geared toward third-party developers to incorporate into their products and build complimentary applications. However, any Bitly user can play around with some of that new-and-vastly-improved functionality using a demo interface at rt.ly.com. “We’re data hackers, not UI designers,” Ms. Mason warned us, but the site is very user-friendly.

Go to rt.ly and you’ll find a number of filters that let you search all Bitly links by city, social network it was shared on, key words, languages, and even topics. You can also “create a story” for a particular search, like our favorite subject “Eric Schmidt,” [see below]. That pops up a report with a map showing click rate, the geographical distribution of readers, link referrers, related stories, top domains, and more. “People who are really social web savvy have been using this already to discover the kind of content they might want to share or if they are writing content to see how it’s being distributed,” she told Betabeat by phone.

It’s obvious after spending some time on rt.ly that Bitly thinks about search differently than your standard Google query. “Our search lets you do things like describe the criteria you’re interested in and then just see what’s popular in that subset right now. An example–and one that I like to show off–is links about the topic food being clicked statistically disproportionately from Brooklyn. So you get a lot of the artisanal pickle style stuff and restaurant reviews.”

Bitly’s search function is also distinguished by its emphasis on real-time. “There’s the stream and we just sort of let you filter it and watch content pop in as it becomes popular and drop out as people stop paying attention to it,” she said.

The interface also lets users peek at the content Bitly analyzes on the other side at the other side of its shortened link. “We do things like pull out the key phrases in that content and pull out the topics,” she said.

The “bursting phrases” API is a even more complex. “Basically we calculate the click rate across all of Bitly’s data on every web page containing any significant phrase in every language that we track,” she said. “What that lets us do is look for the phrases that are getting a disproportionate amount of attention right now.” It’s sort of like Twitter’s trending topics, she explained, in that, “we can tell you what people are paying attention to that we did not expect based on historical behavior.”

Some of the data revealed by the new social APIs will start to show up on Bitly.com, she said. “You can expect to see much better stats for consumers that will draw off of this data, as well as better discovery tools on the enterprise side and audience analysis tools.”

Ms. Mason didn’t seem anxious about Twitter’s sudden zeal for cracking down on third-party apps. “We’re always concerned about it but because of the way Bitly works–you wrap the link in a Bitly link before you share it on Twitter–we haven’t had to deal with any of those issues,” she said. Even after Twitter started shortening links, 30 percent of all links shared on the service are still powered by Bitly. The company powers around 30,000 different domains for people. “The examples we usually give are the New York Times, the Dalai Lama, and Mariah Carey on the celebrity side,” she added. “So it’s not all bit.ly links by a lot.”

Besides, she noted, Bitly isn’t all about Twitter. “We see almost the same amount of data for Facebook as well.” Any links shared privately, however, won’t be exposed by the API.

Developers chomping at the bit to work with Bitly’s API will have a chance at a hackathon at its office next week, although the exact dates and times haven’t been around.

If all this data is too much for you to handle, you can always take the old school approach, as Ms. Mason reminded us. Add a plus sign to any Bitly link and it takes you to a stats page for just that article.

Screen Shot 2013-01-08 at 1.17.51 PM

Screen Shot 2013-01-08 at 1.17.15 PM

Follow Nitasha Tiku on Twitter or via RSS. ntiku@observer.com