Twitter developers have been working on a way to make tweets posted longer than 7 days visible in the search engine (previously if you were searching for tweets, those posted 7 days or longer would not show up in the search engine). But thanks to DataSift, they will be provided with not only the full “Twitter firehose,” but historical tweets as well.
Erick Schonfeld from TechCrunch reports, “DataSift’s historical service will give developers social media monitoring companies, marketers, and brands access to 60 days of tweets for the Alpha, which can be analyzed and filtered beyond simple keyword search. When the service launched more broadly later next year, it will go back as far as two years. DataSift allows for all sorts of data analysis because it pours all the tweets into a structured database.”
So for example, you could give the Historical Service a query, like “Show me all of the tweets that mention Kim Kardashian who also follow @TheHuffPost,” and it will generate a bundle of those tweets.
DataSift’s database is HUGE! Collecting 1 terabyte—a unit of measurement translating into 1,000,000,000,000 bytes—of digital information in just one day. Nick Halstead, founder of DataSift, says “[DataSift] is a real ‘big data’ engine—and we are making it simple—we are taking advantage of map reduce—but this is our own bespoke processing engine.”
Developers interested in this service can sign up for the Alpha of DataSift’s Historical Data starting today and the actual service itself will roll out in Q1 of next year.
(via TechCrunch; photo via eWeek Labs)
Related posts:
