Posts Tagged ‘baidu’

Who Indexes Tweets

I was curious who is indexing the links that people tweet on Twitter. It’s obvious someone does since links get ‘clicks’ almost immediately after submission. To do this presumably they are tapping into the xmpp firehose. Lets take a look: 66.xxx.xxx.xxx – – [06/Dec/2009:20:17:43 +0000] “GET /test HTTP/1.1” 301 20 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)” […]