Twitter Friend Feed Authentication with Yahoo Pipes

Revert to Firefox's Add-on Delicious Bookmarks 2.0.64 →

Twitter Friend Feed Authentication with Yahoo Pipes

Aug 5th 2008 at 3:21 PM

Update 12/2009: Twitter's changed things around since I first made this post. Now, prefer the home_timeline.rss instead of friends_timeline.rss if you want retweets included. Also, "mentions" are no longer included in the home_timeline or friends_timeline. So you'll have to make a similar "Fetch Feed" hierarchy for mentions.rss, and use the Union filter to take both feeds and join them together.

Yahoo Pipes can also be used for feed authentication (like Twitter Friend feeds). You could embed the username and password directly into the URL (in which case, everybody can see them), or you can hide them within a private string object, like this.

With a feed like that (with your own username and password), you can view your friends feed from Google Reader.

Tags: geek, programming, web

Comments

crazy_cityhere on Aug 5th 2008 at 3:21 PM

(Deleted)

jasontromm on Sep 16th 2008 at 9:08 PM

Thanks Dave. Worked like a charm! Now I can catch up with my friends tweets at my own pace.

(no name) on Oct 16th 2008 at 10:08 AM

Really helpful - thanks David!

ext_130996 on Oct 30th 2008 at 7:51 PM

This technique is totally awesome: it works! Unfortunately, Google Reader checks my Twitter Pipes feed every 3 hours. Is there a way to get Google Reader to check it every 15 minutes?

dblume on Oct 30th 2008 at 8:05 PM

Not that I know of. Keep in mind that you also have to deal with the delay on Yahoo's end, too. It's a long pipeline from the original tweet to your eyes. For example: Friend tweets at 01:00 Yahoo runs the pipe to check the private friends feed at the next regular interval, say 03:00, and updates its public feed. Google Reader checks that public feed at its next regular interval, say 05:00. You check Google Reader when you get around to it, say at 07:00. I don't know what the actual numbers would be on average. If you got really lucky, it'd be something more like this: Friend tweets at 01:00 Yahoo runs the pipe to check the private friends feed at the next regular interval, say 01:01, and updates its public feed. Google Reader checks that public feed at its next regular interval, say 01:02. You check Google Reader when you get around to it, say at 01:03. You can take full control of making the feed public if you do it yourself with something like curl. Here's an example of the CURL way.

ext_130996 on Oct 30th 2008 at 9:25 PM

I'm trying it both from my own server and through Yahoo Pipes. Both are working to pull the feed correctly. I have my cron job on my server pull the feed every 15 minutes, while the Pipes pulls it every hour or so. Unfortunately, Google Reader is pulling both feeds at 3 hour intervals, which is just too long! I found this article describing how I can ping Google Blogsearch to notify that my feed is ready, but I'm unsure if it's having any effect to increase the frequency of GR updating my server's and Pipes' tweet feed. Here (at the bottom of the post): http://www.bolinfest.com/changeblog/2007/12/14/get-all-your-tweets-in-google-reader/ and here: http://www.google.com/help/blogsearch/pinging_API.html Just wondering if you or anyone else knows if pinging is supposed to increase the frequency of GR pulling specific feeds.

(no name) on Nov 8th 2008 at 2:45 AM

I found a solution! If you have another Google account, simply use that alternate account's Google Reader service to subscribe to the authenticated feed. My feed are coming in less than an hour now!

(no name) on Sep 11th 2009 at 8:26 AM

Thanks for the explanation. Has the situation changed in the past year? Is it now possible to subscribe to my friend's timeline and not miss any tweets (due to more often updates - e.g. every 15 minutes)

datavortex on Jan 19th 2009 at 3:05 PM

Thanks, this is really handy.

unloveable on Jan 28th 2009 at 5:21 AM

Thank you for this. One question, though: can Yahoo Pipes authenticate over SSL? E.g., https://username:password@twitter.com/statuses/friends_timeline.rss ... I get a "problem parsing response" error when I try. Or do you just have to trust that Yahoo's connection to Twitter (or any other service requiring authentication) is reasonably secure?

dblume on Jan 28th 2009 at 4:28 PM

In the end, I'm not using the pipes solution, I'm simply running curl in a cron job. I don't know the answer to your question. If you figure it out, will you post what you discover in a reply? FWIW, if I need SSL to protect data, I wouldn't give the data to Yahoo pipes to store on their disk in the first place.

(no name) on Apr 25th 2009 at 7:04 PM

This worked so well, thank you for sharing!

(no name) on May 1st 2009 at 4:49 PM

Unfortunately, this technique uses "basic authentication", which sends your password over the Internet in easily reversible base64 encoding. This would be ok if the session were encrypted (https), but pipes doesn't support it at this time.

seotoday on Mar 4th 2010 at 2:50 PM

cool i will embed it in my blog thanks

← So Tweet It Is

Revert to Firefox's Add-on Delicious Bookmarks 2.0.64 →

Backup of David's Livejournal

Recent Entries

Navigation Tips

Twitter Friend Feed Authentication with Yahoo Pipes

Comments

Tags