Derrick's Web Snacks - Process

When I started work on the "Derrick's Web Snacks" feature on OWF, a sidebar item with interesting web tidbits, I wanted to build something that was easy for me to use. If it took too many mouse clicks to get a link into the Snack area, I would never update it; if I could do it quickly, I would keep it fresh. My ideal solution would allow me to select a link, press a button, and see it show up in the Snacks later that day. (It didn't bother me if it took a while to show up on OWF after that.)

I wanted to automatically incorporate links on OWF, and I figured if I could put them somewhere with an RSS feed, I could show that feed on OWF. (I hand-waved over that point, figuring there must be some simple way to make that happen. Boy, was I wrong.)

I looked at Bloglines's "Clippings" and "Blog" features, but the former wasn't flexible enough—it only works with clips I make from within Bloglines—and the latter required only a few less steps than making the changes in Blogger, though it was somewhat faster.

I settled on a private "snacks" blog to which I could publish links. I have a Dashboard widget that publishes to Blogger, so I could snag a link, bring up Dashboard, paste the link, write a description, and hit Publish. The widget keeps my login information, knows which blog I want to use, and publishes faster than my browser can manage. It takes me 10 to 15 seconds from grabbing a URL to publishing a post.

At some point, as I imagined myself using the feature, I realized that certain items would always get "snacked." In particular, I'd always want to link to the latest digests from Ethicurean. If I wanted to always include them, why should I do it manually? There's a running joke that the best programmers are the laziest ones: We don't see any reason to do work that a computer should be able to manage.

And then I remembered Yahoo! Pipes, a tool the company released in February that allows users to create "mashups" of feeds. I noticed the hubbub when it came out, but I didn't play with it. I knew it could combine feeds in some way and create a new feed from the output, but I didn't know if it could do what I wanted.

I have fallen in love with it. Within an evening, and starting with no knowledge of the product, I had created a series of modules that did what I wanted. My pipe pulls the 8 latest posts from my private links blog, doing some simple transformations in the process. It also grabs the latest Ethicurean digest (blogs and news) and changes the title to "Ethicurean Blogs (or News): [date]" where [date] is the Digest's date, so that users know at a glance if they've already seen it. The pipe smushes it all into a single feed. All I had to do was publish that on OWF.

I wrote a simple set of JavaScript functions that used the XMLHttpRequest object to load the data from the pipe. It would have worked perfectly, except for one tiny problem: For security reasons, you can't have an XMLHttpRequest object fetch data from a site other than the one where the page came from. In other words, because my page came from www.obsessionwithfood.com, I couldn't load data from pipes.yahoo.com. Hm.

I tried creating an iframe object that would contain the feed's output. Safari rendered it in its RSS reader, and Firefox freaked out about the fact that I had a ".run" URL.

I started to wonder if my pipe was doomed to die. I looked for advice about incorporating RSS feeds and realized that the solutions all used PHP, which my ISP has made available for me, something I discovered while working on this.

I've never gotten around to learning PHP—I went straight to the enterprise Java server world—but I figured out enough of the language to build a script that grabs the RSS feed from the pipe, parses it, and spits out HTML.

Too bad Blogger doesn't give you a way to incorporate PHP. I added a PHP include to my template. I added a server-side include. Nothing worked the way I wanted. But then I remembered: XMLHttpRequest doesn't work for addresses on a different server, but my PHP script now sat on obsessionwithfood.com. A few lines of JavaScript, and my snack section showed up.