Yahoo Pipes for Scientists

Yahoo pipes logo (TM)

I’ve talked about Yahoo Pipes before. It was Berkeley chemist and Sciencebase regular Mitch Garcia who turned me on to them with his clever pipe that allows anyone to search as many chemistry journals as possible that offer ASAP and in press papers online and to produce a live feed from the results that is automatically updated as new ToCs and ASAPs come online.

Various journal users have also added their own personal spin of the pipe, for example, All My Eye demonstrates how to use Yahoo Pipes with Ingenta newsfeeds specifically. Pedro Beltrão has used Pipes to filter journal RSS feeds by biological community too. Mass spectrometrists have also climbed into Pipes.

So, what is Yahoo Pipes. Well my good friend Roddy MacLeod at the Internet Resources Newsletter (IRN) quotes it best:

“Yahoo Pipes is an interactive feed aggregator and manipulator. Pipes is a free online service that lets you remix popular feed types and create data mashups using a visual editor. You can use Pipes to run your own web projects, or publish and share your own web services without ever having to write a line of code.”

Today, I thought I’d introduce you to a few more courtesy of some other Scientific Pipers.

Alan Cann who runs the Science of the Invisible blog has created a Yahoo Pipe to create a simple “vanity” superfeed. A vanity superfeed as the name would suggest allows someone with an active net life to track mentions of specific keywords (usually their name) across the blogosphere, the web, and among forums. It’s not quite as vain sounding as you might think though. It’s very useful from the marketing and search engine optimisation (SEO) points of view to know where you, your personal brand, is being mentioned on the net. It might lead to new collaborations or opportunities as you discover like minded individuals, it could help you detect plagiarism, and it allows you to put the record straight if someone has misquoted you. Of course, it also lets you see just how popular you really are! You can out more about Alan’s vanity pipe here.

Technology writer Wayne Smallman pointed out recently that Yahoo Pipes no incorporates geotagging, which means that you can mash up Pipes with Google Maps and Google Earth.

Another tech savvy chemist, Cambridge’s Peter Murrary-Rust describes some of the benefits of Yahoo Pipes as an almost coding-free approach to improving workflows in finding chemical information.

Meanwhile, Read/Write web discusses the advent of Yahoo Pipes and the world-wide web as database ripe for mining. The article’s authors draw some interesting parallels between Yahoo Pipes and relational databases, which underpin vast numbers of ecommerce, corporate and other websites concluding that, with pipes, the Web essentially becomes a giant database that can be queried and remixed in any number of ways.

As I trawled the web for science-related Pipes, it occurred to me that I should create a Pipe for finding science pipes in all the various fields. It seems that at the moment, the vast majority mashup journal ToCs and RSS feeds, but the relationship with relational databases is where the real power lies. How about a Pipe for gene sequences and protein products linking with their substrate molecules via InChis & NMR spectra, connected to the literature from all bio/chem publishers?

Once you’ve done that you might like to run InternetDuctTape’s Yahoo Pipe Cleaner Greasemonkey script to tidy up the output and make it easier to scan in your browser.