We’ve made some major updates to Pipes. The big news is that we have 8 new modules to announce!

There are 2 new modules under Sources:

• The Feed Auto-Discovery module detects feed URLs that are embedded in a webpage using auto-discovery links (such as link rel tags). This example demonstrates how you could use the Feed Auto-Discovery module to find the RSS feed for a given webpage and then fetch its content.
  
• The Item Builder module allows you to create a new single item data source by defining a set of key-value attributes. This example demonstrates how you can use it to build up a new feed.
  

There are 4 new modules under Operators:

• The Loop module will run an enclosed sub-module or Pipe one time for each item in the input feed. Its output options can be set to ‘emit’ or ‘assign’ results. Emit results will replace the content of the original input item, here’s an example. Assign results will insert the content into a specified attribute, here’s an example. We introduced this module in order to support a wider range of modules, and data types, that can be looped over. You can use the Loop module with any module or Sub-Pipe except User Inputs, Operators, or Deprecated modules. Note: Old Skool Pipes users will notice that this module is similar to the For Each modules.
  
• The Reverse module reverses the order of the items contained in the input feed. This example demonstrates how it works.
  
• The Sub-element module will emit only the specified attribute or array for each item in the input feed. This example demonstrates how you could use it to take an RSS feed and emit only a specific item attribute for each item.
  
• The Tail module emits the last ‘n’ items from the input feed. This example demonstrates how you could use it to emit only the last 5 items in a feed.
  

There are 2 new modules under String:

• The Translate module automatically translates a string from one language into another. Select the language combination you desire from the pulldown menu. Note: Old Skool Pipes users will notice that this module is similar to the BabelFish module. Unlike the BabelFish module, the Translate module can be used within a Loop module and operate on any specified element attribute. This example demonstrates how you could use it to translate the Title and Description of a feed from one language into another.
  
• The String Regex module allows you to apply Perl-like regular expressions to attributes in a string. This example takes the delicious RSS feed and replaces the title for each item with its “dc:subject” element.
  

Some other updates worth noting:

• The String Concatenate module has been renamed the String Builder. It works exactly as before but if you’re new to it here’s an example that demonstrates how you can use it to build up strings!
• In a few cases several older modules have been Deprecated and replaced by what we consider to be improved modules. The For Each: Annotate and For Each: Replace modules have been replaced by the Loop module. And the BabelFish module has been replaced by the Translate module. The Deprecated modules will continue to work but we’re discouraging their use.
• In the Editor, we’ve updated how you can modify the properties for your Pipe. In the past, there was a button labeled Publish in its place there’s a new Properties button that brings up a dialog window that allows you to modify your Pipes name, description and tags. You now have the option to either make these changes so only you see them or to publish them along with your Pipe.
  
• And, as usual we also fixed miscellaneous bugs that have either been reported or that we stumbled upon. Thanks for continuing to share your suggestions!

We worked with the Yahoo! Maps team to put together two examples that demonstrate how you can take the output from a Pipe containing GeoData and overlay its content on a Y! Map. Using these examples, you can learn how you can start putting Pipes powered Maps on your website.

The Pipe being called in the below examples is called Where2 with flickr. This Pipe aggregates event listings in the San Francisco Bay Area from upcoming.org. Then, uses the location information for each event to search Y! Local for nearby restaurants and flickr for nearby pictures. The output of this Pipe contains all the data you need to decide where to head out to!

Both examples demonstrate how to take the output from a Pipe and overly its data on a Y! Map:

• The first example uses the JSON output from a Pipe.
• The second example uses the RSS output from a Pipe.

To learn more about how each example was made you can view there respective sources. There’s more information about the Yahoo! Maps AJAX API on the Yahoo! Developer Network website. If you have specific questions about the Y! Maps AJAX API we suggest you visit the yws-maps-ajax mailing list.

We’re eager to answer any questions you might have about these examples so start a thread anytime on the Pipes Message Boards.

The IBM developerWorks website has posted a feature article on Pipes that includes both an in-depth description of the project and a great tutorial. Here’s how they describe the article:

This tutorial provides a basic understanding of Yahoo Pipes, a service that filters, transforms, and aggregates content feeds. A complete run-through of the toolset available is included, as are three demonstrations that illustrate the capabilities and features of the service. Finally, step-by-step instructions for one of the demonstrations complete the tutorial, along with a recorded movie showing the steps in detail.

The intent of this tutorial is simply to describe the Yahoo Pipes service for managing available content feeds, focusing on the features, tools, and capabilities available.

This evening we pushed a number of updates to Pipes and there are some exciting new things to tell you about. In brief, we added new ways to browse and explore Pipes, updated the design of the Pipes website, added a number of new features, introduced two new modules, and fixed many bugs.

Many users have asked for additional ways to explore all the Pipes that are being created. When you are on the Browse page you now have ability to browse all the Pipes on that page by their Tags, and by the Sources and Modules contained in them. Look for the links to these options in the sidebar.

Here are a few examples:

• browse all published Pipes tagged with ‘Digg’
• browse all published Pipes containing the Source ‘digg.com’
• browse all published Pipes containing the Module ‘urlbuilder’

We have updated the look, and some functionality, of the Pipes website:

• On the Browse page all key actions for a Pipe (run, edit, delete, publish, clone) are now a single click away, the descriptions for each Pipe are now automatically exposed, and it’s possible to browse a Pipe by its unique tags and sources. To see these options hover over any of the listed Pipes.
• The sites navigation has been moved horizontally to the top of the page and the homepage has been revamped.
• We’ve incorporated user feedback and redesigned the runpage so that it’s easier to use.
• You can now add tags to your Pipe on its runpage, look for this feature in the sidebar.
  

We also introduced two new modules in the Pipes Editor:

• The String Replace Module lets you find and replace the first, last or all occurrences of a string. You can manually enter the text in the input boxes or pipe it in from an appropriate User Input module.
• The Sub String Module cuts a range of consecutive characters from a string. Enter the numeric character starting position and the length of the characters you want to match. Here’s an example using both new modules to get you started.

Finally, we’ve squashed many bugs that were reported on the Suggestions and Discussion Boards. Thank you for taking the time to report them.

As always, we look forward to hearing your Feedback and appreciate you helping us make Pipes a more useful product.

On Wednesday, June 6th, Pipes will be offline for approximately 2 hours starting at 6:30 PM PST. We’ll be performing a routine, scheduled update and expect regular service to return by around 8:30 PM PST. Click here to find out what time the site will be offline where you are located.

We thought it would be useful to showcase a few Pipes that use the new Geo features we announced.

• Apartment Near Something was one of the original examples we created to demonstrate the usefulness of Pipes. Now you can search for an apartment close to something and plot the output on a map!
• Kiva Loans by Location takes a listing of open microloans on Kiva.org and GeoCodes each item based on the nationality of the entrepreneur requesting a loan. The end result allows users to browse a interactive Yahoo! map to see where Kiva is currently fund-raising.
• Photos Near Napa Wineries annotates Yahoo! Local results for Napa Wineries with images from Flickr that were taken nearby.
• Socialight Flickrin’ takes location-based Sticky Notes from Socialight.com and searches for pictures from Flickr based on the content of the Sticky Note. This example demonstrates how to use the Location Extractor Module to ensure that a feed containing GeoData is properly marked up.

We can’t wait to see how you use the new Geo features in Pipes! As always, we hope you’ll share your feedback and suggestions with us.

The Pipes team is proud to announce two new features that we’ve been working on that enhance our support for GeoData.

• First, we added a interactive Yahoo! Map to the runpage of any Pipe containing GeoData. To see this feature in action run the Apartment Near Something Pipe. Here’s a still image of what the map looks like:
• Second, we added an additional output renderer which allows Pipes containing GeoData to be emitted as a KML file. If there’s GeoData in a Pipe you can find a link to the KML output at the bottom of a Pipes runpage listed in the Tools section. It’s possible to use the KML output in a variety of ways including in Google Earth.
  

We posted a followup entry that highlights some example Pipes that make use the new Geo features.

In addition to our enhanced geolocation support there are some other improvements to announce:

• You can now inline edit the Title and Description text listed on the runpage for your Pipes.
• We’ve added the ability to filter dates using the options “before” and “after” in the Filter Module. These parameters can accept dates in a variety of relative formats such as “n days ago”, “now” and “today”.

When we launched Pipes one of our featured examples let you combine multiple feeds on upcoming.org. A couple of weeks ago upcoming.org announced that they’re using this Pipe to power the same feature natively on their site!

To access to it, click on the link to your upcoming.org username and then click on the link you see below labeled ‘All My Stuff in RSS.’

For those of you that aren’t familiar with upcoming.org, it’s a great social events site that offers feeds for things like your subscribed events, your friends’ events, recent activity related to you and your friends, and general news happening on the upcoming.org site.

The Pipe we created aggregates all of those items into a single feed so that you can get all your upcoming.org goodness in one place.

We’ve always felt that, in addition to being a great tool for personal use, Pipes is a great way to quickly and easily enable more functionality in a web application.

We’re glad to see that the team at upcoming.org agree!

Thanks to everyone who has continued to provide the Pipes team with valuable suggestions. Today, we pushed a number of improvements that we want to tell you more about.

1) You can now use subfields in the Regex and Rename Modules. It’s also possible to refer to an item using dot syntax: ${published.timezone}.

If something in the preview pane is nested three levels deep, foo->bar->baz, you can refer to it as ${foo.bar.baz} in Regex replacements. Here’s an example to get you started.

2) Titles in a Pipe that contain quotes and/or angle brackets no longer get mangled

3) You can now use URLs that have urlencoded characters (like %22) in the Fetch Module.

4) Modules can now contain arbitrary text. In the past, fields that contained values that looked like tags (for example: <something>) were getting unexpectedly escaped or stripped.

5) Certain BBC feeds that were broken now work.

We pushed several new updates to Pipes based on the comments we’ve been receiving at the Pipes Suggestion Board. They include:

* A new Split Module, for creating duplicate copies of feeds.

* Significant bugfixes to both the Filter and Regex Modules, so that pattern matching will now work correctly with Chinese and Russian text, and Unicode in general.

* A significant bugfix to the Translate Module, it should no longer mangle encoding.

* The introduction of a new URL syntax across the site (note: these changes should be backwards compatible for Pipes that users have already subscribed to.)

* The list of sources associated with a Pipe now appears on the run page.

As always, we look forward to hearing what you think!