While every release of Pipes fixes many bugs, we don’t often talk about them. However the latest Pipes update addresses one of the biggest problems reported on the message boards and now Pipes supports a wider diversity language encodings and character sets found in many feeds. We still have a few remaining types to nail down but many more feeds should now work well.

Another frequently requested enhancement finally arrives on the Regex module, now our power users (you know who you are) can make use of the “m” “i” “g” and “s” flags.

We’ve also enabled run counting - yes, finally they’re back! We’ve re-counted all those hanging chads, errr, runs, and updated the statistics on each Pipe. Take a look at your Pipes and see how popular they have been.

Fed up with that avatar? You can now use any picture or yourself so other developers can see how pretty you really are. You can change your picture without even leaving your Pipes page:

On Monday, October 22nd, Pipes will be offline for approximately 1 1/2 hours starting at 9:00 PM PST. We’ll be performing a routine, scheduled update (lookout for some cool new stuff and fixes on Tuesday). We expect regular service to return by around 10:30 PM PST. Click here to find out what time the site will be offline where you are located.

An unexpected hiccup means that we won’t be having any schedule downtime tonight.

On Wednesday, October 17th, Pipes will be offline for approximately 1 1/2 hours starting at 9:00 PM PST. We’ll be performing a routine, scheduled update and expect regular service to return by around 10:30 PM PST. Click here to find out what time the site will be offline where you are located.

There are some exciting improvements in Pipes to announce including additional modules and improved documentation.

We’ve added a new module under Sources:

• The Fetch Site Feed module uses a web site’s auto-discovery links (link rel tags) to find an RSS, RDF or Atom feed. If multiple feeds are discovered, only the items in the first one discovered are fetched. This module is a simpler alternative to the Feed Auto-Discovery Module which returns feed URLs (not the content in them). Here’s an example that uses the Fetch Site Feed module to retrieve the first five items from a feed. The feed is auto-discovered, with the site’s URL provided by the user via a URL User Input module.

We’ve also added a new module under String:

• The String Tokenizer module splits a string into an array of strings. A delimiter string (often just a single character) tells the module where to split the input string. The delimiter string doesn’t appear in the output list. Here’s an example that connects a Text User Input module to the String Tokenizer to create an array of search terms. The term delimiter is a comma. These terms are passed into a Loop module containing a Yahoo! Search sub-module. The Search sub-module is site restricted to Wikipedia, and the loop is configured to only output the first result for each search. If you enter the string “frog,vanilla,pulp fiction” into the text user input, the resulting feed will have three items.

Finally, we’ve made some improvements to our Module Reference Documentation. Every Module in the Pipes Editor now has a enhanced description including details about what it does, the types of data it can input and output and links to examples.

The latest updates to Pipes includes a entity detection module and improvements to our existing keyword analysis capabilities.

There are 2 new modules under String:

• The Yahoo! Shortcuts module analyzes the input string for popular entities (for example: person, place, concept, organization, etc). If any entities are discovered additional information about each entity is returned (an example of the data returned could include geodata if the entity detected is a place). You can return the data either as a list of items or a string. Here’s an example.
  
• The Term Extractor module analyzes the content in the input string for significant words or phrases. If any terms are found it appends them as a sub-element under each item containing the results of the analysis. It’s functionally similar to the Content Analysis module but adds the ability to analyze any data element (not just title and description). You can return the data either as a list of items or a string. Here’s an example.
  

Some other updates worth noting:

• The Location Extractor module has been updated to support entity detection, and geocoding, using the Yahoo! Shortcuts API. If the title or description of your data source doesn’t contain one of the other popular geocode formats we look for now we’ll do keyword detection.can return the data either as a list of items or a string. Here’s an example.
• The Content Analysis module has been replaced by the Term Extractor module and can now be found under the list of deprecated modules. Deprecated modules will continue to work but we’re discouraging their use.

We’ve added a new Web Service module to Pipes that allows developers to power their own modules and extend the functionality Pipes offers.

In the Pipes Editor, the new Web Service module is listed under Operators:

• The Web Service module POSTs the items in a pipe in JSON format to an external web service. This allows developers to extend the functionality Pipes offers to do whatever they need. The original items are replaced by the web services JSON or RSS response. Here’s an example.
  

Note: We’ve documented the source code for the web service that we call in the example pipe.

We’ve made some major updates to Pipes. The big news is that we have 8 new modules to announce!

There are 2 new modules under Sources:

• The Feed Auto-Discovery module detects feed URLs that are embedded in a webpage using auto-discovery links (such as link rel tags). This example demonstrates how you could use the Feed Auto-Discovery module to find the RSS feed for a given webpage and then fetch its content.
  
• The Item Builder module allows you to create a new single item data source by defining a set of key-value attributes. This example demonstrates how you can use it to build up a new feed.
  

There are 4 new modules under Operators:

• The Loop module will run an enclosed sub-module or Pipe one time for each item in the input feed. Its output options can be set to ‘emit’ or ‘assign’ results. Emit results will replace the content of the original input item, here’s an example. Assign results will insert the content into a specified attribute, here’s an example. We introduced this module in order to support a wider range of modules, and data types, that can be looped over. You can use the Loop module with any module or Sub-Pipe except User Inputs, Operators, or Deprecated modules. Note: Old Skool Pipes users will notice that this module is similar to the For Each modules.
  
• The Reverse module reverses the order of the items contained in the input feed. This example demonstrates how it works.
  
• The Sub-element module will emit only the specified attribute or array for each item in the input feed. This example demonstrates how you could use it to take an RSS feed and emit only a specific item attribute for each item.
  
• The Tail module emits the last ‘n’ items from the input feed. This example demonstrates how you could use it to emit only the last 5 items in a feed.
  

There are 2 new modules under String:

• The Translate module automatically translates a string from one language into another. Select the language combination you desire from the pulldown menu. Note: Old Skool Pipes users will notice that this module is similar to the BabelFish module. Unlike the BabelFish module, the Translate module can be used within a Loop module and operate on any specified element attribute. This example demonstrates how you could use it to translate the Title and Description of a feed from one language into another.
  
• The String Regex module allows you to apply Perl-like regular expressions to attributes in a string. This example takes the delicious RSS feed and replaces the title for each item with its “dc:subject” element.
  

Some other updates worth noting:

• The String Concatenate module has been renamed the String Builder. It works exactly as before but if you’re new to it here’s an example that demonstrates how you can use it to build up strings!
• In a few cases several older modules have been Deprecated and replaced by what we consider to be improved modules. The For Each: Annotate and For Each: Replace modules have been replaced by the Loop module. And the BabelFish module has been replaced by the Translate module. The Deprecated modules will continue to work but we’re discouraging their use.
• In the Editor, we’ve updated how you can modify the properties for your Pipe. In the past, there was a button labeled Publish in its place there’s a new Properties button that brings up a dialog window that allows you to modify your Pipes name, description and tags. You now have the option to either make these changes so only you see them or to publish them along with your Pipe.
  
• And, as usual we also fixed miscellaneous bugs that have either been reported or that we stumbled upon. Thanks for continuing to share your suggestions!

Our new favorites feature makes it easier for you to share, use, and re-visit Pipes that you like. To favorite a Pipe click on its corresponding star icon.

There are a few ways to access favorites:

• You can view a list of your favorites by clicking on My Pipes. Next to the list of your Pipes you’ll notice the favorites tab.
  
• In the Pipes Editor you have access to your favorites in the Library pane.
  
• You can look at someone else’s favorite Pipes by clicking on their username and navigating to their list of favorites. When someone else looks at your favorites they will only see a list of all the Pipes in your collection that have been Published.

Here’s a list of some of our teams favorite Pipes.

Pipes now supports the ability to process and emit CSV and iCal data.

There’s a new module in the Pipes Editor called Fetch CSV that allows you to retrieve online data sources formatted in CSV (comma separated values). Checkout this example to see how you can start working with CSV data in Pipes.

We have also updated the Fetch Feed module so that it now supports the retrieval of iCal data. Here’s a simple Pipe that demonstrates how to start working with iCal data.

If a Pipe contains an iCal source, or contains the Fetch CSV Module, you’ll notice the appropriate links to “Get as iCal” and “Get as CSV” on the runpage.