Archive for August, 2007

New Modules and Improved Docs

Friday, August 31st, 2007

There are some exciting improvements in Pipes to announce including additional modules and improved documentation.

We’ve added a new module under Sources:

  • The Fetch Site Feed module uses a web site’s auto-discovery links (link rel tags) to find an RSS, RDF or Atom feed. If multiple feeds are discovered, only the items in the first one discovered are fetched. This module is a simpler alternative to the Feed Auto-Discovery Module which returns feed URLs (not the content in them). Here’s an example that uses the Fetch Site Feed module to retrieve the first five items from a feed. The feed is auto-discovered, with the site’s URL provided by the user via a URL User Input module.

We’ve also added a new module under String:

  • The String Tokenizer module splits a string into an array of strings. A delimiter string (often just a single character) tells the module where to split the input string. The delimiter string doesn’t appear in the output list. Here’s an example that connects a Text User Input module to the String Tokenizer to create an array of search terms. The term delimiter is a comma. These terms are passed into a Loop module containing a Yahoo! Search sub-module. The Search sub-module is site restricted to Wikipedia, and the loop is configured to only output the first result for each search. If you enter the string “frog,vanilla,pulp fiction” into the text user input, the resulting feed will have three items.

Finally, we’ve made some improvements to our Module Reference Documentation. Every Module in the Pipes Editor now has a enhanced description including details about what it does, the types of data it can input and output and links to examples.

Pipes Adds Entity Detection

Monday, August 13th, 2007

The latest updates to Pipes includes a entity detection module and improvements to our existing keyword analysis capabilities.

There are 2 new modules under String:

  • The Yahoo! Shortcuts module analyzes the input string for popular entities (for example: person, place, concept, organization, etc). If any entities are discovered additional information about each entity is returned (an example of the data returned could include geodata if the entity detected is a place). You can return the data either as a list of items or a string. Here’s an example.
    Yahoo! Shortcuts Module
  • The Term Extractor module analyzes the content in the input string for significant words or phrases. If any terms are found it appends them as a sub-element under each item containing the results of the analysis. It’s functionally similar to the Content Analysis module but adds the ability to analyze any data element (not just title and description). You can return the data either as a list of items or a string. Here’s an example.
    Term Extractor Module

Some other updates worth noting:

  • The Location Extractor module has been updated to support entity detection, and geocoding, using the Yahoo! Shortcuts API. If the title or description of your data source doesn’t contain one of the other popular geocode formats we look for now we’ll do keyword detection.can return the data either as a list of items or a string. Here’s an example.
  • The Content Analysis module has been replaced by the Term Extractor module and can now be found under the list of deprecated modules. Deprecated modules will continue to work but we’re discouraging their use.

Power Your Own Module

Monday, August 13th, 2007

We’ve added a new Web Service module to Pipes that allows developers to power their own modules and extend the functionality Pipes offers.

In the Pipes Editor, the new Web Service module is listed under Operators:

  • The Web Service module POSTs the items in a pipe in JSON format to an external web service. This allows developers to extend the functionality Pipes offers to do whatever they need. The original items are replaced by the web services JSON or RSS response. Here’s an example.
    Web Service Module

Note: We’ve documented the source code for the web service that we call in the example pipe.

Pipes Blog is powered by WordPress | Entries (RSS) and Comments (RSS).