Let's build a Yahoo! Pipe. I'll show you each step and you'll learn the basics of the Pipes Editor along the way.
Modules are the building blocks of Yahoo! Pipes. In our first example, we'll select a news feed, then use Yahoo Pipes to filter the feed. Click on Sources in the Yahoo! Pipes taskbar. You'll see a list of Source modules. Select the Fetch module from the taskbar by clicking and holding down the left mouse button. Drag the module onto the canvas and release the mouse button. The module expands on the canvas to reveal its attributes.
Fetch has several features common to all modules. You can position Fetch on the canvas by clicking and holding its title bar and dragging it around the canvas. The title bar has three buttons. The first displays help information in the help area (the lower-left pane of the Editor). The middle button collapses the Fetch module so it takes up less real estate on the canvas. The final button deletes the module from the canvas.
Inside the Fetch window is a text entry field labeled url. Here is where we'll enter the URL for our RSS feed (see Figure 2).
If there are thousands of feeds out there, how do you find them? An example may be helpful.
Reuters is a news service that got its start by savvy use of emerging technology. The year was 1851, and the technology was the telegraph. Reuters was one of the first news services to understand how this new technology would transform the world. Reuters made a reputation with several news scoops—they were the first to report U.S. President Abraham Lincoln's assassination in 1865.
It's no surprise that today Reuters has a web presence. Point your browser at www.reuters.com to get to the Reuters home page. Scroll down a bit and look in the left column (see Figure 3).
You'll see a link labeled "RSS", accompanied by a small orange and white icon. On whatever site you're on, look for a link or icon similar to this. For RSS feeds the icon may look like the one above, or it may be an orange rectangle with white letters that say "RSS" or "XML".
In our Reuters example, the link takes us to a page that offers many feeds on a variety of topics: Top News, International, Sports, Technology, and more. We'll use the one called "Oddly Enough". We'll need the URL for this feed—if you right-click on the link, you can select "Copy Link Location" (see Figure 4).
This saves the link URL[1] to your clipboard.
Now we have a URL to enter into our Fetch module. We enter it into the url text entry field.
How do we know if the URL is correct? We use the Debugger pane, below the Canvas. You can expand the Debugger pane by dragging the little arrowed "handle" on the divider between the Canvas and Debugger panes. Click on the Fetch module to make sure it's the active module. Its title bar will turn orange.
Now click on the "Refresh" link in the Debugger. Wait a few seconds and you should see a list of news headlines in the debugger (see Figure 5).
The Debugger pane can display the output from any module on the canvas. Just click on the module, then click on the Debugger's Refresh link. With the Debugger, you can check your Pipe's correctness at each step along the pipeline. This becomes more essential as you build more and more complex Pipes.
The Debugger lists output in an expandable tree format. Click on an item in the Debugger and its contents expand below it (see Figure 6).
Back to our example. Click on Operators in the Modules taskbar. Select the Filter module and drag it to the canvas, dropping it below the Fetch module. If you need to, move down the Pipe Output module so it's positioned below the Filter module (see Figure 7).
By now you may have noticed a small circle centered at the bottom of the Fetch module. A similar circle appears at both the top and bottom of the Filter module. And a fourth circle can be seen on the top of the "Pipe Output" module; it magically appeared when you first dropped Fetch onto the canvas.
These circles are connection points—they let you connect one module to another. You wire together the output (bottom) connector of one module to the input (top) connector of another module.
Click the bottom connector of the Fetch module. Hold the mouse button down and drag toward the top connector of the Filter module (see Figure 8).
Filter's top connector becomes solid blue and glows orange. Release the mouse button and the two modules are now piped together. The output from the Fetch module—that data you saw in the Debugger—now flows into the Filter module.
As its name implies, the Filter module restricts the data passing through it. You define a set of filter rules that tell the module what's allowed to pass through it. The sentence at the top of the Filter defines how these rules are applied:
"Block/Permit items that match all/any of the following"
In our example, we want to find all news stories that contain the word "tour" in their description. First we select Permit from the Block/Permit list box.
Next, we need to build a rule to match the word "tour". We select "description" from the first rule line (see Figure 9).
This is the RSS news feed field this rule applies to. The second list box describes the type of match we want. The default, "Contains", sounds right, so we'll leave it alone. Finally, we type the word "tour" into the text box.
Does our filter work? Click on Refresh in the Debugger to find out (see Figure 10).
Now the news feed has been reduced to just two items, and we can see that "tour" features prominently in both headlines. Success!
But we're not quite done yet. The last step of any Yahoo! Pipes pipeline must be the Pipe Output module. It magically appears on any new Pipes project as soon as you drop your first module on the canvas. To have a complete working Yahoo! Pipe, we need to connect the Filter module to the Pipe Output module. Just click on the Filter's output connector and drag toward the Pipe Output's input connector.
Before you can use your Pipe, you need to save it. In the upper left corner of the Pipes Editor is a tab that says "untitled". Click on this and it becomes a text entry field. Give your Pipe a name. I'm calling mine "Reuters Odd Tours". Click the OK button.
Finally, click on the Save button near the top right of the Pipes Editor. You'll see a brief "Saving..." message in the editor, and when it disappears, you're done. You now have your first Yahoo! Pipe (see Figure 11).
