Jalaj P. Jha Technical & Miscellaneous Ramblings

How to Create Private Yahoo Pipes?

Officially there are no Private Yahoo Pipes! All Pipes that you create can be seen and cloned by any other Yahoo Pipes user, if they know about it, that is they know its address or the pipe id associated with each pipe. So if you want to keep a Pipes created at Yahoo Pipes private you can do that by :

• Not Publishing the Pipe
  You can always use your pipe even if it is not published. Publishing a pipe just means that your pipe becomes the part of the pipes directory which all users can access. If a pipe is published there is every chance that it is discovered by others by search string, modules used in it etc. Using your pipe without publishing keeps you safe on this front.
• Embedding RSS feeds online taking care that the Pipes ID is not revealed
  If you are embedding RSS feeds created by you Yahoo Pipes, make sure that the pipe id is not revealed as a link or in the source code of the page. For example if you embed RSS feed in RSS module available in Blogger your pipe id doesn’t show up anywhere but on a wordpress.com blog an RSS link reveals the pipe url (and thus id) by which anybody can clone the pipe. Take care when you do that and check source code of generated page too.
• Not embedding or linking or using the pipe on other unsecured pipes.
  Pipes can be embedded in other pipes or RSS feeds from pipes retrieved in other pipes. Don’t do with the pipe that you want to keep secure. If the pipes that linked to it or embedded it goes into others’ hands your private pipe is exposed. So if  a pipe uses your private pipe, it to needs to be kept with same security as with this one.
• Not sharing it with anybody else.
  Your pipes is just for you. If you want to keep the source code of pipe private but want to share its functionality with others, you have no chance. As soon as you share your pipe with someone you have exposed the source.

Thanks for patiently reading to this point. Now we start with the unofficial way to create a Private Yahoo Pipe. Yes there is a way!

All we need to do is to create one extra pipe for each pipe that you need to publish or share or embed. We would henceforth call this extra pipe as PipeShield. The PipeShield itself can be published or shared or embedded.

Let’s say I want to secure my Google Trends Scraper pipe, which takes one user input and generates output based on it.

We will create a new yahoo pipe, but before that run the above yahoo pipe and get the RSS output url as given in the link ‘Get as RSS’

Create a new Yahoo Pipe and insert a ‘URL Builder’ module. Paste the url that you got from the previous step into the textbox provided alongside label ‘Base:’. As soon as you press ‘Tab’ button all query parameters present in the url will switch their position.

Insert a ‘Fetch Feed’ module and make it to take url from the url builder module and link to output module.

Now since the pipe that we are shielding here took a user input ‘Date’ so we will need to do same here to. So, add a ‘Text Input’ field and wire it to date field in ‘URL Builder’

Now we have a functional pipe that forms a wrapper around original pipe, but the original pipe is still not secure as its pipe id is visible, in the parameter ‘_id’ of the URL Builder. Here is the final step.

Add a ‘Private Text Input’ module, fill its default value from the value in _id field, check the ‘Private’ checkbox, clear the _id field and make it to take value from Private Text Input module.

Save the pipe and distribute it to all you know and those you don’t without fear of exposing the original pipe and the functions/secrets it carried. This pipeshield will show the value in the ‘Private Text Input’ only to you. Everyone else will see the field blank. Others will not be able to run the pipeshield in Edit mode and thus cannot debug the pipe id, they cannot clone the pipeshield otherwise its link with the underlying pipe will break and the pipeshield will become non-functional. Others can just run the pipe and get the results! Once you have distributed your pipeshield you can rest assured that your original pipe is always.

You can check this pipe in edit mode here. Watch my page on Yahoo Pipes for more pipes. All posts on Yahoo Pipes on this blog will share this archive url