Building a (re)search interface for Yahoo, Bing and Google with YQL

If you do a lot of research using web searches can be frustrating. Different search engines have different results, you need to open things in tabs and in general it can be pretty time consuming to find what you need.

To make this a bit easier I thought it’d be cool to have an interface that searches Yahoo, Google and Bing at the same time and thus I built GooHooBi:

As explained in the screen cast the thing under the hood of GooHooBi is YQL. Instead of fussing about with all the different search APIs all I did was to play with the YQL console and put together the following YQL statement:

select * from query.multi where queries='
  select Title,Description,Url,DisplayUrl 
    from microsoft.bing.web(20) where query="cat";
  select title,clickurl,abstract,dispurl 
    from search.web(20) where query="cat";
  select titleNoFormatting,url,content,visibleUrl 
    from google.search(20) where q="cat"
'

The query.multi table in YQL allows you to list a few queries which will be executed one after the other on the YQL server. The queries themselves I put together by using the different tables in the console and only selecting what I really need from each of them.

You can try this query in the YQL console and you can see the JSON output.

The rest is pretty easy. Cut this up into a parameterized string and do a cURL call:

$query = filter_input(INPUT_GET, 'search', FILTER_SANITIZE_SPECIAL_CHARS);

$queries[] = 'select Title,Description,Url,DisplayUrl '.
             'from microsoft.bing.web(20) where query="'.$query.'"';
$queries[] = 'select title,clickurl,abstract,dispurl '.
             'from search.web(20) where query = "'.$query.'"';
$queries[] = 'select titleNoFormatting,url,content,visibleUrl '.
             'from google.search(20) where q="'.$query.'"';
$url = "select * from query.multi where queries='".join($queries,';')."'";
$api = 'http://query.yahooapis.com/v1/public/yql?q='.
       urlencode($url).'&format=json&env=store'.
       '%3A%2F%2Fdatatables.org%2Falltableswithkeys&diagnostics=false';

$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, $api); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
$output = curl_exec($ch); 
curl_close($ch);
$data = json_decode($output);

Then loop over the results and assemble the HTML output.

You can check the source code of GooHooBi.

• The HTML/JavaScript version JS source JS live.
• The PHP version PHP source PHP live
• The PHP “API” API source API live

In addition to this, here’s a half hour live coding screencast how to build something similar:

Building a search mashup with YQL using Google, Yahoo and Bing – live :) from Christian Heilmann on Vimeo.

The source of the code built in this screencast is also on GitHub.

Tags: api, demo, Google, live, mashup, microsoft, yahoo, yql

This entry was posted on Wednesday, December 9th, 2009 at 4:13 pm and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.