Posts Tagged ‘javascript’

Creating accessible JavaScript menu systems – some basic steps that get forgotten

Tuesday, June 23rd, 2009

OK, so you want to write yet another collapsing menu/section/widget thingamabob. This is really easy and in the following few examples I want to point out some ideas that make it much easier for you.

The following is what we’re going to build – it doesn’t have any bells and whistles like smooth animation, random unicorns, swoosh noises or whatever else you might want to come up with, but it is a good solid base to work from.

Collapsing menu example by  you.

Click the image to go to the live example.

The Markup

The first trick is to use easy to understand and highly styleable markup for your widget. You see a lot of widgets that use endless DIVs, SPANs, IDs on every element and classes on every sub-element. If you build a simple solution (and not a catch-all reusable widget that is part of a framework) none of that is needed. In this case the markup is the following:


<ul id="menu">
  <li><h3>Section 1</h3>
    <ul>
      <li><a href="foo.html">Foo</a></li>
      <li><a href="bar.html">Bar</a></li>
      <li><a href="baz.html">Baz</a></li>
    </ul>
  </li>
  <li><h3>Section 2</h3>
    <ul>
      <li><a href="foo.html">Foo-1</a></li>
      <li><a href="bar.html">Bar-1</a></li>
      <li><a href="baz.html">Baz-1</a></li>
    </ul>
  </li>
  <li><h3>Section 3</h3>
    <ul>
      <li><a href="foo.html">Foo-2</a></li>
      <li><a href="bar.html">Bar-2</a></li>
      <li><a href="baz.html">Baz-2</a></li>
    </ul>
  </li>
  <li><h3>Section 4</h3>
    <ul>
      <li><a href="foo.html">Foo-3</a></li>
      <li><a href="bar.html">Bar-3</a></li>
      <li><a href="baz.html">Baz-3</a></li>
    </ul>
  </li>
</ul>
In other words: a nested list with headings for the different sections. This makes sense without styling or scripting which is still and important point considering mobile devices or environments that have JavaScript turned off (and yes, they are not mythical but do exist).

Assistive access does not mean lack of JavaScript

The first mistake that people do is to think that this is all you need to be accessible – as users with assistive technology have no JavaScript, right? Wrong. Assistive technology is not a browser replacement but in most cases hooks into browser functionality and offers easier access. In other words, what the browser gets the assistive technology gets and you have to make sure it still makes sense without a mouse (to name just one example).

Thinking too complex

The next extra step people take is starting to loop through the menu construct to hide the elements with JavaScript. This is made much easier with clever libraries that don’t use the DOM to do that but piggy-back on the CSS selector engine, but for good old IE6 this is still not an option. And you don’t need to. In order to hide the nested lists in the above example all you need to do is to apply a class to the main list and leave the rest to CSS:


<style type="text/css" media="screen">
#menu.js ul{
  position:absolute;
  left:-9999px;
}
</style>
<script type="text/javascript" charset="utf-8">
var m = document.getElementById('menu');
m.className = 'js';
</script>  

This means that in order to show any of the nested lists, all you need to to is to add a class called “open” to the parent list item. The CSS to undo the hiding is the following:


<style type="text/css" media="screen">
#menu.js li.open ul{
  position:relative;
  left:0;
}  
</style>

You can add the classes on the backend when you render the page – which makes a lot more sense to indicate a current open section of the page than any JavaScript analyzing of window.location or other shenanigans. See the first stage demo. You can also define a “current” class which means you can style it differently and the script will simply quietly skip that section and not collapse it at all.

That is the beauty in leaving all the showing and hiding to the CSS. Of course you can’t animate in CSS (unless you use webkit), but you can make your animation precede the change in classes.

Hiding and showing

So the way to show and hide things is simply to add and remove classes. We need to use event handling to listen for click events to do that. Click events are the only safe ones as they fire with keyboard and mouse. The problem is that a header can be made clickable, but is not keyboard accessible. To work around this, let’s use DOM scripting to inject a button element into the headers. You can use CSS to make it look not like a button:


<style type="text/css">
h3 button{
  border:none;
  background:transparent;
}
</style>

The second stage demo shows how this works and here’s the code.


(function(){

  // get the menu and make sure it exists
  var m = document.getElementById('menu');
  if(m){

    // add a class to allow the CSS magic to happen
    m.className = 'js';

    // loop over all headers
    var headers = m.getElementsByTagName('h3');
    for(var i=0;headers[i];i++){
      // get the content of the current header
      var content = headers[i].innerHTML;
      // create a button, delete the content of the header
      // replace it with the button and set the content to 
      // the cached header content
      var a = document.createElement('button');
      headers[i].innerHTML = '';
      headers[i].appendChild(a);
      a.innerHTML = content;
    }

    // apply a single click event to the menu
    m.addEventListener('click',function(e){

      // find the event target and chec that it was a button
      var t = e.target;
      if(t.nodeName.toLowerCase()==='button'){

        // get the LI the button is in 
        var mom = t.parentNode.parentNode;

        // check if its class name is not content and 
        // remove the open class if it exists, else 
        // add an open class

        if(mom.className!=='current'){
          if(mom.className === 'open'){
            mom.className = '';
          } else {
            mom.className = 'open';
          }
        }

        // don't do the normal things buttons do
        e.preventDefault();
      }
    },true);
  }
})()

This code also takes advantage of event delegation – there is so no need to apply and event handler to each heading – even if that is really easy to do with a library and an each() or batch() command. Event handling is a great concept and the tricks you find when you bother to read the docs are staggering.

Hiding is not removing

This is where most menu scripts stop. Great stuff, it expands and collapses and everybody is happy. Unless you are an unhappy chap and you need to use a keyboard to access it – or an older mobile device. Try it out yourself – use the second example by tabbing from link to link. You’ll find that you have to tab through the invisible links to reach the next section to open or hide. That surely can’t be the idea, right?

The trick to work around is to change the tabindex property of an element. A value of -1 will remove it from the tabbing order and setting it back to 0 will make them available again. So, to make keyboard access easier, let’s remove the tab order upfront and re-set or remove it when we show and hide the menus. This is terribly annoying as we have to loop through the links. I hate loops, but browsers are not our friends.


(function(){

  // get the menu and make sure it exists
  var m = document.getElementById('menu');
  if(m){

    // add a class to allow the CSS magic to happen
    m.className = 'js';

    // loop over all headers
    var headers = m.getElementsByTagName('h3');
    for(var i=0;headers[i];i++){
      // get the content of the current header
      var content = headers[i].innerHTML;
      // create a button, delete the content of the header
      // replace it with the button and set the content to 
      // the cached header content
      var a = document.createElement('button');
      headers[i].innerHTML = '';
      headers[i].appendChild(a);
      a.innerHTML = content;
    }
    
    // loop over all nested lists 
    // and set the taborder of the nested links to -1 to 
    // remove them from the tab order

    var uls = m.getElementsByTagName('ul');
    for(var i=0;uls[i];i++){
      if(uls[i].parentNode.className!=='open' &&
         uls[i].parentNode.className!=='current'){
         var as = uls[i].getElementsByTagName('a');
         tabOrder(as,-1);
      }
    }

    // apply a single click event to the menu
    m.addEventListener('click',function(e){

      // find the event target and chec that it was a button
      var t = e.target;
      if(t.nodeName.toLowerCase()==='button'){

        // get the LI the button is in 
        var mom = t.parentNode.parentNode;

        // check if its class name is not content and 
        // remove the open class if it exists, else 
        // add an open class. Also, remove or add the 
        // links to the tab order

        if(mom.className!=='current'){
          if(mom.className === 'open'){
            mom.className = '';
            tabOrder(as,-1);
          } else {
            mom.className = 'open';
            tabOrder(as,-1);
          }
        }

        // don't do the normal things buttons do
        e.preventDefault();
      }
    },true);
  }  
  
  // remove from or add elements to the tab order
  
  function tabOrder(elms,index){
    for(var i=0;elms[i];i++){
      elms[i].tabIndex = index;
    }
  }
})()

The third demo does exactly that – use your keyboard and marvel at being able to tab from parent element to parent element when sub-sections are collapsed.

Fixing it for Internet Explorer 6 and 7bad old browsers

Now, this is all good and fine but of course we need to fix the code to work around the quirks of browsers that don’t understand the W3C event model or consider an element with a name attribute the same as one with and id. We could fork and fix in the code we have here, but frankly I am tired of this. People give us libraries to work around browser differences and to make our lives easier. This is why we can use YUI for example to make this work reliably cross-browser:


<script type="text/javascript" charset="utf-8">
(function(){
  var config = {
    'id':'menu',        // id of the menu
    'jsenabled':'js',   // class to add to apply design/functionality
    'show':'open',      // show the menu
    'current':'current' // current item (will not be collapsed)
  };
  var YE = YAHOO.util.Event, YD = YAHOO.util.Dom, YS = YAHOO.util.Selector;
  var m = YD.get(config.id);
  if(m){
    YD.addClass(m,config.jsenabled);
    tabOrder(YS.query('#'+config.id+' ul a'),-1);
   
    // set keyboard access for links in open sections - thanks David Lantner
    tabOrder(YS.query('#'+config.id+' li.'+config.current+' a'),0);
    tabOrder(YS.query('#'+config.id+' li.'+config.show+' a'),0);

    YD.batch(YS.query('#'+config.id+' h3'),function(o){
      var c = o.innerHTML;
      var b = document.createElement('button');
      o.innerHTML = '';
      o.appendChild(b);
      b.innerHTML = c;
    });
    YE.on(m,'click',function(e){
      var t = YE.getTarget(e);
      if(t.nodeName.toLowerCase()==='button'){
        var mom = YD.getAncestorByTagName(t,'li');
        if(!YD.hasClass(mom,config.current)){
          var as = YS.query('ul a',mom);
          if(YD.hasClass(mom,config.show)){
            YD.removeClass(mom,config.show);
            tabOrder(as,-1);
          } else {
            YD.addClass(mom,config.show);
            tabOrder(as,0);
          }
        }
      }
    },true);
  }
  function tabOrder(elms,index){
    for(var i=0;elms[i];i++){
      elms[i].tabIndex = index;
    }
  }
})();  
</script>

Good start

This is just a start to make this work. A real menu should also have support for all kind of keys that a menu like this in a real app gives us and notify screen readers of all the happenings. For this, we need ARIA. Right now, I’d be happy to get some comments here :)

Newsmap – using Placemaker to add geo location to a news feed

Friday, May 22nd, 2009

I am right now very excited about the new Placemaker beta – a location extraction web service released at Where2.0. Using Placemaker you can find all the geographical locations in a feed or a text or a web url and you get them back as an array of places.

As a demo I took the Yahoo News feed and ran it through Placemaker. The resulting places are plotted on a map and the map moves from location to location when you hover over the news items.

The result is online at http://isithackday.com/hacks/placemaker/map.php

Yahoo News Map by  you.

Getting the data from the data feed and running it through placemaker is very straight forward. I explained the basic principle in this blog post on the Yahoo Developer Network blog. The only thing to think about is to define the input and output types correctly:


<?php 
$key = 'PASTE YOUR API KEY HERE';
$apiendpoint = 'http://wherein.yahooapis.com/v1/document';
$url = 'http://rss.news.yahoo.com/rss/topstories';
$inputType = 'text/rss';
$outputType = 'rss';
$post = 'appid='.$key.'&documentURL='.$url.
                '&documentType='.$inputType.'&outputType='.$outputType;
$ch = curl_init($apiendpoint);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);  
$results = curl_exec($ch);
?>

If you look at the source of this example you will find that Placemaker injected contentlocation elements in the feed itself:


<cl:contentlocation 
xmlns:georss="http://www.georss.org/georss" xmlns:cl="http://wherein.yahooapis.com/v1/cle"
xmlns:xml="http://www.w3.org/XML/1998/namespace" xml:lang="en">
  <cl:place>
    <cl:woeId>2514815</cl:woeId>
    <cl:name><![CDATA[Washington, DC, US]]></cl:name>
    <cl:latitude>38.8913</cl:latitude>
    <cl:longitude>-77.0337</cl:longitude>
  </cl:place>
  <cl:place>
    <cl:woeId>23424793</cl:woeId>
    <cl:name><![CDATA[Cuba]]></cl:name>
    <cl:latitude>21.511</cl:latitude>
    <cl:longitude>-77.8068</cl:longitude>
  </cl:place>
  <cl:place>
    <cl:woeId>23424977</cl:woeId>
    <cl:name><![CDATA[United States]]></cl:name>
    <cl:latitude>48.8907</cl:latitude>
    <cl:longitude>-116.982</cl:longitude>
  </cl:place>
  <cl:place>
    <cl:woeId>55843872</cl:woeId>
    <cl:name><![CDATA[Guantanamo Bay, Caimanera, 
    Guantanamo, CU]]></cl:name>
    <cl:latitude>19.9445</cl:latitude>
    <cl:longitude>-75.1541</cl:longitude>
  </cl:place>
</cl:contentlocation>

You’ll also notice that the elements are namespaced and the names of the locations in CDATA blocks, both things I hate with a passion. Not because they don’t make sense, but because simplexml can be drag to make understand them.

What I wanted to do with this data was twofold: create a JSON array of geo locations to plot on a map and a display of the news content. This is the PHP that does that:


$places = simplexml_load_string($results, 'SimpleXMLElement', 
                                LIBXML_NOCDATA);  
// if there are elements found
if($places->channel->item){
  // start a JSON array
  $output .= '[';
  // start the HTML output
  $html = '<ul id="news">';
  // set the counter - this will be needed to link news 
  // items and map markers
  $count = 0;
  // loop over RSS items
  foreach($places->channel->item as $p){
    // set inner counter (as there are more locations per news item)
    $innercount = 0;
    // start the HTML list item and give it an ID with the counter
    // value
    $html .=  '<li id="news'.$count.'"';
    // all child elements with the defined namespace
    $locs = $p->children('http://wherein.yahooapis.com/v1/cle');
    // check that there is a location sub-element in this item
    if($locs->contentlocation){
      // if there is one, add a class to the LI
      $html .= ' class="haslocation"';
      // start an array for displaying of the locations under the
      // news items
      $dlocs = array();
      // loop over all the places found for this item
      foreach($locs->contentlocation->place as $pl){
        // append a new JS object with the location data
        // and a unique ID to the locations array 
        $locations[] = '{"name":"'.$pl->name.'","title":"'.
                        preg_replace('/\n+/','',addslashes($p->title)).
                       '","lat":"'.$pl->latitude.
                       '","lon":"'.$pl->longitude.'","id":"m'.
                       $count.'x'.$innercount.'"}';
        // add the location name to the display locations array
        $dlocs[] = $pl->name;                
        // increase the inner count to ensure that every marker has 
        // a unique ID
        $innercount++;
      }
    }
    // append the HTML for the news item
    $html.='><h2><a href="'.$p->link.'">'.$p->title.'</a></h2><p>'.
            $p->description.'</p>';
    // if locations were found, add them 
    if(sizeof($dlocs)>0){
      $html.='<p class="locations">Locations: '.join(',',$dlocs).'</p>';
    }
    // end the list item
    $html.='</li>';
    // increase the counter
    $count++;
  }
  // join the json object data with a comma and close the JSON array
  $output .= join(',',$locations);
  $output .= ']';
// if there are no items simply return nothing
} else {
  $output = '';
}
// and this ends the HTML
$html.= '</ul>';
?>

The result of this can be seen here http://isithackday.com/hacks/placemaker/map-2.php.

The JavaScript to show the map is pretty straight forward and more or less the demo example of the maps API:


// will be called with the array assembled in PHP
function placeonmap(o){
  // if there are locations
  if(o.length > 0){
    // create a new geopoints array to hold all locations
    // this is needed to determine the original zoom 
    // level of the map
    var geopoints = [];
    // add map with controls
    var map = new YMap(document.getElementById('map')); 
    map.addZoomLong();
    map.addPanControl();
    // loop over locations
    for(var i=0;i<o.length;i++){
      // define a new geopoint and store it in the array
      var point = new YGeoPoint(o[i].lat,o[i].lon);
      geopoints.push(point);
      // create a new marker and give it the unique
      // id defined in the PHP. Pop up the title of 
      // the news item and the name of the location when the
      // user hovers over the marker
      var newMarker = new YMarker(point,o[i].id);
      newMarker.addAutoExpand(o[i].title + '('+o[i].name+')');
      map.addOverlay(newMarker);
    }
    // define best zoom level and show map
    var zac = map.getBestZoomAndCenter(geopoints);        
    map.drawZoomAndCenter(zac.YGeoPoint,zac.zoomLevel);
  }
  // add a mouseover handler to the list of results
  YAHOO.util.Event.on('news','mouseover',function(e){
    // remove the "news" text of the ID of the current target
    // as we named the list items news0 to news19
    var id = YAHOO.util.Event.getTarget(e).id.replace('news','');
    // if there is still something left we have one of the news
    // items
    if(id!==''){
      // get the first marker with the ID we defined in the loop.
      var marker = map.getMarkerObject('m'+id+'x0');
      // if there is one, pan the map there and show the message
      // attached to it.
      if(marker){
        map.panToLatLon(marker.YGeoPoint);
        marker.openAutoExpand();
      }
    }
  });
}
// call placeonmap with the JSON array
placeonmap(<?php echo $output;?>);

That’s pretty much it. I am sure it can be refined, but it is amazing how easy it is to get geo information into any text with Placemaker.

TTMMHTM: Events, Latte Art, Full Frontal and Game developers vs. porn stars

Sunday, April 26th, 2009

Things that made me happy this morning:

  • Getting upgraded to business class on my flight back from Hong Kong as BA didn’t have a vegetarian meal for me! Flatbed win with 8 hours straight sleep.
  • Celebrating my birthday with my friends and getting a giant duck: Giant Duck
  • PPK’s slides on JavaScript Events from his presentation at Yahoo
  • Splendid Cappucino Latte Art
  • Meeting colleagues from long long ago randomly in Australia and them blogging about your talk
  • A great presentation by filament group on Access oriented web design
  • Vimeo doing a very nice custom Flash effect for the Let it Shine ad.
  • Sand/Stone is an interesting idea, “a 6,000km-long wall of artificially solidified sandstone architecture that would span the Sahara Desert, east to west, offering a combination of refugee housing and a “green wall” against the future spread of the desert”
  • The Full Frontal conference is now live, come to Brighton in November to get your JavaScript fix.
  • A good comparison of Game Developers and Porn Stars

Is it getting harder and harder to show very easy examples?

Tuesday, April 7th, 2009

I am right now teaching a four day class of DOM and Ajax in Sunnyvale, California and also do some tech editing for Scriptin with JavaScript and Ajax by Charles Wyke-Smith and I find one thing that is pretty worrying: easy examples of web development practices are dangerous to show these days.

I’m talking about practices that make it easy to get quick results and give readers and attendees “I am getting this – this is easy” fuzzy warm feelings.

One very obvious example is form validation and re-rendering of a form using PHP_SELF and displaying user data using $_GET or $_POST. Unfiltered they are a free invitation for any XSS attack and will turn your server into a spam-hub or bot-net drone. Explaining countermeasures of XSS normally is out of scope for an example that only shows how a form would work that you enhance progressively.

The same applies to simply outdated ideas like onevent handlers. It is easy to show an example that uses a few onclick handlers, but explaining event handling really well takes a bit of time. Again, this is something that really does not fit in the scope of a DOM course.

I do however think that it is important to get it in there, as there is no such thing as knowing one technology in the web development stack and being able to use it. There’s a lot of overlap with other areas and in order to be a good developer and play well with others you need to be aware of your effects and areas of overlap with your colleagues’ skill-sets.

The other extreme I find myself doing is being too over-cautious. I went through the tough times of the first browser wars and got a deep-rooted mistrust towards anything some browser tells me is OK to do and use. However, I get the feeling that it doesn’t really matter any more if Internet Explorer has a problem with name vs. ID or whatever other shenanigans we have to be aware of when we build things from scratch.

I do get the distinct feeling that not building on top of a good client-side library is simply a waste of time these days. Libraries allow us to write code, not to work around bugs and wonder what other safety measure we have to put in.

That’s why I started asking people in my courses to use Firefox with Firebug and use a good text editor to code along. Today I managed to breeze through how to write HTML that is ready for internationalisation and works with assistive technology, over simple DOM access to the document and at the end writing a validation script for a form using generated DOM content. By concentrating on how things are meant to work instead of debugging random issues I managed to get the students to reach far into the matter in a day – even those who never touched JavaScript before.

Maybe it is time to get beginners accustomed to a market that builds on working solutions and benefits from browser abstraction via libraries than teaching developing from total scratch – bad browsers and bad people taking advantage of any technology to gain access or spam us seem to have made this way of working redundant.

Building a hack using YQL, Flickr and the web – step by step

Wednesday, March 11th, 2009

As you probably know, I am spending a lot of time speaking and mentoring at hack days for Yahoo. I go to open hack days, university hack days and even organized my own hackday revolving around accessibility last year.

One of the main questions I get is about technologies to use. People are happy to find content on the web, but getting it and mixing it with other sources is still a bit of an enigma.

Following I will go through a hack I prepared at the Georgia Tech University hack day. I am using PHP to retrieve information of the web, YQL to filter it to what I need and YUI to do the CSS layout and add extra functionality.

The main ingredient of a good hack – the idea

I give a lot of presentations and every time I do people ask me where I get the pictures I use from. The answer is Flickr and some other resources on the internet. The next question is how much time I spend finding them and that made me think about building a small tool to make this easier for me.

This is how Slidefodder started and following is a screenshot of the hack in action. If you want to play with it, you can download the Slidefodder source code.

Slide Fodder - find CC licensed photos and funpics for your slides

Step 1: retrieving the data

The next thing I could have done is deep-dive into the Flick API to get photos that I am allowed to use. Instead I am happy to say that using YQL gives you a wonderful shortcut to do this without brooding over documentation for hours on end.

Using YQL I can get photos from flickr with the right license and easily display them. The YQL statement to search photos with the correct license is the following:


select id from flickr.photos.search(10) where text='donkey' and license=4

Retrieving CC licensed photos from flickr in YQL

You can try the flickr YQL query here and you’ll see that the result (once you’ve chosen JSON as the output format) is a JSON object with photo results:


{
 "query": {
  "count": "10",
  "created": "2009-03-11T01:23:00Z",
  "lang": "en-US",
  "updated": "2009-03-11T01:23:00Z",
  "uri": "http://query.yahooapis.com/v1/yql?q=select+*+from+flickr.photos.search%2810%29+where+text%3D%27donkey%27+and+license%3D4",
  "diagnostics": {
   "publiclyCallable": "true",
   "url": {
    "execution-time": "375",
    "content": "http://api.flickr.com/services/rest/?method=flickr.photos.search&text=donkey&license=4&page=1&per_page=10"
   },
   "user-time": "376",
   "service-time": "375",
   "build-version": "911"
  },
  "results": {
   "photo": [
    {
     "farm": "4",
     "id": "3324618478",
     "isfamily": "0",
     "isfriend": "0",
     "ispublic": "1",
     "owner": "25596604@N04",
     "secret": "20babbca36",
     "server": "3601",
     "title": "donkey image"
    }
    [...]
   ]
  }
 }
}

The problem with this is that the user name is not provided anywhere, just their Flickr ID. As I wanted to get the user name, too, I needed to nest a YQL query for that:

select farm,id,secret,server,owner.username,owner.nsid from flickr.photos.info where photo_id in (select id from flickr.photos.search(10) where text='donkey' and license=4)

This gives me only the information I really need (try the nested flickr query here):


{
 "query": {
  "count": "10",
  "created": "2009-03-11T01:24:45Z",
  "lang": "en-US",
  "updated": "2009-03-11T01:24:45Z",
  "uri": "http://query.yahooapis.com/v1/yql?q=select+farm%2Cid%2Csecret%2Cserver%2Cowner.username%2Cowner.nsid+from+flickr.photos.info+where+photo_id+in+%28select+id+from+flickr.photos.search%2810%29+where+text%3D%27donkey%27+and+license%3D4%29",
  "diagnostics": {
   "publiclyCallable": "true",
   "url": [
    {
     "execution-time": "394",
     "content": "http://api.flickr.com/services/rest/?method=flickr.photos.search&text=donkey&license=4&page=1&per_page=10"
    },
    [...]
   ],
   "user-time": "1245",
   "service-time": "4072",
   "build-version": "911"
  },
  "results": {
   "photo": [
    {
     "farm": "4",
     "id": "3344117208",
     "secret": "a583f1bb04",
     "server": "3355",
     "owner": {
      "nsid": "64749744@N00",
      "username": "babasteve"
     }
    }
    [...]
    }
   ]
  }
 }
}

The next step was getting the data from the other resources I am normally tapping into: Fail blog and I can has cheezburger. As neither of them have an API I need to scrape the HTML data of the page. Luckily this is also possible with YQL, all you need to do is select the data from html and give it an XPATH. I found the XPATH by analysing the page source in Firebug:

Using Firebug to find the right xpath to an image

This gave me the following YQL statement to get images from both blogs. You can list several sources as an array inside the in() statement:


select src from html where url in ('http://icanhascheezburger.com/?s=donkey','http://failblog.org/?s=donkey') and xpath="//div[@class='entry']/div/div/p/img"

Retrieving blog images using YQL

The result of this query is again a JSON object with the src values of photos matching this search:


{
 "query": {
  "count": "4",
  "created": "2009-03-11T01:28:35Z",
  "lang": "en-US",
  "updated": "2009-03-11T01:28:35Z",
  "uri": "http://query.yahooapis.com/v1/yql?q=select+src+from+html+where+url+in+%28%27http%3A%2F%2Ficanhascheezburger.com%2F%3Fs%3Ddonkey%27%2C%27http%3A%2F%2Ffailblog.org%2F%3Fs%3Ddonkey%27%29+and+xpath%3D%22%2F%2Fdiv%5B%40class%3D%27entry%27%5D%2Fdiv%2Fdiv%2Fp%2Fimg%22",
  "diagnostics": {
   "publiclyCallable": "true",
   "url": [
    {
     "execution-time": "1188",
     "content": "http://failblog.org/?s=donkey"
    },
    {
     "execution-time": "1933",
     "content": "http://icanhascheezburger.com/?s=donkey"
    }
   ],
   "user-time": "1939",
   "service-time": "3121",
   "build-version": "911"
  },
  "results": {
   "img": [
    {
     "src": "http://icanhascheezburger.files.wordpress.com/2008/09/funny-pictures-you-are-making-a-care-package-very-correctly.jpg"
    },
    {
     "src": "http://icanhascheezburger.files.wordpress.com/2008/01/funny-pictures-zebra-donkey-family.jpg"
    },
    {
     "src": "http://failblog.files.wordpress.com/2008/11/fail-owned-donkey-head-intimidation-fail.jpg"
    },
    {
     "src": "http://failblog.files.wordpress.com/2008/03/donkey.jpg"
    }
   ]
  }
 }
}

Writing the data retrieval API

The next thing I wanted to do was writing a small script to get the data and give it back to me as HTML. I could have used the JSON output in JavaScript directly but wanted to be independent of scripting. The script (or API if you will) takes a search term, filters it and executes both of the YQL statements above before returning a list of HTML items with photos in them. You can try it out for yourself: search for the term donkey or search for the term donkey and give it back as a JavaScript call

I use cURL to get the data as my server has external pulling of data via PHP disabled. This should work for most servers, actually.

Here’s the full “API” code:


<?php
if($_GET['js']==='yes'){
  header('Content-type:text/javascript');
  $out = 'seed({html:\'';    
}
if(isset($_GET['s'])){
  $s = $_GET['s'];
  if(preg_match("/^[0-9|a-z|A-Z|-| |\+|\.|_]+$/",$s)){
    $flickurl = 'http://query.yahooapis.com/v1/public/yql?q=select'.
                '%20farm%2Cid%2Csecret%2Cserver%2Cowner.username'.
                '%2Cowner.nsid%20from%20flickr.photos.info%20where%20'.
                'photo_id%20in%20(select%20id%20from%20'.
                'flickr.photos.search(10)%20where%20text%3D\''.
                $s.'\'%20and%20license%3D4)&format=json';
    $failurl = 'http://query.yahooapis.com/v1/public/yql?q=select'.
               '%20*%20from%20html%20where%20url%20in'.
               '%20(\'http%3A%2F%2Ficanhascheezburger.com'.
               '%2F%3Fs%3D'.$s.'\'%2C\'http%3A%2F%2Ffailblog.org'.
               '%2F%3Fs%3D'.$s.'\')%20and%20xpath%3D%22%2F%2Fdiv'.
               '%5B%40class%3D\'entry\'%5D%2Fdiv%2Fdiv%2Fp%2Fimg%22%0A&'.
               'format=json';
      $out.= '<ul id="collection">';
      $ch = curl_init(); 
      curl_setopt($ch, CURLOPT_URL, $flickurl); 
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
      $output = curl_exec($ch); 
      curl_close($ch);
      $flickrphotos = json_decode($output);
      foreach($flickrphotos->query->results->photo as $a){
          $o = $a->owner;
          $out.= '<li><img src="http://flickr.com/favicon.ico" class="ico">'.
                 '<img src="http://farm'.$a->farm.'.static.flickr.com/'.
                 $a->server.'/'.$a->id.'_'.$a->secret.'.jpg">';
          $href = 'http://www.flickr.com/photos/'.$o->nsid.'/'.$a->id;
          $out.= '<a href="'.$href.'">'.$href.' - '.$o->username.'</a></li>';
      } 
      $ch = curl_init(); 
      curl_setopt($ch, CURLOPT_URL, $failurl); 
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
      $output = curl_exec($ch); 
      curl_close($ch);
      $failphotos = json_decode($output);
      foreach($failphotos->query->results->img as $a){
          $out.= '<li>';
          if(strpos($a->src,'failblog') === 7){
            $out.= '<img src="http://s1.wordpress.com/wp-content/themes/vip/'.
                   'failblog/images/favicon.ico" class="ico">';
          } else {
            $out.= '<img src="http://s1.wordpress.com/wp-content/'.
                   'themes/vip/'.'icanhascheezburger/images/ratings/'.
                   'user.png" class="ico">';
          }
          $out.= '<img src="'.$a->src.'" alt="'.$a->alt.'"></li>';
      } 
      $out.= '</ul>';
      if($_GET['js']==='yes'){
        $out.= '\'})';
      }
      echo $out;    
    } else {
      echo ($_GET['js']!=='yes') ? 
        '<p>Invalid search term.</p>' : 
        'seed({html:"Invalid search Term!"})';
    }
}
?>

Let’s go through it step by step:


<?php
if($_GET['js']==='yes'){
  header('Content-type:text/javascript');
  $out = 'seed({html:\'';    
}

I test if the js parameter is set and if it is I send a JavaScript header and start the JS object output.


if(isset($_GET['s'])){
  $s = $_GET['s'];
  if(preg_match("/^[0-9|a-z|A-Z|-| |\+|\.|_]+$/",$s)){

I get the search term and filter out invalid terms.


    $flickurl = 'http://query.yahooapis.com/v1/public/yql?q=select'.
                '%20farm%2Cid%2Csecret%2Cserver%2Cowner.username'.
                '%2Cowner.nsid%20from%20flickr.photos.info%20where%20'.
                'photo_id%20in%20(select%20id%20from%20'.
                'flickr.photos.search(10)%20where%20text%3D\''.
                $s.'\'%20and%20license%3D4)&format=json';
    $failurl = 'http://query.yahooapis.com/v1/public/yql?q=select'.
               '%20*%20from%20html%20where%20url%20in'.
               '%20(\'http%3A%2F%2Ficanhascheezburger.com'.
               '%2F%3Fs%3D'.$s.'\'%2C\'http%3A%2F%2Ffailblog.org'.
               '%2F%3Fs%3D'.$s.'\')%20and%20xpath%3D%22%2F%2Fdiv'.
               '%5B%40class%3D\'entry\'%5D%2Fdiv%2Fdiv%2Fp%2Fimg%22%0A&'.
               'format=json';

These are the YQL queries, you get them by clicking the “copy url” button in YQL.


      $out.= '<ul id="collection">';

I then start the output list of the results.


      $ch = curl_init(); 
      curl_setopt($ch, CURLOPT_URL, $flickurl); 
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
      $output = curl_exec($ch); 
      curl_close($ch);
      $flickrphotos = json_decode($output);
      foreach($flickrphotos->query->results->photo as $a){
          $o = $a->owner;
          $out.= '<li><img src="http://flickr.com/favicon.ico" class="ico">'.
                 '<img src="http://farm'.$a->farm.'.static.flickr.com/'.
                 $a->server.'/'.$a->id.'_'.$a->secret.'.jpg">';
          $href = 'http://www.flickr.com/photos/'.$o->nsid.'/'.$a->id;
          $out.= '<a href="'.$href.'">'.$href.' - '.$o->username.'</a></li>';
      } 

I call cURL to retrieve the data from the flickr yql query, do a json_decode and loop over the results. Notice the rather annoying way of having to assemble the flickr url and image source. I found this by clicking around flickr and checking the src attribute of images rendered on the page. The images with the “ico” class should tell me where the photo was from.


      $ch = curl_init(); 
      curl_setopt($ch, CURLOPT_URL, $failurl); 
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
      $output = curl_exec($ch); 
      curl_close($ch);
      $failphotos = json_decode($output);
      foreach($failphotos->query->results->img as $a){
          $out.= '<li>';
          if(strpos($a->src,'failblog') === 7){
            $out.= '<img src="http://s1.wordpress.com/wp-content/themes/vip/'.
                   'failblog/images/favicon.ico" class="ico">';
          } else {
            $out.= '<img src="http://s1.wordpress.com/wp-content/'.
                   'themes/vip/'.'icanhascheezburger/images/ratings/'.
                   'user.png" class="ico">';
          }
          $out.= '<img src="'.$a->src.'" alt="'.$a->alt.'"></li>';
      } 

Retrieving the blog data works the same way, all I had to do extra was check for which blog the resulting image came from.


      $out.= '</ul>';
      if($_GET['js']==='yes'){
        $out.= '\'})';
      }
      echo $out;    

I close the list and – if JavaScript was desired – the JavaScript object and function call.


    } else {
      echo ($_GET['js']!=='yes') ? 
        '<p>Invalid search term.</p>' : 
        'seed({html:"Invalid search Term!"})';
    }
}
?>

If there was an invalid term entered I return an error message.

Setting up the display

Next I went to the YUI grids builder and created a shell for my hack. Using the generated code, I added a form, my yql api, an extra stylesheet for some colouring and two IDs for easy access for my JavaScript:


  <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">
  <html>
  <head>
     <title>Slide Fodder - find CC licensed photos and funpics for your slides</title>
     <link rel="stylesheet" href="http://yui.yahooapis.com/2.7.0/build/reset-fonts-grids/reset-fonts-grids.css" type="text/css">
     <link rel="stylesheet" href="slidefodder.css" type="text/css">
  </head>
  <body>
  <div id="doc2" class="yui-t6">
     <div id="hd" role="banner"><h1>Slide Fodder</h1></div>
     <div id="bd" role="main">
      <div class="yui-g">
        <form action="index.php" id="f">
          <div>
            <label for="s">Find Photos of</label>
            <input type="text" id="s" name="s">
            <input type="submit" value="search">
          </div>
        </form>
      </div>
      <div class="yui-ge">
        <div class="yui-u first" id="content">
          <?php include('yql.php');?>
        </div>
        <div id="basket" class="yui-u"></div>
      </div>
    </div>
    <div id="ft" role="contentinfo"><p>Slide Fodder by <a href="http://wait-till-i.com">Christian Heilmann</a>, hacked live at Georgia Tech University Hack day using <a href="http://developer.yahoo.com/yui">YUI</a> and <a href="http://developer.yahoo.com/yui">YQL</a>.</p><p> Photo sources: <a href="http://flickr.com">Flickr</a>, <a href="http://failblog.org">Failblog</a> and <a href="http://icanhascheezburger.com">I can has cheezburger</a>.</p></div>
  </div>
  <script type="text/javascript" src="http://yui.yahooapis.com/combo?2.7.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>
  <script type="text/javascript" charset="utf-8" src="slidefodder.js"></script>
  </body>
  </html>

Rounding up the hack with a basket

The last thing I wanted to add was a “basket” functionality which would allow me to do several searches and then copy and paste all the photos in one go once I am happy with the result. For this I’d either have to do a persistent storage somewhere (DB or cookies) or use JavaScript. I opted for the latter.

The JavaScript uses YUI and is no rocket science whatsoever:


function seed(o){
  YAHOO.util.Dom.get('content').innerHTML = o.html;
}
YAHOO.util.Event.on('f','submit',function(e){
  var s = document.createElement('script');
  s.src = 'yql.php?js=yes&s='+ YAHOO.util.Dom.get('s').value;
  document.getElementsByTagName('head')[0].appendChild(s);
  YAHOO.util.Dom.get('content').innerHTML = '<img src="http://tweeteffect.com/ajax-loader.gif" style="display:block;margin:2em auto">';

  YAHOO.util.Event.preventDefault(e);
});

YAHOO.util.Event.on('content','click',function(e){
  var t = YAHOO.util.Event.getTarget(e);
  if(t.nodeName.toLowerCase()==='img'){
    var str = '<div><img src="'+t.src+'">';
    if(t.src.indexOf('flickr')!==-1){
      str+= '<p>'+t.parentNode.getElementsByTagName('a')[0].innerHTML+'</p>';
    }
    str+='<a href="remove this image">x</a></div>';
    YAHOO.util.Dom.get('basket').innerHTML+=str;
  }
  YAHOO.util.Event.preventDefault(e);
});
YAHOO.util.Event.on('basket','click',function(e){
  var t = YAHOO.util.Event.getTarget(e);
  if(t.nodeName.toLowerCase()==='a'){
    t.parentNode.parentNode.removeChild(t.parentNode);
  }
  YAHOO.util.Event.preventDefault(e);
});

Again, let’s check it bit by bit:


function seed(o){
  YAHOO.util.Dom.get('content').innerHTML = o.html;
}

This is the method called by the “API” when JavaScript was desired as the output format. All it does is change the HTML content of the DIV with the id “content” to the one returned by the “API”.


YAHOO.util.Event.on('f','submit',function(e){
  var s = document.createElement('script');
  s.src = 'yql.php?js=yes&s='+ YAHOO.util.Dom.get('s').value;
  document.getElementsByTagName('head')[0].appendChild(s);
  YAHOO.util.Dom.get('content').innerHTML = '<img '+   
  'src="http://tweeteffect.com/ajax-loader.gif"'+    
  'style="display:block;margin:2em auto">';
  YAHOO.util.Event.preventDefault(e);
});

When the form (the element with th ID “f”) is submitted, I create a new script element,give it the right src attribute pointing to the API and getting the search term and append it to the head of the document. I add a loading image to the content section and stop the browser from submitting the form.


YAHOO.util.Event.on('content','click',function(e){
  var t = YAHOO.util.Event.getTarget(e);
  if(t.nodeName.toLowerCase()==='img'){
    var str = '<div><img src="'+t.src+'">';
    if(t.src.indexOf('flickr')!==-1){
      str+= '<p>'+t.parentNode.getElementsByTagName('a')[0].innerHTML+'</p>';
    }
    str+='<a href="remove this image">x</a></div>';
    YAHOO.util.Dom.get('basket').innerHTML+=str;
  }
  YAHOO.util.Event.preventDefault(e);
});

I am using Event Delegation to check when a user has clicked on an image in the content section and create a new DIV with the image to add to the basket. When the image was from flickr (I am checking the src attribute) I also add the url of the image source and the user name to use in my slides later on. I add an “x” link to remove the image from the basket and again stop the browser from doing its default behaviour.


YAHOO.util.Event.on('basket','click',function(e){
  var t = YAHOO.util.Event.getTarget(e);
  if(t.nodeName.toLowerCase()==='a'){
    t.parentNode.parentNode.removeChild(t.parentNode);
  }
  YAHOO.util.Event.preventDefault(e);
});

In the basket I remove the DIV when the user clicks on the “x” link.

That’s it

This concludes the hack. It works, it helps me get photo material faster and it took me about half an hour to build all in all. Yes, it could be improved in terms of accessibility, but this is enough for me and my idea was to show how to quickly use YQL and YUI with a few lines of PHP to deliver something that does a job :)

Wait till I come! is the blog of Christian Heilmann , a developer evangelist living and working in London, England. Download vcard.

Feed me, Seymour: Entries (RSS) and Comments (RSS).