JoyentJoyent

Day five: Smart XML fun with E4X

We’re going to continue our walkthrough across the different libraries provided by the Smart Platform and some other goodies. As you might now, the Smart Platform comes with all the available features of core JavaScript 1.8, which includes E4X, a programming language extension that adds native XML support to JavaScript.

During this article we’ll learn how to use it for both, processing and generating our XML documents, how to tell the browser about content-types and start using system.http and the datejs library which also comes bundled with Smart.

Generating a OPML representation of our tasks

Our first approach to E4X consist of the generation of an OPML document, including all our tasks, the associated notes, their completion states in such a way we could use the generated document in a software like OmniOutliner. Obviously, our document will not be too complex since right now, our tasks cannot be nested. Either way, enough for us to start playing.

For this article we’re going to use the OPML 1.0 Spec with some modifications intended to make our generated document to play nice with the aforementioned desktop software.

This is the XML skeleton we pretend to generate from our tasks list:

<?xml version="1.0" encoding="utf-8"?>
<opml version="1.0">
  <head>
    <title>Tasks list</title>
    <dateCreated>Thu, 2 Jul 2009 03:24:18 GMT</dateCreated>
    <dateModified>Sat, 4 Jul 2009 09:01:23 GMT</dateModified>
    <ownerName>User Name</ownerName>
    <ownerEmail>user@example.com</ownerEmail>
    <expansionState />
  </head>
  <body>
    <outline text="This is a task title" />
    <outline text="This is another task title" />
    <outline text="And another one" _note="The third task has a note"/>
    <outline text="Task 4" _status="checked" _note="This will be completed"/>
  </body>
</opml>

The absolute first thing to do is to pick a meaningful route and write our bootstrap.js method to capture it:

GET('/tasks.opml', function(){
  // Our code here ...
});

Now, if our application were able to handle multiple users, we would need to get that information from the Data Store but, since we haven’t got more than one user, the only thing on the <head> element we’re concerned about are the dateCreated and dateModified fields. For now, let’s generate our head element without worrying about those fields:

GET('/tasks.opml', function(){
  var opml = <opml version="1.0">
    <head>
      <title>Tasks list</title>
      <ownerName>User Name</ownerName>
      <ownerEmail>user@example.com</ownerEmail>
      <expansionState />
    </head>
  </opml>;

  return opml.toString();
});

(Note we omitted the XML declaration due to a bug on E4X which would raise the error “xml is a reserved identifier”).

The response we’re returning to the browser is pretty obvious: the same XML string we’re writing as the value of the opml variable. One thing to note is we’re returning it as text/html, while we should return it as XML. Let’s alter our previous method to handle the Content-Type header properly:

GET('/tasks.opml', function(){
  var opml = <opml version="1.0">
    <head>
      <title>Tasks list</title>
      <ownerName>User Name</ownerName>
      <ownerEmail>user@example.com</ownerEmail>
      <expansionState />
    </head>
  </opml>;
  this.response.mime = 'application/xml';
  return opml.toString();
});

If you review the request response, you’ll see that by setting this.response.mime, we’re telling the Smart Platform to server the response with the given Content-Type of application/xml.

Ok, let’s add some child nodes to our head element. We want to retrieve the creation date of the tasks list, which should match with the first created task, and the last modification date which should, in turn, match either with the last modified task or with the newest one.

First, we’re going to include the datejs library at the top of our bootstrap.js file:

system.use("com.google.code.date");

Then, we’re going to add the attributes to our XML head element:

GET('/tasks.opml', function(){
  var opml = <opml version="1.0">
    <head>
      <title>Tasks list</title>
      <ownerName>User Name</ownerName>
      <ownerEmail>user@example.com</ownerEmail>
      <expansionState />
    </head>
  </opml>;

  var firstTask = Task.search({}, {sort: 'created', limit: 1})[0];
  var lastCreatedTask = Task.search({}, {sort: 'created', limit: 1, reverse: true})[0];
  var lastModifiedTask = Task.search({}, {sort: 'updated', limit: 1, reverse: true})[0];

  opml.head.dateCreated = firstTask['created'].toString('ddd, dd MMM yyyy HH:mm:ss') +' GMT';
  if ( lastCreatedTask['created'] >= lastModifiedTask['updated'] ) {
    opml.head.dateModified = lastCreatedTask['created'].toString('ddd, dd MMM yyyy HH:mm:ss') +' GMT';
  } else {
    opml.head.dateModified = lastModifiedTask['updated'].toString('ddd, dd MMM yyyy HH:mm:ss') +' GMT'
  }

  this.response.mime = 'application/xml';
  return opml.toString();
});

As you can see, we’re simply setting properties and their values for the new attributes as we do for any other JavaScript attribute, and E4X takes care of setting the proper XML output for us. The format used for the dates is RFC822, as required by the spec. It’s a bit different anyway when we need to manipulate several elements with the same name, like our outline elements. Instead of direct assignment, we need to add the elements as follows:

var allTheTasks = Task.search({}, {sort: 'position'})
for each (task in allTheTasks) {
  status = (task["completed"] == 1) ? '_status="checked" ' : '';
  opml.body.outline+= <outline title={task["title"]} {status}_note={task["notes"]}/>;
}

First thing to notice is we’re accessing the opml.body.outline elements with the += operator, instead of using direct assignation, like we did with the elements present in our document only once. There are also couple more things to notice:

  • We’re interpolating variables into the XML literal using the { } operator.
  • We’re using the for each .. in statement, introduced in JavaScript 1.6 as part of E4X support.

And that’s all!. What about to take advantage of the fact we’re playing with E4X now and do a bit of XML parsing practice?.

Accessing HTTP web services

The Smart Platform provides a convenient way to perform HTTP request using the library system.http. Since it’s pretty well documented, let’s start using it right now in order to retrieve an XML document.

For this example, we’ll use the identi.ca search API, which will retrieve an Atom XML document with the contents of our searches.
First, we need a new page with a form, so our application users can perform searches. In order to keep our application structure the clearest possible, we’ll add a new file js/identica.js and include it from our js/bootstrap.js file:

system.use("identica");

Then, on that new file, we’re going to add a simple action to handle all the GET requests to /identi.ca which, in turn, will simply render a template:

GET('/identi.ca', function() {
  return template("search.html");
});

This is the template being rendered:

[% SET host="http://$request.headers.Host" %]
[% INCLUDE _header.html title="Search identi.ca" %]
  <script type="application/javascript" src="/js/identica.js"></script>
  <h1>Search identi.ca:</h1>
  <form action="[% host %]/identi.ca" method="post" id="search_form">
    <div class="field">
      <label for="search">Search:</label>
      <input type="text" name="search" id="search" class="text" value="@joyent" />
    </div>
    <div class="field">
      <input type="submit" value="Go!" />             
    </div>
  </form>
  <div id="updateme"></div>
[% INCLUDE _footer.html %]

As you can see, nothing too exciting but for another JavaScript file being loaded web/public/js/identica.js which simply contains the logic to perform an AJAX call from the previous form and update the contents of #updateme div element with the results.

And now, the real fun: our function to handle the POST requests to /identi.ca which will, in turn, try to perform a search against the identi.ca API using the system.http library and, on success response, parse it and return it on a manner suitable to be displayed into our browser:

POST('/identi.ca', function() {
  this.entries = new Array();
  uri = system.sprintf("http://identi.ca/api/search.atom?q=%s", this.request.body.search)
  response = system.http.request('GET', uri);
  if ( response.code == 200 && response.content ) {
    var feedXML = response.content.replace(/^<\?xml\s+version\s*=\s*(["'])[^\1]+\1[^?]*\?>/, ""); // bug 336551
    var atom = Namespace("http://www.w3.org/2005/Atom");
    var feed = new XML(feedXML);
    var entry;
    for each (entry in feed.atom::entry) {
      var anEntry = {
        title: entry.atom::title.toString(),
        author: {
          name: entry.atom::author.atom::name.toString(),
          uri: entry.atom::author.atom::uri.toString()
        }
      };
      for each (link in entry.atom::link) {
        if (link.@rel == 'alternate' && link.@type == 'text/html') {
          anEntry.alternate = link.@href;
        } else if (link.@rel == 'related' && link.@type == "image/png") {
          anEntry.icon = link.@href;
        }
      }
      this.entries.push(anEntry);
    }
  return template("results.html");
  }
});

Several things to note here:

  1. We’re using system.sprintf to properly format the search url with the search request coming from our form.
  2. Then, once we perform our HTTP request, we check the response.code to ensure we’ve got a success and the response.content, to ensure we’ve got something to parse before even to try it.
  3. Once we’ve got our response, first thing is to avoid the aforementioned bug regarding xml protocol affecting E4X.
  4. Finally, we loop over all the entry elements into the retrieved Atom document and push them into a Stack variable this.entries, so we’ll be able to use them into our HTML template. The HTML template being used isn’t too different than what you could expect at this point:
<div id="updateme">
[% FOREACH entry = entries %]
<div class="message">
  <img src="[% entry.icon %]" alt="[% entry.author.name %]"/>
  <strong><a href="[% entry.author.uri %]">[% entry.author.name %]</a></strong>:
  <a href="[% entry.alternate %]">[% entry.title %]</a>
</div>
[% END %]
</div>

Being an observant reader, you’ve noticed that we’re using the atom:: namespace in our code, in order to search for the elements of interest. At this point, I’d suggest a good resource regarding E4X: http://rephrase.net/days/07/06/e4x. It contains nice information about this namespaces topic, among others, like elements filtering by attribute values, global function reference on E4X, how to extend it …

And this has been day five!. Keep an eye here, since we’re going to cover other interesting topics regarding the Smart Platform, starting with File and Image manipulation.