Tuesday, July 24, 2012

Speeding things up with HTML5 Web Storage

Cache stuff in Javascript using Modernizr and the HTML5 API. And pronounce cache correctly.

On my brother Gavin's gallery website that I'm working on there's a sidebar which fetches a list of photo sets. Standard stuff. You choose a set and it loads a page with the querystring value of the set's id. It's an oldish WebForms site that I've been doing the jQuery and Web API business on.

I use a Web API project to fetch the list of sets from a Flickr account which Gav has set up, where each group of paintings ("available work", "small paintings in oil") is represented as a set of Flickr photos. It's a double hop: the website hits my Web API project, which hits the Flickr API. And as you can imagine, the list of sets doesn't change that often. So you can probably see this one coming from a looong way down the road. Anyway, in case you can't, here it is. Ready? I thought I'd cache the results.

But not in the .Net code. Because I'm asking for the data in the jQuery ajax() method, and pumping it to the screen on the .done() callback, I thought I'd test use the new (to me anyway) HTML5 Storage feature. It's seems like a good way to cache a small amount of data that is unlikely to change from page to page within the same session, and which is probably going to be pointless, or worse, fetching every single page load.

As a digression, it's weird the way here in Australia devs are wont to pronounce "cache" as "cay-che", not "cash". I grew up in Ireland where one of the stock phrases that you heard during the 80's was "arms cache", as in the British army found one in Northern Ireland, and said "cash", not "cayche". On podcasts and tech shows I only ever hear them say "cash". I spoke to Noel about this when I was back in Ireland a while ago. His wife's a Québécoise, a fluent French speaker, and la belle Catherine pointed out that the English word cache comes from the French word cacher: to hide away, store - and the root of that word is pronounced "cash". The French are losing enough words to internet English as it is - I think we should respect the words we owe to them by honouring the Gallic pronunciation appropriately where we can - just sayin'.

Framework à la carte

Anyway, I've become fascinated with Modernizr of late. I even like the pink fade on their website, the sort of styling that would normally be anathema to me. I can't work out what the logo means - the steps with a quarter circle - but to my great delight it animates when you mouse over it. And I love the way they make you choose what you need before downloading the library. It forces you to pay attention to what you're doing. These days, there are so many Javascript frameworks, plug-ins, and UI helpers, etc. that you can drown in different versions of slightly different libraries on your site. When stuff is free, you just take as much as you can get, I've found, so it's salutary for once to have to go through a list and pick what you need.

Since for the time being I'm only interested in that sessionStorage feature of HTML5, I can get a pretty pared-down version of Modernizr. My custom build amounts to a supermodel-skinny 9kb! This enforced customisation with Modernizr is because of the heterogeneity of HTML5. You are only likely to be interested in checking for the ability of your visitors' browsers to do a few particular things, so why download an exhaustive framework to cover every nook and cranny of the HTML5 features list?

It's code time. In GetSets() I check two things: first, does the user have a half-decent browser? In other words Internet Explorer 8+, Firefox, Opera, Chrome, or Safari? Those browsers all support HTML5 Web Storage. If they do, then see if they have already retrieved the data:
if (Modernizr.sessionstorage && sessionStorage['GLSets'])
If they don't have either, call out to the Web API url to get the sets from Flickr.
GetSets: function () {
        var self = this;

        // retrieve sets from local storage if already cached
        if (Modernizr.sessionstorage && sessionStorage['GLSets'])
        else {
            var url = common.BaseUrl + "api/lavelleartgallery/sets?user=" + common.GLFlickrId;
                dataType: "jsonp",
                url: url
            .done(function (data) {
                // cache sets in local storage
                if (Modernizr.sessionstorage)
                    sessionStorage['GLSets'] = JSON.stringify(data);

            .fail(function (jqXHR, textStatus, errorThrown) {
                // oops!;

ShowSets: function (data) {
        var sets = JSON.parse(data);
        var listOfWorks = $('<ul></ul>');
        $.each(sets, function () {
            var set = {
                id: this.Id,
                name: this.Name,
                numberOfWorks: this.NumberOfPhotos
            listOfWorks.append('<li><a href=\"Set.aspx?id=' + set.id + '\">' + set.name + ' (' + set.numberOfWorks + ')</a></li>');
And as soon as we get the data from Flickr (the .done() callback), stash it in the cache if we can. You'll notice that you have to serialize and deserialize the array of sets to set and get it from sessionStorage. That's because sessionStorage and localStorage only support strings.

Dive into HTML5 Web Storage

You might be thinking to yourself - I could have just saved this as a cookie if I wanted to avail of client side storage. In which case I refer you to the mysterious Mark Pilgrim's erudite "Dive into HTML5" book/website. If you read the chapter on Web Storage, you'll see a couple of ways in which the new Web Storage distinguishes itself from cookies: the data is not sent to and from the server on every request despite persisting beyond page refreshes.

This is all very handy, and browsing GavinLavelle.com, I can see a great improvement in the page load speed - the sets sidebar has no lag whatsoever, but - and it's a big one - the cache expires at the end of the session. I want to upgrade my HTML5 Web Storage to business class. I want to give it a bigger seat and more legroom. I want it to last as long as the data it's caching is current. That could end up being quite tricky. But there is no reliable way to expire localStorage - the counterpart to sessionStorage - as it just sits around for a long time apparently, so you have to come up with some expiration/refreshing strategy if you want to use it.

Monday, July 16, 2012

Making asynchronous Web API calls to a repository

"One of the most common ways in which recursion appears in daily life is when you postpone completing a task in favor of a simpler task, often of the same type."

It's as if Douglas Hofstadter was talking about the new .Net 4 Task type, introduced as part of the asynchronous .Net programming model, when he wrote this in "Gödel, Escher, Bach: an Eternal Golden Braid". There's a chapter in GEB called "Recursive Structures and Processes" which I had a look at again recently, having played around with a jQuery/Web API controller/repository stack for a site I'm doing.

I've been upgrading a lot of my old web services recently, code where I call out to the Flickr API, for instance, to load some photo-related data into one of my web pages. I felt it was time to get on board with the Web API framework, time to get RESTful. It's also time to start programming with the new Task type, introduced in .Net 4 specifically for asynchronous operations.

The starting point for asynchronous web operations is often the trusty jQuery $.ajax() operation, and that's where I'm starting from here too. Maybe it's just out of habit, given that I can do it relatively easily in straight .Net code now. Anyway, here's the JavaScript:
GetFeaturedWork: function () {
        dataType: "jsonp",
        url: "http://www.connemara.com/api/LavelleArtGallery/photos?user=[FlickrUserId]&tags=LAGFrontPage"
    .done(function (data) {
    .fail(function () {
        // show an error message;

ShowFeaturedWork: function (data) {
    var work = {
        Name: data[0]["Name"],
        Url: data[0]["URL"]
    // whack the image HTML into a div on the page...
If you're familiar with jQuery Ajax calls, and are wondering about the unfamiliar-looking .done() and .fail() callbacks, there's an important "Deprecation Notice" in the documentation about the API change (soon) from v1.8 on, so heed ye well this notice.

Also, note the jsonp-osity of my cross-domain Ajax call. This is because I am calling from one domain to another which by default is not possible with vanilla JSON. All you need to do to circumvent this same origin policy restriction is to tell jQuery that the datatype of your request is jsonp, and to have a JsonpMediaTypeFormatter in your service app. Find out how to do it (JSONP with ASP.NET Web API) on StackOverflow. I also touched on jsonp in my post about creating a StackOverflow client.

Flickr photo
FlickrRussian Dolls, by Mike F.. 'Nesting, and variations on nesting...'

This is all straightforward so far, but the next bit I found tricky, so that's why I chose to write it up. It took me a while to work out how to get my Web API controller action to call a repository method asynchronously. It's not that hard, I suppose: I'm just new to the asynchronous methods in MVC4. I can't use the new await keyword as I don't want to upgrade to VS2012 yet - my OS won't support it, and while you can get a standalone download for ASP.Net 4.5 I wanted to see if I could do some new-fangled async programming without all that overhead. Also, in case you think that's copping out, I'd have to upgrade to the 4.5 sandbox on my hosting provider, so...all up, I don't need all that pain.

So, to recap, I'm starting from a page with a jQuery ajax call attached. This hits a controller - an apicontroller - action method, which calls out to a repository to hit the Flickr API.
// GET api/lavelleartgallery/photos?user={user}&tags={tags}
public List GetPhotos(string user, string tags)
    return _repository.GetPhotos(user, tags).Result;
Familiar looking, except for the .Result, I'm guessing. I'm using the "modern" HttpClient, rather than the older Microsft.HttpClient that I had been using. Which means I'm mainly concerned with the HttpResponseMessage that's returned. Or, in this case, a Task<HttpResponseMessage>, because it's asynchronous.
public Task<List<FlickrPhoto>> GetPhotos(string user, string tags)
    var requestUrl = string.Format("http://api.flickr.com/services/rest/?method=flickr.photos.search&api_key={0}&user_id={1}&tags={2}&extras=url_m&{3}", APIKey, user, tags, format);

    var client = new HttpClient();
    return client.GetAsync(requestUrl).ContinueWith(task =>
        // Get HTTP response from completed task. 
        var response = task.Result;

        // Check that response was successful or throw exception 

        return response.Content.ReadAsStringAsync().ContinueWith(readTask =>
            var photosets = JObject.Parse(readTask.Result);
            return photosets["photos"]["photo"].Children().Select(photo => 
                new FlickrPhoto
                    Id = (string)photo["id"],
                    Name = (string)photo["title"],
                    URL = (string)photo["url_m"]
Each of the two generic Task Async methods (client.GetAsync() and response.Content.ReadAsStringAsync()) pushes down a level on the stack, to use the lingo. "To push means to suspend operations on the task you're currently working on, without forgetting where you are - and to take up a new task." - GEB. The associated ContinueWith() continuation functions are what happens after the task pops - "to close operations on one level, and to resume operations exactly where you left off, one level higher". It's hard to envisage all this pushing and popping in the code as it stands in this pattern - the tabbing/indenting doesn't map well to the levels within the stack. This here is more indicative:
(jQuery) $.ajax() calls the (WebAPI) PhotosController
    => PhotosController calls the FlickrRepository
        => FlickrRepository calls the Flickr API
            => Flickr API returns JSON data
        <= FlickrRepository returns a (Task of) List of FlickrPhotos
    <= PhotosController returns a List of FlickrPhotos
<= $.ajax() returns JSON to the .done() callback...
By the way, the terms "push", "pop", and "stack" are basic recursion terminology, and come from the visual image of cafeteria trays. You push down, they pop back up again, etc. From now on, when you place a $5 chicken curry on your café tray you will immediately think of asynchrony, recursion and Russian dolls.

Anyway, I thought it was a useful way to think about the Asynchronous .Net 4 code you can use now, and anything that's gets us all talking about "Gödel, Escher, Bach" is worth it. I love GEB: it's a singular book. There's nothing at all like it, except maybe Alice in Wonderland.