Wednesday, November 16, 2011

Trials and tribulations with HTML5 Canvas on Android

You may not be aware but I recently released a mobile web app called leftanote.com. It is a neat little application that lets you post notes about someones driving abilities based on their license plate number. It leverages several technologies: jquery, jquerymobile, jqScribble (my custom canvas drawing tool), as well as some other things I will get into shortly.

The coolest part of my app, in my opinion, is the jqScribble drawing tool. It is touch enabled, so it will let you draw with either a mouse or your finger. It then provides hooks to let you specify how you want the image saved. In my case I send the image data string to my webservice which creates the image file. To do this the tool leverages the toDataURL function available on the HTML 5 canvas element. Basically toDataURL converts the canvas image as a 64bit string. This canvas feature is amazing and super fast, which I love.

Nearing the final days before my official "launch" one of my co-workers wanted to use the app. It just so happens he has an Android phone. I believe his OS version is 2.2 (not sure which dessert that is). Anyway he wanted to use the drawing tool, which I was expecting to work perfectly, the drawing part worked fine, but the saving part failed. After some research I discovered that up until version 2.3 of Android the toDataURL functionality of canvas was nothing more than an empty function with a comment to the tune of "I don't know what this is supposed to do".

After some obvious ranting and Google cursing, I came across this post by Hans Schmucker. He was having the exact same issue as me and developed his own PNG encoder, which he used to replace toDataURL on Android devices. So I went to work attempting to implement what he did with my code, unfortunately I was never able to get it to work. However he did get me thinking down the right track. This lead me to the realization that toDataURL can be set to return either PNG or JPG formatted data. Now the only trick was finding a JPEG encoder written in javascript. Well that wasn't tough, a good one exists here (Thanks to Arizona for finding a new link after the old one was removed). All that was left was to add some simple detection and execute if the stock toDataURL doesn't work properly.

Here is my code:

var tdu = HTMLCanvasElement.prototype.toDataURL;
HTMLCanvasElement.prototype.toDataURL = function(type)
{
 var res = tdu.apply(this,arguments);
 //If toDataURL fails then we improvise
 if(res.substr(0,6) == "data:,")
 {
  var encoder = new JPEGEncoder();
  return encoder.encode(this.getContext("2d").getImageData(0,0,this.width,this.height), 90);
 }
 else return res;
}

The first line puts the toDataURL function into a variable so that if it works correctly we don't blow it away on the next line where we override toDataURL with our own custom function. The custom function will attempt to call the original toDataURL, then checks the returned value. If it doesn't have the correct header information, we know that the original toDataURL failed, so it creates a new JPEGEncoder instance and encodes, then returns the canvas data.

Is this the best way of doing things? Probably not, it is never a good idea to muck around with stock functionality, but I didn't want to go back and change my code that already used toDataURL, so I made a judgement call.

Finally to break it down for you: If you need to support the toDataURL function of canvas in an Android device < 2.3. You should take a similar approach.

1) Download a JPEG Encoder.
2) Add in your own custom toDataURL handler which will use the Encoder if the original toDataURL fails.

The drawing tool works like a charm, and saving images is still reasonably fast. Although I am dealing with fairly small images.

22 comments:

  1. Works like a charm, thanks a lot

    ReplyDelete
  2. Hey Jim, I'm "A guy" from XDA and I've been getting a couple of reports about issues... but the people always drop off the face of the earth once I explain to them what to do step-by-step (I really don't know the issue... add the script tag to your code and that should be all there is to do).

    If you could describe your problem, I'm sure I'm able to solve the issue...

    hansschmucker@gmail.com

    ReplyDelete
    Replies
    1. Hans, I sent you an email about the issues I ran into. Also I gave you credit in this post. Thanks so much for coming up with that solution it helped me out tremendously.

      Delete
  3. Hi Jim,

    Please help me with step by step instructions to use the encoder to fix the toDataURL problem. Thanks!

    ReplyDelete
    Replies
    1. Once you have the encoder included on your page just copy the code snippet I put in the post. If everything is done in the correct order then it should just work.

      Delete
  4. Thanks. Your works. But I need a PNG encoder.

    ReplyDelete
    Replies
    1. Oh, in that case look at the post made by Hans Schmucker. It's linked above. He developed a PNG encoder, he also left his email address in one of the previous comments to this post. I recommend you talk to him.

      Delete
  5. I did try his png encoder. But that doesnot compress the data. We actually need to save the image in mySQL database and the encoded result is too big to save in db.

    Also, the image data generated from the jpeg encoder (Your solution)is always blank. Any suggestions on this? Thanks for all your help!

    ReplyDelete
  6. Its now working with your jpeg encoder. I did a small mistake before. Thanks for all your help.

    This is really a very useful blog.

    ReplyDelete
  7. Hi Jim,


    I am creating an android application relates with drawing on canvas.But I am unable to get canvas data via toDataURL. I read your blog but i am still getting problems. Can you please explain me , how to implement it with example and can
    you please share the JPEGEncoder script which you were used.


    vijayv2205@gmail.com

    Thanks....

    ReplyDelete
    Replies
    1. Vijay,

      Everything is in the blog post above. Use the code snippet I provided and download the JPEGEncoder from the link I provided. Everything should just work at that point.

      Delete
  8. The image generated could be displayed with an img tag in my desktop browser, but it is blank in Android. Any clue would help.

    ReplyDelete
  9. Hey,

    I have a HTML canvas element on which the user can draw what he wants. I want to store this as an image on the sdcard.

    This is my current code -
    $('#submit').click(function() {

    var tdu = HTMLCanvasElement.prototype.toDataURL; HTMLCanvasElement.prototype.toDataURL = function(type) {

    var res = tdu.apply(this, arguments); if (res.substr(0, 6) == "data:,") { var encoder = new JPEGEncoder(); return encoder.encode( this.getContext("2d").getImageData(0, 0, this.width, this.height), 90); } else return res;
    }

    var canvas = document.getElementById("can");
    var writer = new FileWriter("/mnt/sdcard/sign6.jpeg");
    var img = canvas.toDataURL();
    navigator.notification.alert(img);
    writer.write(img);
    });

    Any help appreciated.

    No errors are displayed on the adb logcat.

    ReplyDelete
  10. That's a pretty neat app. Unfortunately the drawing part doesn't work well on the default browser of my Android phone (Sony Xperia Active ICS). It inserts straight lines, so if I draw a circle it makes it look like a Pacman, and the curve has extra straight lines in it.

    ReplyDelete
    Replies
    1. I have noticed that as well. It's odd since it works on normal browsers and iOS. It seems like maybe Android doesn't have a real good canvas implementation. Which wouldn't be surprising considering the bug I outlined above.

      Delete
  11. Hi Jim,

    It appears that the link you've provided for the JPEGEncoder is no longer a valid link. I've located a "JPEGEncoder" that was written in 11/2009 by Andreas Ritter. It reports itself as version 0.9.

    Is this the same JPEGEncoder you are using?

    Thanks,
    -Arizona

    ReplyDelete
    Replies
    1. Hi Arizona,

      Yes I used the JPEGEncoder by Andreas Ritter. Thanks for the heads up on the link now being dead. Would you mind sending an updated link if you found one? Otherwise any javascript JPEG encoder will work.

      Delete
  12. Hi Jim,

    Below are the following locations at which I found Andreas Ritter's JPEGEncoder:

    http://closure-library.googlecode.com/svn/docs/closure_third_party_closure_goog_jpeg_encoder_jpeg_encoder_basic.js.source.html

    https://gist.github.com/creotiv/1245476

    https://github.com/owencm/js-steg/blob/master/jpg_encoder.js

    Hope this helps...and thank you for your article!

    -Arizona

    ReplyDelete
  13. Hi, can you show me how to use this with jSignature.

    Thanks.

    ReplyDelete
  14. Thanks a lot! That really helped me!

    ReplyDelete