Peter Robinett – Quantifiying productivity from Quantified Self Amsterdam on Vimeo.
Above is the talk on how I track my productivity I gave this Monday at the first every Quantified Self Amsterdam meetup. It was a lot of fun and I think my presentation went quite well considering that I literally spent five minutes preparing.
At the Dutch GTUG HTML5 Hackathon I decided to make an HTML5 TV.
My HTML5 TV uses a whole grab bag of hot web features: CSS 3, SVG, the canvas element, the video element, and HTTP live streaming video. In theory the TV works in all of the most recent versions of the Safari browser variants. However, in Mobile Safari Apple puts a custom overlay over the video element, hiding the static, and prevents videos from playing automatically. I leave it up to you whether this is desirable. Given this, I recommend you use Safari 5 on OS X to view the page.
I can confirm that the latest version of Chrome on OS X does not play HTTP live streaming videos. If you can’t see HTTP live streaming video in your browser and are comfortable with its Javascript console, you can change the channels used like so:
TV.channels = [ { 'name': 'BBB', 'url': 'http://cdn.kaltura.org/apis/html5lib/kplayer-examples/media/bbb_trailer_iphone.m4v' } ]
As you can see, channels is an array of anonymous objects with name and url properties and you can have as many as you want. I recommend doing this before you turn on the TV.
This week I’ve been messing around with the HTML5 video element and have discovered you can do some pretty cool stuff. For instance, Mozilla shows how to do greenscreen image replacement while a video is played. This is very cool but there’s an annoying intermediate step required: since you cannot directly fetch raw frame image data from a video element, you must write each frame to an intermediate canvas element and then use its getImageData method to get raw image data you then transform and write to the final canvas element.
I brought this up in an email to the HTML5 Doctors last night. Remy suggested that I write up a blog post with some benchmarks, so here are some quick findings. You can find my code on GitHub. In the example I take a video and invert all its colors, frame-by-frame.
Here’s my html:
And my Javascript:
Using Chrome 5.0.375.55 on OS X and its developer tools, I took a heap snapshot after loading benchmark.html. There were 2171 code (what, I don’t know) and 12463 objects, taking up 464.70 kb and 866.97 kb respectively. I then started the profiler and executed the following Javascript: var video = document.getElementById("video); video.play();. Once the video appeared to have stopped (confirmed by video.ended) I stopped the profiler and took another heap snapshot. This time there were 2202 code and 13037 objects taking up 471.58 kb and 922.11 kb. Not surprisingly the profile shows that processFrame() takes up the vast majority of execution time (77.63% self, 83.65% total).
This all leads me to believe that, at least for smaller videos like the 480 x 280 px video I used from Apple’s documentation, there are not significant performance issues when transforming videos frame-by-frame in Javascript.
That being said, I still find the process ineffiecent and would like to be able to get a sort of FrameData object directly from video elements, just like you can get an ImageData object from canvas elements. This would mean I would only need to use one canvas element (canvas2 in my example), instead of two.