This week I’ve been messing around with the HTML5 video element and have discovered you can do some pretty cool stuff. For instance, Mozilla shows how to do greenscreen image replacement while a video is played. This is very cool but there’s an annoying intermediate step required: since you cannot directly fetch raw frame image data from a video element, you must write each frame to an intermediate canvas element and then use its getImageData method to get raw image data you then transform and write to the final canvas element.
I brought this up in an email to the HTML5 Doctors last night. Remy suggested that I write up a blog post with some benchmarks, so here are some quick findings. You can find my code on GitHub. In the example I take a video and invert all its colors, frame-by-frame.
Here’s my html:
Using Chrome 5.0.375.55 on OS X and its developer tools, I took a heap snapshot after loading
var video = document.getElementById("video); video.play();. Once the video appeared to have stopped (confirmed by
video.ended) I stopped the profiler and took another heap snapshot. This time there were 2202 code and 13037 objects taking up 471.58 kb and 922.11 kb. Not surprisingly the profile shows that
processFrame() takes up the vast majority of execution time (77.63% self, 83.65% total).
That being said, I still find the process ineffiecent and would like to be able to get a sort of FrameData object directly from video elements, just like you can get an ImageData object from canvas elements. This would mean I would only need to use one canvas element (
canvas2 in my example), instead of two.