2010-02-09
HTML5's canvas Part II: Pixel Manipulation
Your pixels will be
manipulated.
Your browser doesn't support HTML canvas. Get a modern browser such as Chrome, Firefox, Opera, or IE9. Otherwise, you'll just see images of canvases, below, instead of canvases. Admittedly, this isn't much of an issue for this particular demo, but you should upgrade, anyway.
In an earlier post, I wrote about a few of the
things HTML5's
<canvas> element
could do. This post is about one more of them: direct pixel
manipulation. You can basically do the three following things:
- Create an array of empty pixels, or
- Get an array of pixels from an existing canvas, or
- Set the pixels of a canvas from a pixel array
Once you have an array of pixels, you can manipulate it using the standard array accessors "[", "]". The pixels in the array are arranged in row-major order, and are represented as integers between 0 and 255, where each four-integer group represents the four color channels of one pixel: red, green, blue, and alpha (RGBA). What did I just say? Basically, it looks like this... say you have an image that's 3 pixels wide, and 2 pixels high. I know that's tiny, but I want to fit it all in the following super-busy diagram which shows the array indices for given elements of each pixel.
Pixel layout in the pixel array
for a 3-by-2 image of 6 pixels. Each pixel takes 4 elements in the
array for red, green, blue, and alpha, for a total of 24 array elements,
0-23.
We'll get to a quick-and-easy calculation to find your way around the
array in just a moment, but first, how does one create a pixel data
array to be used on a canvas? You use the drawing context's
createImageData()
method. (Use
getImageData()
to get the array from an existing canvas.)
<canvas id="canvas1" width="100", height="100"></canvas>
<script type="text/javascript">
element = document.getElementById("canvas1");
c = element.getContext("2d");
// read the width and height of the canvas
width = element.width;
height = element.height;
// create a new batch of pixels with the same
// dimensions as the image:
imageData = c.createImageData(width, height);
It's going to be useful later to know the width of the image data, and fortunately that's baked into the ImageData object:
imageData.width // width of the image data in pixels
imageData.height // height of the image data in pixels
imageData.data // pixel data array of (width*height*4) elements</pre>
Now we have what we need to write a pixel-setting routine:
function setPixel(imageData, x, y, r, g, b, a) {
index = (x + y * imageData.width) * 4;
imageData.data[index + 0] = r;
imageData.data[index + 1] = g;
imageData.data[index + 2] = b;
imageData.data[index + 3] = a;
}
There you can see how we first calculate the pixel "position" in the
array (which is the row number y times the number of pixels in each
row (the width), plus the column number x) and then we multiply it by
4 to get the first channel for that pixel, namely red, because each
pixel takes up 4 array elements.
Exercise: write a getPixel(x,y) function that returns the RGBA
values for a given pixel as an array, or as a packed 32-bit number
(0x_RRGGBBAA_).
We've seen how to create an image pixel data array, and we've seen how
to access it. The only thing left to do is to "paste" the pixels back
onto the canvas so that we see them. We do this with a call to
putImageData(),
to which we pass the image data object, plus an X and Y offset at which
to stamp the new data. Let's use this to produce a canvas that is
covered with randomly-colored pixels:
<canvas id="canvas1" width="100", height="100">
Random Canvas
</canvas>
<script type="text/javascript">
function setPixel(imageData, x, y, r, g, b, a) {
index = (x + y * imageData.width) * 4;
imageData.data[index+0] = r;
imageData.data[index+1] = g;
imageData.data[index+2] = b;
imageData.data[index+3] = a;
}
element = document.getElementById("canvas1");
c = element.getContext("2d");
// read the width and height of the canvas
width = element.width;
height = element.height;
// create a new pixel array
imageData = c.createImageData(width, height);
// draw random dots
for (i = 0; i < 10000; i++) {
x = Math.random() * width | 0; // |0 to truncate to Int32
y = Math.random() * height | 0;
r = Math.random() * 256 | 0;
g = Math.random() * 256 | 0;
b = Math.random() * 256 | 0;
setPixel(imageData, x, y, r, g, b, 255); // 255 opaque
}
// copy the image data back onto the canvas
c.putImageData(imageData, 0, 0); // at coords 0,0
</script>
But that's so ugly. Let's do something a little prettier, by setting the color of each pixel depending on the sine of the distance it is away from some point on the canvas:
pos = 0; // index position into imagedata array
xoff = width / 3; // offsets to "center"
yoff = height / 3;
// walk left-to-right, top-to-bottom; it's the
// same as the ordering in the imagedata array:
for (y = 0; y < height; y++) {
for (x = 0; x < width; x++) {
// calculate sine based on distance
x2 = x - xoff;
y2 = y - yoff;
d = Math.sqrt(x2*x2 + y2*y2);
t = Math.sin(d/6.0);
// calculate RGB values based on sine
r = t * 200;
g = 125 + t * 80;
b = 235 + t * 20;
// set red, green, blue, and alpha:
imageData.data[pos++] = Math.max(0,Math.min(255, r));
imageData.data[pos++] = Math.max(0,Math.min(255, g));
imageData.data[pos++] = Math.max(0,Math.min(255, b));
imageData.data[pos++] = 255; // opaque alpha
}
}
And that'll give you one of these:
Notice in that case how I didn't bother calling my setPixel()
function—I just walked all the way through the array one element at a
time (indexed by the pos variable.)
What's going on there with all that Math.max() and Math.min() stuff?
That's clamping the value between 0 and 255, inclusive. The min() call
makes sure the value is less than 255, and then the max() call makes
sure that result is greater than 0. Every browser I tried would
automatically clamp the values in the array between 0 and 255, but the
HTML5 draft says thou shalt not use values outside that range, so you'd
better do the clamping yourself.
Exercise: write a script that generates a gradient that goes from black on the top to yellow on the bottom. Then modify it to also make the gradient go from black on the left to blue on the right.
There's just one final thing needed to wrap up this post, I feel, and that's to show manipulation of an existing canvas.
For this demo, we'll load an image, we'll draw that image onto the left
half of the canvas (using the
drawImage()
method), then we'll get the pixel data (with the
getImageData()
method), and copy the image to the right half of the canvas—except along
the way we'll be munging the colors to make it way more blue! Only about
87 million things get more exciting than that, yes?
There's a little bit of JavaScript code in here for loading the image,
and the drawing all takes place in the image's onload callback, which
is called when the image loading is complete.
<canvas id="cancan" width="400", height="200">Canvas Blue Goat</canvas>
<script type="text/javascript">
function imageLoaded(ev) {
element = document.getElementById("cancan");
c = element.getContext("2d");
im = ev.target; // the image, assumed to be 200x200
// read the width and height of the canvas
width = element.width;
height = element.height;
// stamp the image on the left of the canvas:
c.drawImage(im, 0, 0);
// get all canvas pixel data
imageData = c.getImageData(0, 0, width, height);
w2 = width / 2;
// run through the image, increasing blue, but filtering
// down red and green:
for (y = 0; y < height; y++) {
inpos = y * width * 4; // *4 for 4 ints per pixel
outpos = inpos + w2 * 4
for (x = 0; x < w2; x++) {
r = imageData.data[inpos++] / 3; // less red
g = imageData.data[inpos++] / 3; // less green
b = imageData.data[inpos++] * 5; // MORE BLUE
a = imageData.data[inpos++]; // same alpha
b = Math.min(255, b); // clamp to [0..255]
imageData.data[outpos++] = r;
imageData.data[outpos++] = g;
imageData.data[outpos++] = b;
imageData.data[outpos++] = a;
}
}
// put pixel data on canvas
c.putImageData(imageData, 0, 0);
}
im = new Image();
im.onload = imageLoaded;
im.src = "goat200.jpg"; // code assumes this image is 200x200
</script>
Exercise: modify the goat blueifier to instead perform Sobel edge detection on the image. (Follow this link for a tutorial and C code.)
Have fun!
References:
See Also:
I was just curious if using the putImageData() API method was faster for animation purposes?
@David It would depend on the animation, I think. For instance, if you use the CSS transform property to rotate a bitmap and put it on the screen in the DOM, I would think that had a chance of being faster than a putImageData() having done the rotation yourself. (That is, the path to the screen for the bitmap in the DOM might have been a step shorter.)
But I've seen examples with putImageData() that were absolutely fast enough.
One trick you can use is to pass the "dirty" parameters to putImageData(), so that only the updated part of the data is actually drawn.
Another trick (if you're drawing the same data over and over) is to draw your data to a small canvas, and then use drawImage() to stamp it on the display canvas. Again, my gut feeling is this would be faster than putPixelData() for the same usage, but I'm not really sure; it might be implementation-dependent, too.
I wrote a Mandelbrot and Julia set viewer using this technique.
You can see it here:
http://www.scale18.com/canvas2.html
@Kevin Cool! I put a link to it on my Mandelbrot Set article.
Thanks for the post.
Canvas is so disappointing... I cannot even rotate a single bitmap without extremely complex and processor intensive calculations... How can I ever build a seriously engaging user experience with it?
@Filippo Gregoretti Ah, but you can rotate a single bitmap very very quickly with canvas.
You'll want to set the rotation in the drawing context (context.rotate()), then drawImage() the image on the canvas. This uses the fast internal blitting to make it happen in a hurry.
So you can do 2D transformations very quickly with canvas, and even some kinds of compositing very quickly. Other things, like conditionally manipulating each pixel, you will have to do the slow way.
I'm making a javascript/html5/php/mysql online 2d strategy game. It will be posted at cyberempire.net. game site Don't mind the java stuff there, it was the plan to make it in java, but I thought I would try with javascript. My problem before was finding an easy way to rotate images, problem solved with canvas. But, I would appreciate it if anybody had a fast collision detection routine.
@Vidar If you're going HTML5, you also have the option to rotate individual DOM elements (like images) in CSS3 with the "transform" property. (But of course, using canvas is an option, as well.)
As for collision detection, check out this fantastic page at Gamasutra:
http://www.gamasutra.com/view/feature/3383/simple_intersection_tests_for_games.php
Page 5 talks about "Oriented Bounded Boxes" which might be along the lines of what you need.
Great article, and great writing style.
Nice article.
I have a question:
I modify your code as below:
c.translate(50,50);
// put pixel data on canvas
c.putImageData(imageData, 0, 0);
Why is the output image still in (0,0) not (50,50)?
I too have the same question as OrNot, although what I want to do is to read the pixel at (x,y) after a translate and rotation. Is this possible? perhaps there is a way to get the "real" coordinates of x and y after a translate and rotate has been done, and get the pixel at the "real" coordinates?
@OrNot getImageData() and putImageData() operate on a low level, referring to the pixels in the bitmap backing store, and are not affected by the drawing context.
In the spec: "The current path, transformation matrix, shadow attributes, global alpha, the clipping region, and global composition operator must not affect the getImageData() and putImageData() methods."
@michael This is tricky for a couple reasons. (If it's possible, given your needs, to read the pixel at x,y before the transform, it could be a lot better.)
First of all, the math isn't hard, and there are a number of JS libraries to do it. (I've used Sylvester: http://sylvester.jcoglan.com/ ). Basically, you have to run the math again in parallel to get the result.
However, canvas will render to a subpixel. This means that part of your color could be on one pixel, and the rest would be on another. It might be a while pixel, originally, but it might turn to several gray pixels after rotation.
Is it possible to get the raw pixel value of an 8-bit indexed png, not the associated RGBA? I have images where the values (0-255) in the raw pixels have an actual data meaning. Depending on the time of year, the actual index palette changes, (so the displayed colors change) but not the 0-255 values in each pixel. I can do this client side, but with overhead which I don't want.
What I want to do is a browser side calculation. So when a user mouses over an image, the x,y provides the pixel needed, and the 0-255 is extracted. This is then converted to a data value (using a know formula,) which can be displayed for the user (and of course, a coresponding lat long which can be determined since I know the outer lat/long bounds of the image)
@brock
I meant "I can do it server side,..."
@brock I'm unaware of a way to get this low level data from the PNG, short of writing a PNG decoder in JS, and even then, my JS-fu isn't strong enough to know a way to get the binary data in JS.
By the time the data gets exposed, it's already abstracted. :-/
Hi Brock, were you able to find a way to get the raw PNG values in JS? I have a similar requirement and want to use IMG if possible. Thanks!
-dave