2010-02-09

HTML5's canvas Part II: Pixel Manipulation

In an earlier post, I wrote about a few of the things HTML5's <canvas> element could do. This post is about one more of them: direct pixel manipulation. You can basically do the three following things:

1. Create an array of empty pixels, or
2. Get an array of pixels from an existing canvas, or
3. Set the pixels of a canvas from a pixel array

Once you have an array of pixels, you can manipulate it using the standard array accessors "[", "]". The pixels in the array are arranged in row-major order, and are represented as integers between 0 and 255, where each four-integer group represents the four color channels of one pixel: red, green, blue, and alpha (RGBA). What did I just say? Basically, it looks like this... say you have an image that's 3 pixels wide, and 2 pixels high. I know that's tiny, but I want to fit it all in the following super-busy diagram which shows the array indices for given elements of each pixel.

We'll get to a quick-and-easy calculation to find your way around the array in just a moment, but first, how does one create a pixel data array to be used on a canvas? You use the drawing context's createImageData() method. (Use getImageData() to get the array from an existing canvas.)

<canvas id="canvas1" width="100", height="100"></canvas>

<script type="text/javascript">
    element = document.getElementById("canvas1");
    c = element.getContext("2d");

    // read the width and height of the canvas
    width = element.width;
    height = element.height;

    // create a new batch of pixels with the same
    // dimensions as the image:
    imageData = c.createImageData(width, height);

It's going to be useful later to know the width of the image data, and fortunately that's baked into the ImageData object:

    imageData.width   // width of the image data in pixels
    imageData.height  // height of the image data in pixels
    imageData.data    // pixel data array of (width*height*4) elements</pre>

Now we have what we need to write a pixel-setting routine:

function setPixel(imageData, x, y, r, g, b, a) {
    index = (x + y * imageData.width) * 4;
    imageData.data[index + 0] = r;
    imageData.data[index + 1] = g;
    imageData.data[index + 2] = b;
    imageData.data[index + 3] = a;
}

There you can see how we first calculate the pixel "position" in the array (which is the row number y times the number of pixels in each row (the width), plus the column number x) and then we multiply it by 4 to get the first channel for that pixel, namely red, because each pixel takes up 4 array elements.

Eat your heart out, Jackson Pollock.

Exercise: write a getPixel(x,y) function that returns the RGBA values for a given pixel as an array, or as a packed 32-bit number (0x_RRGGBBAA_).

We've seen how to create an image pixel data array, and we've seen how to access it. The only thing left to do is to "paste" the pixels back onto the canvas so that we see them. We do this with a call to putImageData(), to which we pass the image data object, plus an X and Y offset at which to stamp the new data. Let's use this to produce a canvas that is covered with randomly-colored pixels:

<canvas id="canvas1" width="100", height="100">
Random Canvas
</canvas>

<script type="text/javascript">
function setPixel(imageData, x, y, r, g, b, a) {
    index = (x + y * imageData.width) * 4;
    imageData.data[index+0] = r;
    imageData.data[index+1] = g;
    imageData.data[index+2] = b;
    imageData.data[index+3] = a;
}

element = document.getElementById("canvas1");
c = element.getContext("2d");

// read the width and height of the canvas
width = element.width;
height = element.height;

// create a new pixel array
imageData = c.createImageData(width, height);

// draw random dots
for (i = 0; i < 10000; i++) {
    x = Math.random() * width | 0; // |0 to truncate to Int32
    y = Math.random() * height | 0;
    r = Math.random() * 256 | 0;
    g = Math.random() * 256 | 0;
    b = Math.random() * 256 | 0;
    setPixel(imageData, x, y, r, g, b, 255); // 255 opaque
}

// copy the image data back onto the canvas
c.putImageData(imageData, 0, 0); // at coords 0,0

</script>

But that's so ugly. Let's do something a little prettier, by setting the color of each pixel depending on the sine of the distance it is away from some point on the canvas:

pos = 0; // index position into imagedata array

xoff = width / 3; // offsets to "center"
yoff = height / 3;

// walk left-to-right, top-to-bottom; it's the
// same as the ordering in the imagedata array:

for (y = 0; y < height; y++) {
	for (x = 0; x < width; x++) {
		// calculate sine based on distance
		x2 = x - xoff;
		y2 = y - yoff;
		d = Math.sqrt(x2*x2 + y2*y2);
		t = Math.sin(d/6.0);

		// calculate RGB values based on sine
		r = t * 200;
		g = 125 + t * 80;
		b = 235 + t * 20;

		// set red, green, blue, and alpha:
		imageData.data[pos++] = Math.max(0,Math.min(255, r));
		imageData.data[pos++] = Math.max(0,Math.min(255, g));
		imageData.data[pos++] = Math.max(0,Math.min(255, b));
		imageData.data[pos++] = 255; // opaque alpha
	}
}

And that'll give you one of these:

Yummy splashy TRONy goodness. Why use JavaScript to generate the image? How about because the JavaScript is only about 1400 bytes long?

Notice in that case how I didn't bother calling my setPixel() function—I just walked all the way through the array one element at a time (indexed by the pos variable.)

What's going on there with all that Math.max() and Math.min() stuff? That's clamping the value between 0 and 255, inclusive. The min() call makes sure the value is less than 255, and then the max() call makes sure that result is greater than 0. Every browser I tried would automatically clamp the values in the array between 0 and 255, but the HTML5 draft says thou shalt not use values outside that range, so you'd better do the clamping yourself.

Exercise: write a script that generates a gradient that goes from black on the top to yellow on the bottom. Then modify it to also make the gradient go from black on the left to blue on the right.

There's just one final thing needed to wrap up this post, I feel, and that's to show manipulation of an existing canvas.

For this demo, we'll load an image, we'll draw that image onto the left half of the canvas (using the drawImage() method), then we'll get the pixel data (with the getImageData() method), and copy the image to the right half of the canvas—except along the way we'll be munging the colors to make it way more blue! Only about 87 million things get more exciting than that, yes?

Oh, goat! Why so blue?

There's a little bit of JavaScript code in here for loading the image, and the drawing all takes place in the image's onload callback, which is called when the image loading is complete.

<canvas id="cancan" width="400", height="200">Canvas Blue Goat</canvas>

<script type="text/javascript">
function imageLoaded(ev) {
    element = document.getElementById("cancan");
    c = element.getContext("2d");

    im = ev.target; // the image, assumed to be 200x200

    // read the width and height of the canvas
    width = element.width;
    height = element.height;

    // stamp the image on the left of the canvas:
    c.drawImage(im, 0, 0);

    // get all canvas pixel data
    imageData = c.getImageData(0, 0, width, height);

    w2 = width / 2;

    // run through the image, increasing blue, but filtering
    // down red and green:

    for (y = 0; y < height; y++) {
        inpos = y * width * 4; // *4 for 4 ints per pixel
        outpos = inpos + w2 * 4
        for (x = 0; x < w2; x++) {
            r = imageData.data[inpos++] / 3; // less red
            g = imageData.data[inpos++] / 3; // less green
            b = imageData.data[inpos++] * 5; // MORE BLUE
            a = imageData.data[inpos++];     // same alpha

            b = Math.min(255, b); // clamp to [0..255]

            imageData.data[outpos++] = r;
            imageData.data[outpos++] = g;
            imageData.data[outpos++] = b;
            imageData.data[outpos++] = a;
        }
    }

    // put pixel data on canvas
    c.putImageData(imageData, 0, 0);
}

im = new Image();
im.onload = imageLoaded;
im.src = "goat200.jpg"; // code assumes this image is 200x200

</script>

Exercise: modify the goat blueifier to instead perform Sobel edge detection on the image. (Follow this link for a tutorial and C code.)

Have fun!

References:

• HTML5 canvas JavaScript draft spec

See Also:

• HTML5 canvas Part 1