This post is a continuation of a series of posts about WebGL. The first started with fundamentals and the previous was about animation.

How do we apply textures in WebGL? You could probably derive how by reading the articles on image processing but it will probably be easier to understand if we go over it in more detail.

The first thing we need to do is adjust our shaders to use textures. Here are the changes to the vertex shader. We need to pass in texture coordinates. In this case we just pass them straight through to the fragment shader.

#version 300 es
in vec4 a_position;
*in vec2 a_texcoord;

uniform mat4 u_matrix;

+// a varying to pass the texture coordinates to the fragment shader
+out vec2 v_texcoord;

void main() {
  // Multiply the position by the matrix.
  gl_Position = u_matrix * a_position;

+  // Pass the texcoord to the fragment shader.
+  v_texcoord = a_texcoord;
}

In the fragment shader we declare a uniform sampler2D which lets us reference a texture. We use the texture coordinates passed from the vertex shader and we call texture to look up a color from that texture.

#version 300 es
precision highp float;

// Passed in from the vertex shader.
*in vec2 v_texcoord;

*// The texture.
*uniform sampler2D u_texture;

out vec4 outColor;

void main() {
*   outColor = texture(u_texture, v_texcoord);
}

We need to setup the texture coordinates

// look up where the vertex data needs to go.
var positionAttributeLocation = gl.getAttribLocation(program, "a_position");
*var texcoordAttributeLocation = gl.getAttribLocation(program, "a_texcoord");

...

*// create the texcoord buffer, make it the current ARRAY_BUFFER
*// and copy in the texcoord values
*var texcoordBuffer = gl.createBuffer();
*gl.bindBuffer(gl.ARRAY_BUFFER, texcoordBuffer);
*setTexcoords(gl);
*
*// Turn on the attribute
*gl.enableVertexAttribArray(texcoordAttributeLocation);
*
*// Tell the attribute how to get data out of texcoordBuffer (ARRAY_BUFFER)
*var size = 2;          // 2 components per iteration
*var type = gl.FLOAT;   // the data is 32bit floating point values
*var normalize = true;  // convert from 0-255 to 0.0-1.0
*var stride = 0;        // 0 = move forward size * sizeof(type) each iteration to get the next texcoord
*var offset = 0;        // start at the beginning of the buffer
*gl.vertexAttribPointer(
*    texcoordAttributeLocation, size, type, normalize, stride, offset);

And you can see the coordinates we're using which are mapping the entire texture to each quad on our 'F'.

*// Fill the buffer with texture coordinates for the F.
*function setTexcoords(gl) {
*  gl.bufferData(
*      gl.ARRAY_BUFFER,
*      new Float32Array([
*        // left column front
*        0, 0,
*        0, 1,
*        1, 0,
*        0, 1,
*        1, 1,
*        1, 0,
*
*        // top rung front
*        0, 0,
*        0, 1,
*        1, 0,
*        0, 1,
*        1, 1,
*        1, 0,
* ...
*       ]),
*       gl.STATIC_DRAW);

We also need a texture. We could make one from scratch but in this case let's load an image since that's probably the most common way.

Here's the image we're going to use

What an exciting image! Actually an image with an 'F' on it has a clear direction so it's easy to tell if it's turned or flipped etc when we use it as a texture.

The thing about loading an image is it happens asynchronously. We request the image to be loaded but it takes a while for the browser to download it. There are generally 2 solutions to this. We could make the code wait until the texture has downloaded and only then start drawing. The other solution is to make up some texture to use until the image is downloaded. That way we can start rendering immediately. Then, once the image has been downloaded we copy the image to the texture. We'll use that method below.

*// Create a texture.
*var texture = gl.createTexture();
*gl.bindTexture(gl.TEXTURE_2D, texture);
*
*// Fill the texture with a 1x1 blue pixel.
*gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE,
*              new Uint8Array([0, 0, 255, 255]));
*
*// Asynchronously load an image
*var image = new Image();
*image.src = "resources/f-texture.png";
*image.addEventListener('load', function() {
*  // Now that the image has loaded make copy it to the texture.
*  gl.bindTexture(gl.TEXTURE_2D, texture);
*  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA,gl.UNSIGNED_BYTE, image);
*  gl.generateMipmap(gl.TEXTURE_2D);
*});

And here it is

What if we wanted to just use a part of the texture across the front of the 'F'? Textures are referenced with "texture coordinates" and texture coordinates go from 0.0 to 1.0 from left to right across the texture and 0.0 to 1.0 from the first pixel on the first line to the last pixel on the last line. Notice I didn't stay top or bottom. Top and bottom make no sense in texture space because until you draw something and orient it there is no top and bottom. What matters is you supply texture data to WebGL. The start of that data starts at texture coordinate 0,0 and the end of that data is at 1,1

I loaded the texture into photoshop and looked up the various coordinates in pixels.

To convert from pixel coordinates to texture coordinates we can just use

texcoordX = pixelCoordX / (width  - 1)
texcoordY = pixelCoordY / (height - 1)

Here are the texture coordinates for the front.

// left column front
 38 / 255,  44 / 255,
 38 / 255, 223 / 255,
113 / 255,  44 / 255,
 38 / 255, 223 / 255,
113 / 255, 223 / 255,
113 / 255,  44 / 255,

// top rung front
113 / 255, 44 / 255,
113 / 255, 85 / 255,
218 / 255, 44 / 255,
113 / 255, 85 / 255,
218 / 255, 85 / 255,
218 / 255, 44 / 255,

// middle rung front
113 / 255, 112 / 255,
113 / 255, 151 / 255,
203 / 255, 112 / 255,
113 / 255, 151 / 255,
203 / 255, 151 / 255,
203 / 255, 112 / 255,

I also used similar texture coordinates for the back. And here it is.

Not a very exciting display but hopefully it demonstrates how to use texture coordinates. If you're making geometry in code (cubes, spheres, etc) it's usually pretty easy to compute whatever texture coordinates you want. On the other hand if you're getting 3d models from 3d modeling software like Blender, Maya, 3D Studio Max, then your artists (or you) will adjust texture coordinates in those packages using a UV editor.

So what happens if we use texture coordinates outside the 0.0 to 1.0 range. By default WebGL repeats the texture. 0.0 to 1.0 is one 'copy' of the texture. 1.0 to 2.0 is another copy. even -4.0 to -3.0 is yet another copy. Let's display a plane using these texture coordinates.

 -3, -1,
  2, -1,
 -3,  4,
 -3,  4,
  2, -1,
  2,  4,

and here it is

You can tell WebGL to not repeat the texture in a certain direction by using CLAMP_TO_EDGE. For example

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

you can also tell WebGL to mirror the texture when it repeat use gl.MIRRORED_REPEAT. Click the buttons in the sample above to see the difference.

You might have noticed a call to gl.generateMipmap back when we loaded the texture. What is that for?

Imagine we had this 16x16 pixel texture.

Now imagine we tried to draw that texture on a polygon 2x2 pixels big on the screen. What colors should we make those 4 pixels? There are 256 pixels to choose from. In Photoshop if you scaled a 16x16 pixel image to 2x2 it would average the 8x8 pixels in each corner to make the 4 pixels in a 2x2 image. Unfortunately reading 64 pixels and averaging them all together would be way too slow for a GPU. In fact imagine if you had a 2048x2084 pixel texture and you tried to draw it 2x2 pixels. To do what Photoshop does for each of the 4 pixels in the 2x2 result it would have to average 1024x1024 pixel or 1 million pixels times 4. That's way way too much to do and still be fast.

So what the GPU does is it uses a mipmap. A mipmap is a collection of progressively smaller images, each one 1/4th the size of the previous one. The mipmap for the 16x16 texture above would look something like this.

Generally each smaller level is just a bilinear interpolation of the previous level and that's what gl.generateMipmap does. It looks at the biggest level and generates all the smaller levels for you. Of course you can supply the smaller levels yourself if you want.

Now if you try to draw that 16x16 pixel texture only 2x2 pixels on the screen WebGL can select the mip that's 2x2 which has already been averaged from the previous mips.

You can choose what WebGL does by setting the texture filtering for each texture. There are 6 modes

• NEAREST = choose 1 pixel from the biggest mip
• LINEAR = choose 4 pixels from the biggest mip and blend them
• NEAREST_MIPMAP_NEAREST = choose the best mip, then pick one pixel from that mip
• LINEAR_MIPMAP_NEAREST = choose the best mip, then blend 4 pixels from that mip
• NEAREST_MIPMAP_LINEAR = choose the best 2 mips, choose 1 pixel from each, blend them
• LINEAR_MIPMAP_LINEAR = choose the best 2 mips. choose 4 pixels from each, blend them

You can see the importance of mips in these 2 examples. The first one shows that if you use NEAREST or LINEAR and only pick from the largest image then you'll get a lot of flickering because as things move, for each pixel it draws it has to pick a single pixel from the largest image. That changes depending on the size and position and so sometimes it will pick one pixel, other times a different one and so it flickers.

The example above is exaggerated to show the issue. Notice how much the ones on the left and middle flicker where as the ones on the right flicker less. The ones on the right also have blended colors since they are using the mips. The smaller you draw the texture the further apart WebGL is going to pick pixels. That's why for example the bottom middle one, even though it's using LINEAR and blending 4 pixels it flickers different colors because those 4 pixels are from different corners of the 16x16 image depending on which 4 are picked you'll get a different color. The one on the bottom right though stays a consistent color because it's using the 2nd to the smallest mip.

This second example shows polygons that go deep into the distance.

The 6 beams going into the screen are using the 6 filtering modes listed above. The top left beam is using NEAREST and you can see it's clearly very blocky. The top middle is using LINEAR and it's not much better. The top right is using NEAREST_MIPMAP_NEAREST. Click on the image to switch to a texture where every mip is a different color and you'll easily see where it chooses to use a specific mip. The bottom left is using LINEAR_MIPMAP_NEAREST meaning it picks the best mip and then blends 4 pixels within that mip. You can still see a clear area where it switches from one mip to the next mip. The bottom middle is using NEAREST_MIPMAP_LINEAR meaning picking the best 2 mips, picking one pixel from each and blending them. If you look close you can see how it's still blocky, especially in the horizontal direction. The bottom right is using LINEAR_MIPMAP_LINEAR which is picking the best 2 mips, picking 4 pixels from each, and blends all 8 pixels.

You might be thinking why would you ever pick anything other than LINEAR_MIPMAP_LINEAR which is arguably the best one. There are many reasons. One is that LINEAR_MIPMAP_LINEAR is the slowest. Reading 8 pixels is slower than reading 1 pixel. On modern GPU hardware it's probably not an issue if you are only using 1 texture at a time but modern games might use 2 to 4 textures at once. 4 textures * 8 pixels per texture = needing to read 32 pixels for every pixel drawn. That's going to be slow. Another reason is if you're trying to achieve a certain effect. For example if you want something to have that pixelated retro look maybe you want to use NEAREST. Mips also take memory. In fact they take 33% more memory. That can be a lot of memory especially for a very large texture like you might use on a title screen of a game. If you are never going to draw something smaller than the largest mip why waste memory for the smaller mips. Instead just use NEAREST or LINEAR as they only ever use the first mip.

To set filtering you call gl.texParameter like this

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

TEXTURE_MIN_FILTER is the setting used when the size you are drawing is smaller than the largest mip. TEXTURE_MAG_FILTER is the setting used when the size you are drawing is larger than the largest mip. For TEXTURE_MAG_FILTER only NEAREST and LINEAR are valid settings.

Something to be aware of, WebGL2 requires textures to be "texture complete" otherwise they won't render. "texture complete" means that either

1. You've set the filtering so it only uses the first mip level which means setting the TEXTURE_MIN_FILTER to either LINEAR or NEAREST.

2. If you are using mips then they need to be the correct sizes and you have to provide ALL OF THEM down to the 1x1 size.

The easiest way to do that is to call gl.generateMipmap. Otherwise if you provide your own mips you need to provide all of them or you'll get an error.

A common question is "How do I apply a different image to each face of a cube?". For example let's say we had these 6 images.

3 answers come to mind

1. make a complicated shader that references 6 textures and pass in some extra per vertex info into the vertex shader that gets passed to the fragment shader to decide which texture to use. DON'T DO THIS! A little thought would make it clear that you'd end up having to write tons of different shaders if you wanted to do the same thing for different shapes with more sides etc.

2. draw 6 planes instead of a cube. This is a common solution. It's not bad but it also only really works for small shapes like a cube. If you had a sphere with 1000 quads and you wanted to put a different texture on each quad you'd have to draw 1000 planes and that would be slow.

3. The, dare I say, best solution is to put all of the images in 1 texture and use texture coordinates to map a different part of the texture to each face of the cube. This is the technique that pretty much all high performance apps (read games) use. So for example we'd put all the images in one texture possibly like this

and then use a different set of texture coordinates for each face of the cube.

    // select the top left image
    0   , 0  ,
    0   , 0.5,
    0.25, 0  ,
    0   , 0.5,
    0.25, 0.5,
    0.25, 0  ,
    // select the top middle image
    0.25, 0  ,
    0.5 , 0  ,
    0.25, 0.5,
    0.25, 0.5,
    0.5 , 0  ,
    0.5 , 0.5,
    // select to top right image
    0.5 , 0  ,
    0.5 , 0.5,
    0.75, 0  ,
    0.5 , 0.5,
    0.75, 0.5,
    0.75, 0  ,
    // select the bottom left image
    0   , 0.5,
    0.25, 0.5,
    0   , 1  ,
    0   , 1  ,
    0.25, 0.5,
    0.25, 1  ,
    // select the bottom middle image
    0.25, 0.5,
    0.25, 1  ,
    0.5 , 0.5,
    0.25, 1  ,
    0.5 , 1  ,
    0.5 , 0.5,
    // select the bottom right image
    0.5 , 0.5,
    0.75, 0.5,
    0.5 , 1  ,
    0.5 , 1  ,
    0.75, 0.5,
    0.75, 1  ,

And we get

This style of applying multiple images using 1 texture is often called a texture atlas. It's best because there's just 1 texture to load, the shader stays simple as it only has to reference 1 texture, and it only requires 1 draw call to draw the shape instead of 1 draw call per texture as it might if we split it into planes.

A few other very important things you might want to know about textures. One is how texture unit state works. One is how to use 2 or more textures at once. The other is how to use images from other domains.

Next up lets start simplifying with less code more fun.