Textures and Framebuffer Objects

2010, Dr. Lawlor, CS 481/681, CS, UAF

The key trick in a lot of modern graphics applications is "multi-pass rendering": we run one shader, and then read the results in a subsequent shader. The key data structure here is a texture--in GLSL, about the only thing you can read efficiently is texture pixels!

Setting up a Texture

A "texture" in OpenGL is just an image; a 2D array of color pixels. Textures live in graphics card memory, where the graphic card can access them at absurd speed. But this means rather than just a plain C++ array, a texture has to be referenced via an integer "handle", and all your reads and writes to textures need to go through OpenGL. Like most of OpenGL, you operate on a texture by first "bind"ing the texture, which makes all subsequent texture calls modify that texture.

Here's how you set up a new texture of size "w" by "h" pixels in OpenGL:

	GLuint myTex=0; /* openGL "texture handle" */
	glGenTextures(1,&myTex); /* make a new texture */
	glBindTexture(GL_TEXTURE_2D,myTex); /* modify (or draw) our texture */
	GLenum format=GL_RGBA8; /* color, 8 bits per channel */
	glTexImage2D(GL_TEXTURE_2D,0,format, w,h,0, GL_RGBA,GL_UNSIGNED_BYTE,0); /* the texture is w x h pixels */
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,GL_LINEAR); 
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_LINEAR); /* no mipmaps (see below) */
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);  /* no repeat (see below) */
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
	glBindTexture(GL_TEXTURE_2D,0); /* back to default texture */

The last parameter to glTexImage2D can also be a pointer to a CPU array of pixels. If you pass zero, the texture is uninitialized, which is fine as long as you write to the texture using a framebuffer object (see below).

Rendering from a Texture (Texture Reads)

Here's how to draw a texture onscreen: basically just bind the texture, enable texturing, and set up texture coordinates before drawing your vertices:

	glBindTexture(GL_TEXTURE_2D,myTex);
	glEnable(GL_TEXTURE_2D); /* turn on texturing */
	glColor4f(1,1,1,1); /* gets multiplied by texture colors */
	glBegin(GL_QUADS);
	glTexCoord2f(0,0); glVertex3f(0,0,0);
	glTexCoord2f(1,0); glVertex3f(sz,0,0);
	glTexCoord2f(1,1); glVertex3f(sz,sz,0);
	glTexCoord2f(0,1); glVertex3f(0,sz,0);
	glEnd();
	glDisable(GL_TEXTURE_2D);

From GLSL, you have to declare a "uniform sampler2D myTex;" and then look up colors with "vec4 c=texture2D(myTex,texCoords);" with a vec2 texCoords of texture coordinates. You also need to glBindTexture and set up the sampler uniform using "glUniform1iARB(glGetUniformLocationARB(program,"myTex"),0);" from C++. The 0 there is the current 'texture unit'; you can combine several textures at once by calling glActiveTexture to switch units before you bind your texture handle.

Rendering to a Texture (Texture Writes)

You can render pixels into a texture using a framebuffer object. Here's how you make a framebuffer object "fb" hooked up to render into "myTex":

	GLuint fb=0;
	glGenFramebuffersEXT(1, &fb);
	glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fb);
	glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, 
		GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_2D, myTex, 0);

Now you can render stuff into your texture like so:

	glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fb);
	glViewport(0,0,myTex_wid,myTex_ht);

	glMatrixMode(GL_PROJECTION);
	glLoadIdentity();
	/* From 0..1 texture coords to -1..+1 clip coords */
	glTranslatef(-1.0,-1.0,-1.0);
	glScalef(2.0,2.0,2.0);

	glMatrixMode(GL_MODELVIEW);
	glLoadIdentity();

	// ... glClear, glBegin / glEnd, etc will now render into myTex!

Be sure to reset the rendering state afterwards by binding the zero framebuffer object (the screen):

	glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);
	glViewport(0,0,glutGet(GLUT_WINDOW_WIDTH),glutGet(GLUT_WINDOW_HEIGHT));

If you want to enable GL_DEPTH, you need to make a depth texture (or a "renderbuffer") and attach that to your framebuffer as well.

Texture Coordinate Wrapping

You can change what happens outside normal texture coordinate bounds, on both the S (x axis) and T (y axis) texture coordinate axes.

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
Works like fract(texCoords). Texture coordinates wrap back around to 0.0 when they exceed 1.0, which causes the texture to repeat. This is the default and most common wrap mode.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
Works like clamp(texCoords,0.0+0.5/pixelSize,1.0-0.5/pixelSize), where "pixelSize" is the corresponding dimension of the texture in *texture* pixels. This has the effect of extending out the edge pixels. With GL_NEAREST filtering, it's identical to GL_CLAMP, but it's different for GL_LINEAR. I claim CLAMP_TO_EDGE is usually what you want if you don't want REPEAT mode.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
Works like clamp(texCoords,0.0,1.0). Texture coordinates outside the normal range are flattened to 0.0-1.0.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_BORDER);
Works like clamp(texCoords,0.0-0.5/pixelSize,1.0+0.5/pixelSize). The edge pixels linearly blend down to the "border color" (usually a transparent black), which then extends outward. This is probably what GL_CLAMP should have been.

Texture Filtering

You can also change how colors interpolate between pixels in the texture map when blowing up the texture map onscreen ("magnification" mode):

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,GL_NEAREST);
Return the color of the nearest pixel. Results in ugly boxy shapes, but it's really fast, and never blurry.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,GL_LINEAR);
Take a bilinear blend of neighboring pixels to create output values. Gives nice smooth output, but may be strange around the edges without GL_CLAMP_TO_BORDER or GL_REPEAT.

The same options exist when shrinking a texture map down onscreen ("minification" mode), along with extra "mipmap" options. You set up mipmaps above with glGenerateMipmapEXT(GL_TEXTURE_2D) on the current bound texture, or from the CPU side with gluBuild2DMipmaps or a whole set of glTexImage2D calls (one per level). BEWARE! If you enable one of these mipmap filtering modes, but you didn't set up all the mipmap levels in your texture, then OpenGL will IGNORE your texture, resulting in a white polygon!

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_LINEAR);
Linearly interpolate pixels in the mipmaps, and then linearly interpolate between mipmaps. The smoothest version by far, and the recommended one to avoid aliasing.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_NEAREST);
Linearly interpolate pixels in the mipmaps, but only use the closest mipmap level. Slightly faster than linear-linear on older hardware.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_NEAREST_MIPMAP_NEAREST);
Use nearest-neighbor pixels in the mipmaps, and only use the closest mipmap level. The ugliest approach, but good for checking for mipmap problems.
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,GL_NEAREST_MIPMAP_LINEAR);
Use nearest-neighbor pixels in the mipmaps, but blend between mipmap levels. Rarely useful.

Another cool and very easy feature is "anisotropic filtering", which combines several mipmaps to get sharper results from steeply tilted surfaces.

	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAX_ANISOTROPY_EXT,16);

Both the texture wrapping and filtering modes above apply to the currently bound texture--if you switch textures, your wrapping and filtering modes change too.

Deferred Shading Raytracing

The basic idea is to first render the ray-geometry intersection point P to a texture, then read adjacent pixels of the texture to determine the local geometry tangent vectors and hence surface normal. See the example code for the gory details.

This general notion of rendering to a texture, then reading various pixels of the texture to get work done, is extremely common and powerful!