Reading the OpenGL backbuffer to system memory

Sometimes you are in the need to read back the OpenGL backbuffer, or other framebuffers. In my case the question was how to read back a downsampled framebuffer resp. its texture to system memory. There are different methods for this and I wrote a small benchmark to test them on various systems.

UPDATE #1: A thing I’ve totally overlooked is that you can actually blit the backbuffer to the downsampled framebuffer directly, saving you the FBO overhead (memory, setup, rendering time). I’ve updated the page and the code reflecting that.
UPDATE #2: The code now works on Windows and Linux (via GLX). Maybe I’ll port it to GLES2 for the Raspberry Pi too… 🙂
UPDATE #3: The code works on the Raspberry Pi now using OpenGL ES 2.0. Reading the framebuffer is quite slow though even with an overclocked device. It’s not the actual reading that is slow, but that we have to render to another framebuffer first and back to the screen. Atm all buffers and textures use RGBA8888, which might slow things down. I’ll try RGB565 in the future. Also there are no PBOs and no glGetTexImage(), so only the glReadPixels method is working… If you have any ideas on how to make the code faster, I’d love to hear them.
UPDATE #4: Tinkered around with color formats a bit and updated the Pi’s firmware and MESA implementation. I’ve tried changing the FBOs color format to 16bit, but that didn’t change much. The secret seems to lie in using a RGB565 EGL display/framebuffer format with no alpha, no depth, no stencil and to remove the glDiscardFramebuffer calls. The FBO backbuffer and the downsampled FBO still have RGBA8888. Together with using a screen size of 640×480 this nearly quadrupled the frame rate. Blitting the FBO to the screen now seems to be much, much faster… I updated the code accordingly and also did some cleanup and found a safer way to get function addresses on Linux systems.
UPDATE #5: This is old. There are new options now, even for OpenGL.

---

Prequisites

I needed to downsample the framebuffer. This can be done by blitting to a smaller framebuffer that will be read back. For setting up framebuffers, see here. In your rendering loop, do:

// set viewport to window size
glViewport(0, 0, width, height);

// draw stuff here

// blit backbuffer to downsampled buffer
context->glBindFramebuffer(GL_READ_FRAMEBUFFER, 0);
context->glBindFramebuffer(GL_DRAW_FRAMEBUFFER, smallId);
context->glBlitFramebuffer(0, 0, width, height, 0, 0, smallWidth, smallHeight, GL_COLOR_BUFFER_BIT, GL_LINEAR);

// read back buffer content here (Method #1, #2, #3)
// unbind again
glBindFramebuffer(GL_FRAMEBUFFER, 0);

Blitting the framebuffer using glBlitFramebuffer isn’t much faster than binding a texture and rendering a quad to another framebuffer, but it has much less setup needed and you don’t find yourself setting up projection or modelview matrices and trashing the state.

Method #1 – glReadPixels

The first method is to use a standard glReadPixels. This is the common way to read the backbuffer since ancient OpenGL times. Do (depending on you framebuffer format):

// bind downsampled buffer for reading
glBindFramebuffer(GL_FRAMEBUFFER, smallId);
glReadPixels(0, 0, smallWidth, smallHeight, GL_RGBA, GL_UNSIGNED_BYTE, downsampleData);
// unbind buffer again 
glBindFramebuffer(GL_FRAMEBUFFER, 0);

The data will end up in downsampleData in system memory. Note that you need to allocate space for that data before!

Method #2 – glGetTexImage

The second method does not read the actual buffer, but the texture attached to it. That can be done using glGetTexImage. Do (depending on you framebuffer format):

// bind downsampled texture
glBindTexture(GL_TEXTURE_2D, downsampleTextureId);
// read from bound texture to CPU
glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE, downsampleData);
// unbind texture again
glBindTexture(GL_TEXTURE_2D, 0);

The data will end up in downsampleData in system memory again. Note that you need to allocate space for that data before!

Method #3 – PixelBufferObjects

This is the most complicated, but a slightly faster method than the previous two. You read the downsampled framebuffer using glReadPixels again, but now you read it to a PixelBuffer object asynchronously. For that you set up two PBOs and alternate between them so the GPU can go on rendering to the other while you process the first. The download of the data to system memory happens using DMA and does not block the CPU. To do this, first start up creating two PBOs:

int readIndex = 0;
int writeIndex = 1;
GLuint pbo[2];

// create PBOs to hold the data. this allocates memory for them too
glGenBuffers(2, pbo);
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo[0]);
glBufferData(GL_PIXEL_PACK_BUFFER, smallWidth, smallHeight * smallDepth, 0, GL_STREAM_READ);
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo[1]);
glBufferData(GL_PIXEL_PACK_BUFFER, smallWidth, smallHeight * smallDepth, 0, GL_STREAM_READ);

// unbind buffers for now
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);

Now that you have created the PBOs, go to your rendering loop and there you read back data to the PBOs there.

// bind downsampled fbo
downsample->bind();
// swap PBOs each frame
writeIndex = (writeIndex + 1) % 2;
readIndex = (writeIndex + 1) % 2;
// bind PBO to read pixels. This buffer is being copied from GPU to CPU memory
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo[writeIndex]);
// copy from framebuffer to PBO asynchronously. it will be ready in the NEXT frame
glReadPixels(0, 0, downsample->getWidth(), downsample->getHeight(), GL_RGBA, GL_UNSIGNED_BYTE, nullptr);
// now read other PBO which should be already in CPU memory
glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo[readIndex]);
// map buffer so we can access it
downsampleData = (unsigned char *)glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY);
if (downsampleData) {
    // ok. we have the data now available and could process it or copy it somewhere
    // ...
    // unmap the buffer again
    downsampleData = nullptr;
    glUnmapBuffer(GL_PIXEL_PACK_BUFFER);
}
// back to conventional pixel operation
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
// unbind downsampled fbo
downsample->unbind();
// now in the next readIndex and writeIndex will be swapped and we'll read from what is currently writeIndex...

The data will end up in downsampleData in system memory. This method seems to be slightly faster on some systems, but gain is highly dependent on the frame rate.

Results

The results depend on the system, method used and read color format. Here’s a table showing my benchmarking results. The first column is regular rendering to the backbuffer, no FBOs involved. Then follow the tests, every test first without actually reading the buffer, just rendering to the downsampled FBO. Then two tests reading the downsampled buffer to system memory using different formats:

System	No FBOs	#1 no reading	#1 BGRA	#1 RGBA	#2 no reading	#2 BGRA	#2 RGBA	#3 no reading	#3 BGRA	#3 RGBA
Intel Core i7 960 Nvidia Quadro 600, Driver 310.90 OpenGL 4.3, 640×480	0.39	0.54	0.70	0.67	0.54	0.88	0.87	0.55	0.64	0.55
Intel Core i5 480M AMD Radeon 5650M, Driver Catalyst 13.1 OpenGL 4.2, 640×480	0.30	0.41	0.80	0.77	0.41	0.81	0.81	0.42	0.51	0.49
Intel Core i5 480M Integrated HD Graphics, Driver 8.15.10.2827 OpenGL 2.1, 640×480	0.73	0.76	1.75	1.52	0.78	5.26	2.56	0.79	1.77	1.70

System	No FBOs	#1 no reading	#1 BGRA	#1 RGBA	#2 no reading	#2 BGRA	#2 RGBA	#3 no reading	#3 BGRA	#3 RGBA
Intel Core i5 480M Integrated HD Graphics, Driver Mesa DRI Intel Ironlake Mobile OpenGL 2.1, GLX 1.4, MESA 9.0.2, 640×480	1.13	1.46	3.25	16.91	1.97	3.40	16.77	1.89	4.22	17.05
Intel Core i3-2120T Integrated HD Graphics 2000, Driver Mesa DRI Intel Sandybridge Desktop x86/MMX/SSE2 OpenGL 3.0, GLX 1.4, MESA 9.0.2, 640×480	0.78	1.14	1.30	1.77	1.13	1.29	1.55	1.13	1.30	1.77

System	No FBOs	#1 no reading	#1 BGRA	#1 RGBA	#2 no reading	#2 BGRA	#2 RGBA	#3 no reading	#3 BGRA	#3 RGBA
Intel Core i5 480M Integrated HD Graphics, Driver Mesa DRI Intel Ironlake Mobile OpenGL 2.1, GLX 1.4, MESA 17.0.7, 640×480	1.08	1.04	1.48	1.42	1.03	1.44	1.42	1.04	1.07	1.07

System	No FBOs	#1 no reading	#1 BGRA	#1 RGBA	#2 no reading	#2 BGRA	#2 RGBA	#3 no reading	#3 BGRA	#3 RGBA
Intel Core i7 7500U Mesa DRI Intel(R) HD Graphics 620 (Kabylake GT2) OpenGL 3.0, GLX 1.4, MESA 11.2.0, 640×480	0.20	0.22	0.28	0.29	0.22	0.29	0.28	0.22	0.25	0.24

System	No FBOs	#1 no reading	#1 BGRA	#1 RGBA	#2 no reading	#2 BGRA	#2 RGBA	#3 no reading	#3 BGRA	#3 RGBA
Raspberry Pi OpenGL ES 2.0, EGL 1.4, 640×480	2.37	4.49	4.54	6.25	–	–	–	–	–	–

Source code

The source code, a CMake build file and the benchmark results can be found on GitHub and building should work at least on Windows 7, Ubuntu 12.10 (via GLX on X11) and on a current Raspian for the Raspberry Pi (via EGL). The beef is in the Test_… classes. To do a benchmark, run the application from the command line and wait for it to finish. It’ll spit out some frame time values for the different methods and color formats.
I’d love to get your feedback on how else to read to system memory or maybe speed up the methods. Just drop me a comment.