LearnOpenGL

Anti-Aliasing.html (20299B)

---

     <div id="content">
     <h1 id="content-title">Anti Aliasing</h1>
 <h1 id="content-url" style='display:none;'>Advanced-OpenGL/Anti-Aliasing</h1>
 <p>
   Somewhere in your adventurous rendering journey you probably came across some jagged saw-like patterns along the edges of your models. The reason these <def>jagged edges</def> appear is due to how the rasterizer transforms the vertex data into actual fragments behind the scene. An example of what these jagged edges look like can already be seen when drawing a simple cube:
 </p>
 
 <img src="/img/advanced/anti_aliasing_aliasing.png" class="clean" alt="Container with visible aliasing"/>
 
 <p>
   While not immediately visible, if you take a closer look at the edges of the cube you'll see a jagged pattern. If we zoom in you'd see the following:
 </p>
 
 <img src="/img/advanced/anti_aliasing_zoomed.png" alt="Zoomed in on contanier with visible aliasing"/>
 
 <p>
   This is clearly not something we want in a final version of an application. This effect, of clearly seeing the pixel formations an edge is composed of, is called <def>aliasing</def>. There are quite a few techniques out there called <def>anti-aliasing</def> techniques that fight this aliasing behavior by producing <em>smoother</em> edges.
 </p>
 
 <p>
   At first we had a technique called <def>super sample anti-aliasing</def> (SSAA) that temporarily uses a much higher resolution render buffer to render the scene in (super sampling). Then when the full scene is rendered, the resolution is downsampled back to the normal resolution. This <em>extra</em> resolution was used to prevent these jagged edges. While it did provide us with a solution to the aliasing problem, it came with a major performance drawback since we have to draw <strong>a lot</strong> more fragments than usual. This technique therefore only had a short glory moment.
 </p>
 
 <p>
   This technique did give birth to a more modern technique called <def>multisample anti-aliasing</def> or MSAA that borrows from the concepts behind SSAA while implementing a much more efficient approach. In this chapter we'll be extensively discussing this MSAA technique that is built-in in OpenGL.
 </p>
 
 <h2>Multisampling</h2>
 <p>
   To understand what multisampling is and how it works into solving the aliasing problem we first need to delve a bit further into the inner workings of OpenGL's rasterizer.
 </p>
 
 <p>
   The rasterizer is the combination of all algorithms and processes that sit between your final processed vertices and the fragment shader. The rasterizer takes all vertices belonging to a single primitive and transforms this to a set of fragments. Vertex coordinates can theoretically have any coordinate, but fragments can't since they are bound by the resolution of your screen. There will almost never be a one-on-one mapping between vertex coordinates and fragments, so the rasterizer has to determine in some way what fragment/screen-coordinate each specific vertex will end up at.
 </p>
 
 <img src="/img/advanced/anti_aliasing_rasterization.png" alt="Image of a triangle being rasterized in OpenGL"/>
 
 <p>
   Here we see a grid of screen pixels where the center of each pixel contains a <def>sample point</def> that is used to determine if a pixel is covered by the triangle. The red sample points are covered by the triangle and a fragment will be generated for that covered pixel. Even though some parts of the triangle edges still enter certain screen pixels, the pixel's sample point is not covered by the inside of the triangle so this pixel won't be influenced by any fragment shader.
 </p>
 
 <p>
   You can probably already figure out the origin of aliasing right now. The complete rendered version of the triangle would look like this on your screen:
 </p>
 
 <img src="/img/advanced/anti_aliasing_rasterization_filled.png"  alt="Filled triangle as a result of rasterization in OpenGL"/>
 
 <p>
   Due to the limited amount of screen pixels, some pixels will be rendered along an edge and some won't. The result is that we're rendering primitives with non-smooth edges giving rise to the jagged edges we've seen before. 
 </p>
 
 <p>
   What multisampling does, is not use a single sampling point for determining coverage of the triangle, but multiple sample points (guess where it got its name from). Instead of a single sample point at the center of each pixel we're going to place <code>4</code> <def>subsamples</def> in a general pattern and use those to determine pixel coverage. 
 </p>
 
 <img src="/img/advanced/anti_aliasing_sample_points.png" class="clean" alt="Multisampling in OpenGL"/>
 
 <p>
   The left side of the image shows how we would normally determine the coverage of a triangle. This specific pixel won't run a fragment shader (and thus remains blank) since its sample point wasn't covered by the triangle. The right side of the image shows a multisampled version where each pixel contains <code>4</code> sample points. Here we can see that only <code>2</code> of the sample points cover the triangle.
 </p>
 
 <note>
   The amount of sample points can be any number we'd like with more samples giving us better coverage precision. 
 </note>
 
 <p>
   This is where multisampling becomes interesting. We determined that <code>2</code> subsamples were covered by the triangle so the next step is to determine a color for this specific pixel. Our initial guess would be that we run the fragment shader for each covered subsample and later average the colors of each subsample per pixel. In this case we'd run the fragment shader twice on the interpolated vertex data at each subsample and store the resulting color in those sample points. This is (fortunately) <strong>not</strong> how it works, because this would mean we need to run a lot more fragment shaders than without multisampling, drastically reducing performance. 
 </p>
 
 <p>
   How MSAA really works is that the fragment shader is only run <strong>once</strong> per pixel (for each primitive) regardless of how many subsamples the triangle covers; the fragment shader runs with the vertex data interpolated to the <strong>center</strong> of the pixel. MSAA then uses a larger depth/stencil buffer to determine subsample coverage. The number of subsamples covered determines how much the pixel color contributes to the framebuffer. Because only 2 of the 4 samples were covered in the previous image, half of the triangle's color is mixed with the framebuffer color (in this case the clear color) resulting in a light blue-ish color.
 </p>
 
 <p>
   The result is a higher resolution buffer (with higher resolution depth/stencil) where all the primitive edges now produce a smoother pattern. Let's see what multisampling looks like when we determine the coverage of the earlier triangle:
 </p>
 
 <img src="/img/advanced/anti_aliasing_rasterization_samples.png"  alt="Rasterization of triangle with multisampling in OpenGL"/>
 
 <p>
   Here each pixel contains 4 subsamples (the irrelevant samples were hidden) where the blue subsamples are covered by the triangle and the gray sample points aren't. Within the inner region of the triangle all pixels will run the fragment shader once where its color output is stored directly in the framebuffer (assuming no blending). At the inner edges of the triangle however not all subsamples will be covered so the result of the fragment shader won't fully contribute to the framebuffer. Based on the number of covered samples, more or less of the triangle fragment's color ends up at that pixel.
 </p>
 
 <p>
   For each pixel, the less subsamples are part of the triangle, the less it takes the color of the triangle. If we were to fill in the actual pixel colors we get the following image:
 </p>
 
 <img src="/img/advanced/anti_aliasing_rasterization_samples_filled.png"  alt="Rasterized triangle with multisampling in OpenGL"/>
 
 <p>
   The hard edges of the triangle are now surrounded by colors slightly lighter than the actual edge color, which causes the edge to appear smooth when viewed from a distance.
 </p>
 
 <p>
   Depth and stencil values are stored per subsample and, even though we only run the fragment shader once, color values are stored per subsample as well for the case of multiple triangles overlapping a single pixel. For depth testing the vertex's depth value is interpolated to each subsample before running the depth test, and for stencil testing we store the stencil values per subsample. This does mean that the size of the buffers are now increased by the amount of subsamples per pixel.
 </p>
 
 <p>
   What we've discussed so far is a basic overview of how multisampled anti-aliasing works behind the scenes. The actual logic behind the rasterizer is a bit more complicated, but this brief description should be enough to understand the concept and logic behind multisampled anti-aliasing; enough to delve into the practical aspects.
 </p>
 
 <h2>MSAA in OpenGL</h2>
 <p>
   If we want to use MSAA in OpenGL we need to use a buffer that is able to store more than one sample value per pixel. We need a new type of buffer that can store a given amount of multisamples and this is called a <def>multisample buffer</def>. 
 </p>
 
 <p>
   Most windowing systems are able to provide us a multisample buffer instead of a default buffer. GLFW also gives us this functionality and all we need to do is <em>hint</em> GLFW that we'd like to use a multisample buffer with N samples instead of a normal buffer by calling <fun><function id='18'>glfwWindowHint</function></fun> before creating the window:
 </p>
 
 <pre class="cpp"><code>
 <function id='18'>glfwWindowHint</function>(GLFW_SAMPLES, 4);
 </code></pre>
 
 <p>
   When we now call <fun><function id='20'>glfwCreateWindow</function></fun> we create a rendering window, but this time with a buffer containing 4 subsamples per screen coordinate. This does mean that the size of the buffer is increased by 4.
 </p>
 
 <p>
   Now that we asked GLFW for multisampled buffers we need to enable multisampling by calling <fun><function id='60'>glEnable</function></fun> with <var>GL_MULTISAMPLE</var>. On most OpenGL drivers, multisampling is enabled by default so this call is then a bit redundant, but it's usually a good idea to enable it anyways. This way all OpenGL implementations have multisampling enabled.
 </p>
 
 <pre><code>
 <function id='60'>glEnable</function>(GL_MULTISAMPLE);  
 </code></pre>
 
 <p>
   Because the actual multisampling algorithms are implemented in the rasterizer in your OpenGL drivers there's not much else we need to do. If we now were to render the green cube from the start of this chapter we should see smoother edges:
 </p>
 
 <img src="/img/advanced/anti_aliasing_multisampled.png" class="clean" alt="Image of a multisampled cube in OpenGL"/>
 
 <p>
   The cube does indeed look a lot smoother and the same will apply for any other object you're drawing in your scene. You can find the source code for this simple example <a href="/code_viewer_gh.php?code=src/4.advanced_opengl/11.1.anti_aliasing_msaa/anti_aliasing_msaa.cpp" target="_blank">here</a>.
 </p>
 
 <h2>Off-screen MSAA</h2>
 <p>
   Because GLFW takes care of creating the multisampled buffers, enabling MSAA is quite easy. If we want to use our own framebuffers however, we have to generate the multisampled buffers ourselves; now we <strong>do</strong> need to take care of creating multisampled buffers.
 </p>
 
 <p>
   There are two ways we can create multisampled buffers to act as attachments for framebuffers: texture attachments and renderbuffer attachments. Quite similar to normal attachments like we've discussed in the <a href="https://learnopengl.com/Advanced-OpenGL/Framebuffers" target="_blank">framebuffers</a> chapter.
 </p>
 
 <h3>Multisampled texture attachments</h3>
 <p>
   To create a texture that supports storage of multiple sample points we use <fun><function id='101'><function id='52'>glTexImage2D</function>Multisample</function></fun> instead of <fun><function id='52'>glTexImage2D</function></fun> that accepts <var>GL_TEXTURE_2D_MULTISAPLE</var> as its texture target:
 </p>
 
 <pre class="cpp"><code>
 <function id='48'>glBindTexture</function>(GL_TEXTURE_2D_MULTISAMPLE, tex);
 <function id='101'><function id='52'>glTexImage2D</function>Multisample</function>(GL_TEXTURE_2D_MULTISAMPLE, samples, GL_RGB, width, height, GL_TRUE);
 <function id='48'>glBindTexture</function>(GL_TEXTURE_2D_MULTISAMPLE, 0);  
 </code></pre>
 
 <p>
   The second argument sets the number of samples we'd like the texture to have. If the last argument is set to <var>GL_TRUE</var>, the image will use identical sample locations and the same number of subsamples for each texel.
 </p>
 
 <p>
   To attach a multisampled texture to a framebuffer we use <fun><function id='81'>glFramebufferTexture2D</function></fun>, but this time with <var>GL_TEXTURE_2D_MULTISAMPLE</var> as the texture type:
 </p>
 
 <pre class="cpp"><code>
 <function id='81'>glFramebufferTexture2D</function>(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, tex, 0); 
 </code></pre>
 
 <p>
   The currently bound framebuffer now has a multisampled color buffer in the form of a texture image.
 </p>
 
 <h3>Multisampled renderbuffer objects</h3>
 <p>
   Like textures, creating a multisampled renderbuffer object isn't difficult. It is even quite easy since all we need to change is <fun><function id='88'>glRenderbufferStorage</function></fun> to <fun><function id='102'><function id='88'>glRenderbufferStorage</function>Multisample</function></fun> when we configure the (currently bound) renderbuffer's memory storage:
 </p>
 
 <pre class="cpp"><code>
 <function id='102'><function id='88'>glRenderbufferStorage</function>Multisample</function>(GL_RENDERBUFFER, 4, GL_DEPTH24_STENCIL8, width, height);  
 </code></pre>
 
 <p>
   The one thing that changed here is the extra second parameter where we set the amount of samples we'd like to use; 4 in this particular case.
 </p>
 
 <h3>Render to multisampled framebuffer</h3>
 <p>
   Rendering to a multisampled framebuffer is straightforward. Whenever we draw anything while the framebuffer object is bound, the rasterizer will take care of all the multisample operations. However, because a multisampled buffer is a bit special, we can't directly use the buffer for other operations like sampling it in a shader. 
 </p>
 
 <p>
   A multisampled image contains much more information than a normal image so what we need to do is downscale or <def>resolve</def> the image. Resolving a multisampled framebuffer is generally done through <fun><function id='103'>glBlitFramebuffer</function></fun> that copies a region from one framebuffer to the other while also resolving any multisampled buffers. 
 </p>
 
 <p>
   <fun><function id='103'>glBlitFramebuffer</function></fun> transfers a given <def>source</def> region defined by 4 screen-space coordinates to a given <def>target</def> region also defined by 4 screen-space coordinates. You may remember from the <a href="https://learnopengl.com/Advanced-OpenGL/Framebuffers" target="_blank">framebuffers</a> chapter that if we bind to <var>GL_FRAMEBUFFER</var> we're binding to both the read and draw framebuffer targets. We could also bind to those targets individually by binding framebuffers to <var>GL_READ_FRAMEBUFFER</var> and <var>GL_DRAW_FRAMEBUFFER</var> respectively. The <fun><function id='103'>glBlitFramebuffer</function></fun> function reads from those two targets to determine which is the source and which is the target framebuffer. We could then transfer the multisampled framebuffer output to the actual screen by <def>blitting</def> the image to the default framebuffer like so:
 </p>
 
 <pre class="cpp"><code>
 <function id='77'>glBindFramebuffer</function>(GL_READ_FRAMEBUFFER, multisampledFBO);
 <function id='77'>glBindFramebuffer</function>(GL_DRAW_FRAMEBUFFER, 0);
 <function id='103'>glBlitFramebuffer</function>(0, 0, width, height, 0, 0, width, height, GL_COLOR_BUFFER_BIT, GL_NEAREST); 
 </code></pre>
 
 <p>
   If we then were to render the same application we should get the same output: a lime-green cube displayed with MSAA and again showing significantly less jagged edges:
 </p>
 
 <img src="/img/advanced/anti_aliasing_multisampled.png" class="clean" alt="Image of a multisampled cube in OpenGL"/>
 
 <p>
   You can find the source code <a href="/code_viewer_gh.php?code=src/4.advanced_opengl/11.2.anti_aliasing_offscreen/anti_aliasing_offscreen.cpp" target="_blank">here</a>.
 </p>
   
 <p>
   But what if we wanted to use the texture result of a multisampled framebuffer to do stuff like post-processing? We can't directly use the multisampled texture(s) in the fragment shader. What we can do however is blit the multisampled buffer(s) to a different FBO with a non-multisampled texture attachment. We then use this ordinary color attachment texture for post-processing, effectively post-processing an image rendered via multisampling. This does mean we have to generate a new FBO that acts solely as an intermediate framebuffer object to resolve the multisampled buffer into; a normal 2D texture we can use in the fragment shader. This process looks a bit like this in pseudocode:
 </p>
 
 <pre><code>
 unsigned int msFBO = CreateFBOWithMultiSampledAttachments();
 // then create another FBO with a normal texture color attachment
 [...]
 <function id='81'>glFramebufferTexture2D</function>(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, screenTexture, 0);
 [...]
 while(!<function id='14'>glfwWindowShouldClose</function>(window))
 {
     [...]
     
     <function id='77'>glBindFramebuffer</function>(msFBO);
     ClearFrameBuffer();
     DrawScene();
     // now resolve multisampled buffer(s) into intermediate FBO
     <function id='77'>glBindFramebuffer</function>(GL_READ_FRAMEBUFFER, msFBO);
     <function id='77'>glBindFramebuffer</function>(GL_DRAW_FRAMEBUFFER, intermediateFBO);
     <function id='103'>glBlitFramebuffer</function>(0, 0, width, height, 0, 0, width, height, GL_COLOR_BUFFER_BIT, GL_NEAREST);
     // now scene is stored as 2D texture image, so use that image for post-processing
     <function id='77'>glBindFramebuffer</function>(GL_FRAMEBUFFER, 0);
     ClearFramebuffer();
     <function id='48'>glBindTexture</function>(GL_TEXTURE_2D, screenTexture);
     DrawPostProcessingQuad();  
   
     [...] 
 }
 </code></pre>
 
 <p>
   If we then implement this into the post-processing code of the <a href="https://learnopengl.com/Advanced-OpenGL/Framebuffers" target="_blank">framebuffers</a> chapter we're able to create all kinds of cool post-processing effects on a texture of a scene with (almost) no jagged edges. With a grayscale postprocessing filter applied it'll look something like this:
 </p>
 
 <img src="/img/advanced/anti_aliasing_post_processing.png" class="clean" alt="Image of post-processing on a scene drawn with MSAA in OpenGL"/>
 
 <note>
   Because the screen texture is a normal (non-multisampled) texture again, some post-processing filters like <em>edge-detection</em> will introduce jagged edges again. To accommodate for this you could blur the texture afterwards or create your own anti-aliasing algorithm. 
 </note>
   
 <p>
     You can see that when we want to combine multisampling with off-screen rendering we need to take care of some extra steps. The steps are worth the extra effort though since multisampling significantly boosts the visual quality of your scene. Do note that enabling multisampling can noticeably reduce performance the more samples you use. 
 </p>
   
 <h2>Custom Anti-Aliasing algorithm</h2>
 <p>
    It is possible to directly pass a multisampled texture image to a fragment shader instead of first resolving it. GLSL gives us the option to sample the texture image per subsample so we can create our own custom anti-aliasing algorithms.  
 </p>
   
 <p>
     To get a texture value per subsample you'd have to define the texture uniform sampler as a <fun>sampler2DMS</fun> instead of the usual <fun>sampler2D</fun>:
 </p>
  
 <pre><code>
 uniform sampler2DMS screenTextureMS;    
 </code></pre>
   
   <p>
     Using the <fun>texelFetch</fun> function it is then possible to retrieve the color value per sample:
   </p>
   
 <pre><code>
 vec4 colorSample = texelFetch(screenTextureMS, TexCoords, 3);  // 4th subsample
 </code></pre>
   
 <p>
   We won't go into the details of creating custom anti-aliasing techniques here, but this may be enough to get started on building one yourself.
 </p>
          
 
     </div>