Introduction.html (13496B)
1 <div id="content"> 2 <h1 id="content-title">Introduction</h1> 3 <h1 id="content-url" style='display:none;'>Guest-Articles/2020/OIT/Introduction</h1> 4 <p> 5 In the <a href="https://learnopengl.com/Advanced-OpenGL/Blending" target="_blank">Blending</a> chapter, the subject of color blending was introduced. Blending is the way of implementing transparent surfaces in a 3D scene. In short, transparency delves into the subject of drawing semi-solid or fully see-through objects like glasses in computer graphics. The idea is explained up to a suitable point in that chapter, so if you're unfamiliar with the topic, better read Blending first. 6 </p> 7 8 <p> 9 In this article, we are scratching the surface of this topic a bit further, since there are so many techniques involved in implementing such an effect in a 3D environment. 10 </p> 11 12 <p> 13 To begin with, we are going to discuss about the limitations of the graphics library/hardware and the hardships they entail, and the reason that why transparency is such a tricky subject. Later on, we will introduce and briefly review some of the more well-known transparency techniques that have been invented and used for the past twenty years associated with the current hardware. Ultimately, we are going to focus on explaining and implementing one of them, which will be the subject of the following part of this article. 14 </p> 15 16 <p> 17 Note that the goal of this article is to introduce techniques which have significantly better performance than the technique that was used in the Blending chapter. Otherwise, there isn't a genuinely compelling reason to expand on that matter. 18 </p> 19 20 <h2>Graphics library/hardware limitations</h2> 21 22 <p> 23 The reason that this article exists, and you're reading it, is that there is no direct way to draw transparent surfaces with the current technology. Many people wish, that it was as simple as turning on a flag in their graphics API, but that's a fairy tale. Whether, this is a limitation of the graphics libraries or video cards, that's debatable. 24 </p> 25 26 <p> 27 As explained in the Blending chapter, the source of this problem arises from combining depth testing and color blending. At the fragment stage, there is no buffer like the depth buffer for transparent pixels that would tell the graphics library, which pixels are fully visible or semi-visible. One of the reasons could be, that there is no efficient way of storing the information of transparent pixels in such a buffer that can hold an infinite number of pixels for each coordinate on the screen. Since each transparent pixel could expose its underlying pixels, therefore there needs to be a way to store different layers of all pixels for all screen coordinates. 28 </p> 29 30 <p> 31 This limitation leaves us to think for a way to overcome such an issue and since neither the graphics library nor the hardware gives us a hand, this all has to be done by the developer with the tools at hand. We will examine two methods which are prominent in this subject. One being, <def>ordered transparency</def> and the other <def>order-independent transparency</def>. 32 </p> 33 34 <h2>Ordered transparency</h2> 35 36 <p> 37 The most convenient solution to overcome this issue, is to sort your transparent objects, so they're either drawn from the furthest to the nearest, or from the nearest to the furthest in relation to the camera's position. This way, the depth testing wouldn't affect the outcome of those pixels that have been drawn after/before but over/under a further/closer object. However major the expenditure this method entails for the CPU, it was used in many early games that probably most of us have played. 38 </p> 39 40 <p> 41 For example, the sample image below shows the importance of blending order. The top part of the image produces an incorrect result with unordered alpha blending, while the bottom correctly sorts the geometry. Note lower visibility of the skeletal structure without correct depth ordering. 42 This image is from <a href="http://gpuopen.com/archive/gpu-demos/radeon-hd-5000-series-graphics-real-time-demos/" target="_blank">ATI Mecha Demo</a>: 43 </p> 44 45 <img src="/img/guest/2020/oit/ATI_Mecha_Demo_Screenshot.jpg" width="287" alt="Importance of ordering from ATI Mecha Demo"> 46 47 <p> 48 So far, we have understood that in order to overcome the limitation of current technology to draw transparent objects, we need order for our transparent objects to be displayed properly on the screen. Ordering takes away performance from your application, and since most of 3D applications are running in real-time, this will be so much more evident as you perform sorting at every frame. 49 </p> 50 51 <p> 52 Therefore, we will be looking into the world of order-independent transparency techniques and to find one which better suits our purpose and furthermore our pipeline, so we don't have to sort the objects before drawing. 53 </p> 54 55 <h2>Order-independent transparency</h2> 56 57 <p> 58 Order-independent transparency or for short <def>OIT</def>, is a technique which doesn't require us to draw our transparent objects in an orderly fashion. At first glance, this will give us back the CPU cycles that we were taking for sorting the objects, but at the same time OIT techniques have their pros and cons. 59 </p> 60 61 <p> 62 The goal of OIT techniques is to eliminate the need of sorting transparent objects at draw time. Depending on the technique, some of them must sort fragments for an accurate result, but only at a later stage when all the draw calls have been made, and some of them don't require sorting, but results are approximated. 63 </p> 64 65 <h3>History</h3> 66 67 <p> 68 Some of the more advanced techniques that have been invented to overcome the limitation of rendering transparent surfaces, explicitly use a buffer (e.g. a linked list or a 3D array such as [x][y][z]) that can hold multiple layers of pixels' information and can sort pixels on the GPU, normally because of its parallel processing power, as opposed to CPU. 69 </p> 70 71 <note> 72 The <a href="https://en.wikipedia.org/wiki/A-buffer" target="_blank">A-buffer</a> is a computer graphics technique introduced in 1984 which stores per-pixel lists of fragment data (including <a href="https://en.wikipedia.org/wiki/Micropolygon" target="_blank">micro-polygon</a> information) in a software rasterizer, <a href="https://en.wikipedia.org/wiki/Reyes_rendering" target="_blank">REYES</a>, originally designed for anti-aliasing but also supporting transparency. 73 </note> 74 75 <p> 76 At the same time, there has been hardware capable of facilitating this task by performing on-hardware calculations which is the most convenient way for a developer to have access to transparency out of the box. 77 </p> 78 79 <note> 80 <a href="https://en.wikipedia.org/wiki/Dreamcast#Hardware" target="_blank">SEGA Dreamcast</a> was one of the few consoles that had automatic per-pixel translucency sorting, implemented in its hardware. 81 </note> 82 83 <p> 84 Commonly, OIT techniques are separated into two categories which are <def>exact</def> and <def>approximate</def>. Respectively, exact will result in better images with an accurate transparency which suits every scenario, while approximate although resulting in good-looking images, lacks accuracy in complex scenes. 85 </p> 86 87 <h3>Exact OIT</h3> 88 89 <p> 90 These techniques accurately compute the final color, for which all fragments must be sorted. For high depth complexity scenes, sorting becomes the bottleneck. 91 </p> 92 93 <p> 94 One issue with the sorting stage is <def>local memory limited occupancy</def>, in this case a <a href="https://en.wikipedia.org/wiki/Single_Instruction_Multiple_Threads" target="_blank">single instruction, multiple threads</a> attribute relating to the throughput and operation latency hiding of GPUs. 95 Although, <a href="http://diglib.eg.org/handle/10.2312/PE.PG.PG2013short.059-064" target="_blank">BMA</a> (backwards memory allocation) can group pixels by their depth complexity and sort them in batches to improve the occupancy and hence performance of low depth complexity pixels in the context of a potentially high depth complexity scene. Up to a 3× overall OIT performance increase is reported. 96 </p> 97 98 <note> 99 The sorting stage requires relatively large amounts of temporary memory in shaders that is usually conservatively allocated at a maximum, which impacts memory occupancy and performance. 100 </note> 101 102 <p> 103 Sorting is typically performed in a local array, however performance can be improved further by making use of the GPU's memory hierarchy and sorting in registers, similarly to an <a href="https://en.wikipedia.org/wiki/External_sorting#External_merge_sort" target="_blank">external merge sort</a>, especially in conjunction with BMA. 104 </p> 105 106 <h3>Approximate OIT</h3> 107 108 <p> 109 Approximate OIT techniques relax the constraint of exact rendering to provide faster results. Higher performance can be gained from not having to store all fragments or only partially sorting the geometry. A number of techniques also compress, or reduce, the fragment data. These include: 110 </p> 111 112 <ul> 113 <li>Stochastic Transparency: draw in a higher resolution in full opacity but discard some fragments. Down-sampling will then yield transparency.</li> 114 <li>Adaptive Transparency: a two-pass technique where the first constructs a visibility function which compresses on the fly (this compression avoids having to fully sort the fragments) and the second uses this data to composite unordered fragments. Intel's pixel synchronization avoids the need to store all fragments, removing the unbounded memory requirement of many other OIT techniques.</li> 115 </ul> 116 117 <h3>Techniques</h3> 118 119 <p> 120 Some of the OIT techniques that have been commonly used in the industry are as follows: 121 </p> 122 123 <ul> 124 <li><a href="https://en.wikipedia.org/wiki/Depth_peeling" target="_blank">Depth peeling</a>: Introduced in 2001, described a hardware accelerated OIT technique which utilizes the depth buffer to peel a layer of pixels at each pass. With limitations in graphics hardware the scene's geometry had to be rendered many times.</li> 125 <li><a href="http://developer.download.nvidia.com/SDK/10/opengl/src/dual_depth_peeling/doc/DualDepthPeeling.pdf" target="_blank">Dual depth peeling</a>: Introduced in 2008, improves on the performance of depth peeling, still with many-pass rendering limitation.</li> 126 <li><a href="http://jcgt.org/published/0002/02/09/" target="_blank">Weighted, blended</a>: Published in 2013, utilizes a weighting function and two buffers for pixel color and pixel reveal threshold for the final composition pass. Results in an approximated image with a decent quality in complex scenes.</li> 127 </ul> 128 129 <h2>Implementation</h2> 130 131 <p> 132 The usual way of performing OIT in 3D applications is to do it in multiple passes. There are at least three passes required for an OIT technique to be performed, so in order to do this, you'll have to have a perfect understanding of how <a href="https://learnopengl.com/Advanced-OpenGL/Framebuffers" target="_blank">Framebuffers</a> work in OpenGL. Once you're comfortable with Framebuffers, it all boils down to the implementation complexity of the technique you are trying to implement. 133 </p> 134 135 <p> 136 Briefly explained, the three passes involved are as follows: 137 </p> 138 139 <ol> 140 <li>First pass, is where you draw all of your solid objects, this means any object that does not let the light travel through its geometry.</li> 141 <li>Second pass, is where you draw all of your translucent objects. Objects that need alpha discarding, can be rendered in the first pass.</li> 142 <li>Third pass, is where you composite the images that resulted from two previous passes and draw that image onto your backbuffer.</li> 143 </ol> 144 145 <p> 146 This routine is almost identical in implementing OIT techniques across all different pipelines. 147 </p> 148 149 <p> 150 In the next part of this article, we are going to implement weighted, blended OIT which is one of the easiest and high performance OIT techniques that has been used in the video game industry for the past ten years. 151 </p> 152 153 <h2>Further reading</h2> 154 <ul> 155 <li><a href="https://en.wikipedia.org/wiki/Dreamcast#Hardware" target="_blank">SEGA Dreamcast Hardware</a>: Dreamcast was one of the few consoles that had hardware implemented order-independent transparency.</li> 156 <li><a href="https://en.wikipedia.org/wiki/Order-independent_transparency" target="_blank">Order-independent transparency</a>: A series of techniques that have a great performance and produce nice results even with the approximated methods.</li> 157 <li><a href="http://casual-effects.blogspot.com/2014/03/weighted-blended-order-independent.html" target="_blank">Weighted, blended order-independent transparency</a>: One of the easiest OIT techniques in terms of implementation while producing highly acceptable images for complex scenes.</li> 158 </ul> 159 160 <author> 161 <strong>Article by: </strong>Mahan Heshmati Moghaddam<br/> 162 <strong>Contact: </strong><a href="mailto:mahangm@gmail.com" target="_blank">e-mail</a> 163 </author> 164 165 </div> 166