Normal-Mapping.html (45025B)
1 <!DOCTYPE html> 2 <html lang="ja"> 3 <head> 4 <meta charset="utf-8"/> 5 <title>LearnOpenGL</title> 6 <link rel="shortcut icon" type="image/ico" href="/favicon.ico" /> 7 <link rel="stylesheet" href="../static/style.css" /> 8 <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml.js"> </script> 9 <script src="/static/functions.js"></script> 10 </head> 11 <body> 12 <nav> 13 <ol> 14 <li id="Introduction"> 15 <a href="https://learnopengl.com/Introduction">はじめに</a> 16 </li> 17 <li id="Getting-started"> 18 <span class="closed">入門</span> 19 <ol> 20 <li id="Getting-started/OpenGL"> 21 <a href="https://learnopengl.com/Getting-started/OpenGL">OpenGL </a> 22 </li> 23 <li id="Getting-started/Creating-a-window"> 24 <a href="https://learnopengl.com/Getting-started/Creating-a-window">ウィンドウの作成</a> 25 </li> 26 <li id="Getting-started/Hello-Window"> 27 <a href="https://learnopengl.com/Getting-started/Hello-Window">最初のウィンドウ</a> 28 </li> 29 <li id="Getting-started/Hello-Triangle"> 30 <a href="https://learnopengl.com/Getting-started/Hello-Triangle">最初の三角形</a> 31 </li> 32 <li id="Getting-started/Shaders"> 33 <a href="https://learnopengl.com/Getting-started/Shaders">シェーダー</a> 34 </li> 35 <li id="Getting-started/Textures"> 36 <a href="https://learnopengl.com/Getting-started/Textures">テクスチャ</a> 37 </li> 38 <li id="Getting-started/Transformations"> 39 <a href="https://learnopengl.com/Getting-started/Transformations">座標変換</a> 40 </li> 41 <li id="Getting-started/Coordinate-Systems"> 42 <a href="https://learnopengl.com/Getting-started/Coordinate-Systems">座標系</a> 43 </li> 44 <li id="Getting-started/Camera"> 45 <a href="https://learnopengl.com/Getting-started/Camera">カメラ</a> 46 </li> 47 <li id="Getting-started/Review"> 48 <a href="https://learnopengl.com/Getting-started/Review">まとめ</a> 49 </li> 50 </ol> 51 </li> 52 <li id="Lighting"> 53 <span class="closed">Lighting </span> 54 <ol> 55 <li id="Lighting/Colors"> 56 <a href="https://learnopengl.com/Lighting/Colors">Colors </a> 57 </li> 58 <li id="Lighting/Basic-Lighting"> 59 <a href="https://learnopengl.com/Lighting/Basic-Lighting">Basic Lighting </a> 60 </li> 61 <li id="Lighting/Materials"> 62 <a href="https://learnopengl.com/Lighting/Materials">Materials </a> 63 </li> 64 <li id="Lighting/Lighting-maps"> 65 <a href="https://learnopengl.com/Lighting/Lighting-maps">Lighting maps </a> 66 </li> 67 <li id="Lighting/Light-casters"> 68 <a href="https://learnopengl.com/Lighting/Light-casters">Light casters </a> 69 </li> 70 <li id="Lighting/Multiple-lights"> 71 <a href="https://learnopengl.com/Lighting/Multiple-lights">Multiple lights </a> 72 </li> 73 <li id="Lighting/Review"> 74 <a href="https://learnopengl.com/Lighting/Review">Review </a> 75 </li> 76 </ol> 77 </li> 78 <li id="Model-Loading"> 79 <span class="closed">Model Loading </span> 80 <ol> 81 <li id="Model-Loading/Assimp"> 82 <a href="https://learnopengl.com/Model-Loading/Assimp">Assimp </a> 83 </li> 84 <li id="Model-Loading/Mesh"> 85 <a href="https://learnopengl.com/Model-Loading/Mesh">Mesh </a> 86 </li> 87 <li id="Model-Loading/Model"> 88 <a href="https://learnopengl.com/Model-Loading/Model">Model </a> 89 </li> 90 </ol> 91 </li> 92 <li id="Advanced-OpenGL"> 93 <span class="closed">Advanced OpenGL </span> 94 <ol> 95 <li id="Advanced-OpenGL/Depth-testing"> 96 <a href="https://learnopengl.com/Advanced-OpenGL/Depth-testing">Depth testing </a> 97 </li> 98 <li id="Advanced-OpenGL/Stencil-testing"> 99 <a href="https://learnopengl.com/Advanced-OpenGL/Stencil-testing">Stencil testing </a> 100 </li> 101 <li id="Advanced-OpenGL/Blending"> 102 <a href="https://learnopengl.com/Advanced-OpenGL/Blending">Blending </a> 103 </li> 104 <li id="Advanced-OpenGL/Face-culling"> 105 <a href="https://learnopengl.cm/Advanced-OpenGL/Face-culling">Face culling </a> 106 </li> 107 <li id="Advanced-OpenGL/Framebuffers"> 108 <a href="https://learnopengl.com/Advanced-OpenGL/Framebuffers">Framebuffers </a> 109 </li> 110 <li id="Advanced-OpenGL/Cubemaps"> 111 <a href="https://learnopengl.com/Advanced-OpenGL/Cubemaps">Cubemaps </a> 112 </li> 113 <li id="Advanced-OpenGL/Advanced-Data"> 114 <a href="https://learnopengl.com/Advanced-OpenGL/Advanced-Data">Advanced Data </a> 115 </li> 116 <li id="Advanced-OpenGL/Advanced-GLSL"> 117 <a href="https://learnopengl.com/Advanced-OpenGL/Advanced-GLSL">Advanced GLSL </a> 118 </li> 119 <li id="Advanced-OpenGL/Geometry-Shader"> 120 <a href="https://learnopengl.com/Advanced-OpenGL/Geometry-Shader">Geometry Shader </a> 121 </li> 122 <li id="Advanced-OpenGL/Instancing"> 123 <a href="https://learnopengl.com/Advanced-OpenGL/Instancing">Instancing </a> 124 </li> 125 <li id="Advanced-OpenGL/Anti-Aliasing"> 126 <a href="https://learnopengl.com/Advanced-OpenGL/Anti-Aliasing">Anti Aliasing </a> 127 </li> 128 </ol> 129 </li> 130 <li id="Advanced-Lighting"> 131 <span class="closed">Advanced Lighting </span> 132 <ol> 133 <li id="Advanced-Lighting/Advanced-Lighting"> 134 <a href="https://learnopengl.com/Advanced-Lighting/Advanced-Lighting">Advanced Lighting </a> 135 </li> 136 <li id="Advanced-Lighting/Gamma-Correction"> 137 <a href="https://learnopengl.com/Advanced-Lighting/Gamma-Correction">Gamma Correction </a> 138 </li> 139 <li id="Advanced-Lighting/Shadows"> 140 <span class="closed">Shadows </span> 141 <ol> 142 <li id="Advanced-Lighting/Shadows/Shadow-Mapping"> 143 <a href="https://learnopengl.com/Advanced-Lighting/Shadows/Shadow-Mapping">Shadow Mapping </a> 144 </li> 145 <li id="Advanced-Lighting/Shadows/Point-Shadows"> 146 <a href="https://learnopengl.com/Advanced-Lighting/Shadows/Point-Shadows">Point Shadows </a> 147 </li> 148 </ol> 149 </li> 150 <li id="Advanced-Lighting/Normal-Mapping"> 151 <a href="https://learnopengl.com/Advanced-Lighting/Normal-Mapping">Normal Mapping </a> 152 </li> 153 <li id="Advanced-Lighting/Parallax-Mapping"> 154 <a href="https://learnopengl.com/Advanced-Lighting/Parallax-Mapping">Parallax Mapping </a> 155 </li> 156 <li id="Advanced-Lighting/HDR"> 157 <a href="https://learnopengl.com/Advanced-Lighting/HDR">HDR </a> 158 </li> 159 <li id="Advanced-Lighting/Bloom"> 160 <a href="https://learnopengl.com/Advanced-Lighting/Bloom">Bloom </a> 161 </li> 162 <li id="Advanced-Lighting/Deferred-Shading"> 163 <a href="https://learnopengl.com/Advanced-Lighting/Deferred-Shading">Deferred Shading </a> 164 </li> 165 <li id="Advanced-Lighting/SSAO"> 166 <a href="https://learnopengl.com/Advanced-Lighting/SSAO">SSAO </a> 167 </li> 168 </ol> 169 </li> 170 <li id="PBR"> 171 <span class="closed">PBR </span> 172 <ol> 173 <li id="PBR/Theory"> 174 <a href="https://learnopengl.com/PBR/Theory">Theory </a> 175 </li> 176 <li id="PBR/Lighting"> 177 <a href="https://learnopengl.com/PBR/Lighting">Lighting </a> 178 </li> 179 <li id="PBR/IBL"> 180 <span class="closed">IBL </span> 181 <ol> 182 <li id="PBR/IBL/Diffuse-irradiance"> 183 <a href="https://learnopengl.com/PBR/IBL/Diffuse-irradiance">Diffuse irradiance </a> 184 </li> 185 <li id="PBR/IBL/Specular-IBL"> 186 <a href="https://learnopengl.com/PBR/IBL/Specular-IBL">Specular IBL </a> 187 </li> 188 </ol> 189 </li> 190 </ol> 191 </li> 192 <li id="In-Practice"> 193 <span class="closed">In Practice </span> 194 <ol> 195 <li id="In-Practice/Debugging"> 196 <a href="https://learnopengl.com/In-Practice/Debugging">Debugging </a> 197 </li> 198 <li id="In-Practice/Text-Rendering"> 199 <a href="https://learnopengl.com/In-Practice/Text-Rendering">Text Rendering </a> 200 </li> 201 <li id="In-Practice/2D-Game"> 202 <span class="closed">2D Game </span> 203 <ol> 204 <li id="In-Practice/2D-Game/Breakout"> 205 <a href="https://learnopengl.com/In-Practice/2D-Game/Breakout">Breakout </a> 206 </li> 207 <li id="In-Practice/2D-Game/Setting-up"> 208 <a href="https://learnopengl.com/In-Practice/2D-Game/Setting-up">Setting up </a> 209 </li> 210 <li id="In-Practice/2D-Game/Rendering-Sprites"> 211 <a href="https://learnopengl.com/In-Practice/2D-Game/Rendering-Sprites">Rendering Sprites </a> 212 </li> 213 <li id="In-Practice/2D-Game/Levels"> 214 <a href="https://learnopengl.com/In-Practice/2D-Game/Levels">Levels </a> 215 </li> 216 <li id="In-Practice/2D-Game/Collisions"> 217 <span class="closed">Collisions </span> 218 <ol> 219 <li id="In-Practice/2D-Game/Collisions/Ball"> 220 <a href="https://learnopengl.com/In-Practice/2D-Game/Collisions/Ball">Ball </a> 221 </li> 222 <li id="In-Practice/2D-Game/Collisions/Collision-detection"> 223 <a href="https://learnopengl.com/In-Practice/2D-Game/Collisions/Collision-detection">Collision detection </a> 224 </li> 225 <li id="In-Practice/2D-Game/Collisions/Collision-resolution"> 226 <a href="https://learnopengl.com/In-Practice/2D-Game/Collisions/Collision-resolution">Collision resolution </a> 227 </li> 228 </ol> 229 </li> 230 <li id="In-Practice/2D-Game/Particles"> 231 <a href="https://learnopengl.com/In-Practice/2D-Game/Particles">Particles </a> 232 </li> 233 <li id="In-Practice/2D-Game/Postprocessing"> 234 <a href="https://learnopengl.com/In-Practice/2D-Game/Postprocessing">Postprocessing </a> 235 </li> 236 <li id="In-Practice/2D-Game/Powerups"> 237 <a href="https://learnopengl.com/In-Practice/2D-Game/Powerups">Powerups </a> 238 </li> 239 <li id="In-Practice/2D-Game/Audio"> 240 <a href="https://learnopengl.com/In-Practice/2D-Game/Audio">Audio </a> 241 </li> 242 <li id="In-Practice/2D-Game/Render-text"> 243 <a href="https://learnopengl.com/In-Practice/2D-Game/Render-text">Render text </a> 244 </li> 245 <li id="In-Practice/2D-Game/Final-thoughts"> 246 <a href="https://learnopengl.com/In-Practice/2D-Game/Final-thoughts">Final thoughts </a> 247 </li> 248 </ol> 249 </li> 250 </ol> 251 </li> 252 <li id="Guest-Articles"> 253 <span class="closed">Guest Articles </span> 254 <ol> 255 <li id="Guest-Articles/How-to-publish"> 256 <a href="https://learnopengl.com/Guest-Articles/How-to-publish">How to publish </a> 257 </li> 258 <li id="Guest-Articles/2020"> 259 <span class="closed">2020 </span> 260 <ol> 261 <li id="Guest-Articles/2020/OIT"> 262 <span class="closed">OIT </span> 263 <ol> 264 <li id="Guest-Articles/2020/OIT/Introduction"> 265 <a href="https://learnopengl.com/Guest-Articles/2020/OIT/Introduction">Introduction </a> 266 </li> 267 <li id="Guest-Articles/2020/OIT/Weighted-Blended"> 268 <a href="https://learnopengl.com/Guest-Articles/2020/OIT/Weighted-Blended">Weighted Blended </a> 269 </li> 270 </ol> 271 </li> 272 <li id="Guest-Articles/2020/Skeletal-Animation"> 273 <a href="https://learnopengl.com/Guest-Articles/2020/Skeletal-Animation">Skeletal Animation </a> 274 </li> 275 </ol> 276 </li> 277 <li id="Guest-Articles/2021"> 278 <span class="closed">2021 </span> 279 <ol> 280 <li id="Guest-Articles/2021/CSM"> 281 <a href="https://learnopengl.com/Guest-Articles/2021/CSM">CSM </a> 282 </li> 283 <li id="Guest-Articles/2021/Scene"> 284 <span class="closed">Scene </span> 285 <ol> 286 <li id="Guest-Articles/2021/Scene/Scene-Graph"> 287 <a href="https://learnopengl.com/Guest-Articles/2021/Scene/Scene-Graph">Scene Graph </a> 288 </li> 289 <li id="Guest-Articles/2021/Scene/Frustum-Culling"> 290 <a href="https://learnopengl.com/Guest-Articles/2021/Scene/Frustum-Culling">Frustum Culling </a> 291 </li> 292 </ol> 293 </li> 294 <li id="Guest-Articles/2021/Tessellation"> 295 <span class="closed">Tessellation </span> 296 <ol> 297 <li id="Guest-Articles/2021/Tessellation/Height-map"> 298 <a href="https://learnopengl.com/Guest-Articles/2021/Tessellation/Height-map">Height map </a> 299 </li> 300 </ol> 301 </li> 302 </ol> 303 </li> 304 </ol> 305 </li> 306 <li id="Code-repository"> 307 <a href="https://learnopengl.com/Code-repository">Code repository </a> 308 </li> 309 <li id="Translations"> 310 <a href="https://learnopengl.com/Translations">Translations </a> 311 </li> 312 <li id="About"> 313 <a href="https://learnopengl.com/About">About </a> 314 </li> 315 </ol> 316 </nav> 317 <main> 318 <h1 id="content-title">Normal Mapping</h1> 319 <h1 id="content-url" style='display:none;'>Advanced-Lighting/Normal-Mapping</h1> 320 <p> 321 All of our scenes are filled with meshes, each consisting of hundreds or maybe thousands of triangles. We boosted the realism by wrapping 2D textures on these flat triangles, hiding the fact that the polygons are just tiny flat triangles. Textures help, but when you take a good close look at the meshes it is still quite easy to see the underlying flat surfaces. Most real-life surface aren't flat however and exhibit a lot of (bumpy) details. 322 </p> 323 324 <p> 325 For instance, take a brick surface. A brick surface is quite a rough surface and obviously not completely flat: it contains sunken cement stripes and a lot of detailed little holes and cracks. If we were to view such a brick surface in a lit scene the immersion gets easily broken. Below we can see a brick texture applied to a flat surface lit by a point light. 326 </p> 327 328 <img src="/img/advanced-lighting/normal_mapping_flat.png" class="clean" alt="Brick surface lighted by point light in OpenGL. It's not too realistic; its flat structures is now quite obvious"/> 329 330 <p> 331 The lighting doesn't take any of the small cracks and holes into account and completely ignores the deep stripes between the bricks; the surface looks perfectly flat. We can partly fix the flat look by using a specular map to pretend some surfaces are less lit due to depth or other details, but that's more of a hack than a real solution. What we need is some way to inform the lighting system about all the little depth-like details of the surface. 332 </p> 333 334 <p> 335 If we think about this from a light's perspective: how comes the surface is lit as a completely flat surface? The answer is the surface's normal vector. From the lighting technique's point of view, the only way it determines the shape of an object is by its perpendicular normal vector. The brick surface only has a single normal vector, and as a result the surface is uniformly lit based on this normal vector's direction. What if we, instead of a per-surface normal that is the same for each fragment, use a per-fragment normal that is different for each fragment? This way we can slightly deviate the normal vector based on a surface's little details; this gives the illusion the surface is a lot more complex: 336 </p> 337 338 <img src="/img/advanced-lighting/normal_mapping_surfaces.png" class="clean" alt="Surfaces displaying per-surface normal and per-fragment normals for normal mapping in OpenGL"/> 339 340 <p> 341 By using per-fragment normals we can trick the lighting into believing a surface consists of tiny little planes (perpendicular to the normal vectors) giving the surface an enormous boost in detail. This technique to use per-fragment normals compared to per-surface normals is called <def>normal mapping</def> or <def>bump mapping</def>. Applied to the brick plane it looks a bit like this: 342 </p> 343 344 <img src="/img/advanced-lighting/normal_mapping_compare.png" alt="Surface without and with normal mapping in OpenGL"/> 345 346 <p> 347 As you can see, it gives an enormous boost in detail and for a relatively low cost. Since we only change the normal vectors per fragment there is no need to change the lighting equation. We now pass a per-fragment normal, instead of an interpolated surface normal, to the lighting algorithm. The lighting then does the rest. 348 </p> 349 350 <h2>Normal mapping</h2> 351 <p> 352 To get normal mapping to work we're going to need a per-fragment normal. Similar to what we did with diffuse and specular maps we can use a 2D texture to store per-fragment normal data. This way we can sample a 2D texture to get a normal vector for that specific fragment. 353 </p> 354 355 <p> 356 While normal vectors are geometric entities and textures are generally only used for color information, storing normal vectors in a texture may not be immediately obvious. If you think about color vectors in a texture they are represented as a 3D vector with an <code>r</code>, <code>g</code>, and <code>b</code> component. We can similarly store a normal vector's <code>x</code>, <code>y</code> and <code>z</code> component in the respective color components. Normal vectors range between <code>-1</code> and <code>1</code> so they're first mapped to [<code>0</code>,<code>1</code>]: 357 </p> 358 359 <pre><code> 360 vec3 rgb_normal = normal * 0.5 + 0.5; // transforms from [-1,1] to [0,1] 361 </code></pre> 362 363 <p> 364 With normal vectors transformed to an RGB color component like this, we can store a per-fragment normal derived from the shape of a surface onto a 2D texture. An example <def>normal map</def> of the brick surface at the start of this chapter is shown below: 365 </p> 366 367 <img src="/img/advanced-lighting/normal_mapping_normal_map.png" alt="Image of a normal map in OpenGL normal mapping"/> 368 369 <p> 370 This (and almost all normal maps you find online) will have a blue-ish tint. This is because the normals are all closely pointing outwards towards the positive z-axis \((0, 0, 1)\): a blue-ish color. The deviations in color represent normal vectors that are slightly offset from the general positive z direction, giving a sense of depth to the texture. For example, you can see that at the top of each brick the color tends to be more greenish, which makes sense as the top side of a brick would have normals pointing more in the positive y direction \((0, 1, 0)\) which happens to be the color green! 371 </p> 372 373 <p> 374 With a simple plane, looking at the positive z-axis, we can take <a href="/img/textures/brickwall.jpg" target="_blank">this</a> diffuse texture and <a href="/img/textures/brickwall_normal.jpg" target="_blank">this</a> normal map to render the image from the previous section. Note that the linked normal map is different from the one shown above. The reason for this is that OpenGL reads texture coordinates with the y (or v) coordinate reversed from how textures are generally created. The linked normal map thus has its y (or green) component inversed (you can see the green colors are now pointing downwards); if you fail to take this into account, the lighting will be incorrect. Load both textures, bind them to the proper texture units, and render a plane with the following changes in the lighting fragment shader: 375 </p> 376 377 <pre><code> 378 uniform sampler2D normalMap; 379 380 void main() 381 { 382 // obtain normal from normal map in range [0,1] 383 normal = texture(normalMap, fs_in.TexCoords).rgb; 384 // transform normal vector to range [-1,1] 385 normal = normalize(normal * 2.0 - 1.0); 386 387 [...] 388 // proceed with lighting as normal 389 } 390 </code></pre> 391 392 <p> 393 Here we reverse the process of mapping normals to RGB colors by remapping the sampled normal color from [<code>0</code>,<code>1</code>] back to [<code>-1</code>,<code>1</code>] and then use the sampled normal vectors for the upcoming lighting calculations. In this case we used a Blinn-Phong shader. 394 </p> 395 396 <p> 397 By slowly moving the light source over time you really get a sense of depth using the normal map. Running this normal mapping example gives the exact results as shown at the start of this chapter: 398 </p> 399 400 <img src="/img/advanced-lighting/normal_mapping_correct.png" class="clean" alt="Surface without and with normal mapping in OpenGL"/> 401 402 <p> 403 There is one issue however that greatly limits this use of normal maps. The normal map we used had normal vectors that all pointed somewhat in the positive z direction. This worked because the plane's surface normal was also pointing in the positive z direction. However, what would happen if we used the same normal map on a plane laying on the ground with a surface normal vector pointing in the positive y direction? 404 </p> 405 406 <img src="/img/advanced-lighting/normal_mapping_ground.png" class="clean" alt="Image of plane with normal mapping without tangent space transformation, looks off in OpenGL"/> 407 408 <p> 409 The lighting doesn't look right! This happens because the sampled normals of this plane still roughly point in the positive z direction even though they should mostly point in the positive y direction. As a result, the lighting thinks the surface's normals are the same as before when the plane was pointing towards the positive z direction; the lighting is incorrect. The image below shows what the sampled normals approximately look like on this surface: 410 </p> 411 412 <img src="/img/advanced-lighting/normal_mapping_ground_normals.png" class="clean" alt="Image of plane with normal mapping without tangent space transformation with displayed normals, looks off in OpenGL"/> 413 414 <p> 415 You can see that all the normals point somewhat in the positive z direction even though they should be pointing towards the positive y direction. One solution to this problem is to define a normal map for each possible direction of the surface; in the case of a cube we would need 6 normal maps. However, with advanced meshes that can have more than hundreds of possible surface directions this becomes an infeasible approach. 416 </p> 417 418 <p> 419 A different solution exists that does all the lighting in a different coordinate space: a coordinate space where the normal map vectors always point towards the positive z direction; all other lighting vectors are then transformed relative to this positive z direction. This way we can always use the same normal map, regardless of orientation. This coordinate space is called <def>tangent space</def>. 420 </p> 421 422 <h2>Tangent space</h2> 423 <p> 424 Normal vectors in a normal map are expressed in tangent space where normals always point roughly in the positive z direction. Tangent space is a space that's local to the surface of a triangle: the normals are relative to the local reference frame of the individual triangles. Think of it as the local space of the normal map's vectors; they're all defined pointing in the positive z direction regardless of the final transformed direction. Using a specific matrix we can then transform normal vectors from this <em>local</em> tangent space to world or view coordinates, orienting them along the final mapped surface's direction. 425 </p> 426 427 <p> 428 Let's say we have the incorrect normal mapped surface from the previous section looking in the positive y direction. The normal map is defined in tangent space, so one way to solve the problem is to calculate a matrix to transform normals from tangent space to a different space such that they're aligned with the surface's normal direction: the normal vectors are then all pointing roughly in the positive y direction. The great thing about tangent space is that we can calculate this matrix for any type of surface so that we can properly align the tangent space's z direction to the surface's normal direction. 429 </p> 430 431 <p> 432 Such a matrix is called a <def>TBN</def> matrix where the letters depict a <def>Tangent</def>, <def>Bitangent</def> and <def>Normal</def> vector. These are the vectors we need to construct this matrix. To construct such a <em>change-of-basis</em> matrix, that transforms a tangent-space vector to a different coordinate space, we need three perpendicular vectors that are aligned along the surface of a normal map: an up, right, and forward vector; similar to what we did in the <a href="https://learnopengl.com/Getting-Started/Camera" target="_blank">camera</a> chapter. 433 </p> 434 435 <p> 436 We already know the up vector, which is the surface's normal vector. The right and forward vector are the tangent and bitangent vector respectively. The following image of a surface shows all three vectors on a surface: 437 </p> 438 439 <img src="/img/advanced-lighting/normal_mapping_tbn_vectors.png" class="clean" alt="Normal mapping tangent, bitangent and normal vectors on a surface in OpenGL"/> 440 441 <p> 442 Calculating the tangent and bitangent vectors is not as straightforward as the normal vector. We can see from the image that the direction of the normal map's tangent and bitangent vector align with the direction in which we define a surface's texture coordinates. We'll use this fact to calculate tangent and bitangent vectors for each surface. Retrieving them does require a bit of math; take a look at the following image: 443 </p> 444 445 <img src="/img/advanced-lighting/normal_mapping_surface_edges.png" class="clean" alt="Edges of a surface in OpenGL required for calculating TBN matrix"/> 446 447 <p> 448 From the image we can see that the texture coordinate differences of an edge \(E_2\) of a triangle (denoted as \(\Delta U_2\) and \(\Delta V_2\)) are expressed in the same direction as the tangent vector \(T\) and bitangent vector \(B\). Because of this we can write both displayed edges \(E_1\) and \(E_2\) of the triangle as a linear combination of the tangent vector \(T\) and the bitangent vector \(B\): 449 </p> 450 451 \[E_1 = \Delta U_1T + \Delta V_1B\] 452 \[E_2 = \Delta U_2T + \Delta V_2B\] 453 454 <p> 455 Which we can also write as: 456 </p> 457 458 \[(E_{1x}, E_{1y}, E_{1z}) = \Delta U_1(T_x, T_y, T_z) + \Delta V_1(B_x, B_y, B_z)\] 459 \[(E_{2x}, E_{2y}, E_{2z}) = \Delta U_2(T_x, T_y, T_z) + \Delta V_2(B_x, B_y, B_z)\] 460 461 <p> 462 We can calculate \(E\) as the difference vector between two triangle positions, and \(\Delta U\) and \(\Delta V\) as their texture coordinate differences. We're then left with two unknowns (tangent \(T\) and bitangent \(B\)) and two equations. You may remember from your algebra classes that this allows us to solve for \(T\) and \(B\). 463 </p> 464 465 <p> 466 The last equation allows us to write it in a different form: that of matrix multiplication: 467 </p> 468 469 \[\begin{bmatrix} E_{1x} & E_{1y} & E_{1z} \\ E_{2x} & E_{2y} & E_{2z} \end{bmatrix} = \begin{bmatrix} \Delta U_1 & \Delta V_1 \\ \Delta U_2 & \Delta V_2 \end{bmatrix} \begin{bmatrix} T_x & T_y & T_z \\ B_x & B_y & B_z \end{bmatrix} \] 470 471 <p> 472 Try to visualize the matrix multiplications in your head and confirm that this is indeed the same equation. An advantage of rewriting the equations in matrix form is that solving for \(T\) and \(B\) is easier to understand. If we multiply both sides of the equations by the inverse of the \(\Delta U \Delta V\) matrix we get: 473 </p> 474 475 \[ \begin{bmatrix} \Delta U_1 & \Delta V_1 \\ \Delta U_2 & \Delta V_2 \end{bmatrix}^{-1} \begin{bmatrix} E_{1x} & E_{1y} & E_{1z} \\ E_{2x} & E_{2y} & E_{2z} \end{bmatrix} = \begin{bmatrix} T_x & T_y & T_z \\ B_x & B_y & B_z \end{bmatrix} \] 476 477 <p> 478 This allows us to solve for \(T\) and \(B\). This does require us to calculate the inverse of the delta texture coordinate matrix. I won't go into the mathematical details of calculating a matrix' inverse, but it roughly translates to 1 over the determinant of the matrix, multiplied by its adjugate matrix: 479 </p> 480 \[ \begin{bmatrix} T_x & T_y & T_z \\ B_x & B_y & B_z \end{bmatrix} = \frac{1}{\Delta U_1 \Delta V_2 - \Delta U_2 \Delta V_1} \begin{bmatrix} \Delta V_2 & -\Delta V_1 \\ -\Delta U_2 & \Delta U_1 \end{bmatrix} \begin{bmatrix} E_{1x} & E_{1y} & E_{1z} \\ E_{2x} & E_{2y} & E_{2z} \end{bmatrix} \] 481 482 <p> 483 This final equation gives us a formula for calculating the tangent vector \(T\) and bitangent vector \(B\) from a triangle's two edges and its texture coordinates. 484 </p> 485 486 <p> 487 Don't worry if you do not fully understand the mathematics behind this. As long as you understand that we can calculate tangents and bitangents from a triangle's vertices and its texture coordinates (since texture coordinates are in the same space as tangent vectors) you're halfway there. 488 </p> 489 490 <h3>Manual calculation of tangents and bitangents</h3> 491 <p> 492 In the previous demo we had a simple normal mapped plane facing the positive z direction. This time we want to implement normal mapping using tangent space so we can orient this plane however we want and normal mapping would still work. Using the previously discussed mathematics we're going to manually calculate this surface's tangent and bitangent vectors. 493 </p> 494 495 <p> 496 Let's assume the plane is built up from the following vectors (with 1, 2, 3 and 1, 3, 4 as its two triangles): 497 </p> 498 499 <pre><code> 500 // positions 501 glm::vec3 pos1(-1.0, 1.0, 0.0); 502 glm::vec3 pos2(-1.0, -1.0, 0.0); 503 glm::vec3 pos3( 1.0, -1.0, 0.0); 504 glm::vec3 pos4( 1.0, 1.0, 0.0); 505 // texture coordinates 506 glm::vec2 uv1(0.0, 1.0); 507 glm::vec2 uv2(0.0, 0.0); 508 glm::vec2 uv3(1.0, 0.0); 509 glm::vec2 uv4(1.0, 1.0); 510 // normal vector 511 glm::vec3 nm(0.0, 0.0, 1.0); 512 </code></pre> 513 514 <p> 515 We first calculate the first triangle's edges and delta UV coordinates: 516 </p> 517 518 <pre><code> 519 glm::vec3 edge1 = pos2 - pos1; 520 glm::vec3 edge2 = pos3 - pos1; 521 glm::vec2 deltaUV1 = uv2 - uv1; 522 glm::vec2 deltaUV2 = uv3 - uv1; 523 </code></pre> 524 525 <p> 526 With the required data for calculating tangents and bitangents we can start following the equation from the previous section: 527 </p> 528 529 <pre><code> 530 float f = 1.0f / (deltaUV1.x * deltaUV2.y - deltaUV2.x * deltaUV1.y); 531 532 tangent1.x = f * (deltaUV2.y * edge1.x - deltaUV1.y * edge2.x); 533 tangent1.y = f * (deltaUV2.y * edge1.y - deltaUV1.y * edge2.y); 534 tangent1.z = f * (deltaUV2.y * edge1.z - deltaUV1.y * edge2.z); 535 536 bitangent1.x = f * (-deltaUV2.x * edge1.x + deltaUV1.x * edge2.x); 537 bitangent1.y = f * (-deltaUV2.x * edge1.y + deltaUV1.x * edge2.y); 538 bitangent1.z = f * (-deltaUV2.x * edge1.z + deltaUV1.x * edge2.z); 539 540 [...] // similar procedure for calculating tangent/bitangent for plane's second triangle 541 </code></pre> 542 543 <p> 544 Here we first pre-calculate the fractional part of the equation as <var>f</var> and then for each vector component we do the corresponding matrix multiplication multiplied by <var>f</var>. If you compare this code with the final equation you can see it is a direct translation. Because a triangle is always a flat shape, we only need to calculate a single tangent/bitangent pair per triangle as they will be the same for each of the triangle's vertices. 545 </p> 546 547 <p> 548 The resulting tangent and bitangent vector should have a value of (<code>1</code>,<code>0</code>,<code>0</code>) and (<code>0</code>,<code>1</code>,<code>0</code>) respectively that together with the normal (<code>0</code>,<code>0</code>,<code>1</code>) forms an orthogonal TBN matrix. Visualized on the plane, the TBN vectors would look like this: 549 </p> 550 551 <img src="/img/advanced-lighting/normal_mapping_tbn_shown.png" class="clean" alt="Image of TBN vectors visualized on a plane in OpenGL"/> 552 553 <p> 554 With tangent and bitangent vectors defined per vertex we can start implementing <em>proper</em> normal mapping. 555 </p> 556 557 <h3>Tangent space normal mapping</h3> 558 <p> 559 To get normal mapping working, we first have to create a TBN matrix in the shaders. To do that, we pass the earlier calculated tangent and bitangent vectors to the vertex shader as vertex attributes: 560 </p> 561 562 <pre><code> 563 #version 330 core 564 layout (location = 0) in vec3 aPos; 565 layout (location = 1) in vec3 aNormal; 566 layout (location = 2) in vec2 aTexCoords; 567 layout (location = 3) in vec3 aTangent; 568 layout (location = 4) in vec3 aBitangent; 569 </code></pre> 570 571 <p> 572 Then within the vertex shader's <fun>main</fun> function we create the TBN matrix: 573 </p> 574 575 <pre><code> 576 void main() 577 { 578 [...] 579 vec3 T = normalize(vec3(model * vec4(aTangent, 0.0))); 580 vec3 B = normalize(vec3(model * vec4(aBitangent, 0.0))); 581 vec3 N = normalize(vec3(model * vec4(aNormal, 0.0))); 582 mat3 TBN = mat3(T, B, N); 583 } 584 </code></pre> 585 586 <p> 587 Here we first transform all the TBN vectors to the coordinate system we'd like to work in, which in this case is world-space as we multiply them with the <var>model</var> matrix. Then we create the actual TBN matrix by directly supplying <fun>mat3</fun>'s constructor with the relevant column vectors. Note that if we want to be really precise, we would multiply the TBN vectors with the normal matrix as we only care about the orientation of the vectors. 588 </p> 589 590 <note> 591 Technically there is no need for the <var>bitangent</var> variable in the vertex shader. All three TBN vectors are perpendicular to each other so we can calculate the <var>bitangent</var> ourselves in the vertex shader by taking the cross product of the <var>T</var> and <var>N</var> vector: <code>vec3 B = cross(N, T);</code> 592 </note> 593 594 <p> 595 So now that we have a TBN matrix, how are we going to use it? There are two ways we can use a TBN matrix for normal mapping, and we'll demonstrate both of them: 596 </p> 597 598 <ol> 599 <li>We take the TBN matrix that transforms any vector from tangent to world space, give it to the fragment shader, and transform the sampled normal from tangent space to world space using the TBN matrix; the normal is then in the same space as the other lighting variables.</li> 600 <li>We take the inverse of the TBN matrix that transforms any vector from world space to tangent space, and use this matrix to transform not the normal, but the other relevant lighting variables to tangent space; the normal is then again in the same space as the other lighting variables.</li> 601 </ol> 602 603 <p> 604 Let's review the first case. The normal vector we sample from the normal map is expressed in tangent space whereas the other lighting vectors (light and view direction) are expressed in world space. By passing the TBN matrix to the fragment shader we can multiply the sampled tangent space normal with this TBN matrix to transform the normal vector to the same reference space as the other lighting vectors. This way, all the lighting calculations (specifically the dot product) make sense. 605 </p> 606 607 <p> 608 Sending the TBN matrix to the fragment shader is easy: 609 </p> 610 611 <pre><code> 612 out VS_OUT { 613 vec3 FragPos; 614 vec2 TexCoords; 615 mat3 TBN; 616 } vs_out; 617 618 void main() 619 { 620 [...] 621 vs_out.TBN = mat3(T, B, N); 622 } 623 </code></pre> 624 625 <p> 626 In the fragment shader we similarly take a <code>mat3</code> as an input variable: 627 </p> 628 629 <pre><code> 630 in VS_OUT { 631 vec3 FragPos; 632 vec2 TexCoords; 633 mat3 TBN; 634 } fs_in; 635 </code></pre> 636 637 <p> 638 With this TBN matrix we can now update the normal mapping code to include the tangent-to-world space transformation: 639 </p> 640 641 <pre class="cpp"><code> 642 normal = texture(normalMap, fs_in.TexCoords).rgb; 643 normal = normal * 2.0 - 1.0; 644 normal = normalize(fs_in.TBN * normal); 645 </code></pre> 646 647 <p> 648 Because the resulting <var>normal</var> is now in world space, there is no need to change any of the other fragment shader code as the lighting code assumes the normal vector to be in world space. 649 </p> 650 651 <p> 652 Let's also review the second case, where we take the inverse of the TBN matrix to transform all relevant world-space vectors to the space the sampled normal vectors are in: tangent space. The construction of the TBN matrix remains the same, but we first invert the matrix before sending it to the fragment shader: 653 </p> 654 655 <pre><code> 656 vs_out.TBN = transpose(mat3(T, B, N)); 657 </code></pre> 658 659 <p> 660 Note that we use the <fun>transpose</fun> function instead of the <fun>inverse</fun> function here. A great property of orthogonal matrices (each axis is a perpendicular unit vector) is that the transpose of an orthogonal matrix equals its inverse. This is a great property as <fun>inverse</fun> is expensive and a transpose isn't. 661 </p> 662 663 <p> 664 Within the fragment shader we do not transform the normal vector, but we transform the other relevant vectors to tangent space, namely the <var>lightDir</var> and <var>viewDir</var> vectors. That way, each vector is in the same coordinate space: tangent space. 665 </p> 666 667 <pre><code> 668 void main() 669 { 670 vec3 normal = texture(normalMap, fs_in.TexCoords).rgb; 671 normal = normalize(normal * 2.0 - 1.0); 672 673 vec3 lightDir = fs_in.TBN * normalize(lightPos - fs_in.FragPos); 674 vec3 viewDir = fs_in.TBN * normalize(viewPos - fs_in.FragPos); 675 [...] 676 } 677 </code></pre> 678 679 <p> 680 The second approach looks like more work and also requires matrix multiplications in the fragment shader, so why would we bother with the second approach? 681 </p> 682 683 <p> 684 Well, transforming vectors from world to tangent space has an added advantage in that we can transform all the relevant lighting vectors to tangent space in the vertex shader instead of in the fragment shader. This works, because <var>lightPos</var> and <var>viewPos</var> don't update every fragment run, and for <var>fs_in.FragPos</var> we can calculate its tangent-space position in the vertex shader and let fragment interpolation do its work. There is effectively no need to transform a vector to tangent space in the fragment shader, while it is necessary with the first approach as sampled normal vectors are specific to each fragment shader run. 685 </p> 686 687 <p> 688 So instead of sending the inverse of the TBN matrix to the fragment shader, we send a tangent-space light position, view position, and vertex position to the fragment shader. This saves us from having to do matrix multiplications in the fragment shader. This is a nice optimization as the vertex shader runs considerably less often than the fragment shader. This is also the reason why this approach is often the preferred approach. 689 </p> 690 691 <pre><code> 692 out VS_OUT { 693 vec3 FragPos; 694 vec2 TexCoords; 695 vec3 TangentLightPos; 696 vec3 TangentViewPos; 697 vec3 TangentFragPos; 698 } vs_out; 699 700 uniform vec3 lightPos; 701 uniform vec3 viewPos; 702 703 [...] 704 705 void main() 706 { 707 [...] 708 mat3 TBN = transpose(mat3(T, B, N)); 709 vs_out.TangentLightPos = TBN * lightPos; 710 vs_out.TangentViewPos = TBN * viewPos; 711 vs_out.TangentFragPos = TBN * vec3(model * vec4(aPos, 1.0)); 712 } 713 </code></pre> 714 715 <p> 716 In the fragment shader we then use these new input variables to calculate lighting in tangent space. As the normal vector is already in tangent space, the lighting makes sense. 717 </p> 718 719 <p> 720 With normal mapping applied in tangent space, we should get similar results to what we had at the start of this chapter. This time however, we can orient our plane in any way we'd like and the lighting would still be correct: 721 </p> 722 723 <pre><code> 724 glm::mat4 model = glm::mat4(1.0f); 725 model = <function id='57'>glm::rotate</function>(model, (float)<function id='47'>glfwGetTime</function>() * -10.0f, glm::normalize(glm::vec3(1.0, 0.0, 1.0))); 726 shader.setMat4("model", model); 727 RenderQuad(); 728 </code></pre> 729 730 <p> 731 Which indeed looks like proper normal mapping: 732 </p> 733 734 <img src="/img/advanced-lighting/normal_mapping_correct_tangent.png" class="clean" alt="Correct normal mapping with tangent space transformations in OpenGL"/> 735 736 <p> 737 You can find the source code <a href="/code_viewer_gh.php?code=src/5.advanced_lighting/4.normal_mapping/normal_mapping.cpp" target="_blank">here</a>. 738 </p> 739 740 <h2>Complex objects</h2> 741 <p> 742 We've demonstrated how we can use normal mapping, together with tangent space transformations, by manually calculating the tangent and bitangent vectors. Luckily for us, having to manually calculate these tangent and bitangent vectors is not something we do too often. Most of the time you implement it once in a custom model loader, or in our case use a <a href="https://learnopengl.com/Model-Loading/Assimp" target="_blank">model loader</a> using Assimp. 743 </p> 744 745 <p> 746 Assimp has a very useful configuration bit we can set when loading a model called <var>aiProcess_CalcTangentSpace</var>. When the <var>aiProcess_CalcTangentSpace</var> bit is supplied to Assimp's <fun>ReadFile</fun> function, Assimp calculates smooth tangent and bitangent vectors for each of the loaded vertices, similarly to how we did it in this chapter. 747 </p> 748 749 <pre><code> 750 const aiScene *scene = importer.ReadFile( 751 path, aiProcess_Triangulate | aiProcess_FlipUVs | aiProcess_CalcTangentSpace 752 ); 753 </code></pre> 754 755 <p> 756 Within Assimp we can then retrieve the calculated tangents via: 757 </p> 758 759 <pre><code> 760 vector.x = mesh->mTangents[i].x; 761 vector.y = mesh->mTangents[i].y; 762 vector.z = mesh->mTangents[i].z; 763 vertex.Tangent = vector; 764 </code></pre> 765 766 <p> 767 Then you'll have to update the model loader to also load normal maps from a textured model. The wavefront object format (.obj) exports normal maps slightly different from Assimp's conventions as <var>aiTextureType_NORMAL</var> doesn't load normal maps, while <var>aiTextureType_HEIGHT</var> does: 768 </p> 769 770 <pre><code> 771 vector<Texture> normalMaps = loadMaterialTextures(material, aiTextureType_HEIGHT, "texture_normal"); 772 </code></pre> 773 774 <p> 775 Of course, this is different for each type of loaded model and file format. 776 </p> 777 778 <!--It's also important to realize that <var>aiProcess_CalcTangentSpace</var> doesn't always work. Calculating tangents is based on texture coordinates and some 3D artists do certain texture tricks like mirroring a texture surface over a model; this gives incorrect results when the mirroring is not taken into account. The nanosuit model for instance doesn't produce proper tangents as it has mirrored texture coordinates. Assimp gives us a multiplication factor (<code>1</code> or <code>-1</code>) in the tangent's <code>w</code> coordinate that we can use to multiply the bitangent with to account for the mirroring. 779 </p>--> 780 781 <p> 782 Running the application on a model with specular and normal maps, using an updated model loader, gives the following result: 783 </p> 784 785 <img src="/img/advanced-lighting/normal_mapping_complex_compare.png" alt="Normal mapping in OpenGL on a complex object loaded with Assimp"/> 786 787 <p> 788 As you can see, normal mapping boosts the detail of an object by an incredible amount without too much extra cost. 789 </p> 790 791 <p> 792 Using normal maps is also a great way to boost performance. Before normal mapping, you had to use a large number of vertices to get a high number of detail on a mesh. With normal mapping, we can get the same level of detail on a mesh using a lot less vertices. The image below from Paolo Cignoni shows a nice comparison of both methods: 793 </p> 794 795 <img src="/img/advanced-lighting/normal_mapping_comparison.png" alt="Comparrison of visualizing details on a mesh with and without normal mapping"/> 796 797 <p> 798 The details on both the high-vertex mesh and the low-vertex mesh with normal mapping are almost indistinguishable. So normal mapping doesn't only look nice, it's a great tool to replace high-vertex meshes with low-vertex meshes without losing (too much) detail. 799 </p> 800 801 <h2>One last thing</h2> 802 <p> 803 There is one last trick left to discuss that slightly improves quality without too much extra cost. 804 </p> 805 806 <p> 807 When tangent vectors are calculated on larger meshes that share a considerable amount of vertices, the tangent vectors are generally averaged to give nice and smooth results. A problem with this approach is that the three TBN vectors could end up non-perpendicular, which means the resulting TBN matrix would no longer be orthogonal. Normal mapping would only be slightly off with a non-orthogonal TBN matrix, but it's still something we can improve. 808 </p> 809 810 <p> 811 Using a mathematical trick called the <def>Gram-Schmidt process</def>, we can <def>re-orthogonalize</def> the TBN vectors such that each vector is again perpendicular to the other vectors. Within the vertex shader we would do it like this: 812 </p> 813 814 <pre><code> 815 vec3 T = normalize(vec3(model * vec4(aTangent, 0.0))); 816 vec3 N = normalize(vec3(model * vec4(aNormal, 0.0))); 817 // re-orthogonalize T with respect to N 818 T = normalize(T - dot(T, N) * N); 819 // then retrieve perpendicular vector B with the cross product of T and N 820 vec3 B = cross(N, T); 821 822 mat3 TBN = mat3(T, B, N) 823 </code></pre> 824 825 <p> 826 This, albeit by a little, generally improves the normal mapping results with a little extra cost. Take a look at the end of the <em>Normal Mapping Mathematics</em> video in the additional resources for a great explanation of how this process actually works. 827 </p> 828 829 <h2>Additional resources</h2> 830 <ul> 831 <li><a href="http://ogldev.atspace.co.uk/www/tutorial26/tutorial26.html" target="_blank">Tutorial 26: Normal Mapping</a>: normal mapping tutorial by ogldev.</li> 832 <li><a href="https://www.youtube.com/watch?v=LIOPYmknj5Q" target="_blank">How Normal Mapping Works</a>: a nice video tutorial of how normal mapping works by TheBennyBox.</li> 833 <li><a href="https://www.youtube.com/watch?v=4FaWLgsctqY" target="_blank">Normal Mapping Mathematics</a>: a similar video by TheBennyBox about the mathematics behind normal mapping.</li> 834 <li><a href="http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-13-normal-mapping/" target="_blank">Tutorial 13: Normal Mapping</a>: normal mapping tutorial by opengl-tutorial.org.</li> 835 </ul> 836 837 </div> 838 839 <div id="hover"> 840 HI 841 </div> 842 <!-- 728x90/320x50 sticky footer --> 843 <div id="waldo-tag-6196"></div> 844 845 <div id="disqus_thread"></div> 846 847 848 849 850 </div> <!-- container div --> 851 852 853 </div> <!-- super container div --> 854 </body> 855 </html> 856 </main> 857 </body> 858 </html>