Types of normal maps

Normal maps are a crucial technique in 3D graphics that add significant detail and realism to surfaces.
They do this by storing information about the direction of surface normals, which affects how light interacts with the surface.

RGB Channels

---

Shaders can use different techniques to render tangent-space normal maps, but the normal map directions are usually consistent within a game.

Usually the red channel of a tangent-space normal map stores the X axis (pointing the normals predominantly leftwards or rightwards), the green channel stores the Y axis (pointing the normals predominantly upwards or downwards), and the blue channel stores the Z axis (pointing the normals outwards away from the surface).

If you see lighting coming from the wrong angle when you’re looking at your normal-mapped model, and the model is using a tangent-space normal map, the normal map shader might be expecting the red or green channel (or both) to point in the opposite direction.
To fix this either change the shader, or simply invert the appropriate color channels in an image editor, so that the black pixels become white and the white pixels become black.

Some shaders expect the color channels to be swapped or re-arranged to work with a particular compression format.
This re-arranging of the normal map axes is called swizzling.

Swizzle

---

Swizzling means to change the order of the color channels in a texture. For example, moving the red channel into the alpha channel, to prepare an image for DXT5nm compression.

Usually swizzling is done to create better results when using image compression, since the green channel sometimes uses more bits.

This is because the human visual system is biased to perceive more values in the green range than in red or blue.

Here are the main types of normal maps

Tangent-Space Normal Maps

---

• Most Common Type: These are the most widely used normal maps.
• How They Work: They store normal vectors relative to the surface’s tangent space, a local coordinate system defined by the surface’s orientation. This makes them independent of the object’s position and rotation in the world.  
• Advantages: Highly versatile, work well with deformations and animations.  

Object-Space Normal Maps

---

• Less Common: These store normal vectors in the object’s coordinate system.
• How They Work: They are less flexible than tangent-space maps because they are tied to the object’s orientation.
• Advantages: Can be simpler to create and use in some cases.

World-Space Normal Maps

---

• Rarely Used: These store normal vectors in the world’s coordinate system.
• How They Work: They are even less flexible than object-space maps, as they are tied to the world’s coordinate system.
• Advantages: Can be useful in very specific situations where the object’s orientation is fixed.

2-Channel Tangent Space Normal Maps: A Space-Saving Technique

---

A 2-channel tangent space normal map is an optimization technique where only two channels (typically red and green) of the texture are used to store normal vector information.
The third channel (blue) is left empty or filled with a constant value.

– How it Works:

• Tangent Space: Normal vectors are stored relative to the surface’s tangent space, a local coordinate system defined by the surface’s orientation.
• Vector Reconstruction: The shader can reconstruct the missing third component of the normal vector using the known tangent and bitangent vectors. This is possible because these vectors are orthogonal (perpendicular) to each other.

Advantages:

• Reduced Memory Usage: By using only two channels, you can significantly reduce the memory footprint of your normal maps. This is especially beneficial for mobile devices or platforms with limited memory.
• Faster Texture Loading: Smaller texture files load faster, which can improve performance, especially for games with many textures.

Disadvantages:

• Slight Loss of Precision: Reconstructing the third component can introduce minor inaccuracies, potentially leading to slight artifacts in the final rendering. However, these artifacts are often negligible and can be mitigated with careful implementation.

Implementation:

• Shader Support: Your shader must be specifically designed to handle 2-channel normal maps. It needs to include the logic to reconstruct the missing component of the normal vector.
• Texture Compression: Consider using appropriate texture compression formats that can further reduce file size without significantly impacting quality.

In Conclusion: 2-channel tangent space normal maps offer a valuable optimization technique for reducing memory usage and improving performance.
While they may introduce minor artifacts, the benefits often outweigh the drawbacks, especially in resource-constrained environments.

Bent Normal Maps: Enhancing Specular Occlusion

---

Understanding Bent Normals

Bent normal maps are a specialized type of normal map designed to improve the accuracy of specular highlights on surfaces with intricate details or complex geometry. They go beyond traditional normal maps by incorporating information about how surrounding geometry affects the reflection of light.

How They Work:

1. Specular Occlusion: When light hits a surface, the surrounding geometry can block or “occlude” the reflection of light. This is particularly noticeable in areas with deep crevices, sharp corners, or intricate patterns.
2. Bent Normal Encoding: Bent normal maps encode information about these occlusions directly into the normal map data. Instead of just storing surface normals, they store “bent” normals that are influenced by the surrounding geometry.
3. Shader Implementation: In the shader, these bent normals are used to determine the direction of specular highlights. By taking into account the occlusions, the shader can produce more realistic and accurate reflections.

Mikkt Tangent Space Normal Maps: A Universal Standard

---

• The Issue: Traditionally, calculating tangent spaces for normal mapping could vary between software and even different versions of the same software. This inconsistency led to issues like shading seams and incorrect lighting when using normal maps.
• The Solution: Mikkt tangent space was introduced to address this problem. It’s a standardized method for calculating tangent spaces, ensuring consistent results across different tools and platforms.

Key Advantages of Mikkt Tangent Space

• Consistency: Mikkt tangent space guarantees that normal maps baked using this method will render correctly in any software or engine that also uses Mikkt tangent space.
• Reduced Shading Artifacts: By eliminating inconsistencies in tangent space calculation, Mikkt helps minimize shading seams and other artifacts that can occur when using normal maps.
• Industry Standard: Many popular 3D modeling, baking, and game development tools now support Mikkt tangent space, making it the de facto standard for normal mapping.

How Mikkt Tangent Space Works

1. Tangent Frame Calculation: At each vertex of the mesh, a local coordinate system (tangent frame) is calculated. This frame consists of three vectors:
   • Tangent: The direction of the primary texture coordinate (usually ‘U’).
   • Bitangent: The direction of the secondary texture coordinate (usually ‘V’).
   • Normal: The surface normal at that vertex.
2. Normal Map Encoding: The normal vectors stored in the normal map are encoded relative to this Mikkt tangent frame.

In Summary: Mikkt tangent space normal maps are essential for achieving consistent and accurate results when using normal maps in 3D graphics.
By using a standardized method for calculating tangent spaces, Mikkt helps ensure that your normal maps will render correctly across different software and platforms.

Key Points:

• Tangent-Space is King: For most applications, tangent-space normal maps are the preferred choice due to their flexibility and versatility.  
• Normal Map Formats: Normal maps are typically stored in image formats like PNG or DDS, with specific encoding methods to store normal vector information efficiently.
• Creation: Normal maps are often created from high-resolution 3D models or by sculpting techniques.

16-bit Normal Maps: A Deeper Dive

---

16-bit normal maps offer a significant increase in precision compared to their 8-bit counterparts. Here’s a breakdown:

Why 16-bit?

• Increased Precision: Each color channel in a 16-bit normal map has 65,536 possible values, compared to 256 in 8-bit. This translates to finer gradients and smoother transitions in the normal map data.
• Reduced Banding: Banding artifacts, visible as noticeable steps or lines in the lighting and shading, are significantly reduced with 16-bit precision. This leads to a more visually appealing and realistic appearance.
• Preserving Detail: 16-bit normal maps can better preserve fine details and subtle variations in surface normals, especially when dealing with highly detailed models or intricate textures.

When to Use 16-bit Normal Maps:

• High-Detail Models: For models with intricate geometry or fine surface details, 16-bit normal maps can significantly improve the visual quality.
• Subtle Shading Variations: If you need to capture subtle variations in lighting and shading, 16-bit precision is essential.
• Minimizing Banding: In situations where banding is a concern, such as with highly reflective surfaces or under strong lighting, 16-bit normal maps can help eliminate these artifacts.

Considerations:

• File Size: 16-bit normal maps are larger than their 8-bit counterparts, which can increase storage requirements and potentially impact loading times.
• Hardware Support: Ensure that your target hardware and software support 16-bit normal maps. Not all platforms or rendering engines may fully utilize the increased precision.

In Summary: 16-bit normal maps offer a significant improvement in quality and visual fidelity, especially for high-detail models and scenes where subtle shading variations are important.
While they come with a slight increase in file size, the benefits in terms of reduced banding and improved detail often outweigh the drawbacks.

---

By understanding these different types of normal maps, you can make informed decisions about which one is best suited for your specific 3D graphics project.