Normal maps as well as bump/height maps

[Hastouki]

Hi, really hoping to get a conversation going here. Over the last few days I've been playing around with texture painting in 3DCoat and have hit a bit of a wall when dealing with PBR normal maps. This is odd considering normal maps are standard practice with PBR materials, especially when preparing assets for real-time rendering / game engines. I've reached out and discussed this with a number of people, including 3DCoat staff who consistently repeat to me the following:

1. Normal maps are supported, select your normal map in the "Depth" field when making your smart material and it will convert it for you.
2. Height/depth maps are "better" and I should be using those instead of normals.
   1. They are different things and both have their place. Especially when using heightmaps to displace geometry.
3. Normal maps don't make sense when texture painting directly because of tiling or something?

I've put together a few screenshots of what my one of my paid PBR materials looks like when used in 3DCoat in a couple of ways, as well as what it looks like when used via Marmoset Toolbag 4 for reference. I can correctly use this PBR material using ArmorPaint, Substance and Marmoset and Quixel, all of which allow me to apply a normal texture along with albedo, rough, metal etc. You basically just create a fill layer and provide a normal map as part of the material you are filling the layer with. I had a response from support@3dcoat.com which basically told me to convert the map, when I asked for some more details I basically got silence/ignored. Is this something that the 3DCoat dev team is looking to add at some point? Is there an actual technical reason why you can't sample from normal maps when painting? I used the shader materials to bake in the normals prior to texturing but vertex painting those details is just not a solution.

 

3DCoat - Using PBR provided height map

No surface scratches, only general bump information. Scratches may be only slightly visible because they are present in albedo and roughness details.

3DCoat - Using generated depth map from normal map source

This is just wrong, looks like a normal map I would have generated from a colour image or something in GIMP. Notice the weird warping artifacts around the plates where they connect, this is unusable.

 

 

Marmoset - No normals, No Depth/Bump (Just for reference)

 

 

Marmoset - Depth/Bump Only

Take a look at what details the texture artist captured in the height map, scratches are not accentuated in this map, just general material bumpiness. The scratches seen only because they are present in the albedo and roughness textures details.

 

Marmoset - Normals Only

Scratches and other small details are accentuated in the normal map. These are completely different artistic details that are present in this PBR material that are to be replicated using lighting/normals rather than height/displacement or bump.

 

Marmoset - Height and Normals

Edited March 4, 2023 by Hastouki

[Dmitriy Nos]

Could you share a set of these textures to create a material and test?

[Hastouki]

Thanks for the reply, I've attached the ZIP file to this comment, hope that's ok since I've purchased these.

black_metal_plating_28_73.zip

[Hastouki]

Just also wanted to quickly add, I'm not hung up on surface scratches, just used that as an example of where artists may put certain details in a PBR material. Here is a material that uses the normal map to clearly define the crystalline structure of the PBR material, which can't be reproduced with the height map.

Normal Map

Height/Bump Map

[Dmitriy Nos]

3 hours ago, Hastouki said:

Thanks for the reply, I've attached the ZIP file to this comment, hope that's ok since I've purchased these.

I created a material with your textures. Please see if the normal map looks right?

[Hastouki]

Correct me if I'm wrong, but you created the exact same depth map I demonstrated in my 2nd screenshot by simply letting 3DC convert the normal map. This isn't correct, you simply cannot take a normal map which holds XYZ vector information per texel, flatten it down to a height map value and expect correct light shading to occur as if you had the original normal map. I write a lot of graphics/vulkan/engine code and can tell you from experience that during a lighting subpass, the normals are required to get the correct details on the surface. If you destructively remove the direction information from a material and replace it with a depth/height value, you can no longer shade it correctly.

 

Has this not come up in other posts or discussions? I'm honestly quite surprised that this doesn't exist or that more people aren't discussing it. Did you see the last screenshots I posted?

[MJonathan]

I agree with you Hostouki, bump maps and normal maps are differents. The same with displacements and vector displacements. I think they need to put your observation in a wish list to develop a solution. 

[phoenixart]

2 hours ago, Hastouki said:

[...]

Has this not come up in other posts or discussions? I'm honestly quite surprised that this doesn't exist or that more people aren't discussing it. Did you see the last screenshots I posted?

I was, and still am, a bit confused about the absence of the normal map channel in the 3D Coat workflow.

Not sure if it's something simply being ignored, missing for whatever reason, or if there's a way to get a proper normal map which still eludes me.

 

[Andrew Shpagin]

Looks like displacement recovering algorithm chosen incorrect type of normalmap (of two), so produced incorrect result. I will add checkbox to the dialog to force channel invertion before recovering.  

Will be available in 2023.03

[Hastouki]

Hey @Andrew Shpagin, thank you for taking the time to reply. I know this is a lot of text and sorry for all the screenshots, but I think this is an important topic and I imagine many people would benefit from a true normal input channel in 3DC. I hope this post is more helpful overall than a chore to deal with.

Looking at your image, it still looks quite strange due to the conversion, I don't believe the issues stem just from the normal map format. The normal maps I provided are D3D format (I think Substance painter format in 3DC), but I choose OGL as my bake type in 3DC because that's what I want to ultimately export for my own game engine. Btw, even if I purposely choose the wrong handedness direction in Substance, the normal maps look better and less artifacty. Of course they look inverted, but still do not have the weird fading/bending than the converted maps in 3DC (see below).

 

How can we expect correct shading with a simple depth value. In my own GLSL shader code there is texture sampler with per-fragment vec3 available for dot products and other calculations, I can't just replace that with a simple depth value. If you could do that, I'd save on quite a bit of video memory in my own deferred renderer, but I absolutely can't. Perhaps its my own ignorance, but I don't know what mathematics one can use to correctly calculate lighting using a depth/scalar value. Perhaps you generate the height map, then regenerate the normals during painting with some procedural (incorrect) normals that weren't what the original PBR material contained.

 

3DC Generated Depth Map (from Normals)

There is no way this information is sufficient or correct, look at the fading where the bolts are, I don't think this is just from Y direction.

 

Original height map provided with PBR Material Pack

This map would be useful for geometry displacement, but not really not that interesting for surface details.

 

Further Testing

I just created a box similar to what you posted and applied the material using a fill layer in Substance Painter. The difference in the way the normals are presented is quite evident:

 

3DCoat Zoom from your image

 

3DCoat box I just created

I understand that there was an incorrect normal map format, just pointing out the strange bending look at the junction points on top of the inverted looking lighting.

 

Substance Painter w/ Normal Map

 

 

Substance Painter, choosing incorrect normal format  on purpose (for reference)

Inverted and wrong, still no weird bending at junctions/bolts as seen in 3DC converted map.

 

[Hastouki]

Just adding some examples using Marmoset.

Correct Normal Format

 

Incorrect Normal Format

Scratches look inverted, etc

[Andrew Shpagin]

Have you tried the 04 build? It recovers depth from normal correctly with option I pointed.

[Hastouki]

Hey Andrew, just finished more testing and put together some more screen grabs. Things definitely look better than what 3DC was generating before, but there seem to be issues. I spent a lot of time putting together my previous posts, I would really appreciate if perhaps you can touch on why I should be expecting an accurate given the conversion of the normals.

 

 

New 3DC Generated Depth Map

Looks much better, can still see warping near bolts. You can see in my previous comments how this material looks in other software packages.

 

Close-up of the new 3DC generated map

Much better than the previous result posted above, but there should not be fading/gradient/warping present, but its clearly in the generated depth map.

Another example, using Marmoset and the actual normal map

 

And now the 3DC generated bump map, please notice the circular artifacts inside the crevices, probably caused by the banding seen below:

and this is the bump map it generated, its hard to see the banding, but its there.

[Andrew Shpagin]

Why 3DCoat need depth? Because you may paint with different materials over each other even on same layer and 3DCoat may mix them. To mix them I need depth. From that summary depth I generate normalmap that you see as the final render. So, if you have depth channel, use depth for the smart material. The "ladder" problem really may be improved, look for the upcoming 06 build.

[Andrew Shpagin]

Please drop me that blue spots normalmap for tests.