Normal maps as well as bump/height maps

[Hastouki]

Hi, really hoping to get a conversation going here. Over the last few days I've been playing around with texture painting in 3DCoat and have hit a bit of a wall when dealing with PBR normal maps. This is odd considering normal maps are standard practice with PBR materials, especially when preparing assets for real-time rendering / game engines. I've reached out and discussed this with a number of people, including 3DCoat staff who consistently repeat to me the following:

1. Normal maps are supported, select your normal map in the "Depth" field when making your smart material and it will convert it for you.
2. Height/depth maps are "better" and I should be using those instead of normals.
   1. They are different things and both have their place. Especially when using heightmaps to displace geometry.
3. Normal maps don't make sense when texture painting directly because of tiling or something?

I've put together a few screenshots of what my one of my paid PBR materials looks like when used in 3DCoat in a couple of ways, as well as what it looks like when used via Marmoset Toolbag 4 for reference. I can correctly use this PBR material using ArmorPaint, Substance and Marmoset and Quixel, all of which allow me to apply a normal texture along with albedo, rough, metal etc. You basically just create a fill layer and provide a normal map as part of the material you are filling the layer with. I had a response from support@3dcoat.com which basically told me to convert the map, when I asked for some more details I basically got silence/ignored. Is this something that the 3DCoat dev team is looking to add at some point? Is there an actual technical reason why you can't sample from normal maps when painting? I used the shader materials to bake in the normals prior to texturing but vertex painting those details is just not a solution.

 

3DCoat - Using PBR provided height map

No surface scratches, only general bump information. Scratches may be only slightly visible because they are present in albedo and roughness details.

3DCoat - Using generated depth map from normal map source

This is just wrong, looks like a normal map I would have generated from a colour image or something in GIMP. Notice the weird warping artifacts around the plates where they connect, this is unusable.

 

 

Marmoset - No normals, No Depth/Bump (Just for reference)

 

 

Marmoset - Depth/Bump Only

Take a look at what details the texture artist captured in the height map, scratches are not accentuated in this map, just general material bumpiness. The scratches seen only because they are present in the albedo and roughness textures details.

 

Marmoset - Normals Only

Scratches and other small details are accentuated in the normal map. These are completely different artistic details that are present in this PBR material that are to be replicated using lighting/normals rather than height/displacement or bump.

 

Marmoset - Height and Normals

Edited March 4, 2023 by Hastouki

[Oleg Shapov]

Could you share a set of these textures to create a material and test?

[Hastouki]

Thanks for the reply, I've attached the ZIP file to this comment, hope that's ok since I've purchased these.

black_metal_plating_28_73.zip

[Hastouki]

Just also wanted to quickly add, I'm not hung up on surface scratches, just used that as an example of where artists may put certain details in a PBR material. Here is a material that uses the normal map to clearly define the crystalline structure of the PBR material, which can't be reproduced with the height map.

Normal Map

Height/Bump Map

[Oleg Shapov]

3 hours ago, Hastouki said:

Thanks for the reply, I've attached the ZIP file to this comment, hope that's ok since I've purchased these.

I created a material with your textures. Please see if the normal map looks right?

[Hastouki]

Correct me if I'm wrong, but you created the exact same depth map I demonstrated in my 2nd screenshot by simply letting 3DC convert the normal map. This isn't correct, you simply cannot take a normal map which holds XYZ vector information per texel, flatten it down to a height map value and expect correct light shading to occur as if you had the original normal map. I write a lot of graphics/vulkan/engine code and can tell you from experience that during a lighting subpass, the normals are required to get the correct details on the surface. If you destructively remove the direction information from a material and replace it with a depth/height value, you can no longer shade it correctly.

 

Has this not come up in other posts or discussions? I'm honestly quite surprised that this doesn't exist or that more people aren't discussing it. Did you see the last screenshots I posted?

[MJonathan]

I agree with you Hostouki, bump maps and normal maps are differents. The same with displacements and vector displacements. I think they need to put your observation in a wish list to develop a solution. 

[phoenixart]

2 hours ago, Hastouki said:

[...]

Has this not come up in other posts or discussions? I'm honestly quite surprised that this doesn't exist or that more people aren't discussing it. Did you see the last screenshots I posted?

I was, and still am, a bit confused about the absence of the normal map channel in the 3D Coat workflow.

Not sure if it's something simply being ignored, missing for whatever reason, or if there's a way to get a proper normal map which still eludes me.

 

[Andrew Shpagin]

Looks like displacement recovering algorithm chosen incorrect type of normalmap (of two), so produced incorrect result. I will add checkbox to the dialog to force channel invertion before recovering.  

Will be available in 2023.03

[Hastouki]

Hey @Andrew Shpagin, thank you for taking the time to reply. I know this is a lot of text and sorry for all the screenshots, but I think this is an important topic and I imagine many people would benefit from a true normal input channel in 3DC. I hope this post is more helpful overall than a chore to deal with.

Looking at your image, it still looks quite strange due to the conversion, I don't believe the issues stem just from the normal map format. The normal maps I provided are D3D format (I think Substance painter format in 3DC), but I choose OGL as my bake type in 3DC because that's what I want to ultimately export for my own game engine. Btw, even if I purposely choose the wrong handedness direction in Substance, the normal maps look better and less artifacty. Of course they look inverted, but still do not have the weird fading/bending than the converted maps in 3DC (see below).

 

How can we expect correct shading with a simple depth value. In my own GLSL shader code there is texture sampler with per-fragment vec3 available for dot products and other calculations, I can't just replace that with a simple depth value. If you could do that, I'd save on quite a bit of video memory in my own deferred renderer, but I absolutely can't. Perhaps its my own ignorance, but I don't know what mathematics one can use to correctly calculate lighting using a depth/scalar value. Perhaps you generate the height map, then regenerate the normals during painting with some procedural (incorrect) normals that weren't what the original PBR material contained.

 

3DC Generated Depth Map (from Normals)

There is no way this information is sufficient or correct, look at the fading where the bolts are, I don't think this is just from Y direction.

 

Original height map provided with PBR Material Pack

This map would be useful for geometry displacement, but not really not that interesting for surface details.

 

Further Testing

I just created a box similar to what you posted and applied the material using a fill layer in Substance Painter. The difference in the way the normals are presented is quite evident:

 

3DCoat Zoom from your image

 

3DCoat box I just created

I understand that there was an incorrect normal map format, just pointing out the strange bending look at the junction points on top of the inverted looking lighting.

 

Substance Painter w/ Normal Map

 

 

Substance Painter, choosing incorrect normal format  on purpose (for reference)

Inverted and wrong, still no weird bending at junctions/bolts as seen in 3DC converted map.

 

[Hastouki]

Just adding some examples using Marmoset.

Correct Normal Format

 

Incorrect Normal Format

Scratches look inverted, etc

[Andrew Shpagin]

Have you tried the 04 build? It recovers depth from normal correctly with option I pointed.

[Hastouki]

Hey Andrew, just finished more testing and put together some more screen grabs. Things definitely look better than what 3DC was generating before, but there seem to be issues. I spent a lot of time putting together my previous posts, I would really appreciate if perhaps you can touch on why I should be expecting an accurate given the conversion of the normals.

 

 

New 3DC Generated Depth Map

Looks much better, can still see warping near bolts. You can see in my previous comments how this material looks in other software packages.

 

Close-up of the new 3DC generated map

Much better than the previous result posted above, but there should not be fading/gradient/warping present, but its clearly in the generated depth map.

Another example, using Marmoset and the actual normal map

 

And now the 3DC generated bump map, please notice the circular artifacts inside the crevices, probably caused by the banding seen below:

and this is the bump map it generated, its hard to see the banding, but its there.

[Andrew Shpagin]

Why 3DCoat need depth? Because you may paint with different materials over each other even on same layer and 3DCoat may mix them. To mix them I need depth. From that summary depth I generate normalmap that you see as the final render. So, if you have depth channel, use depth for the smart material. The "ladder" problem really may be improved, look for the upcoming 06 build.

[Andrew Shpagin]

Please drop me that blue spots normalmap for tests.

[Hastouki]

Hey Andrew, I understand why the depth is important, my question was more around why we use ONLY depth, disregarding the normal vectors during shading. It would be great to allow for normal map sampling directly. On top of the issues we're discussing, the conversion is very slow, productivity killer.

I'll send you an email with a link to the texture files.

 

Thank you!

Edited March 8, 2023 by Hastouki

[Andrew Shpagin]

In 06 recovered displacement will be saved as 16-bit tiff, so no ladders problem anymore.

[Hastouki]

Great, I'll give it a shot, thank you!

[Carlosan]

3DCoat-2023-06 

- If you import normamap into smart material depth channel, if will be converted to 16-bit tif to prevent "ladder" effect.

[Hastouki]

Thanks @Carlosan, I'll be doing some testing. The long conversion times are huge pain when trying out materials and being creative, but I appreciate the effort @Andrew Shpagin  has put into this so far.

 

Thank you!

[Hastouki]

Ok I took some time to test a few things. I generated a new TIF bump/depth map using the new features @Andrew Shpaginhas provided, then took the texture and performed an exact render using Marmoset Toolbag 4. This is so that I can demonstrate the differences between rendering the PBR using a normal map and the generated bump map. Overall, its a definite improvement from the issues presented previously, but I've also noticed some strangeness where the generated bump map doesn't match the details of the original diffuse/normal/roughness etc maps. Please take a very close look at the following images. I'm forced to upload them on my own server as the forum here has a limit on uploads.

 

Rendered using Original Normal Map

* Everything looks pretty good

https://photos.ensoft.io/picture.php?/13/category/6

 

Rendered using Generated Bump Map (Invert option selected)

* Please see how the features/indents on this map do not match the original normal map features, they do not line up. Take a look at the 3DCoat Generated map image below. The features also look "softer" and less sharp, which is why I'm testing with a crystal texture that requires sharp and exact edge details on the surface.

https://photos.ensoft.io/picture.php?/12/category/6

 

NEW 3DCoat Inverted Generated Map

* Please see how the features on this map DO NOT match up with the original normal map, 

https://photos.ensoft.io/picture.php?/15/category/6

 

Original Height/Bump Map (Provided with my PBR material)

https://photos.ensoft.io/picture.php?/14/category/6

 

OLD 3DCoat Generated Map (Non inverted) for references

* Pretty screwed up

https://photos.ensoft.io/picture.php?/16/category/6

 

Edited March 12, 2023 by Hastouki

[Carlosan]

 did you use this option?

[Hastouki]

31 minutes ago, Carlosan said:

 did you use this option?

 

Absolutely, though I should have been more clear in my post. Sorry, too much other stuff to post/write

[Andrew Shpagin]

Looks like difference is that generated displacement is flipped vertically and has a little bigger contrast.

Please give me link to color/normal/displacement for the fine-tuning. Of course generated displacement will be different from original up to some scaling factor.

[Hastouki]

In theory you're doing a 1:1 conversion of the normal map to a displacement map correct? None of the smart material settings should affect what the conversion looks like, right? Meaning, I would assume I'm not doing something on my end to cause the displacement map to be generated flipped. I can't imagine any of the smart material settings would affect the conversion of the normal -> displacement as that should probably be a 1:1 conversion based on the source normal map.

 

The texture maps of the PBR material are the same ones I emailed you few days ago, I'll email the link again in case your email inbox is a disaster

 

Thank you!