Jump to content
3D Coat Forums
toomanydemons

[Solved] The Normal Map Preset issue

Recommended Posts

I baked some mesh maps in Substance Painter for an old project. It's an atlas of small items for detailing an environment (bottles, books, cans)

In Substance, the OpenGL maps bake perfectly. Here are screenshots from Substance Painter:

 

85a98b72bd4ac61a66d5807d652045e7-png.jpg

 

I decided that there were a lot of cereal boxes, logos and things I would need to paint in an external image editor, and I liked Coat's linking option (for Affinity). So I ported the project over to Coat.

I imported the normal map with the correct settings. The normal map doesn't show up correctly. I double-checked and toggled the green channel to be sure. The dark halos around the triangles are horrible in 3D Coat. Here's a screenshot:

 

365516fe8648d1766f7f3341c543535f-jpg.jpg

 

So I tried re-importing the normal map and even creating a new project and changing the UV smoothing settings. To no avail.

Then I discovered the problem.

 

bc9d5f7b7b028981f7bb3a7cf160456b-png.jpg

 

The software preset Blender and Unity yield different results, even though they should both be exactly the same (an OpenGL normal map with an inverted green channel). SO, I created my own normal map preset and called it OpenGL. I didn't do anything at all, I just made a new one.

Now everything shows up great! This is a screenshot from 3D Coat:

eebfd6ffc693d91070d7a926e47209b4-jpg.jpg

 

There should not be a Unity normal map preset vs a Blender normal map preset. They should not be different. There should only be an OpenGL and a DirectX setting for normal maps. Also, imported OpenGl maps using those presets show up wrong, and it will anger a lot of new users coming from other software. I'm also happy to share the model and normal map for anyone to verify my findings here. I'm using 4.8.32-SL(GL64)

 

 

 

Atlas_1_Bake.obj

MAT_RandomProps_Atlas_1_normal_base.png

Share this post


Link to post
Share on other sites

Edit: After some testing, I still can't get an imported normal map to show up correctly. there are always shading issues, however subtle. Also worth noting is that the normal map looks fine in every other software I try it in.

This is a serious show-stopper for me. I have been looking for a viable replacement since Adobe bought Allegorithmic. But there are normal map issues in 3DC that don't exist in any other software I port to. I can bring the normal maps into Unity, Blender, UE4, Substance (Painter & Designer), even sketchfab. 3D Coat is not reading normal maps correctly.

I wonder how long this has been an issue and no one noticed...

Edited by toomanydemons
More information acquired.

Share this post


Link to post
Share on other sites

Also, here is the high poly model if you would like to bake directly in 3D Coat. The normal map shows up fine in this case, but doesn't look as perfect in any of the other aforementioned softwares.

https://www.dropbox.com/s/rk4fj6stn0s43xo/Atlas 1_high.obj?dl=0

I hope these normal map issues get fixes asap. 3D Coat's front page looks juicy and may draw a large market, but normal maps have to be flawless for that to stick.

Share this post


Link to post
Share on other sites

Hi

You use a different export setting.

If you export it "wrong" you have to flip it.

Take a look into Edit > Preferences and choose a different export setting.

Then it should be exported correctly.

 

preferences.jpg

Share this post


Link to post
Share on other sites
2 hours ago, Carlosan said:

Hi

You use a different export setting.

If you export it "wrong" you have to flip it.

Take a look into Edit > Preferences and choose a different export setting.

Then it should be exported correctly.

 

preferences.jpg

I'm not sure if you actually read my post or just glossed over it.... this response has nothing at all to do with my post 0_0.  The normal map software preset doesn't fix the normal mapping. I've had several people on 3D Coat's discord channel verify my findings so far. Download the provided models and see for yourself. 

Share this post


Link to post
Share on other sites

My problem is not with exporting normal maps, it's with importing them. They don't show up correctly. Also changing the Normal Map Software Preset does not work.

Edited by toomanydemons

Share this post


Link to post
Share on other sites

I am sorry to be rude. Try this:

Make a simple model in Blender.

Make an OpenGL normal map in Blender or in Substance Painter.

Import the model and normal map into 3D Coat.

 

The normal map will not look correct. It will only look correct if it was baked in 3D Coat.

Also, the normal map will look fine everywhere else except 3D Coat. 

I understand it is easy for beginners to mess up. So I must say this: I have 10+ years experience in CG and do not mess up normal maps. I know what they are, the differences between OpenGL and DirectX, and how to use them in all software.

Edited by toomanydemons

Share this post


Link to post
Share on other sites

Sorry i cant understand your workflow.

Please try Textures > Import > Normal Map 

Can you see this popup ? 

Are you working with latest version 4.8.33 ?

 

 

NM.jpg

Share this post


Link to post
Share on other sites

This will help. Lock your normals in the import dialog box. I tested both Unity and Blender import settings.

Without locking your normals  3DC will recompute them. This is important when using an imported normal map. Not in all cases but this one for sure.

The only trouble I had were the rectangle boxes. That could be solved by unwrapping them a little different or adding edge loops to soften the 90 degree normals of the edges.

I generally do not have problems between 3DC, SP, Blender and Unity though I tend to add edges loops so the normal map does not have to try and completely fake the edge plus to help shading issues.

The above are suggestions only which made or made not work for your workflow.

unity.jpg

  • Like 1

Share this post


Link to post
Share on other sites
52 minutes ago, digman said:

This will help. Lock your normals in the import dialog box. I tested both Unity and Blender import settings.

Without locking your normals  3DC will recompute them. This is important when using an imported normal map. Not in all cases but this one for sure.

The only trouble I had were the rectangle boxes. That could be solved by unwrapping them a little different or adding edge loops to soften the 90 degree normals of the edges.

I generally do not have problems between 3DC, SP, Blender and Unity though I tend to add edges loops so the normal map does not have to try and completely fake the edge plus to help shading issues.

The above are suggestions only which made or made not work for your workflow.

unity.jpg

Locking normals also does not work. In fact, I've tried everything in every single menu that relates to importing or normals. The fact is, 3D Coat's viewport renderer simply does not see normals correctly.

Things in your screenshot may seem correct, but take a look over here. Here is your screenshot with an indicator: 

 

unity.thumb.jpg.6135cf85267eb09678557ee528f15b27.png

Share this post


Link to post
Share on other sites

Let me better show you. Here is a link to a gif, of me trying to fix the issue: 

https://gyazo.com/e90a0ab735832b55b22858159f314095

The triangles leave a dark halo no matter how perfect you may seemingly get. This is the reason no one has really noticed this error until me. Although if you are experienced in 3D Coat you may have noticed this whole time that your normal maps seem a little off in other software.

So, this is what your box objects look like:

f748c251b484042113c42e680df5c291-jpg.jpg

 

You see the problem?

 

 

Edited by toomanydemons

Share this post


Link to post
Share on other sites

In all other software, those surfaces are perfectly flat:

https://gyazo.com/afcb33cb529cbc8ac887d3ec1e330462

 

https://gyazo.com/93fd4f7a6f01a5f7e008e0b64825caf3

 

This sort of error is hard to spot, especially on organic models like most 3DC users would be accustomed to creating. Factor in that most users only use 3DC for a few of its rooms in their workflow, it may be that fully half or more don't even paint in 3DC. So you can see how this sort of error can escape detection for years.

Edited by toomanydemons

Share this post


Link to post
Share on other sites

Now, if you bake in 3D Coat, your objects will look perfect. However, the exported normal map will seem a little off in other software.

In a nutshell:

 

When I bake normal maps in 3D Coat, they look a little off in other software, even though they look fine in 3D Coat.

When I import normal maps into 3D Coat, they look a little off, even though they look fine in other software.

 

The lack of continuity here cannot be fixed by any option from any menu from my thorough (thorough...) examination. I spent nearly a whole day trying to figure this out. And even if I did find some sort of workaround, software shouldn't work that way. It should work as intended and not rely on any sort of hacky fixes or workarounds that would snag newer users and force a google search. If the most obvious path isn't default then there's a problem with the software.  If "lock normals" did in fact make a difference, then it should've been checked by default since this workflow is common.

Edited by toomanydemons

Share this post


Link to post
Share on other sites

I am reporting this issue to support@3dcoat.com

Share this post


Link to post
Share on other sites
8 minutes ago, Carlosan said:

I am reporting this issue to support@3dcoat.com

I'd also like to request a C4D preset for normals that can be selected in a new PPP project.  I would of created it myself If I knew the operations of C4D. I don't have it but I work with  AAA content/textures created by it and would like absolute compatibility with the libraries I have then export to other pipelines as needed.

Share this post


Link to post
Share on other sites

As stated I already mentioned the rectangle boxes and their problems. I gave my solution for the rectangle boxes. Also mentioned that how I work might not be what you needed. 

I did not fix the rectangle boxes as a normal map was already created for them. 

I hope you find the solutions you need...  

Share this post


Link to post
Share on other sites
19 minutes ago, digman said:

As stated I already mentioned the rectangle boxes and their problems. I gave my solution for the rectangle boxes. Also mentioned that how I work might not be what you needed. 

I did not fix the rectangle boxes as a normal map was already created for them. 

I hope you find the solutions you need...  

Thanks ^_^ I hope it gets solved too.

As for the rectangle boxes, I should not have to do anything to make those work. I should not have to add edge loops or unwrap differently. The main reason is that those normal maps look great in Substance Painter, Designer, Unity, Unreal, Sketchfab, Marmoset, and Blender. 

I shouldn't have to change the vertices and UV's of a model to make it work in 3D Coat. That is an obvious sign that things are broken. 

I've been keeping an eye on 3D Coat as a viable replacement for PBR painting. I hope that happens in the future :) but in order for that to work, the normal map continuity between software must be addressed. It is extremely important, for 3D Coat as well (and everyone who tries the software in the future).

Given the tantalizing advertisement on 3D Coat's home page, it all looks very appetizing!! This normal map issue must be addressed ASAP, and put at the very top of the list, while people are still jumping ship from the Adobe acquisition of Allegorithmic. 

Share this post


Link to post
Share on other sites

I am sorry to tell you, but the problem is your model, not 3DCoat. But Maya and Marmoset Toolbag present the same issue.

I will explain why the best I can, but it starts for the way you are exporting your model from Blender, but it will happen the same on any other software if you do the same.

Yes, normal map might fix a little the shading issues, for how the normal maps are read by software, and that might make the issue a little less too noticeable, but the issue it is still there, you are just ignoring the real issue and then it seems you aren't applying some of the rules that would help you to make easier the job and life easier by baking better normal maps.

 

By opening the model on maya, right away the model looks terrible, I noticed that most normals were average and few ones were hard, which was weird to see. But that's clearly the the main reason of your problems here, how you are exporting the smoothing groups for your model and how they are being read in other software. First, the boxes can't  be soft edges or average, they are 90° angles, and no matter what you do, they will always have this lighting issue if they are not set to hard edges. In the past, before more advanced baking programs like Marmoset appeared to make it easier,  you had to always make a cage, and the cage would be the one that had the average vertex normals, so the cage would be used to do the average 'searching' of the high poly, without causing errors on the baking since the box would remind hard edges. Of course now Marmoset has the checkbox "smooth cage" doing the same and now you don't need a cage unless you really need a cage.

 

So there are two 'rules' here and they are linked as well. 1. when you are doing models with 90° angles, they should be always hard. Of course you can also do a little beveling and avoid the 90° angle edges. this will not only follow the shape closely of the high res model, but also, you can use the soft edges on the box and make it look nicer and with less issues, and rounded and prettier and all that.

2. Hard edges (which pretty much should be 90° angles) should always be split on the UV to avoid issues as well, you might not really notice it on your model, but it is there, and it has to do with how the pixels meet at the 90° angle and how pixels might get blended and they are just different so there is not enough space like a padding would do to fix the small issues that might happen. Yes, it depends on the resolution and UVs and all that and that might make it more noticeable than others, but yes, in a case like this where you are doing boxes, you would have to split all those 90° edges, and yes, that means 6 box sides = 6 islands to avoid issues.


Talking about 3DCoat and how it works with presets and all that. What I understand about it is that by using the presets like Blender or Unity, you are actually using a predefined file with not only how it will treat the Normal map, which you are right, it will be Green flipped or not. But also Tangent Space, the triangulation, and how it exports the normals and some other options. If you use the Unknown option, you will be able to use the ones on preferences, the ones you set. 

If you go to C:\Program Files\3D-Coat-V4.8XXX\ToolsPresets\ExtTools  you will see the XML files and how Unity differs from Blender, because Blender preset uses MikkTSpace while unity is set to LengyelAreaAngleWeightedSpace. That is the only difference. but apparently enough for you to see it because they are being interpreted differently. Technically they shouldn't affect much anything and only the way it reads the normal map if it is 'DirectX' or' OpenGL',  but the root of the shading problem is not about the normal map, it is about the model and it will always be the model.

 

I hope I explained decently what the issues are here so it might help you on your future normal map bakings. Good luck and have a good day!

Edited by Emi
  • Like 4

Share this post


Link to post
Share on other sites
20 minutes ago, Emi said:

I am sorry to tell you, but the problem is your model, not 3DCoat. But Maya and Marmoset Toolbag present the same issue.

I will explain why the best I can, but it starts for the way you are exporting your model from Blender, but it will happen the same on any other software if you do the same.

Yes, normal map might fix a little the shading issues, for how the normal maps are read by software, and that might make the issue a little less too noticeable, but the issue it is still there, you are just ignoring the real issue and then it seems you aren't applying some of the rules that would help you to make easier the job and life easier by baking better normal maps.

 

By opening the model on maya, right away the model looks terrible, I noticed that most normals were average and few ones were hard, which was weird to see. But that's clearly the the main reason of your problems here, how you are exporting the smoothing groups for your model and how they are being read in other software. First, the boxes can't  be soft edges or average, they are 90° angles, and no matter what you do, they will always have this lighting issue if they are not set to hard edges. In the past, before more advanced baking programs like Marmoset appeared to make it easier,  you had to always make a cage, and the cage would be the one that had the average vertex normals, so the cage would be used to do the average 'searching' of the high poly, without causing errors on the baking since the box would remind hard edges. Of course now Marmoset has the checkbox "smooth cage" doing the same and now you don't need a cage unless you really need a cage.

 

So there are two 'rules' here and they are linked as well. 1. when you are doing models with 90° angles, they should be always hard. Of course you can also do a little beveling and avoid the 90° angle edges. this will not only follow the shape closely of the high res model, but also, you can use the soft edges on the box and make it look nicer and with less issues, and rounded and prettier and all that.

2. Hard edges (which pretty much should be 90° angles) should always be split on the UV to avoid issues as well, you might not really notice it on your model, but it is there, and it has to do with how the pixels meet at the 90° angle and how pixels might get blended and they are just different so there is not enough space like a padding would do to fix the small issues that might happen. Yes, it depends on the resolution and UVs and all that and that might make it more noticeable than others, but yes, in a case like this where you are doing boxes, you would have to split all those 90° edges, and yes, that means 6 box sides = 6 islands to avoid issues.


Talking about 3DCoat and how it works with presets and all that. What I understand about it is that by using the presets like Blender or Unity, you are actually using a predefined file with not only how it will treat the Normal map, which you are right, it will be Green flipped or not. But also Tangent Space, the triangulation, and how it exports the normals and some other options. If you use the Unknown option, you will be able to use the ones on preferences, the ones you set. 

If you go to C:\Program Files\3D-Coat-V4.8XXX\ToolsPresets\ExtTools  you will see the XML files and how Unity differs from Blender, because Blender preset uses MikkTSpace while unity is set to LengyelAreaAngleWeightedSpace. That is the only difference. but apparently enough for you to see it because they are being interpreted differently. Technically they shouldn't affect much anything and only the way it reads the normal map if it is 'DirectX' or' OpenGL',  but the root of the shading problem is not about the normal map, it is about the model and it will always be the model.

 

I hope I explained decently what the issues are here so it might help you on your future normal map bakings. Good luck and have a good day!

There are no hard edges on the model. I'd be happy to jump on call with you personally and explore the situation. (https://gyazo.com/3386ce721adff5f6ead5849bdbed9373)

According to its documentation and settings explained by Allegorithmic staff, Unity uses MikkTSpace.

I have never had a problem with hard angles on a model if the cage is constructed correctly. Also, I have built at least a thousand hard surface models. (https://sketchfab.com/toomanydemons) (https://www.artstation.com/toomanydemons)

My dicord handle is TooManyDemons #2125. Cheers!

All of that being said, I have run some very recent test bakes using just cubes on very new geometry, considering the fact that those previous models are rebuilt from ones downloaded on Blendswap. I still ran into issues.

Additionally, those issues do not exist in Substance or Sketchfab, sorry.

 

 

 

Edited by toomanydemons

Share this post


Link to post
Share on other sites

All of that being said, I am aware of the hard angle rules for baking normal maps. Considering that these are super tiny low-poly cereal boxes, I didn't think it would be an issue. I will take your advice and investigate further. 

All in all however, I'm getting some pretty nice results everywhere else:

https://gyazo.com/184d39716173d4e68588122d26205d5c

Edited by toomanydemons

Share this post


Link to post
Share on other sites

It seems when I change the lighting on Sketchfab, the normal map errors are becoming apparent. So you may be right, this may be an issue with this model. Thank you. I've had several people take a look at this issue on Discord, I will run some more tests with them. Cheers!

PS. I am curious if you can get anything to look good using the import texture button to bring in a normal map, using the Unity (or Blender) preset. If you can, please tell me how you did it! 

Edited by toomanydemons

Share this post


Link to post
Share on other sites

OK so yea :) it does look like the import normals option for Unity and Blender are broken. However, a newly created blank preset does read correctly.

That's a relief. So as long as I avoid 3DC's presets I will do fine.

Thanks for your insight about the hard angles @Emi After running more tests I can confirm this the problem with the box objects. 

Edited by toomanydemons

Share this post


Link to post
Share on other sites

Leaving aside the UVs rules, the issues it cause are minimal, if you are aware of that, then it's your choice saving some UV space and ignoring the edges and the issue that might happen vs doing them correctly since you would have to get too close to notice the issues. sometimes there is a big gap between the edges, has happened to me but in this case they are not that noticeable. 

 

But looking at your screenshot, it was exactly my point. you can't have those boxes as average vertex normals or soft, they have to look like hard edged boxes because they are 90° angle edges. if you want to use soft edges you have to create a bevel. Shading error exists on your screenshot and until you don't fix it the problem it will only be covered by your normal map, but the error is still there. Boxers should be 100% hard edges, Cylinders should be Top and bottom Hard and the sides Soft.
When I opened the model in Maya, it seems even worse what what I thought because all sides were average and others hard. so it is a mess for any engine to deal with it. 

The lowest poly are usually the models with more issues because you are dealing with few polygons that can't deal with the high res topology that well, especially if you go the lowest possible poly count and you do a 6 faces 90° angle edges boxes like in this case, that's why I suggest making bevels, just tiny bevel will make a big difference on many cases, especially in 2019 where there is no excuse to have a box with no bevel at all for something like this.

 

And no, 3Dcoat presets like I said, are just small variations that you can even manually select by using unknown, you dont need the presets, they are just to help 3DCoat interpret the normal map and the model, but I never trust the presets, because like you say Unity preset should be set to MikkTSpace. But my point is that it's not just about OpenGL vs DirectX normal map. so that's not the issue here.

So, when I use 3DCoat which is usually to paint, everything works fine, but that's because I make sure everything on my low poly is good as well, I mean, to make good normal maps and make them work everywhere you need a good low poly that can catch the whatever high poly details you made. There is no magic, it just works because I make sure of everything, like the smoothing groups and UVs and all that.

  • Like 3

Share this post


Link to post
Share on other sites
Just now, Emi said:

Leaving aside the UVs rules, the issues it cause are minimal, if you are aware of that, then it's your choice saving some UV space and ignoring the edges and the issue that might happen vs doing them correctly since you would have to get too close to notice the issues. sometimes there is a big gap between the edges, has happened to me but in this case they are not that noticeable. 

 

But looking at your screenshot, it was exactly my point. you can't have those boxes as average vertex normals or soft, they have to look like hard edged boxes because they are 90° angle edges. if you want to use soft edges you have to create a bevel. Shading error exists on your screenshot and until you don't fix it the problem it will only be covered by your normal map, but the error is still there. Boxers should be 100% hard edges, Cylinders should be Top and bottom Hard and the sides Soft.
When I opened the model in Maya, it seems even worse what what I thought because all sides were average and others hard. so it is a mess for any engine to deal with it. 

The lowest poly are usually the models with more issues because you are dealing with few polygons that can't deal with the high res topology that well, especially if you go the lowest possible poly count and you do a 6 faces 90° angle edges boxes like in this case, that's why I suggest making bevels, just tiny bevel will make a big difference on many cases, especially in 2019 where there is no excuse to have a box with no bevel at all for something like this.

 

And no, 3Dcoat presets like I said, are just small variations that you can even manually select by using unknown, you dont need the presets, they are just to help 3DCoat interpret the normal map and the model, but I never trust the presets, because like you say Unity preset should be set to MikkTSpace. But my point is that it's not just about OpenGL vs DirectX normal map. so that's not the issue here.

So, when I use 3DCoat which is usually to paint, everything works fine, but that's because I make sure everything on my low poly is good as well, I mean, to make good normal maps and make them work everywhere you need a good low poly that can catch the whatever high poly details you made. There is no magic, it just works because I make sure of everything, like the smoothing groups and UVs and all that.

Wau. I learned a lot from you. Thanks so much for taking time to explain these things. 

Share this post


Link to post
Share on other sites
1 hour ago, Emi said:

Leaving aside the UVs rules, the issues it cause are minimal, if you are aware of that, then it's your choice saving some UV space and ignoring the edges and the issue that might happen vs doing them correctly since you would have to get too close to notice the issues. sometimes there is a big gap between the edges, has happened to me but in this case they are not that noticeable. 

 

But looking at your screenshot, it was exactly my point. you can't have those boxes as average vertex normals or soft, they have to look like hard edged boxes because they are 90° angle edges. if you want to use soft edges you have to create a bevel. Shading error exists on your screenshot and until you don't fix it the problem it will only be covered by your normal map, but the error is still there. Boxers should be 100% hard edges, Cylinders should be Top and bottom Hard and the sides Soft.
When I opened the model in Maya, it seems even worse what what I thought because all sides were average and others hard. so it is a mess for any engine to deal with it. 

The lowest poly are usually the models with more issues because you are dealing with few polygons that can't deal with the high res topology that well, especially if you go the lowest possible poly count and you do a 6 faces 90° angle edges boxes like in this case, that's why I suggest making bevels, just tiny bevel will make a big difference on many cases, especially in 2019 where there is no excuse to have a box with no bevel at all for something like this.

 

And no, 3Dcoat presets like I said, are just small variations that you can even manually select by using unknown, you dont need the presets, they are just to help 3DCoat interpret the normal map and the model, but I never trust the presets, because like you say Unity preset should be set to MikkTSpace. But my point is that it's not just about OpenGL vs DirectX normal map. so that's not the issue here.

So, when I use 3DCoat which is usually to paint, everything works fine, but that's because I make sure everything on my low poly is good as well, I mean, to make good normal maps and make them work everywhere you need a good low poly that can catch the whatever high poly details you made. There is no magic, it just works because I make sure of everything, like the smoothing groups and UVs and all that.

I still don't understand why you say my normals have hard edges. Am I missing something? can I see a screenshot of this, because according to my Blender file here, there is only one smoothing group.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×