Jump to content
3DCoat Forums

Scary M's Sketch book


Scary M
 Share

Recommended Posts

  • Advanced Member

Hold breath another long one

Hi Don I will ping him l, my time was spent on it purely because I did not expect it to be the problem it's one of those bang your head against the wall kind of things where you start to question your own sanity I am working heavily modified Doom 3 code but I know that a one Mikktspace normal should at least be the same or very similar to another and I had been extremely careful (or so I thought) to remove all smoothing from all meshes at each stage so the models can render properly in game. 

The way I diagnosed and confirmed the issue (because it not really ab apparent problem until it is if you get my meaning) is to create a complex model in retopo room and bake pp with normal map with auto smoothing turned off. So the normal map produced looked swollen and biased on the model in game. I was pretty sure of my process by now, so I tried another baker on my shelf nowadays and the first I tried was Topogun loading the mesh over an exported scene from the voxel room, and the normal map looked perfect. So this then became my pipe for these model but I still didn't really understand why Topogun would work so well against 3dc?

So last week I got to playing around with 3dcoat painting weight in a area of influence before a bake is commited the model was particularly complex and rather than build a cage mesh for Topogun I thought I'd give baking in 3d coat another as in prior project the weighting of ray casting  prior to bake has been a life save and sure enough it worked and I was able to illiminate baking pinch in awkward areas in the result in 3d coat. But on load into engine map still looked bias and swollen back to Topogun and cage for baking. But I decided to try other software see if I can find a better less labour intensive solution so I tried substance p, and knald and to my surprise each one of these produced the same swollen bias'd map. 

So I then figured that it must be something wrong with the mesh from 3d coat and the topogun upon import removes smoothing angles. 

Over to blender for an import and export after removing smoothing then import into sub p, knald to bake and test and normal map produced is as expected and looks good in game. 

Import into 3dc and bake - funky bias.

By this time I had searched for removing mesh  smoothing angles in 3d coat and a possible solution was Import object into paint room and bake sculpt to paint room. There are several issues with this walk around 

1st being that I have to use a walk around, 

2nd if I import a mesh for ppp I lose the sculpt in the sculpt room and have to re import from 3b file and scaling is all fecked,

3rd is that if I import just the object into the paint room no UV are loaded. And nothing bakes across this might be that I have a retopo mesh with UV setup. 

Anyway at this point I gave up trying and had a rant. 

Conclusion is you simply cannot retopo and bake to non smoothed mesh. You cannot produce a mesh in 3dc at all that does not require a silly little step of export remove smoothing and no simple way to import and without mesh being re smoothed.

The worrying thing to me (and I am passionate about 3d coat) is that how many people have tried this common workflow and either just dropped the software because of its (and this is a quote from the tinternet) 'weird normal maps' 

I know in my case here it's Game engine specific (because I have worked in a few engines now and smoothing is not a problem for some not others) but it does illustrate a fundamental unexpected issue. Furthermore after diagnosing the problem I see there are in fact multiple posts re this from far FAR back in the forum archive.

Link to comment
Share on other sites

  • Advanced Member
9 minutes ago, Gorbatovsky said:

Today at the rally we discussed this message for 10 minutes and did not understand what needs to be done.
Please make a small video so that your technical process is clear. And write what needs to be done.
Thank.

Its difficult because it's not apparent initially until we test the mesh in game the viewport display and rendering in 3dc is perfect. The bias created by smoothing angles isn't visible in 3dc. I will drop here link to my 3d coat file where I found initial issue on this project. And a link to the obj I exported and removed smoothing angles from so you can analyse. This OBJ without smoothing was successfully loaded into 3 other tested applications and produced a working Mikktspace tangent space map without bias.

Another test would be bake something from retopo with per pixel and set auto smoothing groups to off, then export mesh remove smoothing bake from this mesh in another application then import the mesh for ppp, import baked normal from other application and look at the effect smoothing has on the normal map. I think one of the areas of difficulty is 3dc has soo many conflicting normal map settings. 

And how a about an Industry standard Mikktspace default that you can just return too. For the artist who doesn't know the exact requirements of the software preset and tangent space.

It says a lot that I can pick up 3 different software and bake out 3 perfectly good and consistent normal maps with default settings in these softwares using the corrected unsmoothed mesh.

Yet the struggle I have had in 3d coat 

1 to find a workable set of parameters, and

2 to diagnose the smoothing angle issue.

If I wasn't so invested in 3d coat It would be game over.

Link to comment
Share on other sites

  • Advanced Member

our version of RBdoom Requires no smoothing on mesh so its an ideal test to illustrate 3d coat adding its own unspecified smoothing to the mesh before the map is baked.

Both maps were calculated using same mesh one imported into retopo the other imported directly into topogun both calculated from same highres in 3d coat case the sculpt room then in topogun an exported sculpt room scene from 3d coat, can we see the difference?

Edited by Scary M
  • Thanks 1
Link to comment
Share on other sites

I uploaded the new build (72), sharp edges kept when you import the model to retopo room. But I don't think it will solve the problem. Because they was kept before as well if UV map was present in the mesh.

The essence of the problem -

1) If you import to paint room you may lock normals. Then baking will work well because regardless of normals/tangent space you set in prefs normals will be kept, so baking will not depend much on settings.

2) If you import to retopo room I can't lock normals, retopo intended to change geometry. I am sure sharp edges kept, especially for the newest build, but you should set the correct algorithm for normals/tangents calculation in preferences. That is the key point to get the correct result.

Also, for sharp edges, it is better to set naive padding in prefs.

If you will send me the correct model (obj/fbx)+ correctly looking normal map + picture how it looks I may help to find correct settings (if it is possible in principle).

Link to comment
Share on other sites

  • Reputable Contributor
58 minutes ago, Scary M said:

Hi Andrew the problem when importing to the paint room is that the sculpt room geometry disappears, I can re load the sculpt from a 3b file but the scaling no longer correlates, unless I'm doing something wrong?

In the Paint Room VIEW menu, is SHOW VOXELS IN PAINT ROOM unchecked? After baking, I have never had a sculpt model just disappear (from the Sculpt Room), so I can only presume this option is somehow unchecked. 

Link to comment
Share on other sites

  • Reputable Contributor
1 hour ago, Scary M said:

Hi Andrew the problem when importing to the paint room is that the sculpt room geometry disappears, I can re load the sculpt from a 3b file but the scaling no longer correlates, unless I'm doing something wrong?

If you are not talking about "importing into the Paint Room" during the Retopo baking process, but importing directly into the Paint Workspace from the FILE > IMPORT menu, can you try this and see if it works as you expect it to?

 

Link to comment
Share on other sites

  • Advanced Member

I don't understand why the mesh from the retopo room need to have this strange additional smoothing added to it, as the mesh was initially made in 3d coat (I only exported it out fo fix the smoothing problem. importing the mesh is not really the issue here. I would want to bake my retopo mesh without the smoothing can we not have the ability to remove smoothing right there in the bake menu from retopo room? The break in workflow comes more in the fact that I would have to export my retopology out of 3d coat and bring it back into the software with smoothing removed. Surely there should be a way to do this without the extra step of exporting out of software?

I stress I'm pretty sure it's not a bake settings problem, because if I bake any mesh made in retopo room export the mesh and and bake normals map in other software I get the same result as the unusable map I baked in 3d coat, however because the idtech 4 has to have no smoothing at all the mesh from 3d coat cannot be used without removing smoothing first. 

So to break it down again if I make something in the retopo room and bake it with auto smoothing not checked I still get smoothing on the mesh and it bakes weird normal maps regardless of the software.

However If I export the baked result out of the paint room and bring it in to another software (say blender) and remove smoothing. I can then take the mesh into other baking software and bake perfectly good maps that do work.

Link to comment
Share on other sites

  • Advanced Member

Please walk me through the process? I have a sculpt in sculpt room and I have a mesh to import, what do I do?

In paint room when I 'import object' from the file menu it has no UV so when I try and bake from sculpt to paint room the result does nothing.

If I import object for per pixel paint with UV it creates a new scene removing what's in the sculpt room. What am I doing wrong?

 

Link to comment
Share on other sites

  • Advanced Member

i had to re export my model then import into blender again and remove smoothing, then import again into paint room. For some reason it did not work correctly the first time (imported model with no dialouge or UV), I wonder if it was because i had model in retopo room with same UV set name. who knows!.. the main thing is it works its an extra step but it works.

So that work-around works Not only does it work it produces the best maps, However it is still a work around and we need to use another application to remove smoothing in order to do it. 

The normal map generated with 3DC with the import to paint room and locking normals work-around - NICE

778223960_BaseProfileScreenshot2021_03.15-21_59_49_31.thumb.png.c9039dee9c8f91a00dfb0161986d91df.png

Edited by Scary M
  • Like 1
Link to comment
Share on other sites

  • Advanced Member

No that's not it you have to have voxels visible in paint room active when you initially load up 3d coat, if you start the app and this wasn't checked when you import your mesh into the paint room no dialogue appears and the mesh pops in without UV data. If you have this issue set show voxels in paint room on close 3d coat and start it again then you can import mesh , dialogue shows and you can lock your normals set your smoothing etc

  • Thanks 1
Link to comment
Share on other sites

  • 2 weeks later...
  • Advanced Member
2 hours ago, Scary M said:

Hey Chap Not seen you in a while.

I've been here, haven't been posting as much lately work is carrying me in different directions.  Still here though!  Nice to see you posting again lately.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...