Jump to content
3DCoat Forums

bake errors along edges placed in crevices: how to avoid?


Gian-Reto
 Share

Recommended Posts

  • Advanced Member

Hi guys...

 

3D Coat has become pretty great at baking my retopoed models without baking errors lately... but one thing I still struggle with on a regular basis: edges placed in crevices.

Sometimes you have that big fat crevice that runs around a model where you just have to place an edge in because the crevice is pretty important for the overall mesh silhouette. Yet the resulting mesh bend in on itself in an outside angle smaller than 90 degrees. Now you almost HAVE to place the seam inside the crevice as else you might get massive squash and stretch on the UV map somewhere else... But if you place the seam in the crevice, you get massive normal map errors along the seam in the baked model.

Now, I have seen these problems before, currently I am stuck with the retopo of a greek pillar. The pillar has bulges at the end which rest upon square stone boxes. where the bulge meets the box is where the trouble is seated. I tried both placing a seam in that crevice which results in the said normal map errors, and removing the seam which results in massive squash and stretch in the upper side of the box, which results in normal map errors there.

What I didn't try yet was to activate auto smooting groups, an option I last tried 3 years ago, when it would result in massive problems with visible seams in Unity, so I never touched it again. Maybe I should give it another try now that both Unity and 3D Coat had new versions come out.

And I understand if you say pictures would help. Couldn't take pictures yet as I was in kind of a hurry, but figured maybe somebody had the problem before and knew what I was talking about without pictures. I can add pictures when I get home in the evening.

 

Apart from the auto smooting groups, anything else I could/should try? Anyone had the problem before too and found a solution to it? Currently when something like this crops up I easely waste more time on resolving the issue with additional tris, different seam setup and shifting vertices around than time needed for sculpt, retopo AND subsequent texture paint.

 

As always, any help is appreciated a lot. Thanks in advance for that.

 

Gian-Reto

Link to comment
Share on other sites

  • Advanced Member
56 minutes ago, Carlosan said:

Can you to detach the top of the pillar from the box ?

Your texture baking have waviness ?

 

Well, its not the normal waviness problem. Its an actual baking error. Looks like a "rip" in the model in the 3D Coat Painting room, and if you look at the normal map in another app the normal is messed up from this rip down to the next edge.

 

But true enough I COULD separate the voxel object and create different meshes... I originally dismissed that because I thought that the AO wouldn't be baked correctly for the overlapping meshes, but now that I think about it, I did this some years ago and it worked fine with AO being baked as expected.

Guess that would be the easiest way to fix this problem. Going to give it a try tomorrow.

Nevertheless, are there other options? I can upload images of the problem in the evenening if its not clear what problems I am talking about.

Link to comment
Share on other sites

  • Advanced Member
6 minutes ago, Carlosan said:

Is solved by adding control loops on either side of the shape ?

Any pic can help for sure.

Are you using the latest version ?

The IN/OUT cages are ok ?

 

BakeSet.jpg

 

Added an edge loop above the problematic area, it made the problematic area "smaller" as the problem stayed in the lower half of the now split polygon, but the problem did not go away.

I THOUGHT I had the latest version, but looking at the image you attached, probably not. This looks different than the dialog I see. Have there been new scan depth altering tools added? I only see the sphere of influence tools and these did not help. Guess I should download the latest version and give it a spin.

 

Outer cage looked okay from a quick glance. Should probably check the inner cage too. Wasn't sure if the polygons/voxel surfaces set at an outer angle smaller than 90° might confuse the raytracer though....

Edited by Gian-Reto
Link to comment
Share on other sites

  • Reputable Contributor

In your case, some pictures are necessary or if you can share the model with me, I can take a look at it. Pm me if interested.

Oh yes, we now using a paint projection cage method, spheres of influence are gone. The new method is far better plus your projection painting is saved in the 3DC file unlike the spheres. You can manually save out the projection painting as well.

I recreated a pillar based upon your words and had no problems in my uv set islands or normal map. I of course created my own seams in the areas I knew would be good. I was only guessing your model's structure though, I would need access to the file to really know where the problems lies.

Edited by digman
Link to comment
Share on other sites

  • Advanced Member

Screengrabbed some picture of the problem...

 

Pillar_Problem_01.thumb.png.2d9d0418b51e6d0e99d6852b0e21bc7c.png

This is the Problem as visible in the Paint Room

 

Pillar_Problem_02.thumb.png.98daf3dc098135727ee4731ee5b3f095.png

Here is the retopo mesh of the problematic area.

 

Pillar_Problem_03.png.479b2fdd0f42ed9e9a003273378a383d.png

And this is how it looks in the game engine once imported, and only showing the scene normals.

 

Next I'll give the newer 3D Coat version a test run.

 

 

EDIT: new bake tools didn't work for me. Checking the inner cage I think I know now what is the culprit, the inner cage slightly clips through the voxel model to the outside at the problematic edges, and I cannot get it to stay inside by increasing the inside distance. But the new tools also did nothing when I tried. I can make it move outside more, but not inside.

After that I tried setting the preset to Maya instead of Unity. Immidiatly the errors are gone after the bake. But I now get a ton of weird errors when painting normals on the model in the paint room.

 

I think I'll just separate into two meshes and call it a day. As this is a static level prop, most probably not worth further tries when it all gets put into a single batch inside the engine anyway.

Edited by Gian-Reto
Link to comment
Share on other sites

  • Reputable Contributor

Here is my layout on a similar model to avoid the seam problem. That would be a problem area as the normal directions between the base of the pillar and the square support is large plus the base long polygons creating tension between the normals.

I created your same problem by putting a uv seam there. In the picture you see where I put my seam.

On the square support  after unwrapping, which by default is GU, I unwrapped just that uv island again using LSCM. Then I used the transform tool to give it the same texel density by scaling it down just a little.

Still, having it has two separate pieces that share the same material is not a bad idea either. They can be combine to one object as you stated and  be batched together.

Either way works.

Captureee.JPG

Capture33.JPG

Edited by digman
  • Like 1
Link to comment
Share on other sites

Hi

This is my attempt trying to recreate your issue and using the latest version.

- I created your model in Blender -low poly for retopo room-

And created the same model with edge bevels to be used as Sculpt Mesh -highpoly-

- Imported the LP mesh in retopo room and mark seams in 3DC retopo room

Imported the HP mesh in sculpt room

- Using this bake scan settings i bake NM sculpt mesh over retopomesh

And the result in Paint room, show no issues.

Can you please make the same test using my models ?

thx !

 

model.jpg

retopo.jpg

BakingScanSettings.jpg

paint.jpg

 

 

Pilar.zip

Link to comment
Share on other sites

  • Advanced Member

Thanks Carlosan, will give it a go tomorrow.

 

I have now a mesh that works, separated the boxes into their own mesh, worked fine, just had to add some fake geometry to the hidden parts of box and column so I still get some AO there.

But while testing the mesh I had big problems with cubical projection when flood filling the mesh... I tried to get some small normal details on the mesh. Thing is, the boxes currently have no bevels around the edges. And at some edges the cubical projection gets stretched increasingly as we go toward the edges. Is this a known problem? Anything I can do against it? The UV Projection seem to work only with very specific models, and actually also seem to have a problem along those edges in my case... here the projection does not get stretched, but the normals get more intense.

Image of the problem area with UV Projection (sadly have non of the even screwier cubical projection)... you see the normals getting more intense towards the lower edge, yet they should be uniform. That part of the mesh should be flat in the UV. Seams placed along the vertical edges. Any Idea?

3dcoaut_flood_fill_problem.png.4bd33d2f8b997b756e6af32fc3c63ffc.png

Link to comment
Share on other sites

  • Advanced Member
3 hours ago, Carlosan said:

Can be very helpful if you can share your bake settings.

try inner/outer = 01 to begin some test

is the model very small ?

Your scene scale is ok ? Sculpt room > geometry

 

[SNIPPED IMAGES]

 

 

Model AFAIK is not very small. Seems to be about 1:1 size when imported into game engine (Unity)... but I will check the settings you mentioned. I am not sure about the scene scale as I usually do not touch this.

 

I can screengrab the bake settings. Inner Outer currently set to 3, AO baking to 5, traingulate selected, other than that everything is left on default. How could the bake influence problems like the one described in my last post? Or are you still talking about the baking errors I originally described? Kinda lost what problems we are talking about here (which totally is my fault for not opening a new thread for different problems showing up :) ).

Link to comment
Share on other sites

  • Advanced Member
On 20.6.2017 at 4:23 PM, Carlosan said:

Hi

This is my attempt trying to recreate your issue and using the latest version.

- I created your model in Blender -low poly for retopo room-

And created the same model with edge bevels to be used as Sculpt Mesh -highpoly-

- Imported the LP mesh in retopo room and mark seams in 3DC retopo room

Imported the HP mesh in sculpt room

- Using this bake scan settings i bake NM sculpt mesh over retopomesh

And the result in Paint room, show no issues.

Can you please make the same test using my models ?

thx !

 

model.jpg

retopo.jpg

BakingScanSettings.jpg

paint.jpg

 

Pilar.zip

Hi Carlosan

When I look at the 3b file you uploaded, I see no Voxel Object and no mesh object, just a surface object in the voxtree. When I hide the voxtree, the paint room is empty. What should this test?

When I try to bake it, I run into the same errors I get with my voxel model. Bake settings: distance 3 both (setting them to 1 gives me black areas at some places), other scan settings doesn't matter (tested them all), and setting the triangulate option (not setting the triangulate option leads to the same artefacts though).

 

On the other topic of the stretched normals if cubical projection is used, seems 3D Coat cannot handle non bevelled cubes when doing these kind of projections. Which is understandable given how a not that sharp voxel edge is mapped to a supersharp mesh edge, which leads to a distorted normal map... no visible normally, but seems to screw with the cubical projection.

I'll try to add bevels in these places and see if it helps. Not ideal for ingame models, but given the total polygon budget of the asset, hardly going to make much of a difference (besides more subpixel polygons). I'll see if it helps with the projection.

 

EDIT:

Bevels worked... no more normal stretching in paint room when using cubical projection. Was able to save some polygons elsewhere, so only adding bevels to clean 90° angles cost me about 10% more tris overall. Hardly ideal for an ingame model, especially because they are small tris, but meh, good enough for a max LOD asset. The lower LODS will most probably not need the normal map details so I can drop the bevels there.

Maybe I'll give smoothing groups a try to later to see if it still leads to visible seams along the edges like it did with earlier versions of Unity.

Edited by Gian-Reto
Link to comment
Share on other sites

Dont use Concave angles in your low poly mesh to avoid projection errors.

How Does Projection Work?

Projection Diagram

To bake texture maps, the geometry from the high poly source mesh is projected (ray traced) onto the low poly mesh. The best way to visualize this is to imagine a camera looking at the high poly and capturing the direction of the surface for every pixel in the normal map. To make sure the projection does not hit the wrong elements, a cage, or projection mesh must be set up to limit the distance of the projection. For each pixel, the angle of the camera is set by the normal direction of the cage.

Link to comment
Share on other sites

  • Reputable Contributor

@Gian-Reto    You are asking a lot of the normal map in a low polygon model with no bevels and some sharp angles... Possible yes... Each artist has his reasons for their 3DC to Unity 5 workflow, so my method does not mean you have to choose it.

I took Carlosan pillar and this is how I would lay it out for this model with the no bevel design to avoid normal map problems.

No Beveling as it appears you want to avoid it.

I am not trying to get perfect flatten uv islands.  I accept a tiny bit of stretching. Really so little you will not see it in the textures.  If it looks good, it is good...

I added two loops to help the problem with waves of the normal map and seam problems. You noticed where I placed my seam cut not right where the pillar and support base meet. That would be a problem area so just avoid it. Add a loop and put the seam there so the meeting of the the pillar and support are not divided where they meet. Also helps in avoiding the concave problem.

The total vertex count with the uv vertices is 329 vertices and 508 tris. I am more concern with vertex count than polygon count as the GPU renderers vertices.

 

uvs.JPG

good.JPG

 

fine.JPG

loops.JPG

Edited by digman
  • Like 3
Link to comment
Share on other sites

  • Advanced Member

Had another test run, and it seems auto smoothing groups also do the trick.

The nice part is that from my testing, visual seams on the edges are now pretty hard to see from a certain distance in Unity... you have to zoom in quite close even in the non-AA-ed editor view to see them. Given that I couldn't see a difference between the bevelled and smooth grouped model ingame from the intended viewing distance, I guess I will start working with smoothing groups for now. Saving another 10% tris gives me more headroom for particle effects or stuff like that, so I'll gladly take that :)

 

@digman: yeah, I guess these additional loops are what is necessary to make such sharp angles work for baking. Must ask some more up to date guys in the Unity forums what the newest wisdom on subpixel tris in a running game is... I know some of the latest AMD cards have a stage in the pipeline that filters them out before they cost much, and frankly I am not sure anymore if the reason why people were not happy about those was because of the cost vs. benefit relationship (geometry that is not visible but has to be calculated is was pretty bad when the geometry budget was still quite limited).

Probably "hacks" like hat in game assets don't matter nearly as much anymore.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...