Jump to content
3DCoat Forums

How do I stop this from happening? (Baking artifacts)


Grimmy
 Share

Recommended Posts

  • Advanced Member

Hi all, apologies for my amount of posts recently, but here is an issue that some of you may be able to help me with and although I've seen a lot of videos and read the manual I can't recall seeing a workaroundfor this problem.

I've tried increasing the texture resolution and playing with various settings, but I still can't seem to get rid of this baking by-product.(..Dots down many of my edges. See attached)

Cheers

post-37556-0-25560500-1370259200_thumb.p

Edited by Grimmy
Link to comment
Share on other sites

  • Reputable Contributor

Hi all, apologies for my amount of posts recently, but here is an issue that some of you may be able to help me with and although I've seen a lot of videos and read the manual I can't recall seeing a workaroundfor this problem.

I've tried increasing the texture resolution and playing with various settings, but I still can't seem to get rid of this baking by-product.(..Dots down many of my edges. See attached)

Cheers

Andrew released this "Happy Baking Guide" recently when he tried to improve the baking process, including better seamlessness when baking to PPP..

Happy_baking_guide.pdf

Link to comment
Share on other sites

  • Reputable Contributor

Can we see a picture of your retopo mesh to help figure things out...

Here is a possible solution without seeing your retopo mesh...

At the edges where you are having the problem add a small bevel... In the real world there is almost always small bevels on anything. Nothing is really a straight 90 degrees at the corners...

Normal maps do not like sharp 90 degree or so edges... The normal map routine can not quite figure out which way to place the normal direction, hence artifacts... This is just not a 3DCoat problem...

Here is how to make retopo bevels.

Retopo Room:

Select edges in the top menu.

Select your edges you want to bevel.

In the left tool bar under "Commands" choose bevel.

Set your bevel distance.

Done.

3DCoats toolbars are context sensitive. When you choose edges in the top menu, then new items will appear under the Commands menu...

I tested a cube without any bevels and had the same problem you had which I figured I would have.

I created some small bevels in the retopo mesh and remerged... " I first deleted the first model from the paint room and all layers except layer zero before remerging. This is required and I think this is what you have been doing.

Picture shows the result...

Now there this could work but without having access to your 3DCoat file, Im just shooting the arrow straight as I can with blindfolds on... :D

post-518-0-08863900-1370284618_thumb.jpg

Link to comment
Share on other sites

  • Advanced Member

Yeah cheers Digman it looks like that is the issue. Bevelling the edges on the retopo will solve the problem but because this is game art, I need to keep polys as low as possible. I suppose I could bevel>bake>remove bevel but that seems like a long way around. What do you think?

Link to comment
Share on other sites

  • Reputable Contributor

I am not an experienced game artist but it seems with today;s hardware and even smart phone games a few bevels would not be that hard on the game. I do not know of course what your polygon limit is per prop for the game... I know that in most cases there is a base limit and then a upper range you can not exceed.

If you remove the bevel in the final mesh it will screw up your uv maps for that model but there is something you could try, explained below.

Bake the bevel's model textures and normal map to the model that has no bevels and see if it works out.

You must use the Texturing baking tool in the paint room to do the above... There are a couple of things you need to know for it to work successfully

Top Menu / Textures / Texture Baking Tool

Mesh to receive projected textures--- You must choose the obj model with the uv-set that is non-beveled. This is required.

Deselect "use current low poly mesh instead of external mesh" if it is selected

The rest I think is very easy to figure out...

You will have to test to see if it will bake correctly. Import the model and it's normal map / texture map in the paint room in a new scene once the baking is done...

Skype anyone?..... :dash2:

Link to comment
Share on other sites

  • 2 months later...
  • Member

I have no idea where my attempt at a post vanished to but it's "charged" me 1 post for the day so it will likely show up some time but is not currently showing up.

 

Anyways on topic, I am having a different looking type of glitch show up and in unexpected areas where there was no conflict between baked inner and outer layers.  It also messes up in the Occlusion layer etc so I'm not entirely sure what the cause would be for this.

 

any tips would be appreciated.  I do intend to look through the happy baking guide, am just wanting to see if there is an obvious cause to you veteran bakers out there.

 

I also notice there are extra layers there and I've no idea what they are for.. I get the layer for the occlusion, normal map and base mesh.. and I'm guessing another layer is for my subdivided mesh though am not sure which.

 

I'm also unsure as to which layer I am meant to be painting onto if I'm wanting to just paint the texture for my character.  At a guess I do not touch the occlusion or normal map layers unless I want to paint in extra normal map or occlusion information.

 

p.s. sorry Digman I swear I was paying attention when you showed me this stuff.. I just have a rubbish memory or something and don't remember all these extra layers showing up before or the glitching happening on that sphere example.

 

 

 

p.p.s  I've read that guide through now for the happy baking and my baking is still not happy.  It doesn't really cover or show the 2 problems I get.  

 

1.  Holes in my baked normals and occlusion map.

 

2.  Artifacts in the Baked normals and Occlusions

 

 

Now I am guessing holes appear when the inner depth protrudes outside of your reference mesh or your outer depth does not fully cover the reference mesh.  OR inner depth and outer depth intersect somehow.

 

Is it normally this hard to get a simple normal bake to work on a standard biped mesh you sculpted or is it just bad luck I'm having here?

post-37795-0-63133000-1377147429_thumb.j

Edited by Jax_Cavalera
Link to comment
Share on other sites

  • Member

What i've tried since the previous post is to turn on snapping again so that the retopo mesh sticks to the reference mesh as this is typically the way I've seen other's work and I was trying to get a better result by producing a larger "shell" but this didn't work.  I also was not able to find a quick way to simply expand the retopo mesh so that it extruded away from the reference mesh by a set distance uniform.  something like that would save a lot of hours I spent looking for small parts of the ref mesh sticking out through my retopo mesh.

 

Anyhow.. what I have attached is the before in process and after of trying to get the feet to play nice for Baking with snap back on this time.

 

In picture 1 it shows the mesh as expected pre bake

 

Picture 2 shows how the mesh looks when I try to set the outer depth.. lots of intersecting which I can NOT get rid of without the outer depth not properly extruding out beyond the surface of the reference mesh.  I'm lost for inspiration on how to fix this as it will look so wrong if I move the toes apart much further and the retopo mesh is a tight squeeze without any increased depth beyond 1.5 when I tried to manually sort it out... and with this setting it overlaps way too much .. not sure if that is a problem or not though... can't imagine why it won't cause issues.

 

What do you pro users normally do for that kind of situation?  do you do all the foot as a single area and not worry about going in between toes?

 

 

I've tried going with 20 for the occlusion depth so i'm not sure if that's another cause as occlusion wasn't mentioned in the happy baking pdf that I saw.

 

Pic 3 shows the settings I use for the norm bake.

 

the other pictures are more about the results.  and they also show that odd seam around the shoulder area I was trying to mention earlier as well.

 

 

 

 

post-37795-0-09683600-1377155196_thumb.j

post-37795-0-96904700-1377155205_thumb.j

post-37795-0-88766200-1377155216_thumb.j

post-37795-0-53878400-1377155225_thumb.j

post-37795-0-87920500-1377155234_thumb.j

post-37795-0-39717500-1377155244_thumb.j

Link to comment
Share on other sites

  • Member

Hi, thank you for this information, I have tried it with the setting of no subdivision and it comes out with no claws which was one of the issues I received using the subdivision level so that's good though the other complications with the holes / artifacts in the hand and feet are not resolved.

 

I did fix the shoulder seam issue by taking the mesh back into voxel mode.. smoothing and filling holes etc  then taking it back into retopo room so that's another positive so far.

 

just those other 2 issues now and it's all sorted  ~~

Link to comment
Share on other sites

  • Member

Oh cool, so the toes aren't normally meant to be spread apart like that.  I did suspect this may have been the case but wasn't sure so I thought I would try to do them as shown in case.  That's one less problem now.  Just the remaining artifacts.

 

I did notice that in areas on the hand, when I deleted the existing topology and then replaced it with slightly different arrangements that the artifacts in the replaced zones was fixed so I'm left to assume that it is probably caused by the poles I have on the palm of my retopo mesh's hand.. or was.. though it's also happening around the sides of the hand too and there aren't any poles on the top of the hand so i'm still not exactly sure what needs to change there to make that all better.

 

I am getting a lot closer though now.

Link to comment
Share on other sites

  • Member

I have created a video that shows my latest progress thank you to all your help to get me this far.  If anyone is able to point out where I can improve to get a problem free Bake of my mesh with 3D Coat I would be really appreciative.

 

The video at the time of posting this still has 270 minutes remaining before it has completed the upload process and is available to view inside this post just as a heads up.

 

Link to comment
Share on other sites

  • Member

Just a quick update to say that the video in my previous post is now uploaded (4 and a half hours of waiting later :s)  I really need better internet.

 

Also, Below is a recent screen shot at the baked results of my latest attempts.  in this attempt, I tried to set the extrusion to 2 and I ensured that all the reference mesh bits were covered on before entering the baking process.

 

Not really sure why I ended up with stick fingers though :s

 

I'm really struggling to get past the retopology and baking phase successfully.  Does it normally take this long to get the hang of it?  I'm trying to remain as positive as I can and just press on trying things till I get success.  I just don't quite understand where I'm going wrong. 

 

In the attachment, you can see how I think the feet area has slightly improved.. 

 

I think my next step will be to watch the pirate tutorial again and see if there is something that I missed in there that will fix the issues I'm experiencing.

 

EDIT : The pirate tutorial was not applicable so I am looking into the Rat Tutorial and will see where this takes me.

 

EDIT 2 : I have noticed that in a lot of the tutorials, the retopology is way higher than what's allowed for a game model which is likely where I'm having the difficulties here.  It seems that when I subdivide my retopo mesh to the point of it being 6000+ triangles.. the baking goes a lot better .. when it's under or just near 4000 triangles *higher end of what's allowed for a game character mesh in Unity) the baking has these issues again.

 

This leaves me to wonder if 3D Coat is the right tool for creating retopologised game character models and baking their normals or if it's just useful for 3D Animation models where the polycount isn't as critical.

post-37795-0-25447700-1377250775_thumb.j

Edited by Jax_Cavalera
Link to comment
Share on other sites

  • Member

Well I thought it may have been the messy retopo job I did on the hands so I carefully recalculated so that the subdivides would work in well and only need one ring for downsizing between the 2 variations.  The overall topology looks clean now yet it's producing more artifacts than the messy retopology.

 

On a plus side.. the fingers are no longer "mashed" or stick thin like they were in previous attempts..

 

Well, I've been at it for close to 2 days now on 3 hours sleep so I think I will call it a night at this point and hopefully someone may be able to point me in the right direction.

 

 

post-37795-0-29437600-1377270586_thumb.j

Link to comment
Share on other sites

  • Contributor

Hard to tell for sure from your image, but the way you unwrapped your hands looks pretty unusual to me. Maybe unusual seams is causing your baking problems there. Why not just have one island for the palm of that hand, and one island for the top of that hand?

Link to comment
Share on other sites

  • Member

Hmm I will try to do this with the Islands to see if it helps.  I wasn't really aware that the UV map would take it's toll on the baking process though it does make sense that it would have an impact on it and may be part of the problem.

 

I'll give what you recommend a go and report back with the results.  Thank you.

Link to comment
Share on other sites

  • Member

Ok so I have tested out the latest advice and also used it on the latest available release of 3D coat to ensure that I would be only experiencing problems that were not yet addressed.

 

I'm still trying to determine the exact cause of the glitching in the hands area and am hoping someone has a few tips that will help me to avoid the intersection of inner and outer depths between fingers at the join zones.

 

I'm still not having any luck with the feet in between the toe areas so this is another location that some further assistance would be appreciated.

 

Thanks again for all of your help so far guys, I am getting a lot closer now to producing a usable retopologised mesh with baked normals and once I have this sorted out I can confidently purchase the software knowing that it performs as required.

 

My internet connection is still running quite slowly and so the upload of the most recent addition is currently stated on Youtube to still be 92 minutes away from finishing it's upload process.

 

Hopefully someone is able to recognize any remaining problems with my workflow and provide suggestions on how I may improve.

 

 

EDITS :

 

I was able to remove the problem seen on the back by adding an extra sphere for the inner depth at that position and also the problems on the heels of the feet.

 

Is there a way to mirror the effects you do with the region spheres?  I notice that the Right hand of my mesh (the one I did NOT add any spheres to) has identical glitchings in the fingers and looks to me to be the same with respect to overall baking.  same thickness fingers.. same problem areas.. no black bits.. which makes me wonder how much use the spheres are actually doing on the Left Mesh hand.. 

 

Or if they mirror which is why the Right side looks to be identical to the left.

 

On the feet however, I notice that the left and right feet do show differences with the the Black stuff showing up on one more than on the other.  so that tells me that the effects of the region spheres are not mirrored.

 

 

Late Edit :

 

I just realised that a new release has come out since I downloaded the 4.005H.  So am now downloading this newer release in the hopes that it provides a clean bake.  That being said I'm pretty frustrated as to how other experienced users are getting clean bakes on the stable release yet I'm doing so badly at this.  If only the software would work :s oh well all I can do is keep updating progress and trying my best.

Edited by Jax_Cavalera
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...