Texture Baking high res to low poly with multiple layers?

[popwfx]

In case my question got lost in that other thread, I thought I'd rephrase it in case anyone has any thoughts or can clarify some stuff for me.

I have a high res character (~177K poly) consisting of meshes for the guy (his eyes), shirt, pants, and shoes. Each of these high res meshes is textured/painted. The guy has 3 UVs (1 for head, 1 for body and 1 for limbs), the shirt has 1 UV, the pants have 1 UV and the shoes have 1 UV (6 total UV Maps for the high res stuff).

I have retopo'ed a new low poly character out of all of the above (I loaded the dressed hi res character in 3DCoat and retopo'ed across all the meshes - so the new low poly character is one 6.5K poly contiguous mesh ). This means I've incorporated the clothes as part of the low res mesh and not as separate items.

So now I'd like to bake ALL the high res objects textures onto this low res mesh. And, ideally, I'd like to end up with only 3 UV maps completely different than the original ones. I want 1 for head, 1 for torso and 1 for limbs. In the original mesh, there is chest skin textures on the body, but the shirt covers that and has it's own UVs and textures. So in my baked one, I don't care about any skin texture that is "hidden" by clothing. I just want the shirt texture to go to the torso of the retopo'd mesh.

I've watched the couple of texture baking tutorials from AbnRanger, but don't know what to do when multiple UV maps and multiple subobjects are involved. Is the texture baking tool in 3DCoat smart enough to omit visually occluded textures (like the skin) and only project textures from the visually exterior surfaces onto the low poly mesh (like in my case only use the shirt polys to project to the torso, but use the skin where the arms or legs are visible and uncovered?

The two characters are in virtually the same pose - as I have used the high res one to retopo the low res one.

Please help me figure out the best way to transfer these high res textures I've painted across multiple objects (guy & clothing) onto this single low res model. Thanks!!

[alvordr]

Are you using any other app? You tend to use Blender, right? Regardless, if you can combine the meshes to the 3 main parts you want, like say...I can attach various meshes into one in 3ds Max, thereby combining the UV sets, as well...then you could bring them in as a reference mesh for the projected details, and then simply use Photoshop or some other app to combine the texture sets to match the UVs. I'm not aware of an easier way to do this in 3DC with the added requirement of projection on top of all those other things. You could, but then I think you would also have to repaint everything.

Meanwhile, I'm not sure why you'd be concerned about occluding unseen textures, as the added coloring that doesn't get seen won't add THAT much into memory. That said, since you are concerned about it, can you not just use your AO render to mask out the parts in Photoshop or some program like it?

[popwfx]

Thanks for your reply. It's kind of quiet in here..

I use Lightwave - don't use Blender really except for the occasional export if it reads something I need. So not keen on learning Blender unless I absolutely need it for something LW and 3DCoat can't do.

So you think the best way is to take the high res clothed model into Lightwave and delete the skin polys underneath the clothing where it is occluded - and then bring it back in and Texture bake it?? I'm not sure what you mean about using photoshop in this case - how does that help? The shirt UVs are completely different from the chest skin UVs - they are different subobjects I've combined for the high res model.

I'm not concerned about memory, I'm just aware that the source high res model has more uvs and textures that I don't need since the low res character has the clothes as his main body with no need for skin texture underneath - only in the areas where the clothing doesn't cover like the hands and forearms and head and neck etc.

[Carlosan]

the lowres have the 3 UVmaps done and is one object ?

if yes... and if i remember well...

With the highresmesh go to

paint room > textures > textures baking tool

Input mesh

mesh to receive projected textures

select the external lowpolymesh

[AbnRanger]

the lowres have the 3 UVmaps done and is one object ?

if yes... and if i remember well...

With the highresmesh go to

paint room > textures > textures baking tool

Input mesh

mesh to receive projected textures

select the external lowpolymesh

I think he is wanting to do something more complex than that. He wants 3D Coat to bake 5-6 UV maps down to 3, from a high poly group of 5-6 onto a low poly group of 3 or so. Good luck with that, I say. I've never needed to do anything remotely that complicated, or paint through objects in the scene, etc....so I'll leave it to him to explore or someone else to try and help. I would just try to keep it simple and bake one UV/object at a time. Just me, though.

I think 3D Coat would bake from a character with 3 UV sets, to another low poly version with 3, but I don't know about adding a shirt and pants (with their own UVs) into that bake. I don't have time to explore that, so can't be much help with it.

[popwfx]

I think he is wanting to do something more complex than that. He wants 3D Coat to bake 5-6 UV maps down to 3, from a high poly group of 5-6 onto a low poly group of 3 or so. Good luck with that, I say. I've never needed to do anything remotely that complicated, or paint through objects in the scene, etc....so I'll leave it to him to explore or someone else to try and help. I would just try to keep it simple and bake one UV/object at a time. Just me, though.

Yes, this is exactly what I'm trying to do... Maybe me having the high poly character wearing "removable" clothes was a bad idea? I suppose I could delete the chest skin polys and groin/legs skin polys underneath the clothes in an external modeler like Lightwave and then I don't have to worry about whether the chest skin or the shirt will be projected onto the low poly's chest (and I want the shirt to, not the skin).

But assuming I do that, and want to end up with 4096 sized (or 2048 if i have to) uv maps, if I take the high poly mesh with it's 6 UVs and combine them to one UV, then bake that to one low poly mesh UV, won't the texture be all lower resolution? I'd really like the head separate so I can keep detail for closeups... When the UVs are separated out like that, I can have my shirt and pants recolored by my game dynamically and the face skin can maintain it's high quality, compressed to 1 UV will eat up all the quality :-/ I'm still not sure how to do all this and keep detail on the areas I'd like to.

Is there another non-baking way of doing it? Like can I use a textured model as a 3d material to manually, like a projector that I can brush stroke from the high poly mesh as a 3d template and paint onto the low poly model based on the normals and closest point matches or am I asking for a feature request here?....and that is what the texture baking tool does but automatically and not per brush stroke like a material....?? I'm stumped.

[alvordr]

All I was saying was to use your 3d program of choice to combine the meshes into the 3 sets you want, assuming you have a program that can combine them with their UV sets intact. You need to make sure they don't overlap. While 3DC can handle overlapping, most other programs, especially game engines, can't. The only reason to get rid of the polys underneath the clothing is if you're planning on doing animation or game assets. I mentioned Photoshop, because after you combine the meshes and UV sets, you'll want to take your existing texture maps and align them with the new UV layouts. Otherwise, you can't satisfy your requirement of not having to repaint everything.

Just my 2 cents.

[AbnRanger]

All I'm saying is that you'll just have to set up your target mesh with the UV maps you want, then open the hi-poly objects in 3DC > have visible in 3DC's paint room, the objects (character/shirt/pants) you want to bake the textures from. Don't complicate matters with secondary elements like belts, shoes, etc. Bake them separately. Just try it. I don't have time to explore all of this. Never needed to do anything this complicated before, with the Texture Baking tool.

[alvordr]

AbnRanger is right. That approach makes more sense. Just bake everything separately, then put it all together in another app, as I suggested above.

[digman]

I will be testing some workflows inside 3DCoat next week about the question asked. I think it is a good idea to have as many ways as possible to accomplish your work inside 3DCoat.

Now testing might find that it is not a practical approach or even possible but without testing you will never know. Also testing helps when you ask for a feature request that you can give Andrew a clear idea of what you are requesting and why...

The below is not meant as an answer to photonvfx as his questions deal with taking multiple sub-objects with several uv sets and baking down to one object with 3 uv sets.

Testing possible worksflows can be productive as an example testing helped me to learn how to take separate objects (sub-objects) with their own uv sets in the paint room and export them out as one object with one uv set.

It was as simple as moving all the uv sets to one uv set plus deleting the unused uv sets and then renaming all the objects under the objects tab to the all same name. Was 3DCoat designed to do this by intent, probably not but it works. Now I could go to Andrew and make a feature request to have this added as integrated with some optimizing of the process into 3DCoat as a new feature.

Why would you want to do this someone might ask as you lose quality of texture. One reason is when you have a high quality model with multiple uv sets and want to place the model in the background of a another scene. There would be no need for the for the high quality textures with several uv sets but just one object and one uv set as the model is far from the camera.

Now testing is not for everybody, but for some reason I just like finding solutions if all at possible. I'm a glutton for punishment...

[popwfx]

All I was saying was to use your 3d program of choice to combine the meshes into the 3 sets you want, assuming you have a program that can combine them with their UV sets intact. You need to make sure they don't overlap. While 3DC can handle overlapping, most other programs, especially game engines, can't. The only reason to get rid of the polys underneath the clothing is if you're planning on doing animation or game assets. I mentioned Photoshop, because after you combine the meshes and UV sets, you'll want to take your existing texture maps and align them with the new UV layouts. Otherwise, you can't satisfy your requirement of not having to repaint everything.

Just my 2 cents.

All I'm saying is that you'll just have to set up your target mesh with the UV maps you want, then open the hi-poly objects in 3DC > have visible in 3DC's paint room, the objects (character/shirt/pants) you want to bake the textures from. Don't complicate matters with secondary elements like belts, shoes, etc. Bake them separately. Just try it. I don't have time to explore all of this. Never needed to do anything this complicated before, with the Texture Baking tool.

Thank you both for your replies. I appreciate your responses. I did not mean to sound like I was asking you guys to do my work, and I guess initially, I did not realize this was a more complex or fringe task. I would have thought in the process of taking high res assets down to low poly game objects that this sort of consolidation is routine. I can see how it is not that common now.

If I understand the approach you guys are suggesting, it is to make the low poly 3 UV sets first, then bake the high poly assets like shirt and head one at a time to the new UV sets on the low poly object. Then go into photoshop and layer them to combine the separate bakes into one final composite with the body wearing the clothes.

This makes more sense and I think I get it now. I'll try it and let you know. Thanks for the help figuring out an approach.

[popwfx]

I will be testing some workflows inside 3DCoat next week about the question asked. I think it is a good idea to have as many ways as possible to accomplish your work inside 3DCoat.

Now testing might find that it is not a practical approach or even possible but without testing you will never know. Also testing helps when you ask for a feature request that you can give Andrew a clear idea of what you are requesting and why...

The below is not meant as an answer to photonvfx as his questions deal with taking multiple sub-objects with several uv sets and baking down to one object with 3 uv sets.

Testing possible worksflows can be productive as an example testing helped me to learn how to take separate objects (sub-objects) with their own uv sets in the paint room and export them out as one object with one uv set.

It was as simple as moving all the uv sets to one uv set plus deleting the unused uv sets and then renaming all the objects under the objects tab to the all same name. Was 3DCoat designed to do this by intent, probably not but it works. Now I could go to Andrew and make a feature request to have this added as integrated with some optimizing of the process into 3DCoat as a new feature.

Why would you want to do this someone might ask as you lose quality of texture. One reason is when you have a high quality model with multiple uv sets and want to place the model in the background of a another scene. There would be no need for the for the high quality textures with several uv sets but just one object and one uv set as the model is far from the camera.

Now testing is not for everybody, but for some reason I just like finding solutions if all at possible. I'm a glutton for punishment...

Digman, as always, thanks for your help with testing and figuring things out. I will also try the approach you mention with naming things the same and seeing how that compares to the above approach.

One more thing though that no one mentioned, but is a side question of me asking this: Is there any way to use a textured 3D model in the same way the 2D materials are used to "projection paint" onto another model? imagine a semi-transparent model that overlays your target model and you brush onto the target model and it uses the point/poly normals to determine the "warping" of the the application of the texture from the other model onto your model. Obviously, the source model would need to be a similar structure, either that or there would need to be a way to translate and rotate the source model as a painting template over the target model for such a method.

Do any of you think this is a good idea for a feature request? Does what I describe make sense? thanks

[AbnRanger]

Digman, as always, thanks for your help with testing and figuring things out. I will also try the approach you mention with naming things the same and seeing how that compares to the above approach.

One more thing though that no one mentioned, but is a side question of me asking this: Is there any way to use a textured 3D model in the same way the 2D materials are used to "projection paint" onto another model? imagine a semi-transparent model that overlays your target model and you brush onto the target model and it uses the point/poly normals to determine the "warping" of the the application of the texture from the other model onto your model. Obviously, the source model would need to be a similar structure, either that or there would need to be a way to translate and rotate the source model as a painting template over the target model for such a method.

Do any of you think this is a good idea for a feature request? Does what I describe make sense? thanks

That's pretty much what the texture baking tool does already.

[digman]

LOL, AbnRanger, you are correct. I knew of course you could bake a single object and a single uv map to another single model and uv set using the texture baking tool. I just had never tested what was asked here.

The question at hand in both threads was multiple objects and uv sets in the Paint Room. Each question asked in the threads had a different set of parameters.

I was not even sure if the Texture Baking Tool could bake multiple objects and uv sets from the paint room to one object and with one or more uv sets...

I tested and the results were good at least on a simple object configuration. I have to try a more complicated model.

The method I mention in my last post was stated that it was not an answer for Photonvfx but how testing is productive...

The advantage of the method is that you do not need to create a new single retopo mesh and uv set, since all you want is to export one object and one uv set from the paint room for a multiple object. We are talking apples to oranges here as the method does not apply to Photonvfx questions...

[alvordr]

Well, if nothing else, the ability to do this is already in 3DC. The question is how hard it would be to get that whole process automated with a click, rather than the round-about way one would otherwise do it.

[popwfx]

AbnRanger,

Thanks for your tutorials - I just viewed this one:

on Sequential Baking and it is very helpful, but can you please clarify some points for me? Can you only do this with Voxel objects to retopo'ed stuff or can you do this with Imported Highres geometry to low poly geometry? Your video shows the VoxTree popup used for "name correspondence" - well obviously if I have imported geomtery that wasn't created in 3DC's sculpt room, I would have an empty Voxtree. So how (if it is even possible) can I sequentially bake hires to lowpoly target objects (with different UVs) in this manner described? or does this only work for voxelized highres objects?

thanks!

[AbnRanger]

AbnRanger,

Thanks for your tutorials - I just viewed this one:

on Sequential Baking and it is very helpful, but can you please clarify some points for me? Can you only do this with Voxel objects to retopo'ed stuff or can you do this with Imported Highres geometry to low poly geometry? Your video shows the VoxTree popup used for "name correspondence" - well obviously if I have imported geomtery that wasn't created in 3DC's sculpt room, I would have an empty Voxtree. So how (if it is even possible) can I sequentially bake hires to lowpoly target objects (with different UVs) in this manner described? or does this only work for voxelized highres objects?

thanks!

I think it has to be a voxel object. Andrew did this using correspondence between the naming of Voxel Layers and Retopo layers. If you can, merge the high poly reference mesh into the Voxel Room, so this can work for you.

[popwfx]

I think it has to be a voxel object. Andrew did this using correspondence between the naming of Voxel Layers and Retopo layers. If you can, merge the high poly reference mesh into the Voxel Room, so this can work for you.

Thanks, I've never done that before, so do I have to stitch up eye-socket holes and stuff like that first?

Also, if I decide not to do the sequential thing, and try to do it one by one, how can I get my high-res model into 3DC ? I mean I have tried importing reference meshes which allow me to retopo (and that's how I got my low poly model which is my target - which I then UV'ed inside 3DC), but the high-res reference LWO meshes always come in grey and don't seem to load their textures. How can I import my high-res model with textures intact so that I can bake the "hard" way (i.e. the non-voxel, non-sequential way)?

thanks for your help

[alvordr]

Actually, you can use the 'Close Holes' command after right clicking on the vox layer you want to do this for. You might still need to sculpt the eye sockets in a little, but that would speed things up.

As far as importing meshes, you can do this several ways. You can:

1. Open 3DC and at the opening screen, just click Voxel Sculpting, then the folder icon and choose your file to import. Press 'Enter' or click 'Apply.'

2. Open 3DC and at the opening screen, just click Voxel Sculpting, then the first icon (which is a grid or X depending on your version of 3DC) and then click the + beside the Root in the VoxTree tab to create a layer....right click the layer and choose Import > Merge Object and find your file to import. Press 'Enter' or click 'Apply.'

3. Open 3DC, go to File > Import > Import mesh for voxelizing. Press 'Enter' or click 'Apply.'

However, for both methods, I suggest that once you find your file to import and before you press 'Enter' or click 'Apply,' at the pop up options that show up after...un-check all options at the bottom, except for "Merge without voxelizing." Yes, I know this is confusing, but it works for me. You may also choose to Auto Scale your import before committing it to the scene.

[popwfx]

Thank for those steps, I appreciate your reply, but I'm beginning to think maybe I should avoid the Scuplt/voxel room for this project, it seems to be giving me more headaches when all I really want to do is bake 1 high-res LWO to another low poly LWO (that was retopo'ed and UV'ed in 3DC). I wanted to bake mutliple meshes like character accoutrements (like sunglasses etc - similar to the visor in AbnRanger's video) but at this stage, I'm happy to get anything working that bakes properly even if I have to do the baking one mesh at a time.

This is what I get when I try to bring my hi-res file in as you say to:

Textures didn't come in here either. So, forgetting voxels for now, how do I get a hi-res (~200K polys) LWO model into 3DC with all UVs and textures intact and showing so that I can bake it to my imported low poly mesh (which I assume I import with "Import Retopo mesh")?

[alvordr]

It is my understanding that you don't see textures in the Voxel Room. The exception would be if you had objects that have been sent to the paint room and are visible when you go to the Voxel room, but objects in the voxel room are separate and can only contain materials, not textures. Again, this is as I understand it. However, I do see options for importing your textures via the Paint Room or Tweak Room under the Textures > Import menu at the top. See if this helps...but you'll likely have them show up as layers in the Paint Room...not sure.

As far as your mesh, what you're seeing is your imported mesh with an overlay of the import on top. Did you click on another brush to make that overlay go away and does it still look messed up?

I also mis-spoke earlier. You can 'Close Holes' under the Voxels menu at the top. Have you tried this? Also, as I'm not a Lightwave user, could it be export settings that you have chosen in Lightwave? ...just curious.

[alvordr]

I see there is also a 'Make mesh closed' option on the Tools Options popup upon import...try that and see if that helps with your open mesh.

If you want, I can take a look at your file and play with it a bit, if you'd rather save yourself some time.

[alvordr]

I created a quick tutorial on this, just now. I don't know if it will help, but sometimes I need to see several videos on the same topic to really comprehend what's happening. Treat it as a supplement to what you already have or just ignore it.

How to Import an Existing Mesh Into the Voxel Room:

[NOTE: I'm not a member of Vimeo. This video is in much higher quality at Vimeo, so click on HD and then follow the direct link to Vimeo to see it more clearly.]

[popwfx]

Thanks for the tutorial, but I'm still having problems.

Is there any way to import a polygon model WITH textures instead of as voxels? I mean I know you can import textures into layers in the Paint room, but how do you assign a particular layer to an existing UV map? My hi-res model I want to import has many UV maps and I want to use that as the source to texture bake to my low poly one.

Plus with voxelizing - I get this weirdness:

then back in the paint room there are no textures, and the model looks really lowres and weird around edges like the sleeves. Also the near clipping plane is making it impossible to work with the object. I realize this is a scaling issue but my model is real-world sized (~1.9m high) and I cannot super scale it up because I need to use this in conjunction with other real-world accessories (like sunglassess) and environments that are specific sizes. The last thing I need is rounding errors in scaling back and forth causing size differences from how they were modeled.

So the chief purpose of getting a model into 3DC with textures is so that I can texture bake to a low poly retopo I made (which I have since also exported, tweaked in LW and added weight maps etc to - and therefore need to bring back in to be the target of the texture baking).

Sorry if I am being obtuse, but I'm not getting the workflow for using high-res externally created models and if I don't plan to sculpt additional details, I don't see the benefit to cover it to voxels if I end up getting additional issues like this:

Thanks for any pointers in getting a textured polygon model into 3DC. I finding 3DCoat's workflow for non-sculpting users using externally created or modified hi-res assets for baking is not entirely smooth...

[popwfx]

Plus as far as I can tell, importing a hi-res poly model like this as voxels doesn't bring in the UV maps too when I switch to the UV room - so without UV maps how can you use it to texture bake?

[popwfx]

Ok, so voxels aside, I've tried a very simple texture bake just to see that I can get it to work. And it sort of works (with some errors):

Here are 2 boxes, one with higher poly count and a different set of UVs than the other

. The high one was painted inside 3DC. The low one here is how it looks in LW after baking:

What's up with the Blue lines? and how could I do this with a larger character model which I don't have a .3b file for (i.e. how can I bring the hi-res character model into 3DC with its textures) so I can start the baking?

thanks

[alvordr]

Plus as far as I can tell, importing a hi-res poly model like this as voxels doesn't bring in the UV maps too when I switch to the UV room - so without UV maps how can you use it to texture bake?

You're correct. In those cases, where you already have UVs and textures, you can simply import your mesh into the retopo room. I believe if you do this after you have a voxel mesh, this will give you the best of both worlds, where the UVs are already set for the retopo mesh, but you can still sculpt high details...I think.

I can't verify this right now, but given enough time I can test it.

[alvordr]

What's up with the Blue lines? and how could I do this with a larger character model which I don't have a .3b file for (i.e. how can I bring the hi-res character model into 3DC with its textures) so I can start the baking?

OK, so this is something I had to learn with the help of others here. I would never have found it, myself, without a ton of experimentation. There are times you'll get those lines, because your retopo work may not have enough edges to support the deformation (in case of displacement) or may still result in the normals looking funny. In this case, however, it look like all you have to do is make sure to check 'Don't snap subdivision vertices to the surface' from the Retopo menu at the Retopo room after choosing Merge with NM. I also recommend not checking the 'Auto smoothing groups,' which I discovered helps, as well. This will often help with the bake and get rid of the lines.