Tutorial Painting Scanned Displacement Textures

[Ascensi]

Hi everyone, I am under a year new to 3DCoat and I love it so far. I purchased it right away knowing what it was capable of and specifically it seemed like the only 3D painting program that can work with Displacement textures and see it in realtime! 

Anyway, there was no tutorial for me, I asked questions and saw bits of info that came from various places and was a bit of a nightmare after buying a new piece of software and not knowing how to use it the way you need. So recently I put together a couple tutorials, a short version and more of a complete version.  This video -the long one, was not intended  to show off my art skills, I'm just dabbing areas with various textures while explaining how to and trouble shooting etc.  I'd still like to improve the ability to have  displacement work faster and maybe display better while painting. Perhaps if the lower poly model can be subdivided once or twice and while you paint a a front facing & distance based tessellation shader is used to help bring out the details that way you could potentially have subdivision on its highest level and have the same performance  and maybe memory conservation with 100% fidelity but with the speed as if you were working on a lower poly model.

So here is the tutorial if any one here has mastered working with Displacement with 3DCoat please let me know how I can improve the workflow or let me know if I explained something incorrectly. I was working with 8K Photo Scanned textures from RDT

 

 

[kwhali]

I am wanting to do similar with 3dcoat but my model have multiple 8k textures for  each uv set. It is very slow to bring in some baked textures and repair or add new details. Any tips? Even without import texture each 8k uv set seem to allocate 8gb of ram.

[Ascensi]

Well I'm running an Intel i7 4GHz with only 16 GB of ram, a GTX 970 Video card and 22 GB of windows virtual memory. I've had problems having an 8K library with many textures and trying to back it up and then restore it so that the textures would load faster from my SSD. So my 8k library limit was around 5 materials per library, of course you can have as many 8K libraries as you like.  It just takes a moment  for 3DCoat to load them because with the scanned textures for example when you choose a textures your loading approximately 4 at the same time (diffuse, depth in the normal channel, specular etc with a high poly mesh loaded to work on. I'd also consider working from cold boot, disable any program that is running in the background that isn't need at that time. There are also some commercial utilities like AVG tune up that can disable common not required processes, do a clean up and put your computer in turbo mode.

Edited December 14, 2017 by Ascensi

[Ascensi]

On 12/12/2017 at 8:57 PM, kwhali said:

I am wanting to do similar with 3dcoat but my model have multiple 8k textures for  each uv set. It is very slow to bring in some baked textures and repair or add new details. Any tips? Even without import texture each 8k uv set seem to allocate 8gb of ram.

I didn't know at first how to reply to your message directly on this forum but I've replied above.

[AbnRanger]

On 12/12/2017 at 12:28 PM, Ascensi said:

Hi everyone, I am under a year new to 3DCoat and I love it so far. I purchased it right away knowing what it was capable of and specifically it seemed like the only 3D painting program that can work with Displacement textures and see it in realtime! 

Anyway, there was no tutorial for me, I asked questions and saw bits of info that came from various places and was a bit of a nightmare after buying a new piece of software and not knowing how to use it the way you need. So recently I put together a couple tutorials, a short version and more of a complete version.  This video -the long one, was not intended  to show off my art skills, I'm just dabbing areas with various textures while explaining how to and trouble shooting etc.  I'd still like to improve the ability to have  displacement work faster and maybe display better while painting. Perhaps if the lower poly model can be subdivided once or twice and while you paint a a front facing & distance based tessellation shader is used to help bring out the details that way you could potentially have subdivision on its highest level and have the same performance  and maybe memory conservation with 100% fidelity but with the speed as if you were working on a lower poly model.

So here is the tutorial if any one here has mastered working with Displacement with 3DCoat please let me know how I can improve the workflow or let me know if I explained something incorrectly. I was working with 8K Photo Scanned textures from RDT

 

 

May I ask why you are using such high texture map sizes? 8k and up is more for Film-level work. If the model is aimed at games or general animation (TV ads, TV shows, CG Shorts, etc), 4k maps are more than sufficient, in most cases.

[kwhali]

6 hours ago, AbnRanger said:

May I ask why you are using such high texture map sizes? 8k and up is more for Film-level work. If the model is aimed at games or general animation (TV ads, TV shows, CG Shorts, etc), 4k maps are more than sufficient, in most cases.

I work on VR experiences, more about high quality visualization of static environment rather than high performance gaming. We have top end hardware to power the experience. We use many 8k textures for visual fidelty of the environment no matter where you are in the scene. Our low poly environment is usually 1-2 million triangles, we process data sets of billions of triangles. Sometimes though there are issues with baking and need small texture fixes.

3D Coat would be perfect for this but it is not very practical at this scale, even if I just want to juggle between two 8k textures, a large amount of memory is allocated and the steps involved to import and resize the UV sets to 8k takes a while(cannot import low poly with 8k uv sets as not even 128GB RAM is enough, coat allocates memory for these even if you have not painted or imported a texture file). I tried to load in just 2 normal map 8k textures to repair seam issue in paint room, on top end Intel hexacore(skylake) and 128GB RAM + NVMe SSD + nvidia 1080 & Titan XP, this still takes 20 minutes or so to setup(most of it waiting for coat while it is unresponsive processing), then I can import the actual texture files reasonably quickly(just have to use dropdown list of uvsets to find the ID I want(there is 100 sets in this case).

There is only one software I know that can do this well currently, Mari. It has support for texture streaming via disk cache, very fast and effective. Other tools have some support but cannot paint across multiple uv sets or have performance/memory issues like coat at this scale. 3DC needs to add some sort of proxy/cache, especially when working with 8k textures and no paint data is applied or texture imported...Otherwise 3DC would be perfect for this type of repair work.

[AbnRanger]

1 hour ago, kwhali said:

I work on VR experiences, more about high quality visualization of static environment rather than high performance gaming. We have top end hardware to power the experience. We use many 8k textures for visual fidelty of the environment no matter where you are in the scene. Our low poly environment is usually 1-2 million triangles, we process data sets of billions of triangles. Sometimes though there are issues with baking and need small texture fixes.

3D Coat would be perfect for this but it is not very practical at this scale, even if I just want to juggle between two 8k textures, a large amount of memory is allocated and the steps involved to import and resize the UV sets to 8k takes a while(cannot import low poly with 8k uv sets as not even 128GB RAM is enough, coat allocates memory for these even if you have not painted or imported a texture file). I tried to load in just 2 normal map 8k textures to repair seam issue in paint room, on top end Intel hexacore(skylake) and 128GB RAM + NVMe SSD + nvidia 1080 & Titan XP, this still takes 20 minutes or so to setup(most of it waiting for coat while it is unresponsive processing), then I can import the actual texture files reasonably quickly(just have to use dropdown list of uvsets to find the ID I want(there is 100 sets in this case).

There is only one software I know that can do this well currently, Mari. It has support for texture streaming via disk cache, very fast and effective. Other tools have some support but cannot paint across multiple uv sets or have performance/memory issues like coat at this scale. 3DC needs to add some sort of proxy/cache, especially when working with 8k textures and no paint data is applied or texture imported...Otherwise 3DC would be perfect for this type of repair work.

Please relay this to Andrew (owner/chief developer), so he knows how important it is. One of his developers was working on a GPU brush engine (even provided Andrew a test build, to which Andrew was impressed), and it's quite possible that could be a game changer. More user input in this regard, could speed things along more rapidly. Send it to: support@3dcoat.com and you could also drop a mention of it on his Twitter acct.

https://twitter.com/AndrewShpagin

[kwhali]

On 12/16/2017 at 2:13 PM, AbnRanger said:

Please relay this to Andrew (owner/chief developer), so he knows how important it is. One of his developers was working on a GPU brush engine (even provided Andrew a test build, to which Andrew was impressed), and it's quite possible that could be a game changer. More user input in this regard, could speed things along more rapidly. Send it to: support@3dcoat.com and you could also drop a mention of it on his Twitter acct.

https://twitter.com/AndrewShpagin

I believe I've already raised it on the forums in the past and notified Andrew. The only competitors I'm aware of are Substance Painter(but can't paint across UDIM/uvsets) and MARI(not particularly nice to use or train staff on, plus expensive, but only one that does texture streaming of this size well and can paint across tiles/sets). I will push for it again in 2018 after holiday season perhaps. It would definitely be a game changer for my industry not much competition out there and 3D Coat is a great, friendly and affordable(monthly subscription/rent to own license would be nice too of course) software to use, all our artists love it over zbrush now.

[Ascensi]

On 12/15/2017 at 1:31 PM, AbnRanger said:

May I ask why you are using such high texture map sizes? 8k and up is more for Film-level work. If the model is aimed at games or general animation (TV ads, TV shows, CG Shorts, etc), 4k maps are more than sufficient, in most cases.

I plan to either use Amplify texture or Graphine to stream HD textures in without performance issues http://graphinesoftware.com/  for VR.   I find  4 K is all right but isn't  realistic enough with displacement. These days the Film Industry are heading to the Unity game engine  to speed up their production because for the most part everything is real-time.  Look at the Adam Demo  https://unity3d.com/pages/adam   Now imagine creating film sets used in games with RDT textures https://www.rd-textures.com   Also Unity 2017 now has a Octane Render Plugin for Photo-realistic Rendering.

[Ascensi]

On 12/15/2017 at 7:51 PM, kwhali said:

I tried to load in just 2 normal map 8k textures to repair seam issue in paint room, on top end Intel hexacore(skylake) and 128GB RAM + NVMe SSD + nvidia 1080 & Titan XP, this still takes 20 minutes or so to setup(most of it waiting for coat while it is unresponsive processing), then I can import the actual texture files reasonably quickly(just have to use dropdown list of uvsets to find the ID I want(there is 100 sets in this case).

Not sure how your workflow is but you might want to add plenty of Virtual memory 20 GB or more as well. I know I didn't set displacement to the highest on import but I think it might help if 3DCoat was to use a tessellation shader rather than generating a hi-poly model. This is possible using a Unity asset called "Megasplat" and has worked very well for me personally and I have not run into performance issues using it. It uses a texture packing mode so you don't have to load 4 textures or more at once to paint. I would use it for just about everything except for the fact that there is no FBX exporter that can export Triplanar textures atm -but it's coming, this is something I've requested from a developer. 3DCoat has far more painting capabilities as well for just about everything so although there is Megasplat, 3DCoat will always be the main painting prog. Substance painter doesn't paint with displacement, at least from what I've known unless they've recently changed this -I've requested the ability to paint with displacement last year and they replied that if I wanted to I could make a shader to use with it
 

 

[Ascensi]

Ok another example. This wasn't that slow or hard to paint.. the initial low poly model was a 26k sphere, not Mb or Gb.. you might apply too much displacement if you already have a high poly model (100 mb or so) and then add another level of displacement in 3DCoat. 

Also for RDT textures it seems ideal to set/lock the depth in at the size the textures state they are (some say 250cm,some say 230cm) so I might make the depth .250 or .230) but sometimes you might even need to crank it to 1.0 -it may just really depend on model scale. 

On a side note I did also submit a request for performance improvements such as real-time tessellation, texture packing & streaming -would be pretty efficient.  Having texture packing could be very useful  now since many Unity assets use them so exporting to particular order could speed things up.

 

[kwhali]

Yeah, texture streaming is a really needed performance feature for this kind of work. Hope to see it in 4.9 release if lucky.

[Ascensi]

Update:  In this instruction, I convert the depth.tif to png otherwise 3DCoat can hang or crash. This is likely another issue people are having with performance, it will definitely kill it!

 

[digman]

I noticed in your first video that you are loading for Per pixel painting which was designed for normal map painting. The show displacement feature was added to give you a visual representation of true displacement if needed.  It is not designed for true displacement painting, You can do some of course but also if you push it too much you will start getting cracks in your model or artifacts in the model.

Watch this video on Micro-vertex painting. It a 3 years old video but still valid at least on the information about microvertex painting. The video bakes from the retopo room but you can import for micro-vertex painting from the New menu.

Micro-vertex painting is true displacement painting in real time and made for that.  You export your low polygon mesh at then end and apply that displacment map in the game or renderer of your choice. You can also export a normal map.

There is no Show displaced mesh in micro-vertex mode. There is no need of it, your depth painting is already displacing as you paint your depth. You  still can tessellate more still but generally it is not necessary.

Do this test.  Go into per pixel mode and turn on show displaced mesh. Turn on wireframe mode and do some depth painting You will notice that the wireframe mesh is not really displaced.

Go into micro-vertex mode, do the same, and you will see the true polygons being displaced on the display mesh subdivision level  you chose when you imported or baked to the paint room.  Remember though that the displacement map that is being created in the paint room is done in real time and applied to the mesh as you paint. The millions of polygons use to create the displacement map on the fly is an internal operation as explained in the video.

This also helps on exporting as you can export the low polygon mesh with how the model was displaced in the paint so that it better fits the model when it is sub-divided in your game engine or renderer. That is an option you can choose.

 

Edited December 20, 2017 by digman

[digman]

The video spends some time on repairing the few errors in the mesh but the knowledge of how micro-vertex works is explained.

I recommend just using a simple model and seeing how things work in micro-vertex painting. 

You can import and export 32 bit Exr files which most game engines support today and of course renderers. Try loading your 16 tiff images too and see if they work better there. 

I generally do not have trouble with tiff files in 3DC but have not tested deeply in quite a while.

Edited December 20, 2017 by digman

[Carlosan]

In my test .tif is 16bits, you need to export .tiff to get 32bits.

Sidenote:

Anyone testing Vector Displacement export ?

[digman]

Was just saying the tiff / tif format I generally do not have problems with. 

I will test exporting a vector displacement from 3DC and importing into Mudbox when time alllows but that is not anytime soon.

Edited December 20, 2017 by digman

[Ascensi]

4 hours ago, digman said:

Was just saying the tiff / tif format I generally do not have problems with. 

I will test exporting a vector displacement from 3DC and importing into Mudbox when time alllows but that is not anytime soon.

The tif files only gave me problems with creating  8k+ libraries but also seemed that when I used those materials 3DCoat would become non-responsive or crash.

I understand using the microvertex mode for new sculpted creations or after converting models  but for existing models with great topology & UVs and models that can't be changed for re-import reasons into their native program such as iClone character body obj PPP mode is ideal. Working with tris aren't that safe for tessellation and is easy to cause cracks but also hard edges and thin areas such as ears. 

Also PPP now has evolved for displacement painting so although a tutorial like this 

 will go into painting depth information in microvertex mode and manually painting cracks and shape , I'm painting the depth information as displacement from the 8K scanned texture s in PPP mode.  Instead of using the normal map in PPP mode I'm using the depth/height map.

In terms of what you said about "tessellating more if I wish" are you implying that there is a tessellation shader built in and instead of subdiving the model I could just crank up tessellation instead? if yes than that is what I've been looking for!  I will try it and see if it works.

 

@Carlosan when exporting the low poly mesh from PPP mode, is it a new mesh with different verts or is it identical from the one imported?  If I have a model that needs to have the UVs remapped on import, I need to have the exact same model exported but with the new UVs..  is this possible?

Edited December 20, 2017 by Ascensi

[Ascensi]

Update: yes! I guess 3DCoat was updated and the subd patch  levels under "texture" on the menu has been renamed to "adjust tessellation" and now I know what it is -thank you!    It works works but seems to be a lot slower. Cubemapping seems to work best, cracks don't show. I've used a non subdivided but tessellated cube to show the difference. Subdividing then working with cubemapping. Again cubemapping seems to work best preventing cracks but it stretches the texture without using subdividing and using tessellation only but with using tessellation only no subd UV mapping shows cracks. So overall subd works best painting it seems over using tessellation by itself with Cubemapping which is apparently the same thing or idea as  triplanar texturing.

 

[Carlosan]

I need to have the exact same model exported but with the new UVs..  is this possible ?

Switch Use source position to ON

[tokikake]

Could someone "teach" me, how 3d coat generate (and bake) Vector displacement maps?

I have serched about vector displacement (as you know blender already offer nodes, so you can see how it work, but not offer way to genearte vector displacement map)

my sculpt skill is still beginner ,but anyway I want to know, how it work. 

eg just use primitive in sculpt room, then how generate "vector displacement map",  to use  low poligon mesh node (in blender cycles) ? 

I tried before but no success, ,, and serched google, I happend to find this topic so I asked here 

[Carlosan]

 VD works only with microvertex painting. Look under the textures menu.

[tokikake]

@Carlosan

I saw another topics about VD in this forum but it is old topic,(2011, 2012 etc)  then I hoped to know clear,,.

So you means, we can not use the VD map, for retopo UV mesh  + sub-D , at least about current 3d coat?

or import sculpt mesh as microvertex paint (when there is sculpt mesh, it show in paint room to micro vertex paint,,)

I tried to generate VD texture  for retopo mesh by "export texture" from paint room, but it simply generate perfect grey texture for me,,

or I need  to use taxture baking tool ?   or you means I can import VD to show disiplacement for micro vertex painting?  

 

And Can I request it in Trello , bake vector displacement map from sculpt high poligon to retopo mesh?

or there is already same request? (I often try to request in Trello but I still can not understand how it work ^^;)

 

[Carlosan]

I found 2 post related to this request

 

[tokikake]

@Carlosan

Thanks, I follow these topics you linked, but it still complex enough for me ><;

I really like to serch 3d coat training video (official and other users offered thanks!)

so, If you can request andrew or project team, I hope,, to see new 3d coat tutoriall video about Vector dsiplacement map,,, 

"how to generate (bake) Vector Displacement map (tangent space)  from high sculpted mesh, , and use  it for your low poligon model (sub-D) in another apricaiton.

I know there should be many case, (mix with normal map etc), but I hope to most basic procedure, with simple sculpted mesh, (eg sculpted cube)

Anyway If I can generate vector displacement map (tangent space) , I can use it in blender,,, with  multires modifier, etc.

(I  only concern about "vector displacement" so hope to know which option I need to use more clear ^^;    do not think about usuall grey-scale displadement map..  I know how 3d coat generate it,, )

Edited December 29, 2018 by tokikake

[Carlosan]

I forwarded your request.

[Carlosan]

Meantime is this video useful for your current workflow ?