custom Mask jagged normals in paint room

[elindell]

Hello, I working on figuring out a good workflow to create tileable normal maps for use in unreal engine. In 3d coat I'm in the paint room trying to use 2d images that I create as masks to paint on normal depth. When I try it I seem to get some sort of banding in the depth. It's hard to explain so hopefully these sample give an idea. Is there some setting I'm missing or a specific file format to import that would smooth these out? 

 

I can't upload the original mask I used since its over 10meg but here's one at half res to show what I was working from. The mask I used was 2k and the UV map I was painting on was also 2k. The mask I created was uploaded from a png file.  Any ideas? 

 

Thanks

 

 

[Carlosan]

Hello

 

paint room

Shift+M

 

select TILE

[elindell]

Thanks for responding carlosan, unfortunately that doesn't really address my issue, I'm having quality issues with the depth layer created in the paint room when using a custom mask. Please look at the pictures I posted, instead of a smooth rise I'm getting those steps. 

[Carlosan]

sorry misunderstood

 

i tested you file in 3DC and another software... all give me bad results

 

will you please test with another image ?

 

ty

[elindell]

Thanks for testing, I've tried numerous texture, anything with a soft gradient result in a pretty noisy depth, even one of the default mask (mask sample 1 and 3 here)

I tried to use 16 bit mask but those don't work at all in 3d coat. 

I will keep testing and if I find a solution will post, maybe there's a setting that increases sampling of masks somewhere? 

 

[Carlosan]

The grey scale have 256 colors

 

if i change the mode to Indexed -external paint app-, the jagged shapes are more noticeable

 

------------------------------------------

I found this info

 

I had this problem recently. When you use masks you need ridiculously high polycounts to get smooth edges when you inflate the details. There are 2 things you could try....

1. Mask with the intensity like you were doing but use the size slider instead of inflate. Then use pinch and smooth to sort out any jaggies. This takes while and you're never going to get perfect edges

2. Something that worked quite well for me was creating an alpha from the design (just turning it onto a black and white image) 
Hope thats a help