Creating tiling textures/models from photogrammetry

[DMG]

Afternoon,

I know I can sculpt my own sculpt and paint my own tiling textures in 3DCoat using the symmetry functions, but I am wondering about assets I have created using photogrammetry. 

As an example, Here's a range of images I have taken:

and here's the textured model I have created in 3DZephyr: (Also great software, by the way) 

It's about 2 million polys, and I can export it with an 8K texture map. I normally then use Shadermap to generate normal maps of the fine surface details from this diffuse texture.

I realise 3DCoat might not even be the best tool for this job, but does anyone have a workflow which might be used to work on this so that it tiles seamlessly, and exports diffuse, normals, displacement, etc.. for use in other 3D software?

This is just one example, and it isn't a pressing issue for production, so I'm in no rush.  I just have a folder of similar assets, and being able to tile them over a larger area would be great. 

One idea I haven't had time to properly explore is to create a mostly flat plane, and bake the diffuse and a Z-depth, then apply these to a flat plane in 3DCoat as using the colour channel for the diffuse, and the metal channel for the z-Depth, and copy/clone, etc. This way the diffuse and depth are manipulated in the same way at the same time.

I have also wondered about converting it to voxels (however this model is not watertight, and is just a shell), and duplicating then splitting off individual rocks, etc to cover up the seams, perhaps relying on the symmetry's tiling functions to keep it repeatable.

Then I thought instead of wasting days or weeks fumbling around, to ask and see if anyone had any better or tested solutions...

Cheers,

Derek

[DMG]

I've been fiddling with this off and on over the past week.

 

I have roughed out a workflow, but there's a bit of jumping to and from between different programs.

In a nutshell, my plan is:

1) create a low-res, UV'ed  cage object which I can use to bake out the diffuse, height/displacement, and the tangent normals of the geometry.

2) Make these baked maps seamless. I'm hoping I can clone parts of these maps in 3DCoat to hide the seams, but each layer has to be cloned in an identical fashion. The free app Materializer has a 'seamless' feature which works well for regular patterns such as bricks, etc, but not so great for irregular textures such as this old wall.

3) Use Shadermap to create a normal map of the surface detail contained in the texture rather than the geometry. Materialize can also do this, but I find Shadermap to have far more features. 

 

So.  Step 1.

I can create my low-res geometry in 3D Coat. However, I need the UV map to perfectly fit the UV space. I've fiddled with the UV settings but can't seem to numerically set the UV point locations, or make the points snap to the area.

 

So, until I find a way to do this, I'll need to export this Retopo object into Blender, fix the UVs and bring it back in...

 

Next... Baking.

[DMG]

Ah.

 

So it doesn't appear that I can bake the diffuse, normal and displacement in one go.  The imported textured object in the paint room can do the diffuse and normals, but I need to find a workflow to bake the displacement, which seems to require the geometry in the sculpt room.

 

[Carlosan]

As i remember, to make a plane that way so I could paint a tile able texture ( non-square) continuously from one side to another it is not allowed.

[DMG]

Thanks Carlosan,

I was hoping I would be able to have the UVs right to the edges, and paint the diffuse, displacement and normal maps all together. I guess that's not likely.  I have seen a video online of someone doing similar, but painting each channel/layer individually, which seemed like quite a laborious process, open to maps not aligning properly.

I shall continue to ponder options...  Though I'm not sure there are any very good ones... 

On the plus side, I did work out that I can import a model into the sculpt room using Import: Import for Sculpt/Vertex Painting/Reference, bake the displacement using that, and save/export it; then import the same .obj using Import: Model for Per Pixel Painting and bake the diffuse and normals from there, then import the displacement.

I ran the exported diffuse map through Shadermap to generate a normal map for the fine surface details, and at the moment it looks like this: (four quad polygons, tesselation set to 16 to show a bit of the displacement)

 

[Carlosan]

WoW !

> https://www.pixplant.com/ works well

[DMG]

Thanks Carlosan,

I had come across pixplant, but it struck me as very similar to Shadermap which I already use, but with some tiling ability. 

I've just watched some tutorials and examples of the tiling tools, and it is light years ahead of anything else I've seen. It's practically witchcraft.

Will almost certainly be picking up a license. Looks a hell of a lot better, faster and easier than painting multiple maps simultanously.

Thanks for the recommendation. 

[DMG]

Update...

I have tried PixPlant. It's brilliant and it will definitely get plenty of use, but the results with this particular rocky wall have been mixed. I'm sure I just haven't found the right settings yet, but different types of rocks were being blended together. If viewed from a distance, it's great. But if it is only tiled two or three times, it's not perfect. Though to be fair, this particular rocky wall was just a test, designed to be almost as awkward as possible.

So. Back to painting seams in 3DCoat... 

I have discovered that if I import a flat plane, with UVs which fill the 0,0 to 1,1 range, I can import that into the paint room, apply a diffuse and my displacement map.  It tiles in the UV window. 

I have loaded the displacement map into the roughness channel of the diffuse layer so I can paint them both simultaneously.  It only needs to be a black and white map, so it is irrelevant which channel 3DCoat thinks it is. I can create a normal map for the geometry from this height map later on, and create a fine detail normal map from the difuse later on, so I only really need my colour and height maps.

I'm painting in the UV window with the clone tool, (which needs the opacity set to 2, rather than the default 1 to get a clean, solid paint-over). I can't CTRL-click to set my clone origin outside the UV map range, but I can paint outside the UV area. So as long as I'm copying texture from the centre of the image, painting over the edges isn't a problem. Also, the origin of the clone tool seems to be in screen space. If I select my copy point, then pan my view, it copies from what is now under the screen coordinates I'd clicked. It would be nice if it had the option to go by the UV coordinates instead, but It's not a massive issue.

I haven't checked to see how well they tile outside of 3DCoat, but it's looking promising...

 

[DMG]

This is one corner where I've cloned a rock.

Opened and offset in Photoshop. Looks like a good join to me... The texture stretching is purely from the texture created by my Photogrammetry software (I simply didn't capture some parts from enough angles—but to be fair, I was on holiday and my family didn't want to wait while I photographed rocks all day..)

 

Hurrah.

Okay, this is all looking pretty good. I think I shall go back to the start and export a high-res model from 3DZephyr; bake top quality diffuse and displacement maps in 3DCoat; load them onto my UVmapped painting plane, and do a bit of cloning to fix the seams. After that, it'll just be a case of generating a fine detail normal map in PixPlant or Shadermap. It may be some time before my next update... 

This all looks like it will work. Each step is going good and pretty straightforward.  The only stumbling block I have come across is getting 3DCoat to create a UV map which perfectly fills the UV space. I can get it close, but not bang on the 0,0 1,1 perimeter. I can easily sort this in Blender and re-import, but if 3DCoat was able to precisely position, snap or enter numeric values for UVs, I could pretty much run this entire workflow inside 3DCoat.

[Carlosan]

Have you tried Tiled Planes ?

Please detail your step by step procedure -above mentioned- and where you got stuck, or what you need to improve with the normal maps (filling UV spaces).

[DMG]

Tiling Planes...? Those have escaped my attention so far...  I will have a quick look.  If they do what I think they might do, then I don't need to leave 3DCoat for anything but the Photogrammetry and the final detailed normal map.

 

Thanks.

[DMG]

Perfect. Tiling Planes work for importing my textures before painting the seams. They have the same behaviour as my imported plane with a UV map where I can clone from inside the UV area and paint inside and outside, but cannot clone from outside the UV area.  That's not a problem though.

My only remaining UV issue is getting my low poly baking object (usually a simple plane of maybe 8x8 quads) to have a UV which fills the UV area exactly. 

I changed my UV settings (as in my second post) ...

... created a plane with an 8x8 grid, and unwrapped it.

When I check the numeric values of the points in Blender, the edge ones are close to the boundary, but not exactly on them. This point (highlighted) should be at 1,1, rather than 0.999,1.

 

 

This grid is for baking my colour and displacement maps to, so any seams or stray pixels at the edges will be painted over so for my purposes here. it's not critical.

Still, it'd be nice to be able to snap UV points to the boundary exactly. If there is a way, I just haven't found it yet. I'm still looking though.

 

Derek

 

[DMG]

Oh, and another situation is if my grid for baking isn't perfectly square, but rectangular:

How could I make sure the UVs fill the UV area and snap to the edges without just eyeballing it?

[Carlosan]

Textures > Import > Color ?

[DMG]

Hi Carlosan, yes, I've been using the Texture: Import functions.

Here's a bit of a breakdown of some workflow points. It might better explain what I'm doing, and/or where I'm going wrong, and an issue I've come across with my displacement maps.

Bit of an epic post, sorry.

UVs:

The UV issue is more to do with creating the UVs for my baking mesh.  Here's an example (it's not from a photogrammetry object, but that doesn't matter for the purpose of this illustration).  The surface I want to capture and turn into a tiling texture could be an irregular or curved surface such as a tree trunk, etc. I'd import my photogrammetry object, and create a retopo object to bake the displacement and colour to. 

Thing is, I want my UV to completely fill the UV area. I can use various tools to straighten the grid up, but I can't see any way to make the UVs expand to fill the area so that: the vertices on the left snap to a U value of 0; the vertices on the right snap to a U value of 1; the vertices on the bottom snap to a V value of 0; and the vertices on the top snap to a U value of 1. I can eyeball it, which is okay, but being able to expand it to fit would be superior and more accurate.

I've also not found a method of using the gizmo in the UV window to restrict the movement of the UV island to left/right or up/down.  There's handles to restrict scaling to those directions, so I'm a bit surprised I can't see similar handles for translation. 

None of this is not an issue when setting up to paint the textures to tile them, as the Paint UV Mapped Mesh: Tile options import a plane with UVs right to the edge. (Thanks for highlighting that option)     

 

Normal Maps

No issues here at all.  I just wanted to highlight the maps I create and use.

The normal map generated by 3DCoat is perfect. However, there's a lot of overlap between it and the displacement map.  The normal map can easily be generated from the displacement map later on. If I'm painting maps to tile them, I'm not going to bother painting three maps when I can do two and derive the third in a single click later on. 

The high-detail normal map I generate later creates tiny surface details from the diffuse map which is not captured in the scan/photogrammetry model. This is best created after the diffuse has been tiled, and it's not functionality that 3DCoat has or needs.  I use Shadermap and now PixPlant to create these maps. Depending on the situation, I could use either of these normal maps, or a mix of both of them.

Here's tiny swatches from the 8K maps:

Diffuse (baked in 3DC): 

Displacement (baked in 3DC):

Normal map (Baked in 3DC):

Normal Map (derived externally from the diffuse):

 

Displacement Maps:

I have realised that my displacement maps need to remain 16bit.  If they are reduced to 8bit textures, then 'stair stepping' artifacts become visible when rendering. I had tested copying the displacement into the roughness channel so I could paint the diffuse and displacement together, but the roughness channel seems to only support 8bit.  

A bit of research late last night suggests that alpha channels in 3DCoat do support 16bit (most of the posts on this forum about them seem to focus on using them to create brushes), but right now I have little experience of working with those, and I am not sure if they could even be painted simultaneously with the diffuse when I try to tile them. I fear this may ruin my plans... More experimentation needed, and suggestions welcome...

 

Cheers,

Derek

 

Edited September 7, 2022 by DMG

[Carlosan]

I can't see any way to make the UVs expand to fill the area

Have you modified Min/Max UVsettings ?

 

[DMG]

Yes, I have changed those settings.

 

(Showed that in my second post)

 

A later post shows that the UV points are close, but not exact... (The inaccuracy varies along the edges. I think it the whole UV island might be rotated by a fractional amount) 

 

..but even though that amount probably falls under the definition of 'close enough', it doesn't help me if the mesh I'm unwrapping isn't square, because I'm then back to manual guesswork to scale and position it.

 

It's not a workflow killer though. Depending on the situation, I can either live with it being 'close enough', or export the retopo mesh to Blender and snap the UV points there.  Would be nice if there was a 'rotate-and-stretch-to-fit-and-snap-to-the-edges-of-the-UV-area button inside 3DCoat though.

 

Cheers.

[DMG]

Was just having a quick look inside PixPlant, and it has the ability to clone simultaneously over multiple maps.

I think this solves my last issue with simultaneous cloning and keeping displacement maps 16bit. I'm not saying there isn't a way to do this inside 3DCoat (and I may very well continue to investigate layers, alpha channels, etc) but PixPlant is an app designed for this very purpose, so it is the logical tool for that part of the job.

Workflow complete. Just need to fix up these test maps.

I'm still open to suggestions regarding the UV issue I highlighted, and cloning 16bit displacement maps at the same time as the diffuse map.

Thanks especially to Carlosan for all his assistance and for pointing me in the direction of PixPlant.