Andrew Shpagin

Painting color over voxels

61 posts in this topic

Disagree with you there. Texture painting on low poly objects is probably more important to me at this moment even than voxel sculpting. This is a feature that doesn't work as it should yet and for games work it is very important. 3dcoat could very well displace the need for bodypaint, modo, and mudbox if it actually gets proper texture painting. That means more customers and a solid push into studio use. As much as I like the voxel sculpting I'm still not convinced I want to use it in place of say zbrush yet. I want to see it replace zbrush too but I also want texture painting that actually works just as much, if not more so. It was promised for 3.0 and I purchased the software in good faith, under that promise. Zbrush polypaint isn't exactly the best solution for all texturing needs. It's pretty much useless when you need to make tweaks or changes on texture assets that are already in game. With 3dcoat being able to paint accurately in pixels AND edit normal maps there will be no other competition from 3d painting apps at any level. None of them can do this.

I was referring to texture painting in relation to the voxel sculpting (the subject of this thread). I know low poly painting is missing right now in 3dcoat, but as Andrew mentionned, it's coming.

I myself use Maya 3dpaint tool for low poly painting work. And frankly even if it works ok, it's a bit outdated (no layers etc), and I can't wait to do it in 3dcoat.

Franck.

Share this post


Link to post
Share on other sites

I may be just crazy here, but I would think poly painting on game res object would be easier to code than the painting we currently have. There's less calculations with no subdividing. Of course I'm no programmer so I could be wrong.

Share this post


Link to post
Share on other sites

As promised here are some screenshots of voxel painting in Claytools. Unfortunately I don't have Freeform Modeling, which allows to paint color.

Here in Claytools the painting tool is only available for selection and masking purposes. Nevertheless, as you can see below it is very powerfull and you can imagine how cool the color painting would be with proper brushes.

I'm very much looking forward to having voxel painting for selection and masking as well in 3dcoat.

First I enter the selection paint tool with a rather large spherical brush:

post-497-1225273639_thumb.jpg

Next I paint and I change the size of the brush along so you can see later the effect of depth.

As you can see at the bottom of the screen, there are various tools where you can fill selection, deselect all and invert selection. There is also a 'select lump of clay' in the main toolbox on the left which enable to select with one click a bunch of isolated voxels in space.

post-497-1225274141_thumb.jpg

Next I use the cut button to remove the selection from the main sculpt. Now you can see the effect of the brush size on depth.

post-497-1225274149_thumb.jpg

Then I paste the selection and move and rotate it to another place. From there I could add it to the main piece or substract it. Or paste it to another piece (separate object in the tree view).

post-497-1225274161_thumb.jpg

Here I choose to add it to the main piece.

post-497-1225274167_thumb.jpg

Next I increase the resolution to paint some masking. I start with a spherical brush, then I add a custom brush pattern to the brush, a grid for instance (any bitmap could do).

post-497-1225274181_thumb.jpg

I click and you can see the brush has been applied with the pattern.

post-497-1225274193_thumb.jpg

Now I can carve with masking. Notice how the depth of the maskin brushes protect the carving.

post-497-1225274213_thumb.jpg

Hope this helps to visualize the power of true 3d voxel painting when combined with sculpting.

Cheers,

Franck.

Share this post


Link to post
Share on other sites

Looks very interesting

Share this post


Link to post
Share on other sites

Thanks for interesting samples!

It is interesting, how is performance in comparison to 3DC?

Share this post


Link to post
Share on other sites
It is interesting, how is performance in comparison to 3DC?

It is much slower on the rendeing side. They have point rendering and a kind of hierarchical LOD (or they just skip points I don't know), that can help a lot but it becomes a bit abstract when working on huge models.

On the brush side, it's ok up to say 10 millions, then it's becoming very slow (laggy brushes create straight lines). But it is a bit hard to compare to 3dcoat because the accuracy is so much more limited due to the Phantom device.

The great part is the ability to split your work into pieces and merge them later. So the brush performance limitation is less damaging.

Memory management is great in 64bits, never hit the ceiling. Still workable with objects of above 100m. I've used it to assemble pieces that I did in mudbox 1.0 last year, and then generate a single object.

Once all features are in, 3dcoat will surpass them in every area, except maybe the fast blocking-out of initial shapes. That's where the Phantom device is very handy.

So My future pipeline will probably be blocking out stuff in Claytools then bring them asap into 3dcoat for further detail and painting... isn't that the whole point of 3dcoat :)

Cheers,

Franck.

EDIT: Just did a quick test at 35m. Actually the brush doesn't get much slower if it is the same size. However as the resolution increases, you might need to use a much larger brush for rougher adjustments. In that case it becomes very slow because it has to move more voxel. Normal thing then. Knowing this, at some point it will become too slow to work with anyway. Like 100m or above, you can't sulpt past that limit as the carve brushes are definitely too laggy.

Share this post


Link to post
Share on other sites

Frankie you are saying that it is easier to blocking out your form in Claytool because of the phatom.

Can you explain it better?

Are you used to block your form in Zbrush or Mudbox and what is the difference between 3Dcoat or Mud/Zb vs Claytool and its haptic device for this?

I never used Claytool and I didn't find the way of blocking out the shape in Zbrush slow, but because I don't have any experience in Claytool I would like to hear your point of view.

At least if it doesn't help 3DCoat I will be smarter than now :)

Btw the voxel painting you presented is really interesting.

Thanks.

Share this post


Link to post
Share on other sites
Frankie you are saying that it is easier to blocking out your form in Claytool because of the phatom.

Can you explain it better?

Are you used to block your form in Zbrush or Mudbox and what is the difference between 3Dcoat or Mud/Zb vs Claytool and its haptic device for this?

I never used Claytool and I didn't find the way of blocking out the shape in Zbrush slow, but because I don't have any experience in Claytool I would like to hear your point of view.

At least if it doesn't help 3DCoat I will be smarter than now :)

Btw the voxel painting you presented is really interesting.

Thanks.

3dcoat's voxel with an haptic device would work pretty much the same way as Claytools does. Mud/Zb however are surfacic and in essence not capable of sculpting any shape from scratch. You need to start with a shape and then can only deform its surface. There lies the surfacic tools' topology constraint.

There are several advantages in the use of the Phantom in combination with a voxel tool. The device reads your 'pen' position and orientation in space at the same time. You're not working in a plane, or along a surface normal, that means you can carve and extrude matter in any direction at once.

With the Scraper tool for instance, you can cut the corner of a cube with a single stroke, without even tumbling the view to align. It's pretty much like if you had the tool in hand, you can rotate it the way you like and then touch the object.

You could even carve on the back side of the object if you wanted as you feel the surface with the force feedback of the Phantom, not that it makes much sense :P

I find it very cool for carving and scraping hard edges such as for creating rock formations.

Navigation is very easy as you control the camera position and orientation at the same time.

The disadvantage comes from the precision of the input if you compare it to a wacom tablet for instance. It is great to create rough shapes, but if you need to precisely detail a surface, it's not accurate enough.

Hope this helps,

Franck.

Share this post


Link to post
Share on other sites

Something like this has probably been suggested already, but: Maybe instead of storing the color value for every voxel you could just store a material reference. The materials in turn could then be projected in a few different ways, like planar, cube etc. (and be masked/blended by alpha) Just my two cents there anyway, I have no technical background so I'm not aware of performance issues.

Thinking about this further.. how's this:

For every voxel you store the information wether it is affected by a certain material (So for every Material you need 1 bit). Then, for every material you should be able to create multiple projectors that you can move around and for every projector there should be an alpha map so you can mask out and blend (this alpha map, ideally, should be modifiable just by painting on the surface).

Share this post


Link to post
Share on other sites
I don't know if this has any relevance. Mudbox 2009's Polypainting is somewhat superior to zbrush's polypainting. Subdivision level doesn't need to be raised to get a better resolution for the texture. Video link demonstrating this.

http://www.dashdotslash.net/Mudbox2k9/pain...20tut%20SM.html

Mudbox 2009 does not have poly painting like ZBrush, whats being shown in the video is just projection painting. ;)

Share this post


Link to post
Share on other sites
Really interesting paper SonK.

Yeah, i'm just reading it now..its like Zbrush polypaint but it also stores color value for edges and faces, I hope Andrew implement this into 3.0. <_<

Share this post


Link to post
Share on other sites
Yeah, i'm just reading it now..its like Zbrush polypaint but it also stores color value for edges and faces, I hope Andrew implement this into 3.0. <_<

Yes, direct painting will be in 3.0.

There are several; approaches of painting:

1) Per vertex painting -ZBrish,3D-Coat

2) Projection painting - Body Paint, ZB, PS CS4. But Painting resolution is dependent on object size in screen space. It is not good.

3) Direct painting - every pixel in UV has some position in space. You are painting directly over UV placed in space. It gives the best quality. But problems - it is easy to project texture on object but difficult to smooth, smudge, clone. Mayebe Mudbox has direct painting but in very basic stage (texture projection, it is most easy task). But 3DC will offer full set of tools for direct painting - texture projection, clone, transform, smudge.

Share this post


Link to post
Share on other sites
Yes, direct painting will be in 3.0.

There are several; approaches of painting:

1) Per vertex painting -ZBrish,3D-Coat

2) Projection painting - Body Paint, ZB, PS CS4. But Painting resolution is dependent on object size in screen space. It is not good.

3) Direct painting - every pixel in UV has some position in space. You are painting directly over UV placed in space. It gives the best quality. But problems - it is easy to project texture on object but difficult to smooth, smudge, clone. Mayebe Mudbox has direct painting but in very basic stage (texture projection, it is most easy task). But 3DC will offer full set of tools for direct painting - texture projection, clone, transform, smudge.

I'm really impressed with the progress you have been making. You have an amazing work ethic. You have the set the bar really high. I appreciate your hard work, it is inspiring. You are surely blessed.

Share this post


Link to post
Share on other sites
2) Paint over voxels in volume. Not easy to implement because:

- exery voxel takes 2 bytes now (16 bit precission). Color is 3 or even 4 bytes more. Much more memory consumption, less speed because I need to preserve color (3 additional channels) during all operration. + layers... uff

So if you have any idea, please share it there.

Sorry for jumping in late on this...

You don't HAVE to have 3 additional channels, nor do they have to be 16 bit. You could have the user specify how many channels and at what depth they are needed. Adding an 8 bit scalar to the existing voxel structure would add only 50% more memory usage. Even if you locked the depths down, adding a single scalar at 16 bit would only double the memory size. The speed hit would be less, since you aren't reconstructing the isosurface using the other channel. Also, the operation is identical on the second scalar, so it's just moving another channel of data through the same instructions, which should be very efficient.

Regarding memory, CUDA hardware currently runs up to 4GB in size in the Tesla range, and the GeForce line will soon be at 1.7GB. As applications for CUDA grow, I'm sure NVidia would love to sell you hardware with even more memory, so I don't think it is an impossible problem.

- Chad

Share this post


Link to post
Share on other sites

this is very interesting. I would like to hear from artists what they would want to use this for? what kinds of projects would benefit from painting voxels? would you bake the voxel painting to a lower poly mesh?

personally I intend to build a retopologised mesh over my voxel model and then bake normal and AO from it. Then my mesh will probably be less than 10 000 polys and possibly have 3 2048x2048 maps. At this stage I would like to use 3D coat to paint my model with the diffuse colour or specular or fix seams. Maybe most computers memory could handle 3 additional channels at 16bit if you are just painting a 10000 poly mesh with 3 maps? just wondering.

Share this post


Link to post
Share on other sites
what kinds of projects would benefit from painting voxels? would you bake the voxel painting to a lower poly mesh?

Technicaly, every project would benefit from voxel painting. It is next step in "artistic free" computer modeling. With voxel modeling, there is no need to bother with good topology, polygon stretching, polyloops, quads etc. But when you want to paint texture, all these old annoying problems are here again.

Of course, its not problem for game creation pipeline, bacause there still has to be a step where you transform your model into (few) polygons (even Larrabee can't render voxels, it still renders triangles) and it is not important whether it will be before painting texture or after that.

But now, evereyone in pipeline (maybe except of concept artist) has to understand the topology etc. In voxel pipeline you have to have only one person (or few) in company who can create good meshes and others can be just artists.

But as I said, I think that hardware nowadays isn't prepare for full voxel painting. Yes, some of us have dual-quad WS with 16GB of RAM a two or more Quadros (or Tesla at its best) and these people can take real advantage of fluent a ultra-detailed voxel painting.

But how many users of 3DC have HW like this? How many would have similar HW in 2009? And is it realy advantage for all the others?

So if I could vote in a poll, I would vote for voxel modeling and direct texture painting (thats what Im waiting for before buying 3dc) in 3dc 3.0 somewhere in Q1 or Q2 2009 and full voxel painting in some upcoming release in 2010 or 2011 when majority of 3dc users will have HW for that.

Share this post


Link to post
Share on other sites
this is very interesting. I would like to hear from artists what they would want to use this for? what kinds of projects would benefit from painting voxels? would you bake the voxel painting to a lower poly mesh?

Two things come to mind right away.

1) You could use real 3D maps. Either explicit OR procedural data could be stored in the voxels and deformed with the voxels. So you could have a 3D skin texture that gets deformed correctly as you sculpt. It wouldn't have to be RGB data, either.

2) You could extrapolate 3D maps from 2D maps. The imported mesh could transfer it's textures, either from UV maps or from per TV colors to a voxel array. That way you could modify an already textured model and preserve the original colors and UV's.

The reason you feel like you are sculpting in wax, clay, bronze, etc is a limitation, not a design intent. If you could model a human face and feel like you were working with a real live ACTOR and not a plaster MAQUETTE, wouldn't that be better?

- Chad

Share this post


Link to post
Share on other sites

What does sound of use as Frankie pointed out, is using it for masking and such. Very wicked tools that would not require a lot of RAM to implement.

But for actual texturing, its of little use -namely because the artist will want to use a 2d paint program in juxtaposition, and also, whatever solution you choose, if the artifacting and filtering degrades the texturing process - it is of little value.

That is one thing that really shines with projection painting in Mudbox....there is very little quality loss.

Share this post


Link to post
Share on other sites
What does sound of use as Frankie pointed out, is using it for masking and such. Very wicked tools that would not require a lot of RAM to implement.

The way Claytools does masking, it's basically an on-or-off thing for masking. If you could implement segmentation of the voxels based on a scalar value, then you could assign properties to the voxels.

So you could have a segment of "skin" and a segment of "bone". The bone voxels would be very stiff, while the skin ones would be stretchy and flexible. So you could push the skin into the bone, but the bone wouldn't move. Or you could push the top of the bone with enough force, and it would move the ENTIRE bone, not just the point where you touched it. With an 8 bit scalar, you could define 256 unique "materials" for your object.

And when you converted the voxels to an isosurface mesh, the texture vertices could have the nearest segmentation scalar assigned to them in a segmentation channel so you would get automatic masking, so you would have a mask for "hair", "skin", "eyeball", "shirt", "scar tissue", "teeth", etc. Would make painting textures much easier then.

Also, you could choose to extract a mesh of JUST the "shirt" channel or whatever, so you could get multiple meshes out of your main voxel object.

- Chad

Share this post


Link to post
Share on other sites
The way Claytools does masking, it's basically an on-or-off thing for masking. If you could implement segmentation of the voxels based on a scalar value, then you could assign properties to the voxels.

Actually, Claytools uses a 8 bit scalar for masking, and that allows for smooth grandient masking and transforming.

Data painting as you suggest might just be yet another step further. Much like Maya's blind data editor, where you can paint any type of data (boolean, scalar, vector, even strings) at vertex/face level. This would be usefull in combination with a scripting language associated with the brushes (custom brushes), or at the shader level (for special rendering based on those data, skinning, procedural stuff or whatever). Those should also be transferable (baked) to maps or verts (of low poly mesh).

However I agree with previous posts that we need UV painting first for V3. Then I hope Andrew will have a go on voxel masking/painting and maybe at some point generalised data painting.

Franck.

Share this post


Link to post
Share on other sites

Out of curiosity, since there is now a voxel-surface workflow, does this open the door (a little bit maybe) for some kind of vertex painting?

The vertex color can be multiplied with the current shader for the output that can be baked.

Also, VC could be used for freezing states too.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now