Jump to content
3DCoat Forums

Leap Motion - The future .....


Recommended Posts

  • Advanced Member

Hi guys,

It's been a while. :)

With launching a new business this year and a one year old son, I have been a little absent from the 3D Coat forums.

I thought I would drop by though and start doing a bit more 3d sculpting again.

I also found out about this: http://live.leapmotion.com/about.html

They created it for 3D modellers and artists :

"Who is LEAP? How did you get started?

The original inspiration behind Leap came from our frustration with 3D modeling— something that took 10 seconds in real life would take 30 minutes with a computer. Molding virtual clay with a computer should be as easy as molding clay in the real world. The mouse and keyboard were simply getting in the way."

It looks like a slice of the future for us artists and they have an SDK available for developers. Had any of you heard of it before?

How do you think it would work with 3D Coat?

Link to comment
Share on other sites

  • Advanced Member

I held my arms straight in front of me at a 90 degree angle for 2 hours and then tried to wiggle my fingers....

They fell off. ;)

I must admit I only thought of the positives when I first saw this but actually pretending to model something with your hands without any feedback certainly does feel very odd.

I am still very happy using my wacom tablet, keyboard and mouse. I just thought this looked cool.

Its a bit like the Wii and Kinnect in a way. It's fun to wave your arms around for a bit but it quickly becomes tiring. I still always prefer playing games with a control pad.

Link to comment
Share on other sites

  • 11 months later...
  • Advanced Member

Can anyone say necro? ... :)

So, I just pre-ordered the Leap Motion Controller which is now due to ship on the WC June 22nd. I just couldn't resist it for so many reasons. If I can't use it for anything really productive it will entertain my 2 year old and his new baby brother.

Do you think it would work with 3D Coat right away or would it need some serious programming to get a plug in up and running?

The SDK is out there...

Link to comment
Share on other sites

The future release of Leap -if there isnt any new delay- will include own development integration to windows desktop.

Hard to take opinion -from my pov- about a hardware that today by today -again from my pov- is vaporware

Link to comment
Share on other sites

  • Advanced Member

No it's not vapourware and somebody here is already creating a 3d coat plug in for it.

http://3d-coat.com/forum/index.php?showtopic=11150

Photonvfx says he's already developing it.

sshhhh! This is hush hush! ;)

Don't tell anyone, but I am already currently developing a prototype of this with Andrew's assistance and oversight (my company just got our Leap Motion developer device yesterday). We will let you know when/if there is something good to show. Don't clamor too hard - the device is not publicly available yet (Leap is still working the kinks out of it with developers like us, there are connections issues, driver issues in general still going on with it), but I can tell you 3D Coat interoperability is a possibility and likely is in the cards...
:)

its' going to be awesome.

Can't wait

Edited by L'Ancien Regime
Link to comment
Share on other sites

  • Advanced Member

It sounds as if they could have actually released the hardware a while ago. If it truly was delayed for the reasons given, then it surely could have been sent out right now given the other concerns are all things that could have been dealt with after the fact (e.g. taken from their forum: "building our developer program, Airspace, OS interaction, etc"). I highly doubt the pre-order folks would mind finally having the finished/polished hardware right now while downloading more stable software down the road. In a way it's a bit like the boy who cried wolf and the kind of thing that can make one start to doubt their sincerity, dangerous considering this particular product is a want and not a need for most people. Hopefully they'll get it right come June.

It's a lot smaller than I thought it was, after looking at some images of it, and could actually be built into a Wacom tablet I think. In fact I could see myself attaching it to the top of my tablet with stick-on velcro strips. What I see in my minds eye is holding the tablet pen in my right hand and sculpting/texturing with it at the same time as I use my left hand to manipulate the mesh (pan, zoom, rotate), kind of like holding one of those plastic models in one hand while painting it using the other. Can't really see myself using it to actually model though.

Link to comment
Share on other sites

  • Advanced Member

Can't really say too much because of the dev agreement. But I'm having a bit of trouble with the hardware I have. The last two SDK updates have improved things, but hopefully the final hardware will be much better. Though there are still software issues to work out in the drivers and APIs. It is an amazing device, but I'm having trouble making bulletproof things for the masses. And I do see big things for it (they have deals with Asus & HP - so expect to see it built into laptops in the future), but the delay until July-ish (was supposed to be out in May - though I've had this dev device since Jan) and my experience say it's not ready for prime time yet- I.e. granny or a five yr old won't find it intuitive yet. I do need to spend more time on working on the plugin (as opposed to work that pays the bills) but was waiting for improvements in the SDK to make things easier for me...

Link to comment
Share on other sites

  • Advanced Member

Btw, I see sculpting in air as a toy and not the real use case of it. That's a cool tech demo, but probably not practical. A lot of devs in the leap forums are forgetting UX stuff and things like arm fatigue. Might be fine for 5 minutes, but you don't want to work 8 hours like Tom Cruise in Minority Report. I'm aiming for more subtle smaller gestures like finger flicks and catches to make it work in conjunction with a wacom tablet or mouse. This magical holodeck sculpting idea is not practical in real use (maybe as a toy, but not for pro work). Physical feedback is important, and most people don't have the body awareness to know exactly what they are pointing at.

I've had 20 people try to calibrate it by pointing chopsticks at dots on their screen and almost all of them were way off from what they were really pointing at and were different among themselves on subsequent attempts. Believe it or not only dancers or people who had extreme body awareness (like their arm is where they think it is in space) did well at doing tasks like this. All this means making a single interface for all people with complex gestures very tricky - and your app has to compensate for what people *think* they are doing. It is not as simple as writing an app and hooking it up. This is a whole new interaction paradigm.

Edited by photonvfx
  • Like 2
Link to comment
Share on other sites

I've had my eye on this for a long time and do plan to get one when they come out. One reason is that the guys at ReconstructMe are trying to make it into a high detail 3D scanner. I think this would be really awesome on top of it's intended uses.

@Zeddicus yes it really is about the size of a cigarette lighter. They've talked about how they want to have it built into laptops.

Link to comment
Share on other sites

  • Contributor

Wow! This device looks really promising. Some means to rotate the model while painting at the same time and for a cheap price is something I was always looking forward to. I think I've also read that it has pressure detection? If it's true then I'm wondering how does it work?

I will be purchasing this for sure, but only after it hits the market and I'm able to read some reviews and opinions about it. I've never been a fan of pre-ordering. I think it's baaaad. :)

Edited by ajz3d
Link to comment
Share on other sites

  • Advanced Member

phil - 3d Scanning is at least a year away for this device - unless some magic occurs between then and now. The Leap does NOT currently give access to point cloud data & there is no RGB camera in the hardware so texture scanning is not possible. Leap themselves are unwilling to pursue that at present and open up the sdk for that because they want to perfect the hand interaction thing for everyone first. They realize 3D scanning is a possibility down the line, but my guess from where their devs are positioning things is that they will do that with a different device later after the leap motion is out for awhile. Don't bet on this tiny device being a scanner. It has amazing precision though and very very fast responsiveness (I can move my hands super fast like mental and it doesn't stop tracking the motion and keeps the precision to the mm) but there's no texture RGB camera so that limits scanning. Your best bet for a low cost scanning solution is for someone (which may have happened in the last month but I don't know of one yet) who have taken the last Microsoft SDK for the Kinect with its Kinect Fusion scanning and made that into a product (as opposed to just the current tech demos that are out there now). Though I suspect Kinect2 will hit before Christmas and may have even better functionality than the Kinect 1. For scanning now though Leap can't do it - unless these guys have reverse engineered something..?

Ajz3d - you're going to have to change that avatar back to your old one, this one is freaking me out too much ;) - but don't know what you mean about pressure detection? It is just a motion detection using infra red depth cameras - no pressure or touching or physical feedback. That being said, for hands it will be a great device. My dev device works really nice and the latest sdks have had nice gesture detection built in so pokes swipes and circles etc are now all built in to the driver. It's good hardware - the software needs work which is why the delay. They also want to release it with a rock solid OS integration which is what they're working on. I also wouldnt worry too much on preorder with these guys. (it won't be like the Lightwave preorder where it was like 2+ years or something before they finally released lol) Their dev guys are really good and their customer service I've dealt with has been impeccable, they are really trying hard to please developers, and are doing well launching internationally even (with many int'l devs treated just as good as the ones stateside) and the hardware I've received (though older than what they currently are going to release) seems well built and worth the money when it will be released. The big thing whether this takes off enough, is if there is enough good software and useful stuff (that is not just gimmicky) that works. and if any one can just rock up to a device and "get it".

That's still a tricky thing, as calibration issues and just plain interaction design will be the hard part. Some of the tech demos and games I've seen seem useless after the first 5 minutes of fun with them, but I have seen javascript addons for websites which enable it to work on websites which has been surprisingly useful. Hard to see where this will all go, but they've got a good shot at revolutionizing things...

Link to comment
Share on other sites

  • Advanced Member

It also has a few issues if your desk is under bright lights - so if you have a ceiling light right above you, it wont work as well. Plus it is quite small and tiny (and wired - they do plan to make a wireless one down the line, but it's too power hungry to spit all that data all those frames per second). The wired aspect means you tend to move it a lot accidentally and then may need to recalibrate before using it again next time - which is a pain. Hopefully this will be sorted by launch. It may turn out to work better once built into Asus & HP laptops eventually. It is a nice device though.

Link to comment
Share on other sites

  • Contributor

Ajz3d - you're going to have to change that avatar back to your old one, this one is freaking me out too much ;)

Done. Sorry about that.

but don't know what you mean about pressure detection? It is just a motion detection using infra red depth cameras - no pressure or touching or physical feedback.

Well, they say you can draw and paint with this device and I think this implicates some kind of pressure detection solution. Either by depth detection or something more advanced.
Link to comment
Share on other sites

  • Advanced Member

thanks lol - I was mostly kidding about your avatar ;)
your last one was much less distressing than this one with its eyeballs falling out :rofl:

Yes there is full 3D depth with any hand motions - you can move side to side, up and down, and in (towards your screen) and out with all five fingers with both hands. So painting would be achieved by pushing in towards the screen a certain amount and lifting the brush off the canvas would be achieved by just pulling your fingers back. How it feels is up to how the developer implements it. And whether or not it is truly useful depends on that and on how they give you visual feedback on screen as you paint - since there is nothing in mid air you can feel...

Link to comment
Share on other sites

  • Advanced Member

Wow,

There has been a lot of activity on this old post sence I last checked!

It is good to hear that you are working on a plugin photonvfx.

I can not wait to get my device. I have also pre-ordered a Lenovo LT1423p wireless monitor which supports 10 touch points and has a wacom digitiser built in.

It only has 256 levels of presure (Poor mans Cintiq) but combining this and the Leap motion with my Lenovo y580, may just give me my perfect 3d coat set up.

It also works over USB 3.0 so I should be able to get a lag-free work flow.

I love the tablet form factor and the fact that It will be like working on an awesomely over powered Ipad!

I have become a bit of a pre-order junky over the last few months. It's a cool time to be a future tech art geek with a little bit of money to spend. :)

Edited by Marc Wakefield
  • Like 1
Link to comment
Share on other sites

phil - 3d Scanning is at least a year away for this device - unless some magic occurs between then and now. The Leap does NOT currently give access to point cloud data & there is no RGB camera in the hardware so texture scanning is not possible. Leap themselves are unwilling to pursue that at present and open up the sdk for that because they want to perfect the hand interaction thing for everyone first. They realize 3D scanning is a possibility down the line, but my guess from where their devs are positioning things is that they will do that with a different device later after the leap motion is out for awhile. Don't bet on this tiny device being a scanner. It has amazing precision though and very very fast responsiveness (I can move my hands super fast like mental and it doesn't stop tracking the motion and keeps the precision to the mm) but there's no texture RGB camera so that limits scanning. Your best bet for a low cost scanning solution is for someone (which may have happened in the last month but I don't know of one yet) who have taken the last Microsoft SDK for the Kinect with its Kinect Fusion scanning and made that into a product (as opposed to just the current tech demos that are out there now). Though I suspect Kinect2 will hit before Christmas and may have even better functionality than the Kinect 1. For scanning now though Leap can't do it - unless these guys have reverse engineered something..?

-----

Oh I know it's not going to be available immediately, but since ReconstructMe said they were trying to do it, I thought that would be an awesome bonus.

Link to comment
Share on other sites

  • Advanced Member

-----

Oh I know it's not going to be available immediately, but since ReconstructMe said they were trying to do it, I thought that would be an awesome bonus.

--

it sure would be, but the lack of textures from no RGB camera means even if they could get point cloud data and scanning to work, there'd be no color recorded so all you'd get is white models - and that's too bad, because scanning would be great for faces. Looks like Kinect is the only hope for the short term.

Edited by photonvfx
Link to comment
Share on other sites

  • Advanced Member

I would be very happy with white models if they had high enough resolution.

I wonder about the size of the subject matter though. How big is the scanning range of the Leap?

--

If you mean scanning range for 3d scanning, well that hasn't been done yet so not possible, but if it does happen down the line, it will have to be handheld and waved over your object.

For hand motion interaction, if you placed it in front of your keyboard on a desk, the range is a bit wider than shoulder width and say the distance to your monitor and roughly the same distance back if you lean back in the chair that amount. Up and down I'd say it is from desk height where the device is to say head height. And this is all in a wide cone shape from the device. Not sure if that explanation is clear enough or makes sense though... :)

Link to comment
Share on other sites

  • 2 months later...
  • Contributor

I tried my Leap with 3d coat just a bit ago.  Even without any specialized support, using just the Touchless app, it actually works pretty well.  You can certainly see the potential of full integration.  I hope this comes online soon.  

 

What about proving you can make something worthwhile with it before asking for a full integration ?

Tired arms, no tactile feedback, I don't know a single sculptor/painter which can work with those.

Surely a nice gadget cool using for 30 minutes, but definitely not at good interaction device with 3dcoat for DOING something in it.

I understand you want to use your device, or at least try use cases with it in almost all apps. But this thing is doomed to fail on ergonomics alone.

Link to comment
Share on other sites

  • Advanced Member

Tired arms, no tactile feedback, I don't know a single sculptor/painter which can work with those.

 

The ergonomics are no worse than actual painting, if you think about it.  Holding up a paintbrush while painting onto an easel can also get tiring.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...