IT Security Lab The Playground for IT Security Specialists and Pentesters

27Dec/1018

Having Fun with Kinect and Delphi (examples of 2D and 3D visualization)

Well well well... All signs in the Sky and on the Earth clearly say: it's a Christmas time! So it's time to have some rest and fun, and definitely nothing can be better then to spend some time with family and a new toy. ;-) Actually it's my son who got the XBox with the Kinect, but well... let parents have some fun too, right!

It's not like I am trying to reinvent the wheel (there are plenty of applications using Kinect on PC), but recently I did not find any nice examples of how this incredibly cool thing can be used with Delphi. And you don't think I can leave it "just like that", don't you? ;-) So see the results below (video) and so more technical details of both applications (2D and 3D visualization). So as for today there will be no hacking, boys and girls, but just pure awesome 3D-virtual-reality joy... :-)

Ok, so you want to try to get those samples tweak them probably and run your own code? There is nothing more simple:

Prerequisits:

Everything is installing like a charm. Some subtle obstacles with GLScene, but nothing too complicated to be mentioned really. Important: Before you run anything - be sure that the Kinect device is recognizable by your computer (check it in you Device Management panel).

Ach, almost forgotten: take the source files of my applications from here:

Running all that stuff

Now you can try to compile and test both applications. More details about how those application can be used you have already seeing in the movie. Pre-compiled exe files are already included into ZIP packages, just for your convenience. So at the end this is what we have:

2D data visualization

The experiment showing how to collect, process and draw the Kinect's data on the screen. Rather typical,- you've seen it before for sure. Additional challenge was to write a function which would be able to "track  blobs" - areas on the screen with similar pixels. This is needed to track your hands, fingers, nose or whatever you want to use. It is far far from ideal, but surprisingly works!

One more remark: this DOF function, selecting pixels in certain 3D range - is a part of the application, not the Kinect hardware.

I also hope you will forgive me such eeee... "untypical" way of getting depth data from pixel's color by such innocent transformation: RGB -> HLS -> [custom function] -> range [0..255]

3D data visualization

You can see my room (and actually yours too) in 3D in wobbling 3D virtual screen, containing tiny colorful dots :-) Do you like my Xmass tree?

So you see, Delphi is so nice and (important!) easy language (appropriate for lazy coders), so even writing pretty complex applications can take you just couple of hours. I also hope now even more people will start playing with Kinect and do some cool things, [so more happiness will come on Earth this Christmas, etc, etc.]. Remember: You are the controller®. Amen. :-)

Special thanks for Simon J Stuart for his TKinect Delphi component
and for Jet Noir (http://soundcloud.com/jet-noir) for her music for the video!

This is it. Let me know it you like those crazy apps, and well... Have a nice Christmas and a happy New Year! :-)

Comments (18) Trackbacks (7)
  1. Thank you so much for showing what can be done with my TKinect for Delphi component. I’m glad you are making great use of it :)

    I’ve just tweeted the link to this page so others can see your brilliant demo.

    Just wondering: did you suffer the juddering images on the RGB and Depth feeds when using a standard TBitmap to render the image?

    Would love to take a peek under the cover of your test app, see if anything can be integrated into the component directly (If you’re okay with that, I mean)

    • Hi Simon,

      As you see, it was too tempting not to try to write some piece of Delphi code and play with Kinect! :-)

      Answering your question: I abandoned the idea of using standard graphic components from the very beginning, as they are too slow, by definition. The ones I used (graphic32 and GLScene) are super cool, and IMHO there is no need for anything else. Those libraries are exceptionally well written, easy to use and well documented (especially graphic32, but GLScene has tons of examples). In one sentence: HIGHLY RECOMMENDED. If you are asking about flicker – just use graphic32 and everything will be ok, I promise!
      Please use my sample projects at your will and also feel free to modify it the way you want. I do apologize, they are pretty messy, but this is “a playground” rather than a ready app.
      Let me know if you would be interested in writing something cool together. :-)

      With kind regards,
      Aleks

      • Aleks,
        I have been using GLScene for many years… I’ve even produced a simple scripted game engine using it as the rendering engine (my Lua4Delphi suite formed the scripting engine)… it is brilliant but for one downside: doesn’t provide OGL 2+… still brilliant for the kind of demos you produced :)

        I’ve credited you on the TKinect official page on my blog (http://www.lakraven.com/delphi-stuff/tkinect/) and even used one of your screen captures as the feature image (I particularly like your Depth display with the Kinect icon… very classy)

        I think there are things in your demo which could be integrated into the TKinect component’s source directly… I’m going to go through the sources shortly and see what. Obviously the next update to TKinect will include a credit to yourself and a link to your blog.

        Your demos are fantastic… I’ve tested them, though for reasons I don’t quite understand I still have issues with the camera feeds on my setup. Basically the RGB and Depth footage flicker terribly, and I even get ghosting. These aren’t issues with TKinect or your demos, I’m told, but rather some USB hosts don’t particularly like the Kinect controller. I’m going to test it out on my other systems to see if ANY of them play nicely.

        I’ve added you to my follow list on Twitter and subscribed to your blog’s RSS feed :)

        As regards working on a joint project, definitely interested though it’s worth mentioning that right now I’ve got SO MANY projects on the go… if I take on any more I doubt anything will ever get finished:
        - Lua4Delphi 2.0
        - AGE (Another Game Engine)
        - ADL SDK translation (ATi Drivers’ SDK translation for Delphi)
        - TTwitter (and a whole bunch of other social networking components, along with the “Social Delphi” website)
        - Lnka.me (my URL Shortening service)
        - SAPI 5.3 for Delphi (a complete translation of the Microsoft SAPI for Vista/7 and a set of expanding components on top of that)

        As you can see… a lot of projects ;)

  2. I’m PS3 fan, but I really like Kinect.

    Great job and thanks for this amazing demo.

    Andreano Lanusse

  3. Hope you don’t mind I blogged about this great post on my site. http://blog.marcocantu.com/blog/kinect_delphi_aleks.html

  4. Wow I am a designer and sadly still in the proces of learning programming ,but I would love to see what the future brings with these new toys.
    Tell me is it possible to scan an entire room in 3d with fairly accurate dimensions and export the data to 3ds max or autocad?

    Great job
    new years greetings
    Peter

    • Hi Peter,

      Well, it is possible to scan entire room for sure, but I would have serious concerns regarding the resolution and a quality of such scan. ;-) Honestly – I don’t think Kinect can be used this way, because… it’s rather still a cool toy then a professional lidar or any sort of (http://en.wikipedia.org/wiki/LIDAR). In my opinion – it does what it is designed for: low-resolution human body movement detection (in best option: detection of hand and fingers).

      With kind regards,
      A.

  5. If you should use Delphi XE there is an issue with graphics 32.
    Graphics 32 won’t compile cause of some overloadede hypot and sincos procedures which are now contained in the standard Math-Unit with the similar signature Single vs. TFloat.
    You can probably “solve” the problem by cmmenting out the GR32_MATH version (but I do not know yet if this is causing any negative side effects).

    • Wow! That’s an interesting news. Honestly, I am using pretty old version of Delphi (Turbo Delphi actually), so everything works fine there. As soon as there will be free version of Delphi XE – I would be happy to test it out for sure. :)

  6. Hm … no … I will not start to argue about this :-)

    • Dear Roland – no worries. Actually the framework for accessing the Kinect data (Simon J Stuart and I – we are currently working on it) will not be using Graphics32 at all. Well, optionally you might be able to use is if you want, but definitely you will not be “forced” to. :-)

  7. WOW!!!DUDE you are amazing, just got my kinect and seen that people were making use of it to do some cool things on a pc! Now i know i can put my delphi skills to some use i am deffo gunna be putting some time into this!!! im feeling i need to make some soft to be able to do different controls for emulators. Really just wanna play Duke Nukem 3D as the controller lol hard task maybe but ill give it a go

  8. Hello Aleks,
    will the framework support motion detection (something like a gesture engine for kinect)?
    What would be the preferred algorithm for something like that (Neural Networks, State machines)?
    I wonder how to best implement something like that.
    Thanks a lot for your demos once more.

    Best regards

    Roland

    • As for now – there is no framework in use (as all which exists already should be re-implemented for Delphi and/or wrapped-up). So if you look for anything but Delphi – there are tons of gesture recognition frameworks around. I also suggest you to make yourself familiar with stuff like PCL (Point Cloud Library) and all related.

      Actually the Delphi unit for Kinect will be rewritten as soon as a new official SDK will be available from Micro$oft. Stay tuned and look here and at Simon’s page: http://www.lakraven.com/

  9. http://www.openni.org/documentation

    … looks also interesting considering this.

  10. Hello Aleks,
    there was an interesting post on slahdot considering “Real-Time Human Pose Recognition in Parts from Single Depth Images” …
    this is how “Microsoft Research Cambridge & Xbox Incubation” is solving the problem.
    Do you know if this algorithms will be part of the Kinect SDK and will TKinect wrap them up?

    Best regards

    Roland

  11. What excellent work – thank you. I had toyed with the idea of trying this but it looked like it was going to take ages to do so I forgot about it – so well done for bringing the Kinect to delphi – I can think of loads of fun uses for it!

    So now I get all the fun and none of the development pain. I already use glscene and graphics32 so I can just start playing!


Leave a comment


*