During Summer 2014, I had the chance to work on a Kinect v2 “Speech Bubbles” app, built upon a sample project from Kinect MVP Tom Kerkhove. In the initial version, I added a speech bubble gets displayed above a person’s head when the person walks into view.

However, this created a problem for multiple people. It was displaying the same text for every person who walked into view, up to 6 people. So, I decided to update the program to display a new random message every time a new person is detected.

I quickly discovered that this created yet another problem: all the displayed messages would get updated for everyone, even when only 1 new person entered the frame. Finally, I added an array of displayed messages (one for each detected body), so that everyone gets their own randomized text.

Now, it was ready to test in the wild… so I used it at HackUMBC, a hackathon event at the University of Maryland Baltimore College.

•  More Pics from HackUMBC: https://wakeupandcode.com/hackumbc/

Get the Code, Order a Dev Kit

If you have a Kinect v2 dev kit, head on over to GitHub to get the source code.

• Source Code: https://github.com/shahedc/SpeechBubblesForKinect

If you need to order a dev kit, head on over to the Kinect v2 for WIndows website:

• Order Kinect v2: http://www.microsoft.com/en-us/kinectforwindows/develop/

Try It Yourself

If you want to explore the code, try out the following steps.

1. Start with Tom’s codebase: http://www.kinectingforwindows.com/2014/03/03/gen-ii-kinect-basics-overview/
2. Update it with my initial Speech Bubble code, explained here: https://wakeupandcode.com/my-experience-so-far-with-kinect-for-windows-v2/
3. Follow the additional instructions below.

Additional Instructions

Next, use the step-by-step instructions and the screenshots below to update the code. The green areas indicate new additions while the red areas indicate removals/replacements (via Git).

1. Add variables for tracking multiple bodies and the text to display for each.

2. Add a call to InitializeMessages() in your MainWindow() constructor.

3. Create a new method InitializeMessages(), which initializes messages and body tracking id values.

4. In the method OnBodyFrameArrived(), add logic to update the text only when a new body is detected, just before DrawBody(). An additional parameter has been added when calling DrawBody, to pass on the text to display.

5. Update DrawBody() and DrawJoint() to pass an optional parameter for the displayed message.

6. Inside DrawJoint(), use the variable textToDisplay to assign the text to be displayed.

What’s Next?

Here are some things you can do next:

1. Experiment with different speech bubble sizes and font sizes to make the best fit.
2. Replace the single-line text with multi-line text.
3. Use a text delimiter and draw each new line slightly lower than the previous line.
4. Add functionality to email the picture to an email address.
5. Add functionality to tweet the picture with a hashtag or @ mention.

Enjoy! 🙂