Skip to main content

CakeRobot - A Gesture Driven Robot That Follows Your Hand Movements Using Arduino, C# and Kinect

Over the last few weekends I’ve spend some time building a simple robot that can be controlled using Kinect. You can see it in action below.
Ever since I read this Cisco paper that mentions Internet Of Things will crate a whooping 14.4 Trillion $ at stake, I revamped my interests in hobby electronics and started hacking with DIY boards like Arduino and Raspberry Pi. That turned out to be fun, and ended up with the robot. This post provides you the general steps, and the github code may help you build your own.
Even if you don’t have a Kinect for the controller, you can easily put together a controller using your phone (Windows Phone/Android/iOS) as we are using blue tooth to communicate with the controller and the robot.

Now, here is a quick start guide to build your own. We’ve an app running in the laptop that is communicating to the Robot via blue tooth in this case that pumps the commands based on input from Kinect, you could easily build a phone UI as well.
And if you already got the idea, here is the code – You may read further to build the hardware part.

1 – Build familiarity

You need to build some familiarity with Arduino and/or Netduino – In this example I’ll be using Arduino.


Explore Arduino. The best way is to
Mainly you need to understand the Pins in the Arduino board. You can write simple programs with the Arduino IDE (try the Blink sample to blink an LED in IDE File->Samples). PFB the pins description, from SparkFun website.
    • GND (3): Short for ‘Ground’. There are several GND pins on the Arduino, any of which can be used to ground your circuit.
    • 5V (4) & 3.3V (5): The 5V pin supplies 5 volts of power, and the 3.3V pin supplies 3.3 volts of power. Most of the simple components used with the Arduino run happily off of 5 or 3.3 volts. If you’re not sure, take a look at Spark Fun’s datasheet tutorial then look up the datasheet for your part.
    • Analog (6): The area of pins under the ‘Analog In’ label (A0 through A5 on the UNO) are Analog In pins. These pins can read the signal from an analog sensor (like a temperature sensor) and convert it into a digital value that we can read.
    • Digital (7): Across from the analog pins are the digital pins (0 through 13 on the UNO). These pins can be used for both digital input (like telling if a button is pushed) and digital output (like powering an LED).
    • PWM (8): You may have noticed the tilde (~) next to some of the digital pins (3, 5, 6, 9, 10, and 11 on the UNO). These pins act as normal digital pins, but can also be used for something called Pulse-Width Modulation (PWM). We have a tutorial on PWM, but for now, think of these pins as being able to simulate analog output (like fading an LED in and out).
    • AREF (9): Stands for Analog Reference. Most of the time you can leave this pin alone. It is sometimes used to set an external reference voltage (between 0 and 5 Volts) as the upper limit for the analog input pins.



2 – Get The Components

Again, you could find an online store to buy these components. Also, you could try the nearby local electronic store and buy some bread boards, jumper wires (get some mail to mail & female to female wires) etc as well. Here is the list of components you need to build CakeRobot.
  • A Chassis – I used Dagu Magician Chassis  – From Spark Fun, Rhydolabz – it comes with two stepper motors that can be controlled by our driver board.
  • A Arduino board with Motor Driver – I used Dagu Mini Motor Driver – bought from Rhydolabz in India. For other countries you need to search and find. Some description about the board can be found here – It also has a special slot to plug in dagu blue tooth shield. You could also use Micro Magician
    • You also need a Micro USB cable to connect your PC to the motor driver to upload code.
  • A Bluetooth Shield – Get the Dagu blue tooth module if you can find it. I’ve purchased a Class 2 Rn 42 blue tooth shield From Rhydolabz
  • Few mini modular bread boards
  • Jumper wires – Get a mixed pack with M/M, F/F, M/F – like this one
  • A Tiny blue tooth dongle for your PC/Laptop, like this one, to communicate with the blue tooth shield in the robot (if you don’t have built in blue tooth)
  • Few sensors if you want to have more fun – I had an ultra sonic distance sensor to avoid collisions in my final version. A better alternative is Ping sensor.
  • A battery pack and battery holder that should supply around 6V to the Mini Motor Driver
  • Other components for your later creativity/exploration
    • LEDs
    • Resistors
    • More Sensors
  • Tools
    • Few star screw drivers
    • Duct tapes/rubber bands (yea, we are prototyping so no soldering as of now)
And I found these reads pretty useful

3 – Programming the components

You need to spend some hours figuring out how to program each of the components.
  • To start with, play with Arduino a bit, connecting LEDs, switches etc. Then, understand a bit about programming the Digital and Analog pins. Play with the examples
  • Try programming the ultrasonic sensor if you’ve one using your Arduino, using serial sockets. If you are using Ping sensor, check out this
  • Try programming the blue tooth module (Code I used for the distance sensor and blue tooth module are in my examples below, but it’ll be cool if you can figure things out yourself).

4 – Put the components together

Assemble the Dagu Magician Chassis, and place/screw/mount the mini motor driver and Bluetooth module on top of the same. Connect the components using jumper wires/plugin as required. A high level schematic below.

Here is a low resolution snap of mine, from top.

5 – Coding the Arduino Mini Driver

You can explore the full code in the Github repo  - How ever, here are few pointers. According to the Dagu Arduino Mini driver spec, the following digital pins can be used to control the motors
  • D9 is left motor speed
  • D7 is left motor direction
  • D10 is right motor speed
  • D8 is right motor direction

To make a motor move, first we need to set the direction by doing a digitalWrite of HIGH or LOW (for Forward/Reverse) to the direction pin. Next set the motor speed by doing an analogWrite of 0~255 to the speed pin. 0 is stopped and 255 is full throttle.
In the Arduino code, we are initiating communication via blue tooth, to accept commands as strings. For example, speedl 100 will set the left motor speed to 100, and speedr 100 will set the right motor speed to 100. Relevant code below.
        //Setting up the communication with Bluetooth shield over serial 

        Serial.begin(115200);  // rn42 bt


      //Read the input In getSerialLine (shortened for brevity)

      while(serialIn != '\n')
  if (!(Serial.available() > 0)) 

  serialIn =;
  if (serialIn!='\n') {
   char a = char(serialIn);
   strReceived += a;


        //Process the command (shortened for brevity)
 else if (command=="speedl")
  val=getValue(input,' ',1).toInt();
 else if (command=="speedr")
  val=getValue(input,' ',1).toInt();


Have a look at the full code of the quick Arduino client here. Then, compile and upload the code to your mini driver board.

6 – Coding the Controller & Kinect

Essentially, what we are doing is just tracking the Skeletal frame, and calculating the distance of your hand from your hip to provide the direction and speed for the motors. Skeletal tracking details here
We are leveraging for identifying the Blue tooth shield to send the commands. Please ensure your blue tooth shield is paired with your PC/Laptop/Phone – you can normally do that by clicking the blue tooth icon in system tray in Windows, and clicking Add Device.
       //For each 600 ms, send a new command
       //_btCon is our instance variable for a blue tooth connection, built over the cool 32Feet library
       internal void ProcessCommand(Skeleton skeleton)

            var now = DateTime.Now;
            if (now.Subtract(_prevTime).TotalMilliseconds < 600)

            _prevTime = DateTime.Now;

            Joint handRight = skeleton.Joints[JointType.HandRight];
            Joint handLeft = skeleton.Joints[JointType.HandLeft];
            Joint shoulderRight = skeleton.Joints[JointType.ShoulderRight];
            Joint shoulderLeft = skeleton.Joints[JointType.ShoulderLeft];
            Joint hipLeft = skeleton.Joints[JointType.HipLeft];
            Joint hipRight = skeleton.Joints[JointType.HipRight];
            Joint kneeLeft = skeleton.Joints[JointType.KneeLeft];

            if (handRight.Position.Y < hipRight.Position.Y)
                _btCon.SetSpeed(Motor.Left, 0);

            if (handLeft.Position.Y < hipLeft.Position.Y)
                _btCon.SetSpeed(Motor.Right, 0);

            if (handRight.Position.Y > hipRight.Position.Y)
                var speed = (handRight.Position.Y - hipRight.Position.Y) * 200;
                if (speed > 230) speed = 230;
                _btCon.SetSpeed(Motor.Left, (int)speed);

            if (handLeft.Position.Y > hipLeft.Position.Y)
                var speed = (handLeft.Position.Y - hipLeft.Position.Y) * 200;
                if (speed > 230) speed = 230;
                _btCon.SetSpeed(Motor.Right, (int)speed);

And so, it sets the speed based on your hand movements. Explore the ConnectionHelper and BluetoothConnector classes I wrote.


The code is here in Github. Fork it and play with it, and expand it.

Popular posts from this blog

MVVM - Binding Multiple Radio Buttons To a single Enum Property in WPF

I had a property in my View Model, of an Enum type, and wanted to bind multiple radio buttons to this.

Firstly, I wrote a simple Enum to Bool converter, like this.

public class EnumToBoolConverter : IValueConverter { #region IValueConverter Members public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture) { if (parameter.Equals(value)) return true; else return false; } public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture) { return parameter; } #endregion }

And my enumeration is like

public enum CompanyTypes { Type1Comp, Type2Comp, Type3Comp } Now, in my XAML, I provided the enumeration as the ConverterParameter, of the Converter we wrote earlier, like

Creating a quick Todo listing app on Windows using IIS7, Node.js and Mongodb

As I mentioned in my last post, more and more organizations are leaning towards Web Oriented Architecture (WOA) which are highly scalable. If you were exploring cool, scalable options to build highly performing web applications, you know what Node.js is for.After following the recent post from Scott Hanselman, I was up and running quickly with Node.js. In this post, I’ll explain step by step how I’ve setup Node.js and Mongodb to create a simple Todo listing application.Setting up Node.jsThis is what I’ve done.1 – Goto, scroll down and download node.exe for Windows, and place it in your c:\node folder2 – Goto IIS Node project in Git at, download the correct ‘retail’ link of IIS Node zip file (I downloaded the already built retail package, otherwise you can download and build from the source).3 – Extract the zip file some where, and run the install.bat or install_iisexpress.bat depending on your IIS Version. If you don’t have IIS in…

Top 7 Coding Standards & Guideline Documents For C#/.NET Developers

Some time back, I collated a list of 7 Must Read, Free EBooks for .NET Developers, and a lot of people found it useful. So, I thought about putting together a list of Coding Standard guidelines/checklists for .NET /C# developers as well.As you may already know, it is easy to come up with a document - the key is in implementing these standards in your organization, through methods like internal trainings, Peer Reviews, Check in policies, Automated code review tools etc. You can have a look at FxCop and/or StyleCop for automating the review process to some extent, and can customize the rules based on your requirements.Anyway, here is a list of some good Coding Standard Documents. They are useful not just from a review perspective - going through these documents can definitely help you and me to iron out few hidden glitches we might have in the programming portion of our brain. So, here we go, the listing is not in any specific order.1 – IDesign C# Coding StandardsIDesign C# coding stand…