Thursday, April 30, 2009

The VoiceBot: A Voice Controlled Robot Arm

Brandi House, Jonathan Malkin, Jeff Bilmes
Department of Electrical Engineering, University of Washington

Commented on:
Patrick Clay
Richard Russell
Sarah Gray

Individuals with motor impairments such as paraplegia and spinal cord injuries may have difficulties with daily activities. Assistive technology exists to facilitate their actions and increase their abilities. One of the problems facing this technology is the control options of the technology. A solution that was developed was the spoken language and automatic speech recognition (ASR) system. However, it does not provide adequately for tasks that require smooth continuous control.

By using the Vocal Joystick (VJ), a system was developed to use non-speech voice recognition to control a robot arm. Two experiments were carried out. First experiment was to manipulate a robot arm in 2-dimensions while the second experiment was to manipulate it in 3-D.
Three control models were used for the experiements:

- Forward Kinematic (FK) Model
- Inverse-Kinematic (IK) Model
- Hybrid Model

The forward kinematic model requires the use to control each joint angle explicitly. Advantage is that it is light on computation, but it requires more effort from the user. In the inverse kinematic model, the arm is used to position the end effector in the appropriate location. This model requires more computation but less work for the user. The hybrid model is a combination of the FK and the IK. In this model, the first two joints are controlled using IK and the last one is controlled directly.


Experiment 1: Simulate Arm User Study
Objective: Feasibility and to support their hypothesis that the hybrid model would be the best choice.


5 users were invited to test their system. The test had the users control a robot arm in 2D to move a ball along the ground to four locations. Users were given time to practice for as long as they want before the time trial was conducted. Four of the users had previous experience with VJ.

The results:

Users had the most difficulty control the pitch. Overall, users generally preferred the inverse kinematic model, as it was the fastest in the trial for 3 users out of 5. The forward kinematic model was second, and the least favorite was the hybrid model. These results indicate that users would like prefer not to have to explicitly control each joint angle. With more practice though, users may be able to adapt to the forward kinematic model.

Experiment 2: Robotic Arm Control: The VoiceBot
The VoiceBot is a hobbyist robot that they converted to be controlled by the Vocal Joystick. The VoiceBot is controlled using two modes: position mode and orientation mode. The user can switch between these modes by using a “ck” sound.
In position mode, the arm moves according to three algorithms

- Forward Kinematics (FK)
- Inverse Kinematics - Cartesian
- Inverse Kinematics – Cylindrical


In orientation mode, finer controls are allowed to manipulate the gripper.
In this experiment, 12 users were invited and their task was to use the robot arm and place 2 candies into a target spot. Each user was given two practice sessions and two time trails, each to complete the task with a different control method each. A short interview was conducted at the end.


All users were able to complete the task given with both control methods. 75% of the users preferred the Inverse Kinematic controls as opposed to forward kinematics. Many users felt that IK was more intuitive, although it felt slower than FK and produced jerkier movements. One problem the users described was the discrete sound detection. It induced frustration for a few users as there was a large amount of false detections. Another problem was that the pitch control was a significant challenge for many users. Future attempts could try and refine the control methods and improve the sound detection.


The experiments proved the feasibility of this technology and future studies could benefit from better control methods and better equipment. The results presented are the first instance of a non-verbal voice-controlled robotic arm. Future research into this area can yield a greater system that can help individuals with motor impairments.


  1. What happened to good old fashioned up down left right? It would take some getting used to but maybe it's time to change. Cat boot feet law!

  2. Welcome to join China Service Mall Adsense Union
    Earn money for your own website

    China Service Mall ,the biggest China Service Reservation Center in the world. Provided 2 0, 00 0,000 registered members with the best reservation service.

    1, Easy to join
    If your website is up to the standard of our Union, we will pass your apply in 24 hours. Then you can join our Adsense Union.
    2, Big payback
    We can give you big payback if you join us, we give more money than the present Adsense Unions.
    3, Easy to earn money
    People come to our site and give a reservation by click the advertisement on your site; we will give you much money. For every 100 effective reservations we will pay you 60 US dollars. When the total reservations reach 500, we will pay you extra 100 US dollars as rewards. More and more preferential policy expects your participation.
    4 ,Convenient management of Backgrounds
    The proceeds of each time is clearly visible in our convenient management of backgrounds, and we can ensure the benefits of yours.
    5, Good reputation
    Anyone of you can be at ease to take a part in it.

    If you are interested in our Adsense Union, please click to register your account.

    Yours Sincerely,
    Miss Guan Yuqing

    Marketing Department ,
    China Service Mall Co., Ltd.

    Address: 9th Floor,
    Wenhao Business Building,
    No.699 Xiongchu Avenue,
    Hongshan District,
    Wuhan City,
    Tel: +86-027-87446102


  3. To me it seems like this violates the rule of design that you want the user to have to memorize as little as possible. Basically you end up having to learn how to speak robot.