The VoiceBot: A Voice Controlled Robot Arm

Brandi House, Jonathan Malkin, Jeff Bilmes
Department of Electrical Engineering, University of Washington

Commented on:
Patrick Clay
Richard Russell
Sarah Gray

Problem:
Individuals with motor impairments such as paraplegia and spinal cord injuries may have difficulties with daily activities. Assistive technology exists to facilitate their actions and increase their abilities. One of the problems facing this technology is the control options of the technology. A solution that was developed was the spoken language and automatic speech recognition (ASR) system. However, it does not provide adequately for tasks that require smooth continuous control.

Solution:
By using the Vocal Joystick (VJ), a system was developed to use non-speech voice recognition to control a robot arm. Two experiments were carried out. First experiment was to manipulate a robot arm in 2-dimensions while the second experiment was to manipulate it in 3-D.
Three control models were used for the experiements:

- Forward Kinematic (FK) Model
- Inverse-Kinematic (IK) Model
- Hybrid Model

The forward kinematic model requires the use to control each joint angle explicitly. Advantage is that it is light on computation, but it requires more effort from the user. In the inverse kinematic model, the arm is used to position the end effector in the appropriate location. This model requires more computation but less work for the user. The hybrid model is a combination of the FK and the IK. In this model, the first two joints are controlled using IK and the last one is controlled directly.







Source: http://delivery.acm.org/10.1145/1520000/1518731/p183-house.pdf?key1=1518731&key2=8613211421&coll=ACM&dl=ACM&CFID=29176003&CFTOKEN=94010944

Experiment 1: Simulate Arm User Study
Objective: Feasibility and to support their hypothesis that the hybrid model would be the best choice.


Source: http://delivery.acm.org/10.1145/1520000/1518731/p183-house.pdf?key1=1518731&key2=8613211421&coll=ACM&dl=ACM&CFID=29176003&CFTOKEN=94010944


5 users were invited to test their system. The test had the users control a robot arm in 2D to move a ball along the ground to four locations. Users were given time to practice for as long as they want before the time trial was conducted. Four of the users had previous experience with VJ.

The results:

Source: http://delivery.acm.org/10.1145/1520000/1518731/p183-house.pdf?key1=1518731&key2=8613211421&coll=ACM&dl=ACM&CFID=29176003&CFTOKEN=94010944

Users had the most difficulty control the pitch. Overall, users generally preferred the inverse kinematic model, as it was the fastest in the trial for 3 users out of 5. The forward kinematic model was second, and the least favorite was the hybrid model. These results indicate that users would like prefer not to have to explicitly control each joint angle. With more practice though, users may be able to adapt to the forward kinematic model.

Experiment 2: Robotic Arm Control: The VoiceBot
The VoiceBot is a hobbyist robot that they converted to be controlled by the Vocal Joystick. The VoiceBot is controlled using two modes: position mode and orientation mode. The user can switch between these modes by using a “ck” sound.
In position mode, the arm moves according to three algorithms

- Forward Kinematics (FK)
- Inverse Kinematics - Cartesian
- Inverse Kinematics – Cylindrical


Source: http://delivery.acm.org/10.1145/1520000/1518731/p183-house.pdf?key1=1518731&key2=8613211421&coll=ACM&dl=ACM&CFID=29176003&CFTOKEN=94010944

In orientation mode, finer controls are allowed to manipulate the gripper.
In this experiment, 12 users were invited and their task was to use the robot arm and place 2 candies into a target spot. Each user was given two practice sessions and two time trails, each to complete the task with a different control method each. A short interview was conducted at the end.

Results:

Source: http://delivery.acm.org/10.1145/1520000/1518731/p183-house.pdf?key1=1518731&key2=8613211421&coll=ACM&dl=ACM&CFID=29176003&CFTOKEN=94010944


All users were able to complete the task given with both control methods. 75% of the users preferred the Inverse Kinematic controls as opposed to forward kinematics. Many users felt that IK was more intuitive, although it felt slower than FK and produced jerkier movements. One problem the users described was the discrete sound detection. It induced frustration for a few users as there was a large amount of false detections. Another problem was that the pitch control was a significant challenge for many users. Future attempts could try and refine the control methods and improve the sound detection.

Discussion/Conclusion:

The experiments proved the feasibility of this technology and future studies could benefit from better control methods and better equipment. The results presented are the first instance of a non-verbal voice-controlled robotic arm. Future research into this area can yield a greater system that can help individuals with motor impairments.