Gesture recognition

1
INTRODUCTION
● About Me?
– Besjan Xhika
– My Background
● What this Presentation il About?
– Gesture Recognition Devices
– My Experiments with Leap Motion

2
GESTURE RECOGNITION DEVICES
● BODY
– Kinect (Microsoft)
– Asus Xtion
– Carmine (PrimeSense)
– DS311 (SoftKinetic)
● HANDS
– Leap Motion
– Creative* Interactive Gesture
Camera (Intel)
– Capri (Embedded Solution,
PrimeSense)
– DS325 (SoftKinetic)
– MYO

4
LEAP MOTION
● Detects and Tracks Hands, Fingers and Finger-Like Tools
● Reports Discrete Positions, Gestures and Motion
● The FOV is an Inverted Pyramid Centered on the Device
● The Effective Range Extends From 25 to 600 mm
● Employs a Right-Handed Cartesian Coordinate System
● Provides Updates as a Set or Frame of Data
● Leap Motion Visualizer

5
LEAP MOTION IN A
MEDICAL APPLICATION
● InVesalius is a Medical Software Used to Reconstruct
Structures of the Human Body
● The Leap Device Simulates the Mouse
● Python was Used
● Example Integration of Leap Motion with InVesalius

6
API OVERVIEW
The Applications can use the Leap API to Access the Data of Hands
and Fingers that the Device Captures One Frame at a Time
– Leap.Controller: The interface between the Leap and the application
– Leap.Listener: Used to handle events dispatched by the Leap
– Leap.Frame: Contains a set of hand and finger tracking data
– Leap.Hand: Contains tracking data for a detected hand
– Leap.Finger: Contains tracking data for a detected finger
– Leap.Vector: Represents a 3D position or directional vector
– Leap.Gesture: Represents a recognized gesture.

7
def main():
# Implements callback functions to handle events dispatched by the Leap
listener = Listener()
# The Controller class provides the main interface between the Leap and the application
controller = Leap.Controller()
# Allows the application to run in the background
controller.set_policy_flags(Leap.Controller.POLICY_BACKGROUND_FRAMES)
# Have the sample listener receive events from the controller
controller.add_listener(listener)
# Keep this process running until Enter is pressed
sys.stdin.readline()
# Remove the sample listener when done
controller.remove_listener(listener)
CREATING A CONTROLLER OBJECT

8
class Listener(Leap.Listener):
def on_init(self, controller):
# Dispatched once when the controller to which the listener is registered is initialized
print "Initialized"
def on_connect(self, controller):
# Dispatched when the controller connects to the Leap and is ready to begin sending
frames of motion tracking data
print "Connected"
# Enable gestures
controller.enable_gesture(Leap.Gesture.TYPE_CIRCLE)
def on_disconnect(self, controller):
# Dispatched if the controller disconnects from the Leap
print "Disconnected"
def on_exit(self, controller):
# Dispatched to a listener when it is removed from a controller
print "Exited"
def on_frame(self, controller):
# Dispatched when a new frame of motion tracking data is available
...
SUBCLASSING THE LISTENER CLASS

9
def on_frame(self, controller):
# Get the most recent frame
frame = controller.frame()
if not frame.hands.empty:
# MOUSE CURSOR
firstHand = frame.hands[0]
if not firstHand.fingers.empty:
xPos = 5*firstHand.palm_position.x + self.screenWidth/2
yPos = 1500 - (4*firstHand.palm_position.y - 300 + self.screenHeight/2)
ctypes.windll.user32.SetCursorPos(int(xPos), int(yPos))
GETTING A FRAME OF DATA (Part I)

10
# MOUSE CLICKS
secondHand = frame.hands[1]
if not secondHand.fingers.empty and len(secondHand.fingers) == 1:
for gesture in frame.gestures():
if gesture.type == Leap.Gesture.TYPE_CIRCLE:
circle = Leap.CircleGesture(gesture)
if circle.pointable.direction.angle_to(circle.normal) <= Leap.PI/4: clockwiseness = "clockwise"
else: clockwiseness = "counterclockwise"
if circle.state == Leap.Gesture.STATE_STOP:
# MOUSE CLICK
if clockwiseness == "clockwise":
print "MOUSE CLICK"
ctypes.windll.user32.mouse_event(0x0002, 0, 0, 0, 0)
# MOUSE CLICK + HOLD
else:
if not self.leftclickHold:
print "CLICK + HOLD"
else:
print "CLICK + RELEASE"
self.leftclickHold = not self.leftclickHold
GETTING A FRAME OF DATA (Part II)

18
THANK YOU
● Blog
– bxhika.wordpress.com
● Twitter
– @BesjanXhika
● E-Mail
– xhikab@gmail.com

Gesture recognition

More Related Content

Viewers also liked (10)

Similar to Gesture recognition (20)

Recently uploaded (20)

Gesture recognition