KINECT WITH ROS
2 likes2,556 views
社内で開催したROS勉強会の発表資料です。ROSに接続されたKinectから取得した画像から物体検出をする仕組みについて発表しました。
![KINECT WITH ROS
OPENCVで緑⾊の物体を検出する
#!/usr/bin/env python
import cv2
import numpy
# read png image and convert the image to HSV
image = cv2.imread("/path/to/target.png", cv2.IMREAD_COLOR)
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
# detect green objects
lower_green = numpy.array([133 / 2 - 10, 62 * 255 / 100 - 30, 37 * 255 / 100 - 30], numpy.uint8)
upper_green = numpy.array([133 / 2 + 10, 62 * 255 / 100 + 30, 37 * 255 / 100 + 30], numpy.uint8)
mask = cv2.inRange(hsv, lower_green, upper_green)
cv2.imshow('original', image)
cv2.imshow('mask', mask)
cv2.imwrite("hsv_mask.png", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()](https://image.slidesharecdn.com/kinectwithros-161117124751/85/KINECT-WITH-ROS-13-320.jpg)
KINECT WITH ROS
- 1. KINECT WITH ROS ROS勉強会 @ 2016.11.17
- 2. KINECT WITH ROS AGENDA ▸ 3D Vision System Overview ▸ 3D Sensors ▸ Simple Target Detection with OpenCV ▸ Put It All Together
- 4. KINECT WITH ROS 3D VISION SYSTEM OVERVIEW 3D SENSOR (KINECT) KINECT BRIDGE OPENCV ROS POSITION TRACKER SENSOR IMAGE SENSOR IMAGE 3D POSITION
- 5. 3D SENSORS
- 6. KINECT WITH ROS 3D SENSORS ▸ depth cameraによって距離が計測できる ▸ 距離計測ができることで観測点のx,y,z座標が得られる ▸ ⼊⼿可能な3Dセンサー ▸ Microsoft Kinect ▸ ASUS Xtion ▸ PrimeSense Carmine
- 7. KINECT WITH ROS MICROSOFT KINECT ▸ 製品(カッコ内は発売時期) ▸ Kinect 360 (2010. 11) ▸ Kinect One (2013. 11) ▸ Kinect for Windows v2 (2014. 7) ▸ Driver ▸ Freenect ▸ libfreenect2 (Kinect for Windows v2)
- 8. KINECT WITH ROS KINECT FOR WINDOWS V2をROSから使うには？ ▸ iai_kinect2をインストールすればよい ▸ https://github.com/code-iai/iai_kinect2 ▸ READMEのインストール⼿順参照 ▸ libfreenect2のビルド＆インストール ▸ iai_kinect2のビルド&インストール
- 9. KINECT WITH ROS LIBFREENECT2のインストール ▸ https://github.com/OpenKinect/libfreenect2 $ git clone https://github.com/OpenKinect/libfreenect2.git $ cd libfreenect2 $ sudo apt-get install build-essential cmake pkg-config $ mkdir build && cd build $ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/freenect2 $ make $ make install $ ./bin/Protonect ⼿順 動作確認
- 10. KINECT WITH ROS IAI_KINECT2のインストール ▸ https://github.com/code-iai/iai_kinect2 $ cd ~/catkin_ws/src/ $ git clone https://github.com/code-iai/iai_kinect2.git $ cd iai_kinect2 $ rosdep install -r --from-paths . $ cd ~/catkin_ws $ catkin_make -DCMAKE_BUILD_TYPE="Release" $ roslaunch kinect2_bridge kinect2_bridge.launch $ rosrun kinect2_viewer kinect2_viewer ⼿順 動作確認
- 12. KINECT WITH ROS SIMPLE TARGET DETECTION ▸ Kinectで取得した画像から対象となる物体を検出する ▸ OpenCVを使って⾊(HSV)認識で検出
- 13. KINECT WITH ROS OPENCVで緑⾊の物体を検出する #!/usr/bin/env python import cv2 import numpy # read png image and convert the image to HSV image = cv2.imread("/path/to/target.png", cv2.IMREAD_COLOR) hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV) # detect green objects lower_green = numpy.array([133 / 2 - 10, 62 * 255 / 100 - 30, 37 * 255 / 100 - 30], numpy.uint8) upper_green = numpy.array([133 / 2 + 10, 62 * 255 / 100 + 30, 37 * 255 / 100 + 30], numpy.uint8) mask = cv2.inRange(hsv, lower_green, upper_green) cv2.imshow('original', image) cv2.imshow('mask', mask) cv2.imwrite("hsv_mask.png", mask) cv2.waitKey(0) cv2.destroyAllWindows()
- 15. KINECT WITH ROS KINECTはROSからどう⾒えているのか?(1/2) $ rostopic list | grep kinect2 /kinect2/bond /kinect2/hd/camera_info /kinect2/hd/image_color /kinect2/hd/image_color_rect (中略) /kinect2/qhd/camera_info /kinect2/qhd/image_color /kinect2/qhd/image_color_rect /kinect2/qhd/image_depth_rect (中略) /kinect2/sd/camera_info /kinect2/sd/image_color_rect /kinect2/sd/image_depth /kinect2/sd/image_depth_rect (中略) /kinect2/sd/image_ir /kinect2/sd/image_ir_rect /kinect2/sd/points
- 16. KINECT WITH ROS potix2@ros% rostopic info /kinect2/hd/image_color_rect Type: sensor_msgs/Image Publishers: * /kinect2 (http://ros:42318/) Subscribers: None KINECTはROSからどう⾒えているのか?(2/2)
- 17. KINECT WITH ROS 物体検出の流れ(未完) ▸ kinect_bridgeからimageを取得 ▸ 取得したimageをopencvで処理 ▸ kinectのreference frame上の座標へ変換(未完) ▸ topicとしてpublish(未完)
- 18. KINECT WITH ROS TRACKER.PY import roslib import rospy import cv2 import sys from std_msgs.msg import String from sensor_msgs.msg import Image from cv_bridge import CvBridge, CvBridgeError class PositionTracker: def __init__(self): # subscribe to kinect image messages self.sub = rospy.Subscriber("kinect2/qhd/image_color", Image, self.image_callback, queue_size=1) self.bridge = CvBridge() def image_callback(self,data): try: cv_image = self.bridge.imgmsg_to_cv2(data, "bgr8") except CvBridgeError as e: print(e) cv2.imshow("Image window", cv_image) cv2.waitKey(3)
- 19. def main(args): ic = PositionTracker() rospy.init_node('kinect_tracker') try: rospy.spin() except KeyboardInterrupt: print("Shutting down") cv2.destroyAllWindows() if __name__ == '__main__': main(sys.argv) KINECT WITH ROS TRACKER.PY