Overview #
This article shows how to convert OpenNI2 video frames to OpenCV Mat format. Once you have frame in Mat format, you can leverage all OpenCV functions.
Expect Output #
Tutorial #
- After connect to camera and start each sensor (color, depth and IR). Read the frame into OpenNI frame
color = device.create_color_stream()
color.start()
depth = device.create_depth_stream()
depth.start()
ir = device.create_ir_stream()
ir.start()
rgbFrame = color.read_frame()
depthFrame = depth.read_frame()
irFrame = ir.read_frame()- Convert OpenNI frame to numpy array
rgbMat = np.frombuffer(rgbFrame.get_buffer_as_uint8(), dtype=np.uint8).reshape(rgbFrame.height, rgbFrame.width, 3)
depthMat = np.frombuffer(depthFrame.get_buffer_as_uint16(), dtype=np.uint16).reshape(depthFrame.height, depthFrame.width, 1)
irMat = np.frombuffer(irFrame.get_buffer_as_uint16(), dtype=np.uint16).reshape(irFrame.height, irFrame.width, 1)
- For better visualization, you can do the following conversion
// For color frame
rgbMat = cv2.cvtColor(rgbMat, cv2.COLOR_BGR2RGB)
// For depth frame
depthMat = cv2.convertScaleAbs(depthMat, alpha=255.0 / 1024.0)
depthMat = cv2.applyColorMap(depthMat, cv2.COLORMAP_JET)
// For IR frame
irMat = np.frombuffer(irFrame.get_buffer_as_uint16(), dtype=np.uint16).reshape(irFrame.height, irFrame.width, 1)
irMat = cv2.convertScaleAbs(irMat, alpha=255.0 / 1024.0)