Storeviva is not only an inventory management software, it comes integrated with logistics services.

Measuring parcels with Kinect

Logistics has traditionally been labor intensive, but technology is quickly changing that. Here at Courex, we use technology like crowd sourcing, intelligent delivery dispatching, real-time delivery notification to improve efficiency and customer satisfaction. We’re improving our warehouse operations with a piece of innovation from a surprising source: console gaming. Namely the Microsoft Kinect. Compared to conventional 3D scanners, it costs hundreds rather than tens of thousands.

We developed the Kinect Dimensioning System to measure the size and weight of incoming and outgoing packages. The core of this system, the Kinect, was developed by Microsoft originally for gaming, but we have re-purposed it as a cost-effective 3D scanner. The Kinect’s sensor sends out millions of rays of individual infrared light and times how long each one reaches an object and bounces back; divide the flight time by the speed of light and you get the distance to the object, this is called the time of flight method of measuring distance. The Kinect captures millions of these distance readings per seconds and creates a point cloud with them, and it is this point cloud that we feed to our algorithm to determine how large the package is.

3D Rending of the KDS

Mobile KDS station with Kinect, scale, printer, and an onboard computer

Getting started

Microsoft provides a SDK for the Kinect.

This is the raw data received from the Kinect sensor in the form of a point cloud. The position of each point is determined by the Kinect’s 512 x 424 time of flight camera; the color of each point comes from the Kinect’s 1080p RGB camera.

We process this point cloud using various algorithms from the open source Point Cloud Library.

The sample PCD file is here: sample.pcd(8MB). To view it, you need to follow this PCL tutorial.

What the raw point cloud from the Kinect sensor looks like

Raw point cloud data from the Kinect sensor

Stage 1: Plane Segmentation

First we utilize a technique called plane model segmentation to find the floor plane. After we have the floor plane’s parameters, we remove any point that's below a preset cutoff height. This helps to removes the floor and foreign objects on the floor, along with any low-lying noise points.

Next we linearly shift and rotate the point cloud so that the floor plane becomes the XY plane to simplify things for later steps. This step is not strictly necessary but it makes visualization and debugging much easier.

What the point cloud looks like after removing all points below the cutoff

The point cloud after removing all points below the cutoff height

Stage 2: Cropping

We then crop out all the points that lie outside a pre-set detection region. This step helps to eliminate the extraneous points associated with the KDS station itself.

Cropped to detection region

Cropping down to the detection region

Stage 3: Filtering

Next we filter out any noise among the points using a statistical method by which we iterate through the list of points twice. For the first iteration, we calculate the average distance from each point it to its nearest k neighbors. All the average distances are tabulated to find their mean and standard deviation. Then we set the distance threshold as one standard deviation above the mean. For the second iteration, we eliminate any point that has an average distance to neighbors that is higher than the distance threshold.

Post-filtering point cloud

Outlier points being filtered out

Stage 4: Clustering

Then we use Euclidean cluster extraction to break down the point cloud into cluster of points. This allow us the reject any sizeable foreign object that has not been yet eliminated in the previous rounds of filtering.

We rate each cluster of points by its fitness score, which is computed by multiplying the number of points in the cluster by its distance to the center of the detection region. This metric assigns a high fitness score to large cluster that’s close to the center, and assigns a low fitness score to small clusters, especially ones that are far from the center. Here we have the three main clusters near the origin that make up the detection target and a faraway red cluster representing an inadvertently captured foreign object.

Point cloud with the various clusters colored

Separating the points into clusters

Stage 5: Minimum Bounding Box

Now we take all clusters with a fitness score above a preset threshold and combine them to arrive at the point cloud of the target object. Finally we calculate a minimal bounding box over this point cloud to get the target object's measurements.

Final result bounding box

Merging the target clusters and calculating their minimal bounding box

Conclusion

Thanks to the Kinect Dimensioning System we are able to measure incoming and outgoing pallets and packages quickly and accurately. It is capable of measuring entire pallet loads on the warehouse floor like shown above, and thanks to the Kinect being mounted on a swivel mechanism it is also capable of measuring down to small cellphone-sized packages on KDS station tabletop. The KDS also contains other goodies like a scale to weigh packages and a RFID label printer to print out trackable labels.

All of these components are seamlessly integrated to our Storeviva Inventory Management System. Thanks to these technologies, Courex is able to improve our operational efficiency.