ICRA 2023 Virtual Manipulation Challenge: Bin Picking

Bin Picking Track Tasks

The bin picking track is about localizing objects stored chaotically in a bin, computing a grasping pose and executing the pick up. There will be a fully equipped virtual bin picking cell with robot, vacuum and parallel gripper, camera, box and industrial parts to pick. The box is always filled with one type of industrial parts, which are randomly distributed. The simulation will output the following data to the user: system setup, collision models, RGB-D camera data (colored point cloud, depth and color images), states of actuators. The user-contributed controller can control the following items in simulation: robot arm motion commands (TCP poses or trajectories), gripper selection (vacuum/parallel), gripper state (on/off, open/close). Both tasks require the robot controller to clear all parts from the bin, one by one. This includes object localization, grasp planning (incl. gripper selection) and motion planning capabilities in the robot controller. The task is repeated with different industrial parts and environmental boundary conditions at increasing levels of difficulty. The track is split into a Machine Tending and Kitting task.

© Fraunhofer IPA

Machine Tending

The concrete task is that the robot picks a single object out of the box and puts it as precisely at a target location as possible, repeating this until the box is empty. All objects in this setting are known beforehand and CAD models of all relevant objects in this task are provided to the teams during the preparation phase. Hence, this task is attracting model-based bin picking solutions. Scoring will incorporate all 4 KPIs.


In this task, the robot picks a single object out of the box and puts or drops it into a target box without placement restrictions. The operation repeats until the box is empty. However, this time objects are not known in advance. The teams will receive CAD models of other, exemplary industrial parts during the preparation phase for validation purposes, but will face different objects in the contest. This task addresses model-free picking solutions. The scoring will neglect the precise placement KPI, which is invalid in this setting.


Participating teams can opt to participate solely in either one or in both of these bin picking tasks.


The assessment score will be formed as the sum of the following 4 KPIs:

  1. Picking success: picking success rate in picking exactly one object at time.
  2. Precise placement: we employ the symmetry-aware metric provided by Romain Bregier et al. for evaluating precise placement.
  3. Average cycle time of a pick, mapping the range of 20 s to 5 s linearly to 0 to full points.
  4. Remaining objects in the bin (0 points if 10 or more objects remain in the bin, linear mapping up to 0 remaining parts).

The total score over all test scenarios will determine the winner


Reem Al-Gaifi, Kilian Kleeberger, Richard Bormann

The Virtual Manipulation Challenge is organized by the research project Sim4Dexterity. The Sim4Dexterity project has received funding from the German Federal Ministry of Education and Research (BMBF) through grant 01IS21061 (Sim4Dexterity).