Optimization-based programming of sensor-based robot systems

Existing commercial robot programming systems were mainly designed with well-structured environments, position control, fast and accurate motions, simple motion primitives, a low number of sensors, involving only simple sensor data processing, in mind. Moreover most commercial are installed behind fences, shielded from humans.

The next generation of robot applications shift to ill-structured environments, a combination of position and sensor-based control, robust motions
complex motion primitives and a high number of sensors with sophisticated sensor data processing. In many applications, robots can (but not necessarily) interact with humans.

To prepare robots for the next generation of applications there is a clear need of improved programming methods and tools.

The iTaSC-Skill framework

iTaSC stands for instantaneous Task Specification using Constraints, which is developed at the K.U.Leuven during the past years [1,2,5].

The framework generates motions by specifying constraints in geometric, dynamic or sensor-space between the robots and their environment. These motion specifications constrain the relationships between objects (object frames) and their features (feature frames). Established robot motion specification formalisms such as the Operational Space Approach [3], the Task Function Approach [6], the Task Frame Formalism [4], Cartesian Space control, and Joint Space control are special cases of iTaSC and can be specified using the generic iTaSC methodology.

The key advantages of iTaSC over traditional motion specification methodologies are:

1. composability of constraints: multiple constraints can be combined, hence the constraints can be partial, they do not have to constrain the full 6D relation between two objects;
2. reusability of constraint specification: the constraints specify a relation between feature frames, that have a semantic meaning in the context of a task, implying that the same task specification can be reused on different objects;
3. automatic derivation of the control solution: the iTaSC methodology generates a robot motion that optimizes the constraints by automatically deriving the controllers from that constraint specification.

These advantages imply that the framework can be used for any robotic system, with a wide variety of sensors.

Skills are responsible for the coordinated execution of tasks and the parameter configuration of different instantaneous motion specifications. Consequently, the framework separates the continuous level motion specification and discrete level coordination. One skill coordinates a limited set of constraints, that together form a functional motion. Finite State Machines implement the skill functionality.

Box picking:

In this demo, a human and a two seven degree of freedom robot arms manipulate a box together. In a first step the human aligns the first robot arm (which is actively controlled to be compliant) on one side of the box. In a second step the human triggers the second robot arm (by touch) to align itself autonomously to the opposite side of the box. In a third step, both robots hold the box and compensate its gravity, allowing the human operator to move the box freely. Note that the robots respond to force inputs exerted on the whole robot, not only at the end point.

Teleoperation:

References

• [1] J. De Schutter, T. De Laet, J. Rutgeerts, W. Decre, R. Smits,E. Aertbelien, K. Claes, and H. Bruyninckx. Constraint-based task specification and estimation for sensor-based robot systems in the presence of geometric uncertainty. The International Journal of Robotics Research, 26(5):433–455, 2007.
• [2] W. Decre, R. Smits, H. Bruyninckx, and J. De Schutter. Extending iTaSC to support inequality constraints and non-instantaneous task specification. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, pages 964–971, Kobe, Japan, 2009.
• [3] O. Khatib. The operational space formulation in robot manipulator control. In Proceedings of the 15th International Symposium on Industrial Robots, pages 165–172, Tokyo, Japan, 1985.
• [4] M. T. Mason. Compliance and force control for computer controlled manipulators. IEEE Transactions on Systems, Man, and Cybernetics, SMC-11(6):418–432, 1981.
• [5] J. Rutgeerts. Constraint-based task specification and estimation for sensor-based robot tasks in the presence of geometric uncertainty. PhD thesis, Department of Mechanical Engineering, Katholieke Universiteit Leuven, Belgium, 2007.
• [6] C. Samson, M. Le Borgne, and B. Espiau. Robot Control, the Task Function Approach. Clarendon Press, Oxford, England, 1991.
• [7] R. Smits and H. Bruyninckx. Composition of complex robot applications via data flow integration. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 5576–5580, Shangai, China, 2011.