Augmented Neural Network for Full Robot Kinematic Modelling in SE(3)

Accepted for publication in IEEE Robotics and Automation Letters (RAL), 2022

Description:

Robot kinematics is a fundamental basis for robot control. The kinematic model finds the relationship between actuation space (mtor/joint values) and Cartesian space (robot’s 3D position and orientation).

Generally, for serial-link rigid robots, geometric approaches (e.g. Denavit-Hartenberg convention) are sufficient to obtain accurate an kinematic model. However, robot designs are becoming more and more complex, especially in fields like minimally invasive surgery, where high degree of articulation and miniaturization are required.

Standard serial-link manipulator
Continuum robot

Due to the increasing modelling complexity, machine learning, like Artificial Neural Networks, has become very useful for robot modelling. Generally two approaches are used:

  • Inverse Kinematic (IK) modelling: the inputs are the desired robot’s poses to reach and the outputs are directly the robot’s motor commands;
  • Forward Kinematic (FK) modelling: the inputs are the motor commands and the outputs are the robot’s poses.

Learning the IK allows directly obtaining the robot’s commands, but the output only depends on the training strategy and data. When learning the FK, instead, the robot model can be used in consolidated optimal control approaches (like MPC, QP, etc) that guarantee better system’s stability and allow finding optimal control values for a given task.

The drawback of learning the FK is that derivatives of the model are needed for control purposes and, in general, robot model learning does not consider differential relationships during model training.

Additionally, when the orientation is considered, a proper representation is needed in order to ensure continuity in the model to be built.

In this work we present AugNet, a neural network model that includes differential relationships during model learning for full robot pose (both 3D position and orientation). Additionally, we propose a feasible representation (trigonometric representation) for the orientation to overcome the challenges due to model discontinuities.

AugNet model for position and orientation learning.

Our results show that including differential relationships during model learning allows improving robot model’s accuracy and control performance. Additionally, our trigonometric representation proves feasible in learning orientations.

In our work we employed Roll, Pitch, and Yaw angles. The main limitation from the control perspective is the possibility to encounter singularities due to Gimbal lock.

Simulation results, comparing AugNet with a standard learning approach (FFNet), on a serial-link manipulator and on a soft robot model
Real world experiments controlling a KUKA arm with the learnt AugNet model

Augmenting Loss Functions of Feedforward Neural Networks withDifferential Relationships for Robot Kinematic Modelling

Accepted for publication in IEEE ICAR 2021

Description:

In this work we present two novel strategies for training Feedforward Artificial Neural Networks (ANN) to learn robot’s forward kinematics (3D position only).

Generally, ANN are used to diectly learn the mapping from actuation space to Cartesian space using a cost function that just minimizes the error between the predicted tip position and the measured (ground truth) one.

For control purposes, however, the velocity mapping is exploited. This information is neglected instandard ANN training.

The two novel loss functions include information about the differential reletionship between positions and velocities to incorporate knowledge on the velocity mapping too while training the model for the forward kinematics.

Our results show large improvements in model learning and control when the differential relationship is included.

Exemplary simulation results on using the different loss functions. P Loss is the standard one used in ANN. PV and PVI Loss account for vleocity and inverse velocity mapping

GlobDesOpt: A Global Optimization Framework for Optimal Robot Manipulator Design

Accepted for publication in IEEE Access, 2022

Available in Git-Hub.

Description:

GlobDesOpt’s flow chart

In this work we present GlobDesOpt, a simple-to-use Matlab package for robot manipulator design.

Currently, it only focuses on serial-link manipulators to optimize different user-defined parametrs such as link lenghts, joint offsets, joint types (prismatic or revolute).

It allows the user to choose between three solvers:

  • BO: Bayesian Optimization;
  • PSO: Particle Swarm Optimization
  • GA: Genetic Algorithm

Currently, the optimization cost function aims at maximing the robot dexetrous workspace.

The framework can be used for single or dual-arm robots.

For dual-arm robots, the distance between the two arms can be set as additional optimization variable and the current cost function maximizes the dual-arm dexterity within the common dexterous workspace.

Exemplary results using GlobDesOpt solvers for dual-arm robot design

Kalibrot: A Simple-To-Use Matlab Package for Robot Kinematic Calibration

Accepted for publication in IEEE IROS 2021-International Conference on Intelligent Systems and Robots

Available in Git-Hub.

Description:

Kalibrot’s flow chart

In this work we present Kalibrot, a simple-to-use Matlab package for robot kinematic calibration.

Currently, it only focuses on serial-link manipulators to optimize the Denavit-Hartenberg parameters.

It allows the user to choose between two solvers:

  • Traditional pseudoinverse;
  • Quadratic Programming to include bounds on the variables

The package provides the user with the calibrated parameters, the calibrated kinematic model, the identification matrix

Pre-operative Offline Optimization of Insertion Point Location for Safe and Accurate Surgical Task Execution

Accepted for publication in IEEE IROS 2021-International Conference on Intelligent Systems and Robots

Description:

The optimization framework

In this work we present an optimization approach to identify the optiaml insertion point location for laparascopic surgeies when employing macro-micro robotic manipulators.

Our results show how the location of the Remote Center of Motion (RCM) afects the robot’s perfomance.

The cost function takes into account safety in avoiding nearby organs, maximizing robot’s dexterity, and ensuring precision in perfoming a specified task (path tracking for tumor resection).

Our cost function and resilence to error strategy ensure the optimal location resides in a region with small variations in the perfomance in case of deviations from the optimum.

Results showing the obstacle avoidance and the effects of different RCM locations

Bayesian Neural Network Modeling and Hierarchical MPC for a Tendon-Driven Surgical Robot with Uncertainty Minimization

Accepted for publication in Robotics and Automation Letters (RAL) and ICRA 2021- International Conference on Robotics and Automation.

Description:

The Hi-MPC controller

In this work we present a Hierarchical Model Predictive Control (Hi-MPC) approach in conjunction with Bayesian Neural Networks (BNN) to model and control a tendon-driven surgical robotic tool.

BNN are here employed because they allow obtaining not only the model prediciton, but also the associated model uncertainty.

The Hi-MPC is instead used to solve two prioritized subtasks:

  • primary task: tracking a desired path;
  • secondary task: ezploit the robot redundnacy and minimize the model’s uncertainty so as to reconfigure itself in more reliable state space regions.

Our simulation and real world experiments show that the uncertainty minimization helps imrpoving the control performance and make the path tracking more accurate.

Comparison of the control with only the primary MPC and the Hi-MPC approach.

Model predictive control for a tendon-driven surgical robot with safety constraints in kinematics and dynamics

Accepted for publication in IROS 2020- International Conference on Intelligent Robots and Systems.

Description:

The MPC controller

In this work we evaluate the applicatbility of Model Predictive Control (MPC) to control a tendon-driven surgical robotic tool. The dynamic model of the robot is learnt by means of Artificial Neural Networks (ANN) and used to impose bounds on the dynamics of the system.

We evaluate our approach on different bounding values.

Tracking results in simulation tests

A Novel Approach for Outlier Detection and Robust Sensory Data Model Learning

Accepted for publication in IROS 2019- International Conference on Intelligent Robots and Systems

Description:

The robust algorithm for nonlinear regression with ANN

In this work we developed a novel algorithm for nonlinear regression with Artificial Neural Networks (ANN) capable of rejecting outliers in the dataset and thus improve the modelling.