15
Feb, 2017

Gesture control of robotic arm using the Kinect Module

The aim of this project is to create a remote control system for a robotic arm controlled by using the Kinect v2 sensor, to track the movements of the user arm, without any additional point of measurement (marker-less modality).

The Kinect camera acquires a 3D point cloud of the body and a skeleton representation of the gesture/pose is obtained using the SDK library software provided by the Kinect. The skeleton joints are tracked and used to estimate the angles.

Figure 1 - The point cloud acquired by the Kinect, and the skeleton. Points A, B, and C are the joints.

Point A is the joint of the wrist, point B is the joint of the elbow and point C is the joint of the shoulder. In the three dimensional space, vectors BA and BC are calculated with using the space coordinates of points A, B and C, which are taken from the skeleton. Angle α is calculated by using the dot product of the two vectors. 

The software has been developed in C# in Visual Studio 2015.

Figure 2 - Elbow angle geometry.

Related Publications

Sarikaya, Y.; Bodini, I.; Pasinetti, S.; Lancini, M.; Docchio, F.; Sansoni, G. “Remote control system for 3D printed robotic arm based on Kinect camera“, Congresso Nazionale delle Misure Elettriche ed Elettroniche GMEE-GMMT. 2017

13
Jan, 2017

The Smart Gym Concept

Goals

Subjects with complete spinal cord injury (SCI) experience several limitations in their daily activities. Rehabilitative gait training by means of powered gait orthosis (PGO) has been shown to decrease the risk of secondary pathologies (e.g. skin injuries, osteoporosis, cardiovascular issues) and can significantly improve the quality of life, provided that they are used on a regular basis and in the correct way.
The traditional training is based on three steps: gait trial, data analysis, and gait correction. Gait is monitored using force platforms and dedicated instrumentation directly applied to the patient (EMGs, IMUs, markers). These devices require time to be positioned and care by the patient while walking.
Data analysis and gait correction are totally dependent on the therapist experience. The process must be performed in specialized centers and is very time consuming. This leads to a high cost for the community health services.

Costs and efforts needed for longer training sessions could be reduced by the availability of a SMART GYM environment to:

  1. provide real-time feedback directly to the patient about his/her gait performance;
  2. provide information to clinicians and therapists to remotely monitor the patient;
  3. involve the patient in personalized gait exercises depending on his/her performance in time;
  4. allow the patient to train on his/her own, along paths longer than those permitted by the limited room available in gait laboratories.
Figure 1 - The smart gym objectives.

The SMART GYM components

  • BMM (BioMechanical Model): this component evaluates patient posture and motion in real time. The BMM is fed with (i) the spatiotemporal gait parameters, (ii) the kinematics of the lower limbs, and (iii) the kinematics of the upper limbs;
  • IC (Instrumented Crutches): this system is specifically designed measure the kinematics of the lower limbs and the spatiotemporal gait parameters;
  • OMT (Optical Motion Tracking): this system estimates the upper limb kinematics using a suitable set of Kinect devices;
  • VR (Virtual Reality): this system carries out the self-training of the patient;
  • TD (Therapist Dashboard): a dedicated software service that collects the data from BMM and allows the therapist to follow the patient progress remotely, an to plan new train patterns.
Figure 2 - The SMART GYM components.
Figure 3 - The therapist dashboard architecture.

Novelty of the Project

  • The IC system, based on “A Vision System that walks with the patient” (challenging, but very promising);
  • The OMT based on multiple Kinect devices is new with respect to state of the art, since most applications are performed using a single device.
  • The combination of the OMT in the VR system will lead to the realization of a novel human machine interface, able to accurately render the posture of the whole body.
  • The paradigm of adaptiveness that lays under the BMM system is novel, as it requires new approaches to the estimate of the gait and posture indices.
  • The smart gym project is highly interdisciplinary: knowledge and expertise from the mechanical and the electronic measurement community will allow the development of either sensors (both contact and contact-less), models (both in the biomechanics and in the artificial vision contexts), measurement procedures, virtual reality scenarios and clinical experimentation, which will be fused together and integrated using ICT technologies. This aspect well fits one of the most significant Horizon 2020 priority, i.e., transversality.

Impact of the Project

  • The SMART GYM represents a new environment where the patient can practice autonomously and yet under the therapist control;
  • The SMART GYM will reduce community costs;
  • The SMART GYM will be valuable for elderly people;
  • The SMART GYM will be valuable even for healthy people.
11
Jan, 2017

MAPSS: Multimaterial Additive Production for Structural Sustainability Applications

This project focuses on the e-manufacturing of complex parts by integrating smart additive manufacturing technologies to more traditional manufacturing processes, in view of obtaining sustainable, flexible, completely automatic and fast manufacturing (SMART MANUFACTURING).

The project is funded by the POR FESR 2014-2020: Linea R&S per Aggregazioni 2015 Regione Lombardia, and involves SMEs and Research Laboratories in strong cooperation to demonstrate the feasibility of producing industrial manufacts in the automotive industry.

The role of the laboratory is the study and the implementation of vision tools for the on-line control of the manufacturing process based on the additive approach.