Development of a WEB-based image analysis service for biological and medical images

The aim of this project is to develop and test novel image processing algorithms for image deconvolution and analysis specifically designed for real time, hardware-independent operations. The final goal is to share the image analysis capabilities using cloud computing techniques, by means of a specifically developed web-based system.

This project involves a lasting collaboration between the faculty of Engineering at Università degli Studi di Brescia and the Mechanical Engineering department at MIT. It is supported by the CARIPLO UniBS-MIT-MechE project, a  faculty exchange program agreement designed to promote new long term scientific collaborations and to consolidate existing collaborations between faculty members of UniBS and MIT-MechE.

The faculties involved into this program are Prof. Giovanna Sansoni, for the University of Brescia, and Prof. Forbes Dewey, for the Massachusetts Institute of Technology.

As part of project, Prof. Sansoni of the University of Brescia visited MIT for three months in the early Fall of 2011 to inaugurate the personal exchange program. Prof. Sansoni gave a lecture at MIT, entitled “3D imaging and surroundings“.

Dewey_sansoni
Prof. Giovanna Sansoni's seminar "3D imaging and surroundings"

During her stay at MIT, she had the opportunity to meet a number of Prof. Dewey’s collegues. Among them, Prof. George Barbastiathis gave a number of useful suggestions on non-linear image analysis approaches. The exchange she had at MIT made it possible to establish the basics for an extremely stimulating research field, dealing with image denoising, deconvolution and compression to be performed across the web.

Prof. Dewey travelled to Brescia for two weeks in April of 2012. The aim of his visit was twofold: on one hand, he interacted with Prof. Sansoni Laboratory to optimize the collaboration. He also met people from the Laboratory Start-Ups, and realized how rich and powerful is the high-technology pole generated by the Laboratory.

On the other hand, he gave two seminars open to Ph. D. students and researchers of the University of Brescia. In the first seminar, entitled “Quantitative Models of Cellular Biology: Application to Cellular Dynamics and Drug Development“, he gave two examples of how one can begin to put large-scale quantitative biological models together to predict ab into the results of altering drug dosages and creating drug combinations. In the second seminar, enitled “Super-Resolution Optical Microscopy: Beating Photons at Their Own Game“, he addressed the application of measuring the mechanical properties of living cells using 3D stacks from optical microscope images, far beyond the Rayleigh criteria.

Prof. Dewey's seminar "Application to Cellular Dynamics and Drug Development"
Prof. Dewey’s Seminar “Super-Resolution Optical Microscopy: Beating Photons at Their Own Game”
Prof. Dewey and Prof. Sansoni with Prof. Beretta, the director of the CARIPLO-MIT program

Study case: ear reconstruction

The example that we present in this article concerns the reconstruction of an ear. The approach is based on optical acquisition and modeling of the shapes. The final prosthesis is directly obtained from the model by means of rapid prototyping.

The patient’s defect is shown in the Fig. 1. The left ear is seriously damaged in consequence of a burn. To fabricate the prosthetic element, the right ear, shown in Fig. 2, was used as the template.

The test was performed as follows: first, we acquired the right, safe ear. We configured the digital scanner (Vivid 910 , Konica Minolta Inc.) in the MIDDLE configuration. Four views were acquired and aligned together. Then the triangle mesh was obtained and mirrored, in view of using it to model the prosthesis. The corresponding models are shown in Fig 3a, 3b and 3c. As a second step, the defect was gauged. The system configuration was the same as the one in the previous acquisition. Two views were sufficient to cover the whole surface. The mesh was created over the aligned views. The result is presented in Fig 3d.

The third step was the acquisition of the whole patient face in three views, as shown in Fig. 4. This model was used as the skeleton to align the mesh in Fig. 3d to the one in Fig. 3c. The model of the defect was aligned to the skeleton. Then the model of the ear was interactively aligned until the aesthetical appearance on the whole face was judged optimal. At this point, the skeleton was discarded. The two models were edited to fill residual holes and to reconstruct missing surface parts (mainly due to undercuts). Finally, they were finely connected in correspondence with their borders. The result of this step is shown in Fig. 5.

The mesh has been topologically controlled to produce the physical copy. This has been fabricated by means of rapid prototyping technology. The Connex 500 3D Printing System (Objet-Geometries Inc.) has been used. This machine is capable of printing parts and assemblies made of multiple model materials all in a single build. The materials used to fabricate the ear prosthetic element are the TangoBlackPlus Shore A85 for the area corresponding to the auricle surface, and the TangoBlackPlus Shore A27 for the areas at the borders of the ear. The ear was obtained in about one hour; the process is very cheap (the cost is in the order of 70 Euros). Fig. 6 shows the front and the back side of the final prosthesis.

Fig. 7 shows the patient’s face after the application of the prosthetic element. It is worth noting that, in this figure, the prosthesis color is not optimized yet. In fact, we wanted to check its functionality before optimizing it under the aesthetical point of view.

Fig. 7 - Patient face with prosthesis on.

In this process, patient comfort was optimal, since the acquisition step was quick, contactless and safe. The prosthesis try-in was unnecessary. The prototyping step was very cheap, and the overall time required was about six hours, plus the machining of the prosthesis.

Real time acquisition of silhouettes

The aim of this project is to design, implement and charaterise a vision system for the 3D reconstruction of objects rotating with respect to their axis by means of the Silhouette method. The system has been developed using LabView Real-Time 8.6.1 (National Instruments, NI).

 The hardware of the system is composed by the NI Embedded Vision System (EVS), the  Basler Scout scA1390-17gm/gc Gigabit Ethernet camera, and a telecentric lighting system combined with a telecentric optics (Optoengineering, Italy). The software tools dedicated to image acquisition and elaboration have been developed by using the NI-IMAQ vision libraies. The vision system is called  ‘OPTOSILHO‘.

A vision system for a labelling machine

In this work, a vision system specifically designed to monitor the labels on bottling lines for wine production industries is presented. The system is based on smart cameras, Real Time LabView sofware and IMAQ vision libraries. The work has been developed in the frame of a thesis work, in collaboration with Studio Progetti Automation srl (Italy). 

Bottling lines for wine are complex systems, which provide bottle rinsing, filling, capping, labelling, wrapping and pallettising. In this work, a vision system has been studied and implemented for the automatic control of the labelling machine. The bottling line, that is installed at Cantine Leonardo da Vinci (Tuscan, Italy), is characterized by a high level of automation, and is able to output 10.000 bottles per hour. However, the labeling machine is completely operator-dependant: three blocks of labels must be manually positioned on the machine for each wine brand. These are the front label, the back label and the DOCG label. The position of the labels with respect to the bottles is visually controlled and adjusted by the operators.

Due to human errors, it is possible that the operator mount on the machine labels corresponding to a wine brand different with respect to the one programmed on the bottling line. Early detection of this situation is mandatory to reduce downtime and, in the worse case, to avoid that the whole bottling process is performed from scratch.

die_cutter_plebani.jpg

Student Projects 2008-2009

This article reports the projects developed by students in the area of the development and validation of 2D vision systems. The projects are carried out by undergraduated students, for their graduation (I level) or by graduated students, during the attendance of the courses of Optical Measurements and of Electronic Instrumentation B.

The following project has been developed by Thomas Plabani, in 2007-2008

The aim was to study a Pattern Matching software for a 2D vision system to be assembled on an automatic die cutter. Time and precision constraints were rather demanding. 
Download the presentation to have an insight of how the problem has been solved!

The project below has been developed by Matteo Frassine and Stefano Pesce, during the course of Electronic instrumentation, year 2008-2009.

They had to detect the features of the plug in the figure: diameter of the plug and number of the small metallic spheres of the wire.

This project has been developed by Daniele Mazzotti and Diego Rossini, during the course of Electronic instrumentation, year 2008-2009.

They had to detect the features of the clothes peg shown in the figure below. They had to count the pegs in the package, detect the absence of one or more pegs, detect misalignment of them, control the color sequence.

The project below has been developed by Andrea Cadei and Manuel Zenato, during the course of Electronic instrumentation, year 2008-2009

They had to detect the features of the soapdish shown in the figure. The difficulty in this case was to perform the measurement of a transparent object. Suitable illumination and image processing have been used.

This project has been developed by Aurel Fonta, during the course of Electronic instrumentation, year 2008-2009.

The objective was to measure the object geometry, to ‘read’ the brand label, and to read the bar code.

The project below has been developed by Davide Alghisi and Alessio Zanni, during the course of Electronic instrumentation, year 2008-2009

They had to control the integrity of the comb shown below, to count the comb teeth and to measure its dimension.
demonstrator_button.jpg

Development of Laser Slits: Student Projects

The development of systems based on the projecttion of laser stripes is very interesting under the education point of view. Students must learn about active triangulation principlesimaging techniquesprogramming techniques, and metrology.

In addition, computer vision is required for the calibration of the camera projector pair. This article reports examples of this type of projects.

The first one is the development of a laser slit system for the measurement of the central profile of small buttons. The system has been completely developed in LabView.

The students are Emanuele Ferrari and Paolo Bellandi. In this presentation one can learn how they worked for the development of the demonstrator.

The demonstrator developed by the students for the project.

The second project deals with the development of a system based on the projection of two laser stripes and the use of a single camera. The aim is to add redundancy to decrease the influence of shadow, which inherently decrease the performances of the measurement. 

Two Ph. D. students are being working at this project: Paolo Bellandi and Gianluca Cavagnini. The first results are available in the presentation.

The demonstrator developed so far by the Ph. D. students.
auto_orig_e_frange.jpg

Student Projects 2006-2007

This is an example of the typical project in the course of Optical Measurements

The students have been involved into the acquisition of a small car by using the system based on the projection of structured light. The PolyWorks suite of programs was used to acquire the mesh.

The real difficulty in this project was the calibration of OPL-3D, since it requires the knowledge of calibration camera models and the experimental practice with calibration masters  and setup of the projector-camera pair.

The students are Marco Tomasini and Michele Mancini. Download the presentation to have an insight of the work.

The small car to be acquired and renderize by using OPL-3D.

This project has been carried out at the city museum (Museo di Santa Giulia), where the students had to acquire the Winged Victory of Brescia statue using the Vivid 910 Scanner as the measurement sensor.

The statue was already measured in 2001 by using OPL-3D. The aim was to compare the measurement performances when coherent light (Vivid 910) is used instead of incoherent light (OPL-3D). 

The students who carried out the project are Nicola Modonesi and Davide Barba. Here you can download a brief presentation of their work.

The beautiful Winged Victory of Brescia statue to be acquired by the students.

In this project the students had to acquire a bas-relief at the Museo di Santa Giulia of Brescia by using the Vivid 910 Scanner.

The bas-relief is a very large one, representing the patron saints of Brescia, St. Faustino and Giovita. The aim was to produce the triangle mesh of the bass-relief.

The students who carried out the project are Mauro Facchini and Emanuele Tonoli. Here you can download a brief presentation of their work.

The bas-relief of Saints Faustino and Giovita to be acquired by the students.

3D image acquisition of crime scenes for documentation, analysis and medical inspection

In this thesis project the two students involved (Marco Scalvenzi and Gianluca Cavagnini) had the opportunity of using the Vivid 910 laser scanner to document 3D crime scenes at different resolution levels.

It was really interesting to capture medium range details, such as the injury tools and body parts, as well as short range details, like bullet holes, skin lesions, and blood patterns.

A numer of experiences have been performed both in indoor and in outdoor environments. The flexibility, the portability and the ease of use of this system revealed very precious to complete the projects.

present_photogrammetry.jpg

Photogrammetry: experiences carried out in the Laboratory from 2004 to 2007

From 2004 to 2007 a number of students have done projects and hands-on practice in the Laboratory with photogrammetry. The main goal was to understand pros and cons with respect to active vision: as such, 2D and 3D vision has been studied and used to perform measurments and to reconstruct real world scenes using industrial hardware.

A brief presentation of the activities carried out in the Laboratory during these years can be downloaded below.