Contents


  1. Accropode blocks detection from point cloud
    1. Context
    2. Algorithm Overview
  2. Civil Structures Defects detection from orthophoto using ConvNet
    1. Context
    2. Algorithm Overview
  3. Imaging Software for thermal measurement in fusion facilities
    1. Context
    2. Imaging software
    3. Thermal Event detection and tracking
  4. Building Detection from VHR aerial imagery using shadow and image segmentation
  5. Simultaneous shadow/vegetation detection from high resolution aerial images
  6. Analysis of individual migration movements of the European sea bass by data mining with electronic tagging

Keywords: deep learning, object detection and tracking, pattern recognition, image segmentaion, Bayesian inference, Markov random field.



Detection and Tracking of each accropode block on the dam of "La Réunion"

Context:

The context of the project is to detect and track each accropode of the dam (accropodes blocks are used to strengthen the dam) at "La Réunion" island (the project "Nouvelle route du Littoral"). The point clouds are acquired using photogrammetry method (drone aerial survey) or using bathymetric surveys (for the blocks submerged in the water).

Trulli

Overview of the algorithm:

An automated detection of accropode block from a point cloud is presented. The input point cloud is first decomposed into a set of regions, in which spatial and normal information are taken into account. An adjacency graph is then built. An iterative process of region growing/merging is incorporated with ICP (iterative closest point) fitting using the CAD model. The algorithm is capable to detect accropodes whose a very small part is visible.

Trulli
Figure: results of accropode detection. Each accropode is overlaid with an CAD model whose position and orientation are determined.

Patent: David Tran-Thanh NGO, Natacha ROBERT, Elisabeth GARDON, "Détection et suivi de blocs préfabriqués dans les nuages de points", pending registration by Soletanche Freyssinet.




Civil Structures Defects detection from orthophoto using ConvNet

Context of the project:

The infrastructure managers carry out periodically an assessment of the state of their heritage to ensure that the structures do not present a risk and schedule the maintenance operations. Today the cartographies of disorders are manually carried out by Civil Engineering inspectors. The disorders are located and then, their caracteristics (length, surface,...) are mesured or estimated.
The goal of the project is to develop an automated detection of defect (concrete spalling, concrete crack, ...) from the orthophoto using deep learning. So, the inspectors can focus on understanding pathologies of defects and not on their representations.

Trulli Trulli
Figure: the cartographies of defects are carried out by inspector on the ground.

Defect Detection Algorithm:

Abstract: Automated building defect detection is highly required in the field of civil and construction to increase the working efficiency of building surveyors. The current image processing techniques are difficult to capture visual representation of cracks in a complex background due to the irregular crack patterns and size. Hence, deep learning neural network is proposed to automatically identify defects. The proposed model is based on pre-trained CNN classifiers of VGG-16 with class activation mapping (CAM) for object localisation. The proposed model has proven to be robust and able to accurately detect and localise building defects.

Trulli
Figure: flowchart for detection building defects
Trulli
Figure: defect (concrete spalling) is identified using CAM.




Imaging Software for thermal measurement in fusion facilities

Context:

One of the main issues raised by nuclear fusion relates to the safe exploitation of wall materials that can withstand high heat flux. To satisfy these requirements, numerous diagnostics are implemented, especially Infrared (IR) and visible camera to survey and study the behavior of the wall. A powerful Imaging Software is therefore essential in order to analyze the data. In this context, since 2016, within the IRFM (Institute for Magnetic Fusion Research), CEA Cadarache, I am in charge of the conception and development of a software platform for the exploitation of imaging diagnostics in nuclear fusion devices (WEST tokamak of the CEA/Cadarache and Wendelstein 7-X stellarator of IPP Greifswald).


Imaging software:

The software is a C++ framework with Qt library which is aimed to provide a visualization and analysis tool for infra-red data. Some of its functionalites :


Trulli
Figure: thermal events database plugin


Thermal Event detection and tracking


We develop a method to detect, track thermal events using feature extraction (motion and visual patterns) from offline videos. By using the thermal event detection editor plugin, experts annotate thermal events and correct event detection results (remove false alarm and/or add miss-detection object) and saved them into the SQL Database. They create a ground truth to feed the neural network. A CNN (convolutional neural network) is trained with event features. The network is then used to detect and track thermal events in a new IR video.


Trulli
Figure: (left) thermal event detection and tracking, (right) maximum temperature of thermal event over time.




Building Detection from VHR aerial imagery using shadow and image segmentation

Abstract: an automated detection of rectangular buildings from monocular very high resolution aerial images, based on a novel region-growing image segmentation algorithm is proposed. The input image is first decomposed into a set of homogeneous primitive regions. Regions are then grouped into different clusters by a Markov Random Field (MRF)-based region classification. According to the position of the shadows, a merging procedure is performed over regions that belongs to the same cluster to produce regions whose shapes are appropriate to rectangles. This iterative classification-merging stops when no merging happens. Final building regions are extracted from the obtained region growing image segmentation result. The experimental results prove that the proposed method is applicable in various areas (high dense urban, suburban, and rural) and is highly robust and reliable.

Trulli
Figure: An example of first classification-merging: (a) RGB image, (b) oversegmentation (regions are separated by cyan lines) of the image in which shadows and vegetation are masked out, (c) MRF-based classification (clusters are separated by red lines), (d) shadows cast by building, (e) detected building segments (boundaries are delineated by violet lines), (f) after merging.









Simultaneous shadow/vegetation detection from high resolution aerial images

Abstract: A new method for simultaneously detecting shadows and vegetation in remote sensing images, based on Otsu’s thresholding method and Dempster-Shafer (DS) fusion which aims at combining different shadow indices and vegetation indices in order to increase the information quality and to obtain a more reliable and accurate segmentation result. The DS fusion is carried out pixel by pixel and is incorporated in the Markovian context while obtaining the optimal segmentation with the energy minimization scheme associated with the Markov random field (MRF). This new approach is applied on remote sensing images and demonstrates its efficiency.

Figure: Shadow/vegetation detection with the DS fusion and MRF regularization. From left to right : RGB original image, (b) Shadow/vegetation mapping with Dempster-Shafer fusion, after Markov random field regularization.





You can find here the comments of my PhD advisors on my work.



Analysis of individual migration movements of the European sea bass by data mining with electronic tagging

Abstract: European sea bass is a highly mobile species whose spatial dynamics remain poorly understood. In order to analyze adult migration patterns, we carried out tagging experiments (acoustic telemetry and data storage tags) off the coast of west Brittany. Acoustic tracking provided 3D behaviour used to parameterize a geolocation method from DST data (depth and temperature). To date, 21 DSTs out of 246 have been recovered. Our geolocation model (daily predicted probability maps issued from a HMM - hidden Markov model) combines both satellite SST data and the hydrodynamic model MARS3D. A sensitivity analysis demonstrates its robustness to recover consistent movement patterns. Fish with long times at liberty exhibited strong site fidelity to summer feeding ground. Reconstructed individual trajectories revealed different spawning migration patterns (mostly in the Bay of Biscay but also in the Celtic sea/English Channel). The joint analysis of horizontal and vertical movements showed seasonal behavioural shifts. Our results support the hypothesis of a fine scale spatial structuring at the population scale, a key issue poorly addressed by current management frameworks. It also provides key information for the understanding and the modelling of the spatial dynamics of the species under environmental forcing.

Trulli
Figure: Celtic sea migration pattern. Most likely tracks (red) issued from the developed hidden Markov model for 2 bass tagged and recaptured off the coast of west Brittany after ~1 year at liberty.