3.4.1.1. Consolidation and georeferencing

Point cloud registration begins with a consolidation step (also called 3D registration) to assign correspondences between two sets of points. This consolidation is done by identifying homologous points in the various clouds to be consolidated, which can either be targets or homologous geometric entities (planes, spheres, cylinders), or on the basis of specific algorithms such as the DARCES method: Data-Aligned Rigidity-Constrained Exhaustive Search based on RANSAC (RANdom SAmple Consensus) and ICP (Iterative Closest Point) (Landes et al. 2011).

To be able to link a point cloud to a national geographic reference frame, indirect georeferencing is required. In this case, topographic techniques such as tacheometry or GNSS surveys are used to link the clouds to the base geographic reference frame. As an alternative to this indirect solution, some scanners allow direct georeferencing on site through a polygonal path scan (Figure 3.2).

3.4.1.2. Pre-treatment

Once the various point clouds have been assembled in the same frame and the results of the consolidation have been validated, a point cloud cleaning step is necessary to eliminate undesirable points that add noise to the measurements (Figure 3.3). This step is either done manually by using the tools available in point cloud processing software, or automatically using filters that allow the removal of isolated points based on the distance from their surroundings.

Cleaning can also be done with machine learning-based algorithms such as PointCleanNet (Rakotosaona and Polytechnique 2019). More recently, a part of this operation can be done by acquiring two scans one after the other to minimize the effect of objects moving during the survey (the double scan).

At the end of the pre-processing, the point cloud is consolidated, cleaned and sometimes georeferenced. It can then enter a process of segmentation and modeling to produce a BIM model.

3.4.1.3. Segmentation

The 3D reconstruction of urban models from LiDAR data involves two fundamental steps: (1) segmentation and (2) 3D modeling of the resulting classes.

Segmentation is based on the homogeneous characteristics of the scene data with respect to a certain number of criteria in order to separate them into several regions (Deveau 2006). It involves decomposing a cloud into subsets based on similarity or dissimilarity criteria, (Figure 3.4). In the case of point clouds, the 3D position of points, curvature or flatness are criteria that are considered in the segmentation process (Boulaassal 2010). Other criteria concern geometric, spectral, semantic characteristics, etc.

The process of segmentation is well known in the field of image processing. The segmentation algorithms were originally developed and applied to images. Their extension to the case of point clouds by airborne LiDAR is argued by the fact that airborne LiDAR acquisitions are of the 2.5D type, and their transformation into images does not cause a loss of consistent information that could alter their accuracy. However, in the case of terrestrial LiDAR, the scene is acquired in 3D. Hence, the need to adapt the models and algorithms to take into account the 3D character of the scene.

Segmentation approaches

Many segmentation approaches are reported in the literature with several classifications. A first classification allows the following main families of methods to be distinguished: (1) edge-based segmentation; (2) region growing segmentation; (3) model fitting segmentation; and (4) machine learning segmentation (Figure 3.5). A summary of these approaches was provided by Grilli et al. (2017).

Another classification classifies LiDAR data segmentation methods into three main families of approaches: (1) direct approaches based on the raw point cloud; (2) derived approaches based on a cloud-derived product; and (3) hybrid approaches combining several data sources (LiDAR, aerial images, 2D maps, etc.). A state of the art on these different approaches is reported by Ballouch et al. (2020).

The segmentation approaches developed in the literature are based on LiDAR data acquired from several platforms and concern both the interior and exterior of the building.

In the case of airborne LiDAR, we cite the example of Tarsha-Kurdi et al. (2008), who developed a semi-automated approach for the segmentation of point clouds on an urban scene, where a first distinction between the ground and aboveground is made through the generation of an DSM (Digital Surface Model) (Figure 3.6), and then an extraction of buildings from the ground segment is conducted by adopting a region growth algorithm.

In the case of terrestrial LiDAR, Boulaassal (2010) proposes a semi-automatic segmentation of building facades surveyed by a static terrestrial scanner. The planar segments corresponding to the building facades are extracted by the RANSAC algorithm, which has demonstrated its robustness, even in the presence of outliers in the point cloud.

For the case of indoor acquisitions, Anand et al. (2011) propose a segmentation approach based on the use of contextual data, such as local visual appearance, shape cues and geometric relationships, to semantically segment point clouds of building interiors.

Advances in segmentation methods are more notable in the image mode than in the point cloud mode, given the time lag in the development of the two acquisition methods. Thus, several studies attempt to exploit image-based approaches by projecting the point cloud onto images. One approach, proposed by Tabkha et al. (2019), uses the 360° panoramic image based on cylindrical projections to transform a point cloud into images and apply image-based segmentation processes to them.

Semantic segmentation (classification)

When the cloud segmentation approach is not controlled by prior knowledge, the segmentation results lack semantic information (Xie et al. 2019). Once the segmentation process is accomplished, each cluster of points can be semantically augmented by associating a semantic label with it. This is a semantization of the clouds that occurs after the segmentation process. This process is called semantic segmentation. Semantic segmentation is known as classification in the field of photogrammetry and remote sensing.

In computer vision, semantic segmentation is based on subdividing a point cloud into parts with a semantic assignment by associating semantic labels with each class (Lawin et al. 2017) (Figure 3.7). Semantic segmentation of 3D point clouds is a major challenge for understanding and extracting knowledge from urban scenes. This is one of the major challenges in the field of computer vision (Ballouch et al. 2020).

Semantic segmentation approaches are distinguished into: (1) supervised approaches; (2) unsupervised approaches; and (3) interactive approaches. Supervised approaches use data sets or training models for semantic classification of the cloud. These approaches require massive training data to train the classification model. Unsupervised approaches are automatic methods based on user parameterization of the classification algorithm and can lead to imprecise results, while interactive approaches require user intervention to guide the segmentation process (Grilli et al. 2017).

Today, deep learning methods and Convolutional Neural Networks (CNN) allow us to solve many problems related to the thematic exploitation of massive point clouds (image classification and analysis, real-time object detection on videos, semantic segmentation, Natural Language Processing (NLP), audio recognition, etc.). The term Deep Learning (DL) refers to the training of neural networks. It is one of the most important forms of artificial intelligence derived from machine learning, whose basic architecture is inspired by the biological processes of connections between neurons in the brain, assembled together.

The advantage of a CNN is the preservation of the spatial information of the visual data, and also the considerable decrease of the weight and bias parameters, which allows a faster execution and more accurate predictions. The algorithm mainly relies on the detection of features (boundaries, shapes, texture, etc.) in the images through filters based on the convolution operation.

Several semantic segmentation approaches based on DL have been developed in the literature. A state of the art of these approaches is reported in Ballouch et al. (2020).

3.4.2.1. Geometric modeling methods

3D geometric modeling methods are distinguished by the type of geometric primitives used and the relations of the combination adopted to associate them, in order to build a 3D shape. Their choice depends on the intended application: some models are efficient for 3D visualization, others perform well in spatial analysis and 3D simulation applications.

Many 3D geometric models have been proposed in the literature. A first classification of 3D modeling methods by Li (1994) distinguishes the methods into two main categories: (1) surface methods when the object is reconstructed based on surface primitives and (2) volume methods when the object is reconstructed based on solids. Another category classifies the models into: (1) models based on geometric primitives; (2) mesh models; and (3) hybrid models.

Surface methods

The main surface methods are: 2D grid mode, triangulations, NURBS (Non-Uniform Rational B-Splines) methods, B-REP (Boundary Representation) method, etc.

NURBS and B-Rep methods are well suited for modeling buildings and other urban objects. NURBS modeling is based on curves that are defined by control points and constraints imposed on a curve through a precise mathematical definition. The B-Rep method is a boundary-based modeling that describes a 3D shape by the surfaces composing its boundaries: nodes, edges and knots. This method integrates an implicit topological representation.

Volume methods

In this category of methods, we find the CSG (Constructive Solid Geometry) method, the SPR (Spatial Partitioning Representation) method, Primitive Instancing (PI), etc.

The CSG model is based on the combination of geometric primitives for the construction of complex objects through geometric transformations or Boolean operations (union, intersection, difference, etc.). This allows the construction of a good number of urban objects with complex geometry. The SPR method decomposes an object into a set of juxtaposed elementary solids that do not intersect and that can vary in terms of types, sizes, positions, parameters and orientations. In this category, we find the voxel model, which consists of splitting a solid into elementary cubes, and the Octree model, which is an improvement of the “Voxel” model. Octrees allow us to represent objects by a hierarchical succession of cubes of variable sizes. The Octree model allows a hierarchical representation of the object which adapts to a display at different resolutions.

The “Primitive Instancing” method uses complex parameterizable primitives to represent a 3D scene, which are usually stored in an object library.

Mesh modeling

The 3D mesh is one of the most popular representations of 3D shapes. A 3D mesh structure describes the geometry and appearance of the objects it represents with an associated texture. It consists of a set of polygons called faces, described by a diagram showing the location and connectivity of their vertices. The objects can be easily recognized by the human eye, but cannot be distinguished by computer systems (Willenborg et al. 2018). 3D meshes can also be presented as graphically structured data where the nodes of the graph correspond to the vertices of the mesh and the edges represent the connectivity between these vertices. This mode is usually combined with the point cloud mode where the points of the cloud form the vertices of the triangles which are calculated mathematically, in most cases, by Delaunay triangulation. Thus, the mesh, bringing no additional information, remains more easily interpretable than a point cloud with a good visual rendering and a realistic visualization quality. The object format (OBJ) is one of the most popular 3D formats for storing and exchanging 3D mesh models that defines a geometry suitable for 3D modeling and computer graphics (Willenborg et al. 2018).

Parametric modeling

BIM modeling is based on the definition of object classes containing instances whose appearance changes based on parameters associated with the class (common parameters) or object (object-specific parameters), and also the relationships they undertake with other objects. This is the principle of parametric modeling, where representation is made based on parameters and rules that determine the geometric and non-geometric properties of objects (Eastman et al. 2011).

Parametric modeling consists of associating a set of parameters (qualitative and/or quantitative) with an object, that can easily be modified (Figure 3.8). This modeling defines the relationships between the components of an object, which can, for example, be represented by mathematical equations. The set of rules assigned to the objects allows them to be automatically updated following a change of context.

With BIM tools, it is possible to create libraries of intelligent objects that can store and use information about the project. For example, a door, placed in a BIM model, is a simple assembly of lines and surfaces, but represents an object connected to a wall and contains information about materials, costs, suppliers, etc.

3.4.2.2. Selection criteria

The choice of a 3D geometric modeling must allow the representation of a large number of urban objects, a multi-level representation, a storage performance and a 3D spatial analysis (Koussa 2011). In addition, the adopted model must be able to be translated to the schema of a 3D standard, such as CityGML and IFC, to ensure interoperability of exchanges.

In the case of buildings, surface representations such as B-Rep and NURBS are the best known. The B-Rep method, by its intuitive character, allows us to cover a very large domain since most solids can be represented by surfaces. It is well adapted for multi-scale use and has a good topological potential. In terms of interoperability, it is compatible with the geometric primitives (Point, LineString, Polygon, MultiLineString, Multi Polygon, etc.) defined by the OGC specifications. In addition, the B-Rep method can allow 3D functions such as volume calculation in the case where the faces are oriented (Ramos 2003). However, the method is not very compact since the description of some objects requires a large number of surfaces to represent them.

In the urban environment, the use of hybrid methods combining the B-Rep method and other methods such as the CSG method allows the representation of buildings to be extended to model more complex geometries. Most of the methods use variants of hierarchical space partitioning structures, also called multi-resolution structures, such as k-d trees, octrees or quadtrees. Massive point clouds need to make use of suitable structures in order to optimize their management. One of the examples of these structures is the Modifiable Nested Octree (MNO).