354
V. L. Kondarattsev et al.
In this chapter, we consider the problem of semiautomatic detection and classification of three-dimensional objects in three-dimensional scenes. Scene data is
obtained by scanning the surrounding area using depth cameras. Detection (selection) of objects is performed using classical preprocessing methods: noise removal,
deletion of supporting surfaces, and unnecessary objects. Deep learning methods are
used to classify the object highlighted in the scene.
In Sect. 24.2, we present a comparative analysis of existing deep learning architectures designed for semantic segmentation of point clouds. Based on this analysis, we
justify the choice of the LDGCNN architecture [1] for implementing the classification
stage of the model. In Sect. 24.3, we consider the formal formulation of classification
and search problems for 3D models (also named as meshes), and introduce all the
necessary terms and mathematical constructions. In Sect. 24.4, we look at various
aspects of data preprocessing. In particular, to highlight an object of interest on a 3D
scene, we introduce heuristic algorithms for removing the floor and foreign objects.
For the problem of classification using a neural network, we introduce an algorithm
for preprocessing 3D models made by 3D artists, which allows us to diversify the
input data and bring it closer to the real raw data obtained from scanners. Raw data,
which is usually represented in the form of a point cloud, does not allow creating
visualizations of desirable quality. Therefore, separately in Sect. 24.5, we considered
the problem of the automatic completion of a 3D scene with objects from a database
of polygonal 3D models. In order to find the closest-shaped model in the dataset,
we will need to solve the problem of searching in the space of three-dimensional
models. For this purpose, we have developed an algorithm based on the ray casting
method to obtain the descriptive representation of 3D model [2]. In Sect. 24.6, we
present conclusions from the work done and discuss further work on the application
of the considered algorithms and their improvement.
24.2 Related Work
From the point of view of classical machine learning, the task of constructing such
an algorithm is usually divided into two subtasks:
1. Selection of informative features from the entire description of the object.
2. Application of machine learning algorithm (classification, clustering, etc.) to the
selected description.
The first subtask is traditionally based on a good understanding of the subject area
and the specifics of data. Thus, for example, in [3], the authors obtained an effective
algorithm for extracting informative features from a point cloud. Firstly, for each
point from the point cloud, the optimal, in the sense of proximity by some metric,
number of points is the nearest neighbors for the selected subset of points. Then the
covariance matrix is calculated, and its own vectors are used to construct various
information signs.
V. L. Kondarattsev et al.
In this chapter, we consider the problem of semiautomatic detection and classification of three-dimensional objects in three-dimensional scenes. Scene data is
obtained by scanning the surrounding area using depth cameras. Detection (selection) of objects is performed using classical preprocessing methods: noise removal,
deletion of supporting surfaces, and unnecessary objects. Deep learning methods are
used to classify the object highlighted in the scene.
In Sect. 24.2, we present a comparative analysis of existing deep learning architectures designed for semantic segmentation of point clouds. Based on this analysis, we
justify the choice of the LDGCNN architecture [1] for implementing the classification
stage of the model. In Sect. 24.3, we consider the formal formulation of classification
and search problems for 3D models (also named as meshes), and introduce all the
necessary terms and mathematical constructions. In Sect. 24.4, we look at various
aspects of data preprocessing. In particular, to highlight an object of interest on a 3D
scene, we introduce heuristic algorithms for removing the floor and foreign objects.
For the problem of classification using a neural network, we introduce an algorithm
for preprocessing 3D models made by 3D artists, which allows us to diversify the
input data and bring it closer to the real raw data obtained from scanners. Raw data,
which is usually represented in the form of a point cloud, does not allow creating
visualizations of desirable quality. Therefore, separately in Sect. 24.5, we considered
the problem of the automatic completion of a 3D scene with objects from a database
of polygonal 3D models. In order to find the closest-shaped model in the dataset,
we will need to solve the problem of searching in the space of three-dimensional
models. For this purpose, we have developed an algorithm based on the ray casting
method to obtain the descriptive representation of 3D model [2]. In Sect. 24.6, we
present conclusions from the work done and discuss further work on the application
of the considered algorithms and their improvement.
24.2 Related Work
From the point of view of classical machine learning, the task of constructing such
an algorithm is usually divided into two subtasks:
1. Selection of informative features from the entire description of the object.
2. Application of machine learning algorithm (classification, clustering, etc.) to the
selected description.
The first subtask is traditionally based on a good understanding of the subject area
and the specifics of data. Thus, for example, in [3], the authors obtained an effective
algorithm for extracting informative features from a point cloud. Firstly, for each
point from the point cloud, the optimal, in the sense of proximity by some metric,
number of points is the nearest neighbors for the selected subset of points. Then the
covariance matrix is calculated, and its own vectors are used to construct various
information signs.