Object recognition, localization, and tracking play a role of primordial importance in computer vision applications. However, it is still an extremely difficult task, particularly in scenarios where ...objects are attended to using fast-moving UAVs that need to robustly operate in real time. Typically the performance of these vision-based systems is affected by motion blur and geometric distortions, to name but two issues. Gimbal systems are thus essential to compensate for motion blur and ensure visual streams are stable. In this work, we investigate the advantages of active tracking approaches using a three-degrees-of-freedom (DoF) gimbal system mounted on UAVs. A method that utilizes joint movement and visual information for actively tracking spherical and planar objects in real time is proposed. Tracking methodologies are tested and evaluated in two different realistic Gazebo simulation environments: the first on 3D positional tracking (sphere) and the second on tracking of 6D poses (planar fiducial markers). We show that active object tracking is advantageous for UAV applications, first, by reducing motion blur, caused by fast camera motion and vibrations, and, second, by fixating the object of interest within the center of the field of view and thus reducing re-projection errors due to peripheral distortion. The results demonstrate significant object pose estimation accuracy improvements of active approaches when compared with traditional passive ones. More specifically, a set of experiments suggests that active gimbal tracking can increase the spatial estimation accuracy of known-size moving objects, under conditions of challenging motion patterns and in the presence of image distortion.
This article presents a complete solution for autonomous mapping and inspection tasks, namely a lightweight multi-camera drone design coupled with computationally efficient planning algorithms and ...environment representations for enhanced autonomous navigation in exploration and mapping tasks. The proposed system utilizes state-of-the-art Next-Best-View (NBV) planning techniques, with geometric and semantic segmentation information computed with Deep Convolutional Neural Networks (DCNNs) to improve the environment map representation. The main contributions of this article are the following. First, we propose a novel efficient sensor observation model and a utility function that encodes the expected information gains from observations taken from specific viewpoints. Second, we propose a reward function that incorporates both geometric and semantic probabilistic information provided by a DCNN for semantic segmentation that operates in close to real-time. The incorporation of semantics in the environment representation enables biasing exploration towards specific object categories while disregarding task-irrelevant ones during path planning. Experiments in both a virtual and a real scenario demonstrate the benefits on reconstruction accuracy of using semantics for biasing exploration towards task-relevant objects, when compared with purely geometric state-of-the-art methods. Finally, we present a unified approach for the selection of the number of cameras on a UAV, to optimize the balance between power consumption, flight-time duration, and exploration and mapping performance trade-offs. Unlike previous design optimization approaches, our method is couples with the sense and plan algorithms. The proposed system and general formulations can be be applied in the mapping, exploration, and inspection of any type of environment, as long as environment dependent semantic training data are available, with demonstrated successful applicability in the inspection of dry dock shipyard environments.
In this work we propose a holistic framework for autonomous aerial inspection tasks, using semantically-aware, yet, computationally efficient planning and mapping algorithms. The system leverages ...state-of-the-art receding horizon exploration techniques for next-best-view (NBV) planning with geometric and semantic segmentation information provided by state-of-the-art deep convolutional neural networks (DCNNs), with the goal of enriching environment representations. The contributions of this article are threefold, first we propose an efficient sensor observation model, and a reward function that encodes the expected information gains from the observations taken from specific view points. Second, we extend the reward function to incorporate not only geometric but also semantic probabilistic information, provided by a DCNN for semantic segmentation that operates in real-time. The incorporation of semantic information in the environment representation allows biasing exploration towards specific objects, while ignoring task-irrelevant ones during planning. Finally, we employ our approaches in an autonomous drone shipyard inspection task. A set of simulations in realistic scenarios demonstrate the efficacy and efficiency of the proposed framework when compared with the state-of-the-art.
In this work we showcase the design and assessment of the performance of a multi-camera UAV, when coupled with state-of-the-art planning and mapping algorithms for autonomous navigation. The system ...leverages state-of-the-art receding horizon exploration techniques for Next-Best-View (NBV) planning with 3D and semantic information, provided by a reconfigurable multi stereo camera system. We employ our approaches in an autonomous drone-based inspection task and evaluate them in an autonomous exploration and mapping scenario. We discuss the advantages and limitations of using multi stereo camera flying systems, and the trade-off between number of cameras and mapping performance.
In this work we propose a holistic framework for autonomous aerial inspection tasks, using semantically-aware, yet, computationally efficient planning and mapping algorithms. The system leverages ...state-of-the-art receding horizon exploration techniques for next-best-view (NBV) planning with geometric and semantic segmentation information provided by state-of-the-art deep convolutional neural networks (DCNNs), with the goal of enriching environment representations. The contributions of this article are threefold, first we propose an efficient sensor observation model, and a reward function that encodes the expected information gains from the observations taken from specific view points. Second, we extend the reward function to incorporate not only geometric but also semantic probabilistic information, provided by a DCNN for semantic segmentation that operates in real-time. The incorporation of semantic information in the environment representation allows biasing exploration towards specific objects, while ignoring task-irrelevant ones during planning. Finally, we employ our approaches in an autonomous drone shipyard inspection task. A set of simulations in realistic scenarios demonstrate the efficacy and efficiency of the proposed framework when compared with the state-of-the-art.
The use of unmanned aerial vehicles (UAVs) for autonomous inspection tasks has become more prominent in recent years. To make the most of the autonomous inspection, the parameters governing control, ...perception and navigation of the robot should be tuned precisely to the necessary task. Currently, the use of motion capture (mocap) systems is the norm when performing the stringent evaluation of simultaneous localization and mapping (SLAM) and advanced controllers. In this paper, we address the use of a cost-effective solution to ground-truthing and evaluation of said algorithms in large industrial environments. To this end, we use fiducial markers, deployed in known locations, in order to estimate the pose of the vehicle in 6 degrees-of-freedom (6DOF) and test them against a state-of-the-art mocap system. We additionally test the method in the field, by deploying the markers to the environment of interest and applying widely used SLAM implementations to confirm its efficacy by evaluating their performance in two emulated inspection task scenarios. We find that our method is comparable in performance to the state-of-the-art mocap systems without the need for laborious calibration and is capable of providing a pose estimate for evaluating SLAM and underlying UAV control methods.
This paper proposes PredictiveSLAM, a novel extension to ORB-SLAM2, which extracts features from specific regions of interest (ROI). The proposed method was designed with the risk posed both to ...humans and robotic systems in large-scale industrial sites in mind. The ROI are determined through an object detection network trained to detect moving human beings. The method detects and removes humans from feature extraction, predicting their potential future trajectory. This is done by omitting a specific ROI from extraction, deemed to be occluded in consecutive time steps. Two masking methods -static object and moving object trajectories - are proposed. This approach improves tracking accuracy and the performance of SLAM by removing the dynamic features from the reference for tracking and loop closures. The method is tested on data collected in a laboratory environment and compared against a state-of-the-art ground truth system. The validation data was collected from real-time experiments which aimed at simulating the typical human worker behaviours in industrial environments using an unmanned aerial vehicle (UAV). This study illustrates the advantages of the proposed method over earlier approaches, even with a highly dynamic camera setup on a UAV working in challenging environments.
In this paper, a novel strategy for practical inspection planning in dry docks using unmanned aerial vehicles (UAVs) is presented. Planning is a fundamental prerequisite for accurate navigation and ...control of the UAV. The proposed method utilises the random sample consensus (RANSAC) algorithm to extract plane models from a voxel grid representation of the environment. For high-level planning, semantic knowledge of the environment is leveraged in a novel manner to exploit of structured obstacles, such as straight walls and orthogonal corners. In order to deal with lower-level navigation, the approach incorporates a simple graph-based local replanner to generate paths that avoid obstacles in the environment. The proposed method is compared with state-of-the-art graph-based planner in simulation and subsequently evaluated in a real environment. The paper maintains the use case of UAV vessel inspection and presents exhaustive simulation and field testing, which demonstrate the viability of the proposed approach in a fully working large-scale industrial environment.
PDF
