The demand for cone-beam computed tomography (CBCT) imaging in clinics, particularly in dentistry, is rapidly increasing. Preoperative surgical planning is crucial to achieving desired treatment ...outcomes for imaging-guided surgical navigation. However, the lack of surface texture hinders effective communication between clinicians and patients, and the accuracy of superimposing a textured surface onto CBCT volume is limited by dissimilarity and registration based on facial features. To address these issues, this study presents a CBCT imaging system integrated with a monocular camera for reconstructing the texture surface by mapping it onto a 3D surface model created from CBCT images. The proposed method utilizes a geometric calibration tool for accurate mapping of the camera-visible surface with the mosaic texture. Additionally, a novel approach using 3D-2D feature mapping and surface parameterization technology is proposed for texture surface reconstruction. Experimental results, obtained from both real and simulation data, validate the effectiveness of the proposed approach with an error reduction to 0.32 mm and automated generation of integrated images. These findings demonstrate the robustness and high accuracy of our approach, improving the performance of texture mapping in CBCT imaging.
Despite the increasing need of analyzing human poses on the street and in the wild, multi-person 3D pose estimation using monocular static or moving camera in real-world scenarios remains a ...challenge, either requiring large-scale training data or high computation complexity due to the high degrees of freedom in 3D human poses. We propose a novel scheme to effectively track and hierarchically estimate 3D human poses in natural videos in an efficient fashion. Without the need of using labelled 3D training data, we formulate torso estimation as a Perspective-N-Point (PNP) problem, and limb pose estimation as an optimization problem, and hierarchically structure the high dimensional poses to efficiently address the challenge. Experiments show good performance and high efficiency of multi-person 3D pose estimation on real-world videos, including street scenarios and various human daily activities from fixed and moving cameras, resulting in great new opportunities to understand and predict human behaviors.
The technology for Visual Odometry (VO) that estimates the position and orientation of the moving object through analyzing the image sequences captured by on-board cameras, has been well investigated ...with the rising interest in autonomous driving. This paper studies monocular VO from the perspective of Deep Learning (DL). Unlike most current learning-based methods, our approach, called DeepAVO, is established on the intuition that features contribute discriminately to different motion patterns. Specifically, we present a novel four-branch network to learn the rotation and translation by leveraging Convolutional Neural Networks (CNNs) to focus on different quadrants of optical flow input. To enhance the ability of feature selection, we further introduce an effective channel-spatial attention mechanism to force each branch to explicitly distill related information for specific Frame to Frame (F2F) motion estimation. Experiments on various datasets involving outdoor driving and indoor walking scenarios show that the proposed DeepAVO outperforms the state-of-the-art monocular methods by a large margin, demonstrating competitive performance to the stereo VO algorithm and verifying promising potential for generalization.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
With the potential to increase road safety and provide economic benefits, intelligent vehicles have elicited a significant amount of interest from both academics and industry. A robust and reliable ...vehicle detection and tracking system is one of the key modules for intelligent vehicles to perceive the surrounding environment. The millimeter-wave radar and the monocular camera are two vehicular sensors commonly used for vehicle detection and tracking. Despite their advantages, the drawbacks of these two sensors make them insufficient when used separately. Thus, the fusion of these two sensors is considered as an efficient way to address the challenge. This paper presents a collaborative fusion approach to achieve the optimal balance between vehicle detection accuracy and computational efficiency. The proposed vehicle detection and tracking design is extensively evaluated with a real-world data set collected by the developed intelligent vehicle. Experimental results show that the proposed system can detect on-road vehicles with 92.36% detection rate and 0% false alarm rate, and it only takes ten frames (0.16 s) for the detection and tracking of each vehicle. This system is installed on Kuafu-II intelligent vehicle for the fourth and fifth autonomous vehicle competitions, which is called "Intelligent Vehicle Future Challenge" in China.
Excavation is one of the broadest activities in the construction industry, often affected by safety and productivity. To address these problems, it is necessary for construction sites to ...automatically monitor the poses of excavator manipulators in real time. Based on computer vision (CV) technology, an approach, through a monocular camera and marker, was proposed to estimate the pose parameters (including orientation and position) of the excavator manipulator. To simulate the pose estimation process, a measurement system was established with a common camera and marker. Through comprehensive experiments and error analysis, this approach showed that the maximum detectable depth of the system is greater than 11 m, the orientation error is less than 8.5°, and the position error is less than 22 mm. A prototype of the system that proved the feasibility of the proposed method was tested. Furthermore, this study provides an alternative CV technology for monitoring construction machines.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
In-air handwriting based on a monocular camera is an innovative and promising modality for human–computer interaction, offering a plethora of potential applications. However, existing in-air ...handwriting systems based on monocular camera suffer a significant challenge in determining the spatial coordinates of fingertips using two-dimensional images from a monocular camera. Since the size of the fingertip is very small and has very few discriminative features. To tackle this challenge, we propose a Multi-Scale Channel Attention Network (MSCAN). Through weighting multi-scale channels, the MSCAN facilitates the concentration of target detection models on high-resolution, small-scale channels, thus enhancing fingertip localization precision. We integrate the MSCAN with the YOLOv5s model to realize a novel in-air handwriting system based on monocular vision. We conducted comparative experiments on a self-constructed fingertip dataset and several publicly available small-target datasets. Experimental results show that the proposed method can effectively improve the accuracy of fingertip and small-target detection. The detection rate of fingertips reaches 98%, indicating that the proposed in-air handwriting system enables users to write freely and smoothly.
•We proposed a MSCAN to enhance detection accuracy in small target detection.•Incorporating the MSCAN and leveraging monocular vision, we develop an in-air handwriting system.•We gathered in-air handwriting data through the collaboration of volunteers.•Experimental results demonstrate the efficacy of the proposed in-air handwriting system.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Real-time vehicle localization (i.e., position and orientation estimation in the world coordinate system) with high accuracy is the fundamental function of an intelligent vehicle (IV) system. In the ...process of commercialization of IVs, many car manufacturers attempt to avoid high-cost sensor systems (e.g., RTK GNSS and LiDAR) in favor of low-cost optical sensors such as cameras. The same cost-saving strategy also gives rise to an increasing number of vehicles equipped with High Definition (HD) maps. Rooted upon these existing technologies, this article presents the concept of Monocular Localization with Vector HD Map (MLVHM), a novel camera-based map-matching method that efficiently aligns semantic-level geometric features in-camera acquired frames against the vector HD map in order to achieve high-precision vehicle absolute localization with minimal cost. The semantic features are delicately chosen for the ease of map vector alignment as well as for the resiliency against occlusion and fluctuation in illumination. The effective data association method in MLVHM serves as the basis for the camera position estimation by minimizing feature re-projection errors, and the frame-to-frame motion fusion is further introduced for reliable localization results. Experiments have shown that MLVHM can achieve high-precision vehicle localization with an RMSE of 24 cm with no cumulative error. In addition, we use low-cost on-board sensors and light-weight HD maps to achieve or even exceed the accuracy of existing map-matching algorithms.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
This paper proposes a novel multi-sensor-based indoor global localization system integrating visual localization aided by CNN-based image retrieval with a probabilistic localization approach. The ...global localization system consists of three parts: coarse place recognition, fine localization and re-localization from kidnapping. Coarse place recognition exploits a monocular camera to realize the initial localization based on image retrieval, in which off-the-shelf features extracted from a pre-trained Convolutional Neural Network (CNN) are adopted to determine the candidate locations of the robot. In the fine localization, a laser range finder is equipped to estimate the accurate pose of a mobile robot by means of an adaptive Monte Carlo localization, in which the candidate locations obtained by image retrieval are considered as seeds for initial random sampling. Additionally, to address the problem of robot kidnapping, we present a closed-loop localization mechanism to monitor the state of the robot in real time and make adaptive adjustments when the robot is kidnapped. The closed-loop mechanism effectively exploits the correlation of image sequences to realize the re-localization based on Long-Short Term Memory (LSTM) network. Extensive experiments were conducted and the results indicate that the proposed method not only exhibits great improvement on accuracy and speed, but also can recover from localization failures compared to two conventional localization methods.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Autonomous landing of aerial vehicles is challenging, especially in emergency flight scenarios in which precise information about the vehicle and the environment is required for near-to-ground ...maneuvers. In this paper, the optic-flow concept based on feature detection is applied to estimate the vertical distance and the velocity vector of a multirotor UAV (MUAV) for landing. The UAV kinematics, the optical flow equations, and the detected feature states, provided by a low-cost monocular camera, are combined to develop a novel appropriate model for estimation. The proposed algorithm applies the variation of detected features, the angular velocities, as well as the Euler angles, measured by the Inertial Measurement Unit (IMU), to estimate the vertical distance of the UAV to the ground, the MUAV velocity vector, and also to predict the future features position. Extended Kalman filter (EKF) is applied as the estimation method on the coupled optic-flow and kinematic equations. The accuracy of state estimation is enhanced by the idea of multiple-feature tracking. The 6-DOF simulations, laboratory experiments, and comparison of results demonstrate the capability of height and velocity estimation of a MUAV in the landing phase of flight by just applying the low-cost camera information. Monte Carlo simulations have been performed to study the effect of IMU acceleration, and angular velocity measurement noises as well as the number of the detected features on the success probability of the estimation process. The results reveal that increasing the number of detected features, i.e tracking multiple features, increases the estimation accuracy, however, it mainly improves the success probability, which is a more important factor in practical scenarios.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
PDF
