Shared control schemes allow a human driver to work with an automated driving agent in driver-vehicle systems while retaining the driverʼ s abilities to control. The human driver, as an ...essential agent in the driver-vehicle shared control systems, should be precisely modeled regarding their cognitive processes, control strategies, and decision-making processes. The interactive strategy design between drivers and automated driving agents brings an excellent challenge for human-centric driver assistance systems due to the inherent characteristics of humans. Many open-ended questions arise, such as what proper role of human drivers should act in a shared control scheme? How to make an intelligent decision capable of balancing the benefits of agents in shared control systems? Due to the advent of these attentions and questions, it is desirable to present a survey on the decision making between human drivers and highly automated vehicles, to understand their architectures, human driver modeling, and interaction strategies under the driver-vehicle shared schemes. Finally, we give a further discussion on the key future challenges and opportunities. They are likely to shape new potential research directions.
Intelligent transportation system (ITS) collects numerous data for analysis of the transportation system. The data can be used for providing services for travellers and traffic controllers in the ITS ...and optimizing it, for the purpose of making the transportation more efficient and safer. Due to the wide and flexible employment of video cameras in visual surveillance system (VSS), mature edge-cloud resource scheduling for data transmission and analysis, and the fast development of deep learning, computer vision (CV) methods have been employed in the visual-based ITS services successfully. In this paper, we discuss the edge-cloud surveillance resource scheduling for the CV methods and review the deep learning-based CV methods in the VSS, including detection, classification, and tracking methods, for better understanding of the relationship between the CV-based ITS services and these methods. We experimentally compare several state-of-the-art deep learning-based methods, which have been successfully applied in the CV fields under the ITS scenario, on their performance, inference speed, computational quantity, and model size. According to the comparisons, we propose four main challenges of the deep learning-based CV methods applied in the services, as a discussion of the future research directions. Code are available at https://github.com/PRIS-CV/DL-CV-ITS .
Lane detection is a fundamental aspect of most current advanced driver assistance systems ( ADASs ). A large number of existing results focus on the study of vision-based lane detection ...methods due to the extensive knowledge background and the low-cost of camera devices. In this paper, previous vision-based lane detection studies are reviewed in terms of three aspects, which are lane detection algorithms, integration, and evaluation methods. Next, considering the inevitable limitations that exist in the camera-based lane detection system, the system integration methodologies for constructing more robust detection systems are reviewed and analyzed. The integration methods are further divided into three levels, namely, algorithm, system, and sensor. Algorithm level combines different lane detection algorithms while system level integrates other object detection systems to comprehensively detect lane positions. Sensor level uses multi-modal sensors to build a robust lane recognition system. In view of the complexity of evaluating the detection system, and the lack of common evaluation procedure and uniform metrics in past studies, the existing evaluation methods and metrics are analyzed and classified to propose a better evaluation of the lane detection system. Next, a comparison of representative studies is performed. Finally, a discussion on the limitations of current lane detection systems and the future developing trends toward an Artificial Society, Computational experiment-based parallel lane detection framework is proposed.
Autonomous driving is a promising technology to reduce traffic accidents and improve driving efficiency. In this work, a deep reinforcement learning (DRL)-enabled decision-making policy is ...constructed for autonomous vehicles to address the overtaking behaviors on the highway. First, a highway driving environment is founded, wherein the ego vehicle aims to pass through the surrounding vehicles with an efficient and safe maneuver. A hierarchical control framework is presented to control these vehicles, which indicates the upper-level manages the driving decisions, and the lower-level cares about the supervision of vehicle speed and acceleration. Then, the particular DRL method named dueling deep Q-network (DDQN) algorithm is applied to derive the highway decision-making strategy. The exhaustive calculative procedures of deep Q-network and DDQN algorithms are discussed and compared. Finally, a series of estimation simulation experiments are conducted to evaluate the effectiveness of the proposed highway decision-making policy. The advantages of the proposed framework in convergence rate and control performance are illuminated. Simulation results reveal that the DDQN-based overtaking policy could accomplish highway driving tasks efficiently and safely.
As the main component of an autonomous driving system, the motion planner plays an essential role for safe and efficient driving. However, traditional motion planners cannot make full use of the ...on-board sensing information and lack the ability to efficiently adapt to different driving scenes and behaviors of different drivers. To overcome this limitation, a personalized behavior learning system (PBLS) is proposed in this paper to improve the performance of the traditional motion planner. This system is based on the neural reinforcement learning (NRL) technique, which can learn from human drivers online based on the on-board sensing information and realize human-like longitudinal speed control (LSC) through the learning from demonstration (LFD) paradigm. Under the LFD framework, the desired speed of human drivers can be learned by PBLS and converted to the low-level control commands by a proportion integration differentiation (PID) controller. Experiments using driving simulator and real driving data show that PBLS can adapt to different drivers by reproducing their driving behaviors for LSC in different scenes. Moreover, through a comparative experiment with the traditional adaptive cruise control (ACC) system, the proposed PBLS demonstrates a superior performance in maintaining driving comfort and smoothness.
To reduce the discrepancy between the source and target domains, a new multi-label adaptation network (ML-ANet) based on multiple kernel variants with maximum mean discrepancies is proposed in this ...paper. The hidden representations of the task-specific layers in ML-ANet are embedded in the reproducing kernel Hilbert space (RKHS) so that the mean-embeddings of specific features in different domains could be precisely matched. Multiple kernel functions are used to improve feature distribution efficiency for explicit mean embedding matching, which can further reduce domain discrepancy. Adverse weather and cross-camera adaptation examinations are conducted to verify the effectiveness of our proposed ML-ANet. The results show that our proposed ML-ANet achieves higher accuracies than the compared state-of-the-art methods for multi-label image classification in both the adverse weather adaptation and cross-camera adaptation experiments. These results indicate that ML-ANet can alleviate the reliance on fully labeled training data and improve the accuracy of multi-label image classification in various domain shift scenarios.
Driving cycles and road slope are two important factors affecting fuel saving performance of plug-in hybrid electric buses (PHEBs) in Chinese cities. Moreover, onboard auxiliary equipment (e.g., ...Global Position System receiver and General Packet Radio Service (GPRS) wireless module) of PHEB may provide potential means to communicate with the control center of the bus company, allowing for driving cycle prediction through data communication between foregoing buses and the control center. With this general approach in mind, and by utilizing driving data clustering and driving cycle classifier, this paper presents a multimode switched logic control strategy, targeting fuel economy improvement of the PHEB team for a particular city bus route. First, the normal feature parameters are extracted from the sampled driving history cycles, and the composed feature parameters are given by a mapping of normal feature parameters in this approach. A novel improved hierarchical clustering algorithm is applied for driving cycles' data clustering into four groups. Then, on the basis of the clustering results, support vector machine method is used to predict the current driving cycle. Finally, a switched driving controller is presented according to current type of driving cycle and slope information. Simulation results are compared with those of traditional methods in the given real-world driving cycles of city bus, showing significant improvement, which may offer a theoretical solution with engineering application. Experimental results also demonstrate that the proposed control approach is feasible in the tested bus routes.
The cooperation between an autonomous vehicle and a nearby vehicle is critical to ensure driving safety in the lane-exchanging scenario. The nearby vehicle trajectory needs to be predicted, from ...which the autonomous vehicle is controlled to prevent possible collisions. This paper proposes a lane-exchanging driving strategy for the autonomous vehicle to cooperate with the nearby vehicle by integrating vehicle trajectory prediction and motion control. A trajectory prediction method is developed to anticipate the nearby vehicle trajectory. The Gaussian mixture model (GMM), together with the vehicle kinematic model, are synthesized to predict the nearby vehicle trajectory. A potential-field-based model predictive control (MPC) approach is utilized by the autonomous vehicle to conduct the lane-exchanging maneuver. The potential field of the nearby vehicle is considered in the controller design for collision avoidance. On-road driving data verification shows that the nearby vehicle trajectory can be predicted by the proposed method. CarSim
®
simulations validate that the autonomous vehicle can perform the lane-exchanging maneuver and avoid the nearby vehicle using the proposed driving strategy. The autonomous vehicle can thus safely perform the lane-exchanging maneuver and avoid the nearby vehicle.
Driver decisions and behaviors regarding the surrounding traffic are critical to traffic safety. It is important for an intelligent vehicle to understand driver behavior and assist in driving tasks ...according to their status. In this paper, the consumer range camera Kinect is used to monitor drivers and identify driving tasks in a real vehicle. Specifically, seven common tasks performed by multiple drivers during driving are identified in this paper. The tasks include normal driving, left-, right-, and rear-mirror checking, mobile phone answering, texting using a mobile phone with one or both hands, and the setup of in-vehicle video devices. The first four tasks are considered safe driving tasks, while the other three tasks are regarded as dangerous and distracting tasks. The driver behavior signals collected from the Kinect consist of a color and depth image of the driver inside the vehicle cabin. In addition, 3-D head rotation angles and the upper body (hand and arm at both sides) joint positions are recorded. Then, the importance of these features for behavior recognition is evaluated using random forests and maximal information coefficient methods. Next, a feedforward neural network (FFNN) is used to identify the seven tasks. Finally, the model performance for task recognition is evaluated with different features (body only, head only, and combined). The final detection result for the seven driving tasks among five participants achieved an average of greater than 80% accuracy, and the FFNN tasks detector is proved to be an efficient model that can be implemented for real-time driver distraction and dangerous behavior recognition.
Motion planning is one of the most significant technologies for autonomous driving. To make motion planning models able to learn from the environment and to deal with emergency situations, a new ...motion planning framework called as “ parallel planning ” is proposed in this paper. In order to generate sufficient and various training samples, artificial traffic scenes are firstly constructed based on the knowledge from the reality. A deep planning model which combines a convolutional neural network ( CNN ) with the Long Short-Term Memory module ( LSTM ) is developed to make planning decisions in an end-to-end mode. This model can learn from both real and artificial traffic scenes and imitate the driving style of human drivers. Moreover, a parallel deep reinforcement learning approach is also presented to improve the robustness of planning model and reduce the error rate. To handle emergency situations, a hybrid generative model including a variational auto-encoder ( VAE ) and a generative adversarial network ( GAN ) is utilized to learn from virtual emergencies generated in artificial traffic scenes. While an autonomous vehicle is moving, the hybrid generative model generates multiple video clips in parallel, which correspond to different potential emergency scenarios. Simultaneously, the deep planning model makes planning decisions for both virtual and current real scenes. The final planning decision is determined by analysis of real observations. Leveraging the parallel planning approach, the planner is able to make rational decisions without heavy calculation burden when an emergency occurs.
PDF
