Pre-training Graph Neural Networks (GNN) via self-supervised contrastive learning has recently drawn lots of attention. However, most existing works focus on node-level contrastive learning, which ...cannot capture global graph structure. The key challenge to conduct subgraph-level contrastive learning is to sample informative subgraphs that are semantically meaningful. To solve it, we propose to learn graph motifs, which are frequently-occurring subgraph patterns (e.g. functional groups of molecules), for better subgraph sampling. Our framework M ot I f-driven C ontrastive lea R ning O f G raph representations ( MICRO-Graph ) can: 1) use GNNs to extract motifs from large graph datasets; 2) leverage learned motifs to sample informative subgraphs for contrastive learning of GNN. We formulate motif learning as a differentiable clustering problem, and adopt EM-clustering to group similar and significant subgraphs into several motifs. Guided by these learned motifs, a sampler is trained to generate more informative subgraphs, and these subgraphs are used to train GNNs through graph-to-subgraph contrastive learning. By pre-training on the ogbg-molhiv dataset with MICRO-Graph , the pre-trained GNN achieves 2.04<inline-formula><tex-math notation="LaTeX">\%</tex-math> <mml:math><mml:mo>%</mml:mo></mml:math><inline-graphic xlink:href="zhang-ieq1-3364059.gif"/> </inline-formula> ROC-AUC average performance enhancement on various downstream benchmark datasets, which is significantly higher than other state-of-the-art self-supervised learning baselines.
Heterogeneous Graph Transformer Hu, Ziniu; Dong, Yuxiao; Wang, Kuansan ...
Proceedings of The Web Conference 2020,
04/2020
Conference Proceeding
Open access
Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous graphs, in which all nodes and edges ...belong to the same types, making it infeasible to represent heterogeneous structures. In this paper, we present the Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs. To model heterogeneity, we design node- and edge-type dependent parameters to characterize the heterogeneous attention over each edge, empowering HGT to maintain dedicated representations for different types of nodes and edges. To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm—HGSampling—for efficient and scalable training. Extensive experiments on the Open Academic Graph of 179 million nodes and 2 billion edges show that the proposed HGT model consistently outperforms all the state-of-the-art GNN baselines by 9–21 on various downstream tasks. The dataset and source code of HGT are publicly available at https://github.com/acbull/pyHGT.
For the sake of safety, the vehicle path tracking control should not only ensure the stability of the path tracking error containing the lateral offset and the orientation error but also guarantee ...that both the transient and steady states of the lateral offset are within a specified safe boundary. However, the time‐varying uncertainties of a vehicle system make the control design a tough task. This paper develops an adaptive robust control (ARC) which guarantees both the tracking stability and the bounded error property for autonomous vehicles. First, to handle the bounded error requirement, a barrier function based state transformation which converts the constrained lateral offset into an unconstrained state is proposed. Then, the path tracking control task is cast into an equality constraint of the system state. On this basis, a novel adaptive robust constraint‐following controller is developed to make the transformed system follow the proposed equality constraint. Through Lyapunov minimax analysis, it is proved that the resulting control guarantees the approximate constraint‐following performance and the bounded error property despite the presence of system uncertainties. Finally, the main theoretical results are verified through CarSim‐Simulink co‐simulation results.
Full text
Available for:
FZAB, GIS, IJS, KILJ, NLZOH, NUK, OILJ, SAZU, SBCE, SBMB, UL, UM, UPUK
Short-term traffic flow forecasting, an important component of intelligent transportation systems (ITS), is a challenging research direction as forecasting itself is affected by a series of complex ...factors. As more and more attention is paid to the data itself, deep learning methods have attained mainstream popularity for accomplishing traffic flow prediction tasks. In recent years, the attention mechanism has been widely used in various fields thanks to its excellent result interpretation ability and its capability to improve the performance of neural network models. In terms of time series data prediction, LSTM has demonstrated its powerful time feature extraction capability. Because of its ability to efficiently and quickly extract spatial–temporal features, CNN is often used in combination with LSTM and attention mechanisms to obtain accurate traffic flow prediction forecast results. In this paper, we propose a short-term traffic flow prediction model based on self-attention, and test the performance of the model experimentally with real data. The model can achieve the best prediction results compared with other classical models. In addition, the temporal and spatial features extracted by the model have certain physical characteristics making results easier to interpret.
•An accurate traffic flow prediction framework with less training data is proposed.•Model attention is utilized to obtain a more explanatory data correlation matrix.•An easy-to-understand Mask mechanism is designed to capture the time-series data.•Time and space features of traffic flow can be obtained from the proposed model.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
This thesis addresses the intersection of neural and symbolic artificial intelligence systems. Recent deep learning methods could memorize vast amount of world knowledge, but still have their ...limitation to conduct symbolic reasoning over them; while symbolic AI is good at solving reasoning tasks, but is inefficient for adapting to new knowledge. Prior efforts that bridge the two worlds mainly focus on building parsing-based systems, which require lots of annotated intermediate labels and hard to scale.My ultimate research goal is to enable neural model to interact with symbolic reasoning module in a differentiable manner, and train such Neural-Symbolic model end-to-end without intermediate labels. To bring this vision about, I have conducted works on:1. Designing Novel Reasoning Module: design differentiable neural modules that can conduct symbolic reasoning, including knowledge graph reasoning and complex Logical inference.2. Learning via Self-Supervision: train the neural model via self-supervision from structural and symbolic knowledge base without additional annotation.3. Generalizing across Domains: the modular design of neural-symbolic system by its nature help to generalize better for Out-of-Distribution, Out-of-Vocabulary, cross-lingual and cross-type.Putting these pieces together, I am pursuing the ultimate vision to build end-to-end Neural-Symbolic system that has the capacity of reasoning, advancing to true human intelligence.
Traffic signal control is critical for traffic efficiency optimization but is usually constrained by traffic detection methods. The emerging V2I (Vehicle to Infrastructure) technology is capable of ...providing rich information for traffic detection, thus becoming promising for traffic signal control. Based on parallel simulation, this paper presents a new traffic signal optimization method in a V2I environment. In the proposed method, a predictive optimization problem is formulated, and a cellular automata model is employed as traffic flow model. By using genetic algorithm, the predictive optimization problem is solved online to implement receding horizon control. Simulation results show that the proposed method can improve traffic efficiency in the sense of reducing average delay and number of stops. Meanwhile, simulation also shows that greater communication range brings better performance for reducing the average number of stops. Simulation results show that the proposed V2I-based signal control method can improve traffic efficiency, especially when the traffic volume is relatively high. The proposed algorithm can be applied to traffic signal control to improve traffic efficiency.
Sentiment classification typically relies on a large amount of labeled data. In practice, the availability of labels is highly imbalanced among different languages, e.g., more English texts are ...labeled than texts in any other languages, which creates a considerable inequality in the quality of related information services received by users speaking different languages. To tackle this problem, cross-lingual sentiment classification approaches aim to transfer knowledge learned from one language that has abundant labeled examples (i.e., the source language, usually English) to another language with fewer labels (i.e., the target language). The source and the target languages are usually bridged through off-the-shelf machine translation tools. Through such a channel, cross-language sentiment patterns can be successfully learned from English and transferred into the target languages. This approach, however, often fails to capture sentiment knowledge specific to the target language, and thus compromises the accuracy of the downstream classification task. In this paper, we employ emojis, which are widely available in many languages, as a new channel to learn both the cross-language and the language-specific sentiment patterns. We propose a novel representation learning method that uses emoji prediction as an instrument to learn respective sentiment-aware representations for each language. The learned representations are then integrated to facilitate cross-lingual sentiment classification. The proposed method demonstrates state-of-the-art performance on benchmark datasets, which is sustained even when sentiment labels are scarce.
A spatial-dependent robust control strategy is proposed for the on-ramp merging problem based on the coordination of the connected and automated vehicles. In the proposed strategy, the planning stage ...of the merging coordination is weakened while the control stage is strengthened. More specifically, the planning stage mainly forms a virtual platoon containing all vehicles inside the communication zone. In the control stage, the time-varying parameter uncertainties in the model are considered. A spatial-dependent controller with uniform boundedness, uniform ultimate boundedness and robustness is delicately designed for each vehicle in the virtual platoon to analytically calculate the control force in real time. The spatial-dependence means that the safety-related performances of the controller are directly and explicitly bonded with spatial locations such that the collision-avoidance safety is ensured at the most dangerous conflicting merging zone. Since spatial locations in the traffic environment are static, this spatial-dependence endows the proposed strategy more stability and reliability. The effectiveness of the proposed strategy is demonstrated through the verification, ablation and comparison simulation cases.
In this paper, we propose an end-to-end Retrieval-Augmented Visual Language Model (REVEAL) that learns to encode world knowledge into a large-scale memory, and to retrieve from it to answer ...knowledge-intensive queries. Reveal consists of four key components: the memory, the encoder, the retriever and the generator. The large-scale memory encodes various sources of multimodal world knowledge (e.g. image-text pairs, question answering pairs, knowledge graph triplets, etc.) via a unified encoder. The retriever finds the most relevant knowledge entries in the memory, and the generator fuses the retrieved knowledge with the input query to produce the output. A key novelty in our approach is that the memory, encoder, retriever and generator are all pre-trained end-to-end on a massive amount of data. Furthermore, our approach can use a diverse set of multimodal knowledge sources, which is shown to result in significant gains. We show that Reveal achieves state-of-the-art results on visual question answering and image captioning. The project page of this work is reveal. github. io.
This paper proposes a data-driven human-like driver model (HDM) based on the analysis and understanding of human drivers’ behavior in path-tracking tasks. The proposed model contains a visual ...perception module and a decision-making module. The visual perception module was established to extract the visual inputs, including road information and vehicle motion states, which can be perceived by human drivers. The extracted inputs utilized for lateral steering decisions can reflect specific driving skills exhibited by human drivers like compensation control, preview behavior, and anticipation ability. On this basis, an adaptive neuro-fuzzy inference system (ANFIS) was adopted to design the decision-making module. The inputs of the ANFIS include the vehicle speed, lateral deviation in the near zone, and heading angle error in the far zone. The output is the steering wheel angle. ANFIS can mimic the fuzzy reasoning characteristics of human driving behavior. Next, a large amount of human driving data was collected through driving simulator experiments. Based on the data, the HDM was established. Finally, the results of the joint simulation under PreScan/MATLAB verified the superior performances of the proposed HDM.
PDF
