Graph neural networks (GNNs) have been widely used to learn vector representation of graph-structured data and achieved better task performance than conventional methods. The foundation of GNNs is ...the message passing procedure, which propagates the information in a node to its neighbors. Since this procedure proceeds one step per layer, the range of the information propagation among nodes is small in the lower layers, and it expands toward the higher layers. Therefore, a GNN model has to be deep enough to capture global structural information in a graph. On the other hand, it is known that deep GNN models suffer from performance degradation because they lose nodes’ local information, which would be essential for good model performance, through many message passing steps. In this study, we propose multi-level attention pooling (MLAP) for graph-level classification tasks, which can adapt to both local and global structural information in a graph. It has an attention pooling layer for each message passing step and computes the final graph representation by unifying the layer-wise graph representations. The MLAP architecture allows models to utilize the structural information of graphs with multiple levels of localities because it preserves layer-wise information before losing them due to oversmoothing. Results of our experiments show that the MLAP architecture improves the graph classification performance compared to the baseline architectures. In addition, analyses on the layer-wise graph representations suggest that aggregating information from multiple levels of localities indeed has the potential to improve the discriminability of learned graph representations.
This paper proposes a data-driven method based on signal convolution pooling for real-time fault diagnosis in T-type inverters. The model is composed of an auxiliary neural network and a multilayer ...convolution feature classifier (MCFC). The auxiliary neural network can learn and provide filter parameters for an MCFC by learning from a small training dataset. Through shared filter learning and a global average pooling layer, a feedforward MCFC can greatly reduce testing time. This makes the approach suitable for real-time fault diagnosis. A feature processing function is used to retain fault features observed in measured three-phase current signals while avoiding effects of load changes. A multi-signal sequence reconstruction strategy is proposed to transform multiple time-series diagnostic signals into an input feature map for the MCFC. This strategy extends the domain of the MCFC information by increasing the input channel count of the auxiliary neural network. The combined approach increases fault diagnosis accuracy compared to prior work. The performance of the proposed diagnosis method is validated with experiments.
Road extraction from high‐resolution remote sensing images (HRSIs) has great importance in various practical applications. However, most existing road extraction methods have considerable limitation ...in capturing long‐range shape feature of road, and thus, they are ineffective in extracting road region under complex scenes. To address this issue, a novel model called long‐range context‐aware road extraction neural network (LR‐RoadNet) is proposed. LR‐RoadNet takes advantage of strip pooling to capture long‐range context from horizontal and vertical directions, aiming to improve continuity and completeness of road extraction results. Specifically, the LR‐RoadNet consists of two parts: strip residual module (SRM) and strip pyramid pooling module (SPPM). The SRM is built based on residual unit, in which the strip pooling is employed to learn general and long‐range road feature from input image. Then, the SPPM is used to obtain long‐range feature from multiple scales by multiple parallel strip pooling operations. More importantly, a structural similarity (SS) loss function is introduced to further explore road structure for optimizing LR‐RoadNet. The experimental results show that the proposed method achieves great improvement than other state‐of‐the‐art methods on three challenging datasets, Cheng‐Roads, Zimbabwe‐Roads and Mass‐Roads.
Global pooling is one of the most significant operations in many machine learning models and tasks, which works for information fusion and structured data (like sets and graphs) representation. ...However, without solid mathematical fundamentals, its practical implementations often depend on empirical mechanisms and thus lead to sub-optimal, even unsatisfactory performance. In this work, we develop a novel and generalized global pooling framework through the lens of optimal transport. The proposed framework is interpretable from the perspective of expectation-maximization. Essentially, it aims at learning an optimal transport across sample indices and feature dimensions, making the corresponding pooling operation maximize the conditional expectation of input data. We demonstrate that most existing pooling methods are equivalent to solving a regularized optimal transport (ROT) problem with different specializations, and more sophisticated pooling operations can be implemented by hierarchically solving multiple ROT problems. Making the parameters of the ROT problem learnable, we develop a family of regularized optimal transport pooling (ROTP) layers. We implement the ROTP layers as a new kind of deep implicit layer. Their model architectures correspond to different optimization algorithms. We test our ROTP layers in several representative set-level machine learning scenarios, including multi-instance learning (MIL), graph classification, graph set representation, and image classification. Experimental results show that applying our ROTP layers can reduce the difficulty of the design and selection of global pooling - our ROTP layers may either imitate some existing global pooling methods or lead to some new pooling layers fitting data better.
Variation in oxidation of ascorbic acid can be well characterized by multilevel modeling.
Display omitted
•Multilevel modeling allows to characterize variability at various levels.•Averaging of ...results should be avoided because of loss of information.•Experimental variability is a source of information, not a nuisance.•Oxidation of ascorbic acid leads to highly variable experimental results.•Bayesian regression allows to estimate the order of reaction.
Characterization of variation of experimental results is achieved by repeating experiments. Frequently, results are averaged before data are analysed but that may not be the best practice because valuable information is then lost. Three other ways to analyze repetitions are: (1) each experiment is analyzed on its own (no pooling of data), (2) all experiments are analyzed together in one go (complete pooling), (3) data are analyzed together while allowing for similarities as well as differences in the result (partial pooling). Multilevel modeling uses partial pooling by partitioning variance over more than one level. Level 1 consists of the measurements themselves; higher levels consist of groups or clusters of measurements (repetitions, experiments at various temperatures, at various pH values, etc.) and parameters are analyzed both at the population and at the group/cluster level.
The approach is applied to a case study in which heat-induced isothermal degradation of ascorbic acid was studied with 15 repetitions in an aqueous solution, making it a two-level study. The data were analyzed using averaging and complete pooling, complete pooling without averaging, no-pooling at all, and partial pooling. The kinetic model was established by letting the data decide about the order of the reaction, while this was compared to a model where the order was fixed at 1 (first-order model). Results show that both averaging with complete pooling, as well as complete pooling without averaging, strongly underestimate variation. The no-pooling technique overestimates variation, while partial pooling partitions variation over the levels and thus gives a better impression of the variation involved. The kinetics of ascorbic acid appear to be subject to strong variation when each experiment is considered separately because it is a compound that is very sensitive to all kinds of experimental conditions. With multilevel modeling it appeared to be possible to characterize the uncertainties involved much better than with single level modeling. A Bayesian analysis was performed, in which parameters are allowed to be variable, which is useful because multilevel modeling leads to characterization of variation of parameters. The Bayesian method allows to visualize the posterior distribution of parameters, thereby giving more insight in their behaviour. Also, a Bayesian analysis focuses more strongly on predictive accuracy of models, including multilevel models. The predictive accuracy of 4 models describing the same ascorbic acid data was compared and the multilevel model with reaction order estimated from the data performed by far the best in this regard. The pros and cons of multilevel modeling are discussed and it is concluded that multilevel modeling is to be preferred whenever the data allow to perform such an analysis.
Recent years have witnessed the emergence and flourishing of hierarchical graph pooling neural networks (HGPNNs) which are effective graph representation learning approaches for graph level tasks ...such as graph classification. However, current HGPNNs do not take full advantage of the graph’s intrinsic structures (e.g., community structure). Moreover, the pooling operations in existing HGPNNs are difficult to be interpreted. In this paper, we propose a new interpretable graph pooling framework — CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph representation learning process. Specifically, the proposed community pooling mechanism in CommPOOL utilizes an unsupervised approach for capturing the inherent community structure of graphs in an interpretable manner. CommPOOL is a general and flexible framework for hierarchical graph representation learning that can further facilitate various graph-level tasks. Evaluations on five public benchmark datasets and one synthetic dataset demonstrate the superior performance of CommPOOL in graph representation learning for graph classification compared to the state-of-the-art baseline methods, and its effectiveness in capturing and preserving the community structure of graphs.
•1. Introduce average vehicle occupancy ratio into cost calculation to represent more realistic aspects of ridesharing costs subdued by ridesharing drivers and passengers.•Formulate a link-node ...complementarity representation for ridesharing user equilibrium.•Model the presence of HOT lanes.•Employ multi-start strategies to provide a sufficiently good initial solution to our network design problem.
Though the conventional network design is extensively studied, the network design problem for ridesharing, in particular, the deployment of high-occupancy toll (HOT) lanes, remains understudied. This paper focuses on one type of network design problem as to whether existing roads should be retrofit into HOT lanes. It is a continuous bi-level mathematical program with equilibrium constraints. The lower level problem is ridesharing user equilibrium (RUE). To reduce the problem size and facilitate computation, we reformulate RUE in the link-node representation. Then we extend the RUE framework to accommodate the presence of HOT lanes and tolls. Algorithms are briefly discussed and numerical examples are illustrated on the Braess network and the Sioux Falls network, respectively. Results show that carefully selecting the deployment of HOT lanes can improve the overall system travel time.
We investigate pooling problems in which multiple players vie with one another to maximize individual profit in a non-cooperative competitive market. This competitive setting is interesting and ...worthy of study because the majority of prevailing process systems engineering models largely overlook the non-cooperative strategies that exist in real-world markets. In this work, each player controls a processing network involving intermediate tanks (or pools) where raw materials are blended together before being further combined into final products. Each player then solves a pure or mixed-integer bilinear optimization problem whose profit is influenced by other players. We present several bilevel formulations and numerical results of a novel decomposition algorithm.
•Game-theoretic paradigm for studying interaction of noncooperative nonconvex agents.•Bilevel feasibility problems with lower-level nonconvex structures.•Provably optimal bilevel decomposition method to minimize disequilibrium is applied.•Minimum disequilibrium yields certificate of nonexistence if no equilibrium exists.•Traditional deterministic, perfect competition, & Nash–Cournot models are compared.
•Overarching framework that covers non-shared taxi, paratransit, and ridesharing.•Systematic quantification of cost and performance in approximate closed-form formulas.•Insights that could be used to ...explore operating, pricing, and regulatory strategies.•Analytical results are corroborated by agent-based simulations.
The paper presents a general analytic framework to model transit systems that provide door-to-door service. The model includes as special cases non-shared taxi and demand responsive transportation (DRT). In the latter we include both, paratransit services such as dial-a-ride (DAR), and the form of ridesharing (shared taxi) currently being used by crowd-sourced taxi companies like Lyft and Uber. The framework yields somewhat optimistic results because, among other things, it is deterministic and does not track vehicles across space. By virtue of its simplicity, however, the framework yields approximate closed form formulas for many cases of interest.
PDF
