Unmanned aerial vehicles (UAVs) are capable of serving as aerial base stations (BSs) for providing both cost-effective and on-demand wireless communications. This article investigates dynamic ...resource allocation of multiple UAVs enabled communication networks with the goal of maximizing long-term rewards. More particularly, each UAV communicates with a ground user by automatically selecting its communicating user, power level and subchannel without any information exchange among UAVs. To model the dynamics and uncertainty in environments, we formulate the long-term resource allocation problem as a stochastic game for maximizing the expected rewards, where each UAV becomes a learning agent and each resource allocation solution corresponds to an action taken by the UAVs. Afterwards, we develop a multi-agent reinforcement learning (MARL) framework that each agent discovers its best strategy according to its local observations using learning. More specifically, we propose an agent-independent method, for which all agents conduct a decision algorithm independently but share a common structure based on Q-learning. Finally, simulation results reveal that: 1) appropriate parameters for exploitation and exploration are capable of enhancing the performance of the proposed MARL based resource allocation algorithm; 2) the proposed MARL algorithm provides acceptable performance compared to the case with complete information exchanges among UAVs. By doing so, it strikes a good tradeoff between performance gains and information exchange overheads.
Evolutionary multitasking is a recently proposed paradigm to simultaneously solve multiple tasks using a single population. Most of the existing evolutionary multitasking algorithms treat all tasks ...equally and then assign the same amount of resources to each task. However, when the resources are limited, it is difficult for some tasks to converge to acceptable solutions. This paper aims at investigating the resource allocation in the multitasking environment to efficiently utilize the restrictive resources. In this paper, we design a novel multitask evolutionary algorithm with an online dynamic resource allocation strategy. Specifically, the proposed dynamic resource allocation strategy allocates resources to each task adaptively according to the requirements of tasks. We also design an adaptive method to control the resources invested into cross-domain searching. The proposed algorithm is able to allocate the computational resources dynamically according to the computational complexities of tasks. The experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-art algorithms on benchmark problems of multitask optimization.
In the realm of next-generation mobile communication networks, characterized by dynamic and evolving workloads, the efficient resource allocation becomes paramount for achieving optimal performance. ...This paper addresses the intricate challenges of multiagent resource management through the integration of stochastic learning automata and continuous-time optimization techniques, presenting an innovative solution for real-time adaptability and dynamic load balancing. The central aim is to achieve optimal values for local time-varying cost functions while taking into account resource constraints and the feasibility of local allocation. In the context of multi-agent resource allocation systems, where conditions vary over time, the algorithm exhibits a dynamic adaptability, continuously adjusting to the evolving optimal solutions tailored for each agent. Utilizing stochastic learning automata, the algorithm effectively addresses the time-varying load-balancing function in response to dynamic user demands. Furthermore, we introduced an scalable solution for mobility robustness optimization based on deep reinforcement learning (DRL-MRO). This method dynamically identifies network-wide optimal parameter values across the entire network to accommodate diverse mobility patterns, ensuring the effectiveness of load balancing parameters that seamlessly adapt to the dynamic configuration of the network. The overarching goal is to maintain a consistent quality of service level for each agent, thereby enhancing the overall system performance. Simulation outcomes unequivocally affirm the superior performance of the proposed load balancing approach, rooted in continuous-time optimization and stochastic learning automata, surpassing existing schemes across diverse system configurations.
Dynamic resource allocation (DRA) is the key technology to improve the network performance in resource-limited multibeam satellite (MBS) systems. The aim is to find a policy that maximizes the ...expected long-term resource utilization. Existing iterative metaheuristics DRA optimization algorithms are not practical due to the high computational complexity. To solve the problem of unknown dynamics and prohibitive computation, a deep reinforcement learning-based framework (DRLF) is proposed for DRA problems in MBS systems. A novel image-like tensor reformulation on the system environments is adopted to extract traffic spatial and temporal features. A use case of dynamic channel allocation in DRLF is simulated and shows the effectiveness of the proposed DRLF in time-varying scenarios.
For conventional wireless networks, the main target of resource allocation (RA) is to efficiently utilize the available resources. Generally, there are no changes in the available spectrum, thus ...static spectrum allocation policies were adopted. However, these allocation policies lead to spectrum under-utilization. In this regard, cognitive radio networks (CRNs) have received great attention due to their potential to improve the spectrum utilization. In general, efficient spectrum management and resource allocation are essential and very crucial for CRNs. This is due to the fact that unlicensed users should attain the most benefit from accessing the licensed spectrum without causing adverse interference to the licensed ones. The cognitive users or called secondary users have to effectively capture the arising spectrum opportunities in time, frequency, and space to transmit their data. Mainly, two aspects characterize the resource allocation for CRNs: 1) primary (licensed) network protection and 2) secondary (unlicensed) network performance enhancement in terms of quality-of-service, throughput, fairness, energy efficiency, etc. CRNs can operate in one of three known operation modes: 1) interweave; 2) overlay; and 3) underlay. Among which the underlay cognitive radio mode is known to be highly efficient in terms of spectrum utilization. This is because the unlicensed users are allowed to share the same channels with the active licensed users under some conditions. In this paper, we provide a survey for resource allocation in underlay CRNs. In particular, we first define the RA process and its components for underlay CRNs. Second, we provide a taxonomy that categorizes the RA algorithms proposed in literature based on the approaches, criteria, common techniques, and network architecture. Then, the state-of-the-art resource allocation algorithms are reviewed according to the provided taxonomy. Additionally, comparisons among different proposals are provided. Finally, directions for future research are outlined.
Machine-to-machine (M2M) communications, also known as machine-type communications (MTC) in 3GPP LTE systems, provide autonomous connectivity between machines without human intervention to create new ...service, e.g., the Internet of Things and the smart grid. M2M communications normally involve a large number of MTC devices (MTCDs) to support a variety of sensor applications. Consequently, concurrent and massive access attempts of MTCDs to radio access networks (RANs) may cause intolerable delay, packet loss, and even service unavailability. In this paper, we propose a joint optimal physical random access channel (PRACH) resource allocation and access control mechanism to address the performance degradation caused by concurrent and massive access attempts of MTCDs in LTE systems. We define the notion of random access efficiency and formulate an optimization problem for maximization of the random access efficiency with random access delay constraint. We also propose a dynamic resource allocation and access control algorithm based on estimation of the number of MTCDs for a system with dynamically varying numbers of massive MTCDs. Then, an analytical model is provided using a discrete-time Markov chain for the proposed mechanism. The effectiveness of the proposed algorithm is demonstrated via analysis and simulations. The proposed algorithm was able to maintain the optimal random access efficiency while satisfying the average random access delay requirement of MTCDs to handle massive and dynamic MTCDs per cell.
This article investigates the cache-enabling unmanned aerial vehicle (UAV) cellular networks with massive access capability supported by non-orthogonal multiple access (NOMA). The delivery of a large ...volume of multimedia contents for ground users is assisted by a mobile UAV base station, which caches some popular contents for wireless backhaul link traffic offloading. In cache-enabling UAV NOMA networks, the caching placement of content caching phase and radio resource allocation of content delivery phase are crucial for network performance. To cope with the dynamic UAV locations and content requests in practical scenarios, we formulate the long-term caching placement and resource allocation optimization problem for content delivery delay minimization as a Markov decision process (MDP). The UAV acts as an agent to take actions for caching placement and resource allocation, which includes the user scheduling of content requests and the power allocation of NOMA users. In order to tackle the MDP, we propose a Q-learning based caching placement and resource allocation algorithm, where the UAV learns and selects action with soft <inline-formula><tex-math notation="LaTeX">{\varepsilon }</tex-math></inline-formula>-greedy strategy to search for the optimal match between actions and states. Since the action-state table size of Q-learning grows with the number of states in the dynamic networks, we propose a function approximation based algorithm with combination of stochastic gradient descent and deep neural networks, which is suitable for large-scale networks. Finally, the numerical results show that the proposed algorithms provide considerable performance compared to benchmark algorithms, and obtain a trade-off between network performance and calculation complexity.
An optimal model and algorithm for dynamic resource allocation in satellite communication systems is proposed to further promote the system throughput and fairness based on user satisfaction. It ...takes the parameters of application layer, data link layer and physical layer into consideration and constructs a new kind of utility function by using the cross-layer design strategy and converts the resource allocation problem into a utility maximization model. The optimal solution of the model is solved by the combination of the Lagrange multiplier method and the Greedy algorithm, which maximizes the utility and ensures the optimal resources allocation of the satellite communication system in the multiple periods. The simulation results show that the proposed optimal model and algorithm effectively promote the throughput and fairness based on user satisfaction on the premise of ensuring service quality, therefore it can meet the requirements of the variety and the dynamic change of the service in the satellite communication system.
We propose a novel "smart parking" system for an urban environment. The system assigns and reserves an optimal parking space based on the driver's cost function that combines proximity to destination ...and parking cost. Our approach solves a mixed-integer linear programming (MILP) problem at each decision point defined in a time-driven sequence. The solution of each MILP is an optimal allocation based on current state information and is updated at the next decision point with a guarantee that there is no resource reservation conflict and that no driver is ever assigned a resource with a cost function higher than this driver's current cost function value. Based on simulation results, compared with uncontrolled parking processes or state-of-the-art guidance-based systems, our system reduces the average time to find a parking space and the parking cost, whereas the overall parking capacity is more efficiently utilized. We also describe full implementation in a garage to test this system, where a new light system scheme is proposed to guarantee user reservations.
The effective use of limited resources for controlling spreading processes on networks is of prime significance in diverse contexts, ranging from the identification of “influential spreaders” for ...maximizing information dissemination and targeted interventions in regulatory networks, to the development of mitigation policies for infectious diseases and financial contagion in economic systems. Solutions for these optimization tasks that are based purely on topological arguments are not fully satisfactory; in realistic settings, the problem is often characterized by heterogeneous interactions and requires interventions in a dynamic fashion over a finite time window via a restricted set of controllable nodes. The optimal distribution of available resources hence results from an interplay between network topology and spreading dynamics. We show how these problems can be addressed as particular instances of a universal analytical framework based on a scalable dynamic message-passing approach and demonstrate the efficacy of the method on a variety of real-world examples.
PDF
