We study a model of opinion dynamics introduced by Krause: each agent has an opinion represented by a real number, and updates its opinion by averaging all agent opinions that differ from its own by ...less than one. We give a new proof of convergence into clusters of agents, with all agents in the same cluster holding the same opinion. We then introduce a particular notion of equilibrium stability and provide lower bounds on the inter-cluster distances at a stable equilibrium. To better understand the behavior of the system when the number of agents is large, we also introduce and study a variant involving a continuum of agents, obtaining partial convergence results and lower bounds on inter-cluster distances, under some mild assumptions.
We present serial and parallel algorithms for solving a system of equations that arises from the discretization of the Hamilton-Jacobi equation associated to a trajectory optimization problem of the ...following type. A vehicle starts at a prespecified point x/sub o/ and follows a unit speed trajectory x(t) inside a region in /spl Rscr//sup m/ until an unspecified time T that the region is exited. A trajectory minimizing a cost function of the form /spl int//sub 0//sup T/ r(x(t))dt+q(x(T)) is sought. The discretized Hamilton-Jacobi equation corresponding to this problem is usually solved using iterative methods. Nevertheless, assuming that the function r is positive, we are able to exploit the problem structure and develop one-pass algorithms for the discretized problem. The first algorithm resembles Dijkstra's shortest path algorithm and runs in time O(n log n), where n is the number of grid points. The second algorithm uses a somewhat different discretization and borrows some ideas from a variation of Dial's shortest path algorithm (1969) that we develop here; it runs in time O(n), which is the best possible, under some fairly mild assumptions. Finally, we show that the latter algorithm can be efficiently parallelized: for two-dimensional problems and with p processors, its running time becomes O(n/p), provided that p=O(/spl radic/n/log n).< >
We consider continuous-time consensus seeking systems whose time-dependent interactions are cut-balanced, in the following sense: if a group of agents influences the remaining ones, the former group ...is also influenced by the remaining ones by at least a proportional amount. Models involving symmetric interconnections and models in which a weighted average of the agent values is conserved are special cases. We prove that such systems always converge. We give a sufficient condition on the evolving interaction topology for the limit values of two agents to be the same. Conversely, we show that if our condition is not satisfied, then these limits are generically different. These results allow treating systems where the agent interactions are a priori unknown, being for example random or determined endogenously by the agent values.
This paper proposes a simulation-based algorithm for optimizing the average reward in a finite-state Markov reward process that depends on a set of parameters. As a special case, the method applies ...to Markov decision processes where optimization takes place within a parametrized set of policies. The algorithm relies on the regenerative structure of finite-state Markov processes, involves the simulation of a single sample path, and can be implemented online. A convergence result (with probability 1) is provided.
We discuss the temporal-difference learning algorithm, as applied to approximating the cost-to-go function of an infinite-horizon discounted Markov chain. The algorithm we analyze updates parameters ...of a linear function approximator online during a single endless trajectory of an irreducible aperiodic Markov chain with a finite or infinite state space. We present a proof of convergence (with probability one), a characterization of the limit of convergence, and a bound on the resulting approximation error. Furthermore, our analysis is based on a new line of reasoning that provides new intuition about the dynamics of temporal-difference learning. In addition to proving new and stronger positive results than those previously available, we identify the significance of online updating and potential hazards associated with the use of nonlinear function approximators. First, we prove that divergence may occur when updates are not based on trajectories of the Markov chain. This fact reconciles positive and negative results that have been discussed in the literature, regarding the soundness of temporal-difference learning. Second, we present an example illustrating the possibility of divergence when temporal difference learning is used in the presence of a nonlinear function approximator.
Hardness of Low Delay Network Scheduling Shah, D.; Tse, D. N. C.; Tsitsiklis, J. N.
IEEE transactions on information theory,
12/2011, Letnik:
57, Številka:
12
Journal Article
Recenzirano
Odprti dostop
We consider a communication network and study the problem of designing a high-throughput and low-delay scheduling policy that only requires a polynomial amount of computation at each time step. The ...well-known maximum weight scheduling policy, proposed by Tassiulas and Ephremides (1992), has favorable performance in terms of throughput and delay but, for general networks, it can be computationally very expensive. A related randomized policy proposed by Tassiulas (1998) provides maximal throughput with only a small amount of computation per step, but seems to induce exponentially large average delay. These considerations raise some natural questions. Is it possible to design a policy with low complexity, high throughput, and low delay for a general network? Does Tassiulas' randomized policy result in low average delay? In this paper, we answer both of these questions negatively. We consider a wireless network operating under two alternative interference models: (a) a combinatorial model involving independent set constraints and (b) the standard SINR (signal to interference noise ratio) model. We show that unless NP ⊆ BPP (or P = NP for the case of determistic arrivals and deterministic policies), and even if the required throughput is a very small fraction of the network's capacity, there does not exist a low-delay policy whose computation per time step scales polynomially with the number of queues. In particular, the average delay of Tassiulas' randomized algorithm must grow super-polynomially. To establish our results, we employ a clever graph transformation introduced by Lund and Yannakakis (1994).
We provide an example proving that there exists no quadratic Lyapunov function for a certain class of linear agreement/consensus algorithms, a fact that had been numerically verified in . We also ...briefly discuss sufficient conditions for the existence of such a Lyapunov function.
We consider a resource allocation problem where individual users wish to send data across a network to maximize their utility, and a cost is incurred at each link that depends on the total rate sent ...through the link. It is known that as long as users do not anticipate the effect of their actions on prices, a simple proportional pricing mechanism can maximize the sum of users' utilities minus the cost (called aggregate surplus). Continuing previous efforts to quantify the effects of selfish behavior in network pricing mechanisms, we consider the possibility that users anticipate the effect of their actions on link prices. Under the assumption that the links' marginal cost functions are convex, we establish existence of a Nash equilibrium. We show that the aggregate surplus at a Nash equilibrium is no worse than a factor of 4/spl radic/2-5 times the optimal aggregate surplus; thus, the efficiency loss when users are selfish is no more than approximately 34%.
We consider a service provider (SP) who provides access to a communication network or some other form of on-line services. Users initiate calls that belong to a set of diverse service classes, ...differing in resource requirements, demand pattern, and call duration. The SP charges a fee per call, which can depend on the current congestion level, and which affects users' demand for calls. We provide a dynamic programming formulation of the problems of revenue and welfare maximization, and derive some qualitative properties of the optimal solution. We also provide a number of approximate approaches, together with an analysis that indicates that near-optimality is obtained for the case of many, relatively small, users. In particular, we show analytically as well as computationally, that the performance of an optimal pricing strategy is closely matched by a suitably chosen static price, which does not depend on instantaneous congestion. This indicates that the easily implementable time-of-day pricing will often suffice. Throughout, we compare the alternative formulations involving revenue or welfare maximization, respectively, and draw some qualitative conclusions.
We consider a multiarmed bandit problem where the expected reward of each arm is a linear function of an unknown scalar with a prior distribution. The objective is to choose a sequence of arms that ...maximizes the expected total (or discounted total) reward. We demonstrate the effectiveness of a greedy policy that takes advantage of the known statistical correlation structure among the arms. In the infinite horizon discounted reward setting, we show that the greedy and optimal policies eventually coincide, and both settle on the best arm. This is in contrast with the Incomplete Learning Theorem for the case of independent arms. In the total reward setting, we show that the cumulative Bayes risk after T periods under the greedy policy is at most O (log T ), which is smaller than the lower bound of ¿(log 2 T ) established by Lai for a general, but different, class of bandit problems. We also establish the tightness of our bounds. Theoretical and numerical results show that the performance of our policy scales independently of the number of arms.
PDF
