Message from the TPC Co-Chairs: ICDCSW 2022 Yang, Yuanyuan; Ye, Fan
2022 IEEE 42nd International Conference on Distributed Computing Systems Workshops (ICDCSW),
2022-July
Conference Proceeding
Welcome to ICDCS 2022 in Bologna, Italy! It is an exciting moment to return to in-person conference after two years in a row of virtual presentations amid the COVID-19 pandemic. ICDCS is a premier ...venue for research results in distributed computing systems. It is our great pleasure to introduce the ICDCS 2022 program, which consists of excellent keynote talks, high-quality paper presentations, social activities including a reception and a banquet, and many opportunities for researchers around the world to gather and exchange the latest work and ideas in distributed computing.
Today, data analysis drives the decision-making process in virtually every human activity. This demands for software platforms that offer simple programming abstractions to express data analysis ...tasks and that can execute them in an efficient and scalable way. State-of-the-art solutions range from low-level programming primitives, which give control to the developer about communication and resource usage, but require significant effort to develop and optimize new algorithms, to high-level platforms that hide most of the complexities of parallel and distributed processing, but often at the cost of reduced efficiency.
To reconcile these requirements, we developed Renoir, a novel distributed data processing platform written in Rust. Renoir provides a high-level dataflow programming model as mainstream data processing systems. It supports static and streaming data, it enables data transformations, grouping, aggregation, iterative computations, and time-based analytics, and it provides all these features incurring in a low overhead.
In this paper, we present the programming model and the implementation details of Renoir. We evaluate it under heterogeneous workloads. We compare it with state-of-the-art solutions for data analysis and high-performance computing, as well as alternative research products, which offer different programming abstractions and implementation strategies. Renoir programs are compact and easy to write: developers need not care about low-level concerns such as resource usage, data serialization, concurrency control, and communication. At the same time, Renoir consistently presents comparable or better performance than competing solutions, by a large margin in several scenarios.
We conclude that Renoir offers a good tradeoff between simplicity and performance, allowing developers to easily express complex data analysis tasks and achieve high performance and scalability.
•Renoir is a parallel and distributed framework that targets simplicity and performance.•Renoir makes stream processing fast using features of the Rust programming language.•Renoir outperforms state-of-the-art stream processing frameworks.•Renoir can generate fast specialized programs from a generic high-level interface.•Renoir seamlessly runs on multiple machines, while optimizing local communication.
Genome-wide association analysis of cohorts with thousands of phenotypes is computationally expensive, particularly when accounting for sample relatedness or population structure. Here we present a ...novel machine-learning method called REGENIE for fitting a whole-genome regression model for quantitative and binary phenotypes that is substantially faster than alternatives in multi-trait analyses while maintaining statistical efficiency. The method naturally accommodates parallel analysis of multiple phenotypes and requires only local segments of the genotype matrix to be loaded in memory, in contrast to existing alternatives, which must load genome-wide matrices into memory. This results in substantial savings in compute time and memory usage. We introduce a fast, approximate Firth logistic regression test for unbalanced case-control phenotypes. The method is ideally suited to take advantage of distributed computing frameworks. We demonstrate the accuracy and computational benefits of this approach using the UK Biobank dataset with up to 407,746 individuals.
Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the ...proceedings record.
DDoS in the IoT: Mirai and Other Botnets Kolias, Constantinos; Kambourakis, Georgios; Stavrou, Angelos ...
Computer (Long Beach, Calif.),
2017, Letnik:
50, Številka:
7
Journal Article
Recenzirano
The Mirai botnet and its variants and imitators are a wake-up call to the industry to better secure Internet of Things devices or risk exposing the Internet infrastructure to increasingly disruptive ...distributed denial-of-service attacks.
How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a fundamental tradeoff between computation and ...communication in distributed computing, i.e., the two are inversely proportional to each other. More specifically, a general distributed computing framework, motivated by commonly used structures like MapReduce, is considered, where the overall computation is decomposed into computing a set of "Map" and "Reduce" functions distributedly across multiple computing nodes. A coded scheme, named "coded distributed computing" (CDC), is proposed to demonstrate that increasing the computation load of the Map functions by a factor of r (i.e., evaluating each function at r carefully chosen nodes) can create novel coding opportunities that reduce the communication load by the same factor. An information-theoretic lower bound on the communication load is also provided, which matches the communication load achieved by the CDC scheme. As a result, the optimal computation-communication tradeoff in distributed computing is exactly characterized. Finally, the coding techniques of CDC is applied to the Hadoop TeraSort benchmark to develop a novel CodedTeraSort algorithm, which is empirically demonstrated to speed up the overall job execution by 1.97× -3.39×, for typical settings of interest.
Jump Anderson, Robert; Gallup, David; Barron, Jonathan T. ...
ACM transactions on graphics,
11/2016, Letnik:
35, Številka:
6
Journal Article
Recenzirano
Odprti dostop
We present Jump, a practical system for capturing high resolution, omnidirectional stereo (ODS) video suitable for wide scale consumption in currently available virtual reality (VR) headsets. Our ...system consists of a video camera built using off-the-shelf components and a fully automatic stitching pipeline capable of capturing video content in the ODS format. We have discovered and analyzed the distortions inherent to ODS when used for VR display as well as those introduced by our capture method and show that they are small enough to make this approach suitable for capturing a wide variety of scenes. Our stitching algorithm produces robust results by reducing the problem to one of pairwise image interpolation followed by compositing. We introduce novel optical flow and compositing methods designed specifically for this task. Our algorithm is temporally coherent and efficient, is currently running at scale on a distributed computing platform, and is capable of processing hours of footage each day.
PDF
