Cloud-based computing systems can get oversubscribed due to the budget constraints of their users or limitations in certain resource types. The oversubscription can, in turn, degrade the users ...perceived Quality of Service (QoS). The approach we investigate to mitigate both the oversubscription and the incurred cost is based on smart reusing of the computation needed to process the service requests (i.e., tasks). We propose a reusing paradigm for the tasks that are waiting for execution. This paradigm can be particularly impactful in serverless platforms where multiple users can request similar services simultaneously. Our motivation is a multimedia streaming engine that processes the media segments in an on-demand manner. We propose a mechanism to identify various types of "mergeable" tasks and aggregate them to improve the QoS and mitigate the incurred cost. We develop novel approaches to determine when and how to perform task aggregation such that the QoS of other tasks is not affected. Evaluation results show that the proposed mechanism can improve the QoS by significantly reducing the percentage of tasks missing their deadlines and reduce the overall time (and subsequently the incurred cost) of utilizing cloud services by more than 9 percent.
Summary
Along with the rise of domain‐specific computing (ASICs hardware) and domain‐specific programming languages, we envision that the next step is the emergence of domain‐specific cloud ...platforms. Considering multimedia streaming as one of the most trendy applications in the IT industry, the goal of this study is to develop serverless multimedia streaming engine (SMSE), the first domain‐specific serverless platform for multimedia streaming. SMSE democratizes multimedia service development via enabling content providers (or even end‐users) to rapidly develop their desired functionalities on their multimedia contents. Upon developing SMSE, the next goal of this study is to deal with its efficiency challenges and develop a function container provisioning method that can efficiently utilize cloud resources and improve the users' quality of service. In particular, we develop a dynamic method that provisions durable or ephemeral containers depending on the spatiotemporal and data‐dependency characteristics of the functions. Evaluating the prototype implementation of SMSE under real‐world settings demonstrates its capability to reduce both the containerization overhead, and the makespan time of serving multimedia processing functions (by up to 30%) in compare to the function provision methods that are being used in the general‐purpose serverless cloud systems.
Summary
Serverless computing along with Function‐as‐a‐Service (FaaS) is forming a new computing paradigm that is anticipated to found the next generation of cloud systems. The popularity of this ...paradigm is due to offering a highly transparent infrastructure that enables user applications to scale in the granularity of their functions. Since these often small and single‐purpose functions are managed on shared computing resources behind the scene, a great potential for computational reuse and approximate computing emerges that if unleashed, can remarkably improve the efficiency of serverless cloud systems—both from the user's QoS and system's (energy consumption and incurred cost) perspectives. Accordingly, the goal of this survey study is to, first, unfold the internal mechanics of serverless computing and, second, explore the scope for efficiency within this paradigm via studying function reuse and approximation approaches and discussing the pros and cons of each one. Next, we outline potential future research directions within this paradigm that can either unlock new use cases or make the paradigm more efficient.
In heterogeneous distributed computing (HC) systems, diversity can exist in both computational resources and arriving tasks. In an inconsistently heterogeneous computing system, task types have ...different execution times on heterogeneous machines. A method is required to map arriving tasks to machines based on machine availability and performance, maximizing the number of tasks meeting deadlines (defined as robustness). For tasks with hard deadlines (e.g., those in live video streaming), tasks that miss their deadlines are dropped. The problem investigated in this research is maximizing the robustness of an oversubscribed HC system. A way to maximize this robustness is to prune (i.e., defer or drop) tasks with low probability of meeting their deadlines to increase the probability of other tasks meeting their deadlines. In this paper, we first provide a mathematical model to estimate a task’s probability of meeting its deadline in the presence of task dropping. We then investigate methods for engaging probabilistic dropping. We propose methods to dynamically determine task dropping and deferring threshold probabilities. Next, we develop a pruning system and a pruning-aware mapping heuristic, which we extend to engender fairness across various task types. We present the pruning mechanism as an independent component that can be applied to any mapping heuristic to improve the system robustness. To reduce overhead of the pruning mechanism, we propose approximation methods that remarkably reduce the number of mathematical calculations and improve the practicality of deploying the mechanism in heterogeneous or even homogeneous computing systems. We show the cost and energy gains of the pruning mechanism. Simulation results, harnessing a selection of mapping heuristics, show efficacy of the pruning mechanism in improving robustness (on average by ≃22%) and cost in an oversubscribed HC system by up to ≃33%.
•Mathematically modeling impact of task dropping on the probability of other tasks.•Developing a method to determine probabilistic dropping and deferring thresholds.•Proposing a method to engage task dropping in response to oversubscription.•Developing a pruning-aware and a fairness-aware mapping heuristic for an HC system.•Developing a generic pruning mechanism that can be applied to existing HC systems.
Cloud-based serverless computing systems, either public or privately provisioned, aim to provide the illusion of infinite resources and abstract users from details of the allocation decisions.With ...the goal of providing a low cost and a high QoS, the serverless computing paradigm offers opportunities that can be harnessed to attain the goals. Specifically, our strategy in this dissertation is to avoid redundant computing, in cases where independent task requests are similar to each other and for tasks that are pointless to process. We explore two main approaches to (A) reuse part of computation needed to process the services and (B) proactively pruning tasks with a low chance of success to improve the overall QoS of the system.For the first approach, we propose a mechanism to identify various types of "mergeable'' tasks, which can benefit from computational reuse if they are executed together as a group. To evaluate the task merging configurations extensively, we quantify the resource-saving magnitude and then leveraging the experimental data to create a resource-saving predictor.We investigate multiple tasks merging approaches that suit different workload scenarios to determine when it is appropriate to aggregate tasks and how to allocate them so that the QoS of other tasks is minimally affected.For the second approach, we developed the mechanisms to skip tasks whose chance of completing on time is not worth pursuing by drop or defer them. We determined the minimum chance of success thresholds for tasks to pass to get scheduled and executed. We dynamically adjust such thresholds based on multiple characteristics of the arriving workload and the system's conditions. We employed approximate computing to reduce the pruning mechanism's computational overheads and ensure that the mechanism can be used practically. The developed task pruning components are modular and are compatible with various scheduling policies.We develop both approaches in the context of a serverless on-demand media processing platform. However, the ideas and approaches developed in this dissertation are generic and can be applied to serverless systems in other contexts.
Cloud-based serverless computing systems, either public or privately provisioned, aim to provide the illusion of infinite resources and abstract users from details of the allocation decisions. With ...the goal of providing a low cost and a high QoS, the serverless computing paradigm offers opportunities that can be harnessed to attain the goals. Specifically, our strategy in this dissertation is to avoid redundant computing, in cases where independent task requests are similar to each other and for tasks that are pointless to process. We explore two main approaches to (A) reuse part of computation needed to process the services and (B) proactively pruning tasks with a low chance of success to improve the overall QoS of the system. For the first approach, we propose a mechanism to identify various types of "mergeable" tasks, which can benefit from computational reuse if they are executed together as a group. To evaluate the task merging configurations extensively, we quantify the resource-saving magnitude and then leveraging the experimental data to create a resource-saving predictor. We investigate multiple tasks merging approaches that suit different workload scenarios to determine when it is appropriate to aggregate tasks and how to allocate them so that the QoS of other tasks is minimally affected. For the second approach, we developed the mechanisms to skip tasks whose chance of completing on time is not worth pursuing by drop or defer them. We determined the minimum chance of success thresholds for tasks to pass to get scheduled and executed. We dynamically adjust such thresholds based on multiple characteristics of the arriving workload and the system's conditions. We employed approximate computing to reduce the pruning mechanism's computational overheads and ensure that the mechanism can be used practically.
Video streaming is growing in popularity and has become the most bandwidth-consuming Internet service. As such, robust streaming in terms of low latency and uninterrupted streaming experience, ...particularly for viewers in distant areas, has become a challenge. The common practice to reduce latency is to pre-process multiple versions of each video and use Content Delivery Networks (CDN) to cache videos that are popular in a geographical area. However, with the fast-growing video repository sizes, caching video contents in multiple versions on each CDN is becoming inefficient. Accordingly, in this paper, we propose the architecture for Fog Delivery Networks (FDN) and provide methods to federate them (called F-FDN) to reduce video streaming latency. In addition to caching, FDNs have the ability to process videos in an on-demand manner. F-FDN leverages cached contents on the neighboring FDNs to further reduce latency. In particular, F-FDN is equipped with methods that aim at reducing latency through probabilistically evaluating the cost benefit of fetching video segments either from neighboring FDNs or by processing them. Experimental results against alternative streaming methods show that both on-demand processing and leveraging cached video segments on neighboring FDNs can remarkably reduce streaming latency (on average 52%).
Robustness of a distributed computing system is defined as the ability to maintain its performance in the presence of uncertain parameters. Uncertainty is a key problem in heterogeneous (and even ...homogeneous) distributed computing systems that perturbs system robustness. Notably, the performance of these systems is perturbed by uncertainty in both task execution time and arrival. Accordingly, our goal is to make the system robust against these uncertainties. Considering task execution time as a random variable, we use probabilistic analysis to develop an autonomous proactive task dropping mechanism to attain our robustness goal. Specifically, we provide a mathematical model that identifies the optimality of a task dropping decision, so that the system robustness is maximized. Then, we leverage the mathematical model to develop a task dropping heuristic that achieves the system robustness within a feasible time complexity. Although the proposed model is generic and can be applied to any distributed system, we concentrate on heterogeneous computing (HC) systems that have a higher degree of exposure to uncertainty than homogeneous systems. Experimental results demonstrate that the autonomous proactive dropping mechanism can improve the system robustness by up to 20%.
Serverless computing along with Function-as-a-Service (FaaS) is forming a new computing paradigm that is anticipated to found the next generation of cloud systems. The popularity of this paradigm is ...due to offering a highly transparent infrastructure that enables user applications to scale in the granularity of their functions. Since these often small and single-purpose functions are managed on shared computing resources behind the scene, a great potential for computational reuse and approximate computing emerges that if unleashed, can remarkably improve the efficiency of serverless cloud systems -- both from the user's QoS and system's (energy consumption and incurred cost) perspectives. Accordingly, the goal of this survey study is to, first, unfold the internal mechanics of serverless computing and, second, explore the scope for efficiency within this paradigm via studying function reuse and approximation approaches and discussing the pros and cons of each one. Next, we outline potential future research directions within this paradigm that can either unlock new use cases or make the paradigm more efficient.
PDF
