This article presents a novel system that produces multiyear high-resolution irrigation water demand maps for agricultural areas, enabling a new level of detail for irrigation support for farmers and ...agricultural stakeholders. The system is based on a scalable distributed deep learning (DL) model trained on dense time series of Sentinel-2 images and a large training set for the first year of observation and fine tuned on new labeled data for the consecutive years. The trained models are used to generate multiyear crop type maps, which are assimilated together with the Sentinel-2 dense time series and the meteorological data into a physically based agrohydrological model to derive the irrigation water demand for different crops. To process the required large volume of multiyear Copernicus Sentinel-2 data, the software architecture of the proposed system has been built on the integration of the Food Security thematic exploitation platform (TEP) and the data-intensive artificial intelligence Hopsworks platform. While the Food Security TEP provides easy access to Sentinel-2 data and the possibility of developing processing algorithms directly in the cloud, the Hopsworks platform has been used to train DL algorithms in a distributed manner. The experimental analysis was carried out in the upper part of the Danube Basin for the years 2018, 2019, and 2020 considering 37 Sentinel-2 tiles acquired in Austria, Moravia, Hungary, Slovakia, and Germany.
This paper introduces the Hopsworks platform to the entire Earth Observation (EO) data community and the Copernicus programme. Hopsworks is a scalable data-intensive open-source Artificial ...Intelligence (AI) platform that was jointly developed by Logical Clocks and the KTH Royal Institute of Technology for building end-to-end Machine Learning (ML)/Deep Learning (DL) pipelines for EO data. It provides the full stack of services needed to manage the entire life cycle of data in ML. In particular, Hopsworks supports the development of horizontally scalable DL applications in notebooks and the operation of workflows to support those applications, including parallel data processing, model training, and model deployment at scale. To the best of our knowledge, this is the first work that demonstrates the services and features of the Hopsworks platform, which provide users with the means to build scalable end-to-end ML/DL pipelines for EO data, as well as support for the discovery and search for EO metadata. This paper serves as a demonstration and walkthrough of the stages of building a production-level model that includes data ingestion, data preparation, feature extraction, model training, model serving, and monitoring. To this end, we provide a practical example that demonstrates the aforementioned stages with real-world EO data and includes source code that implements the functionality of the platform. We also perform an experimental evaluation of two frameworks built on top of Hopsworks, namely Maggy and AutoAblation. We show that using Maggy for hyperparameter tuning results in roughly half the wall-clock time required to execute the same number of hyperparameter tuning trials using Spark while providing linear scalability as more workers are added. Furthermore, we demonstrate how AutoAblation facilitates the definition of ablation studies and enables the asynchronous parallel execution of ablation trials.
Body area networks (BANs), cloud computing, and machine learning are platforms that can potentially enable advanced healthcare outside the hospital. By applying distributed sensors and drug delivery ...devices on/in our body and connecting to such communication and decision-making technology, a system for remote diagnostics and therapy is achieved with additional autoregulation capabilities. Challenges with such autarchic on-body healthcare schemes relate to integrity and safety, and interfacing and transduction of electronic signals into biochemical signals, and vice versa. Here, we report a BAN, comprising flexible on-body organic bioelectronic sensors and actuators utilizing two parallel pathways for communication and decision-making. Data, recorded from strain sensors detecting body motion, are both securely transferred to the cloud for machine learning and improved decision-making, and sent through the body using a secure body-coupled communication protocol to auto-actuate delivery of neurotransmitters, all within seconds. We conclude that both highly stable and accurate sensing-from multiple sensors-are needed to enable robust decision making and limit the frequency of retraining. The holistic platform resembles the self-regulatory properties of the nervous system, i.e., the ability to sense, communicate, decide, and react accordingly, thus operating as a digital nervous system.
Abstract The purpose of this paper was to examine how muscle architecture parameter (MAP) measurements made using brightness-mode ultrasonography (BMU) differ based on probe orientation. The human ...tibialis anterior muscle was imaged from nine different probe orientations during concentric contractions at four joint angles to determine the effect of probe orientation on the measurement of muscle architecture parameters. Ankle dorsi-flexion torque, tibialis anterior electromyography (EMG), and measures of MAP including fascicle length (FL), pennation angle (PA) and muscle thickness (MT) were collected. Statistically significant differences were found between joint angles for measures of FL and PA and between probe orientations for measures of FL and MT. A comparison of actual MAP values to a geometric model used by researchers to determine an ideal probe orientation show that the actual values vary to a greater extent and do not follow the trend predicted by the model. The results suggest that ultrasound probe orientation affects measures of MAP but the effect either cannot be predicted from a geometric model and/or the error in the measurement technique does not allow a comparison.
ExtremeEarth Meets Satellite Data From Space Hagos, Desta Haileselassie; Kakantousis, Theofilos; Vlassov, Vladimir ...
IEEE journal of selected topics in applied earth observations and remote sensing,
2021, Letnik:
14
Journal Article
Recenzirano
Odprti dostop
Bringing together a number of cutting-edge technologies that range from storing extremely large volumes of data all the way to developing scalable machine learning and deep learning algorithms in a ...distributed manner and having them operate over the same infrastructure poses unprecedented challenges. One of these challenges is the integration of European Space Agency (ESA)'s Thematic Exploitation Platforms (TEPs) and data information access service platforms with a data platform, namely Hopsworks, which enables scalable data processing, machine learning, and deep learning on Copernicus data, and development of very large training datasets for deep learning architectures targeting the classification of Sentinel images. In this article, we present the software architecture of ExtremeEarth that aims at the development of scalable deep learning and geospatial analytics techniques for processing and analyzing petabytes of Copernicus data. The ExtremeEarth software infrastructure seamlessly integrates existing and novel software platforms and tools for storing, accessing, processing, analyzing, and visualizing large amounts of Copernicus data. New techniques in the areas of remote sensing and artificial intelligence with an emphasis on deep learning are developed. These techniques and corresponding software presented in this article are to be integrated with and used in two ESA TEPs, namely Polar and Food Security TEPs. Furthermore, we present the integration of Hopsworks with the Polar and Food Security use cases and the flow of events for the products offered through the TEPs.
Abstract
Objective:We provide an e-Science perspective on the workflow from risk factor discovery and classification of disease to evaluation of personalized intervention programs. As case studies, ...we use personalized prostate and breast cancer screenings.
Materials and Methods:We describe an e-Science initiative in Sweden, e-Science for Cancer Prevention and Control (eCPC), which supports biomarker discovery and offers decision support for personalized intervention strategies. The generic eCPC contribution is a workflow with 4 nodes applied iteratively, and the concept of e-Science signifies systematic use of tools from the mathematical, statistical, data, and computer sciences.
Results:The eCPC workflow is illustrated through 2 case studies. For prostate cancer, an in-house personalized screening tool, the Stockholm-3 model (S3M), is presented as an alternative to prostate-specific antigen testing alone. S3M is evaluated in a trial setting and plans for rollout in the population are discussed. For breast cancer, new biomarkers based on breast density and molecular profiles are developed and the US multicenter Women Informed to Screen Depending on Measures (WISDOM) trial is referred to for evaluation. While current eCPC data management uses a traditional data warehouse model, we discuss eCPC-developed features of a coherent data integration platform.
Discussion and Conclusion:E-Science tools are a key part of an evidence-based process for personalized medicine. This paper provides a structured workflow from data and models to evaluation of new personalized intervention strategies. The importance of multidisciplinary collaboration is emphasized. Importantly, the generic concepts of the suggested eCPC workflow are transferrable to other disease domains, although each disease will require tailored solutions.
This paper presents a novel system that produces multi-year high-resolution irrigation water demand maps for agricultural areas enabling a new level of detail for irrigation support for farmers and ...agricultural stakeholders. The system is based on a scalable distributed Deep Learning (DL) model trained on dense time series of Sentinel-2 images and a large training set for the first year of observation, and fine-tuned on new labeled data for the consecutive years. The trained models are used to generate multi-year crop type maps, which are assimilated together with the Sentinel-2 dense time series and the meteorological data into a physically based agro-hydrological model to derive the irrigation water demand for different crops. To process the required large volume of multi-year Copernicus Sentinel-2 data, the software architecture of the proposed system has been built on the integration of the Food Security TEP and the data-intensive arpngicial intelligence Hopsworks platform. While the Food Security TEP provides easy access to Sentinel- 2 data and the possibility of developing processing algorithms directly in the cloud, the Hopsworks platform has been used to train DL algorithms in a distributed manner. The experimental analysis was carried out in the upper part of the Danube Basin for the years 2018, 2019 and 2020 considering 37 Sentinel-2 tiles acquired in Austria, Moravia, Hungary, Slovakia and Germany.
Hadoop is a popular system for storing, managing,and processing large volumes of data, but it has bare-bonesinternal support for metadata, as metadata is a bottleneck andless means more scalability. ...The result is a scalable platform withrudimentary access control that is neither user- nor developer-friendly. Also, metadata services that are built on Hadoop, suchas SQL-on-Hadoop, access control, data provenance, and datagovernance are necessarily implemented as eventually consistentservices, resulting in increased development effort and morebrittle software. In this paper, we present a new project-based multi-tenancymodel for Hadoop, built on a new distribution of Hadoopthat provides a distributed database backend for the HadoopDistributed Filesystem's (HDFS) metadata layer. We extendHadoop's metadata model to introduce projects, datasets, andproject-users as new core concepts that enable a user-friendly, UI-driven Hadoop experience. As our metadata service is backed bya transactional database, developers can easily extend metadataby adding new tables and ensure the strong consistency ofextended metadata using both transactions and foreign keys.
Emerging use-cases like smart manufacturing and smart cities pose challenges in terms of latency, which cannot be satisfied by traditional centralized infrastructure. Edge networks, which bring ...computational capacity closer to the users/clients, are a promising solution for supporting these critical low latency services. Different from traditional centralized networks, the edge is distributed by nature and is usually equipped with limited compute capacity. This creates a complex network to handle, subject to failures of different natures, that requires novel solutions to work in practice. To reduce complexity, edge application technology enablers, advanced infrastructure and application orchestration techniques need to be in place where AI and ML are key players.
PDF
