Due to a general shift in manufacturing paradigm from mass production towards mass customization, reconfigurable automation technologies, such as robots, are required. However, current industrial ...robot solutions are notoriously difficult to program, leading to high changeover times when new products are introduced by manufacturers. In order to compete on global markets, the factories of tomorrow need complete production lines, including automation technologies that can effortlessly be reconfigured or repurposed, when the need arises. In this paper we present the concept of general, self-asserting robot skills for manufacturing. We show how a relatively small set of skills are derived from current factory worker instructions, and how these can be transferred to industrial mobile manipulators. General robot skills can not only be implemented on these robots, but also be intuitively concatenated to program the robots to perform a variety of tasks, through the use of simple task-level programming methods. We demonstrate various approaches to this, extensively tested with several people inexperienced in robotics. We validate our findings through several deployments of the complete robot system in running production facilities at an industrial partner. It follows from these experiments that the use of robot skills, and associated task-level programming framework, is a viable solution to introducing robots that can intuitively and on the fly be programmed to perform new tasks by factory workers.
•We propose a conceptual model of robot skills and show how this differs from macros.•We show how this approach can enable non-experts to utilize advanced robotic systems.•Concrete industrial applications of the approach are presented, on advanced robot systems.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Stereo vision, resulting in the knowledge of deep information in a scene, is of great importance in the field of machine vision, robotics and image analysis. In this article, an explicit analysis of ...the existing stereo matching methods, up to date, is presented. The presented algorithms are discussed in terms of speed, accuracy, coverage, time consumption, and disparity range. Towards the direction of real-time operation, the development of stereo matching algorithms, suitable for efficient hardware implementation is highly desirable. Implementations of stereo matching algorithms in hardware for real-time applications are also discussed in details.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Object detection has been in the focus of researchers within varying applications propelled by the recent advances in deep learning and neural networks. Many applications require both detection of ...class instances as well as a quantification of the spatial coverage of the class instances. While the performance of deep learning approaches for these tasks has been extensively studied there has not been much effort into creating a unified network structure to achieve both goals. The purpose of this paper is to present a regressor to the faster R‐CNN architecture that can help quantify the spatial coverage estimation of some detected object. The goal of the regressor is to provide a reproducible result of the spatial coverage. To demonstrate the developed architecture, an example use‐case of land cover estimation is used. The experiments conducted in this paper show that the network does not sacrifice object detection accuracy, and indicate that the network is able to estimate the spatial coverage of six different types of land.
Full text
Available for:
FZAB, GIS, IJS, KILJ, NLZOH, NUK, OILJ, SBCE, SBMB, UL, UM, UPUK
Autonomous weeding robots need to accurately detect the joint stem of grassland weeds in order to control those weeds in an effective and energy-efficient manner. In this work, keypoints on joint ...stems and bounding boxes around weeds in grasslands are detected jointly using multi-task learning. We compare a two-stage, heatmap-based architecture to a single-stage, regression-based architecture—both based on the popular YOLOv5 object detector. Our results show that introducing joint-stem detection as a second task boosts the individual weed detection performance in both architectures. Furthermore, the single-stage architecture clearly outperforms its competitors with an OKS of 56.3 in joint-stem detection while also achieving real-time performance of 12.2 FPS on Nvidia Jetson NX, suitable for agricultural robots. Finally, we make the newly created joint-stem ground-truth annotations publicly available for the relevant research community.
Real-time prediction of human location combined with the capability to perceive obstacles is crucial for socially-aware navigation in robotics. Our work focuses on localizing humans in the world and ...predicting the free space around them by incorporating other static and dynamic obstacles. We propose a multi-task learning strategy to handle both tasks, achieving this goal with minimal computational demands. We use a dataset captured in a typical warehouse environment by mounting a perception module consisting of a Jetson Xavier AGX and an Intel L515 LiDAR camera on a MiR100 mobile robot. Our method, which is built upon prior works in the field of human detection and localization demonstrates improved results in difficult cases that are not tackled in other works, such as human instances at a close distance or at the limits of the field of view of the capturing sensor. We further extend this work by using a lightweight network structure and integrating a free space segmentation branch that can independently segment the floor space without any prior maps or 3D data, relying instead on the characteristics of the floor. In conclusion, our method presents a lightweight and efficient solution for predicting human 3D location and segmenting the floor space for low-energy consumption platforms, tested in an industrial environment.
Machine learning and specifically deep learning techniques address many of the issues faced in visual object detection and classification tasks. However, they have the caveat of needing large amounts ...of annotated training data. In the maritime domain one may encounter objects fairly infrequently, depending on weather and location. This creates an issue of data collection. Areas such as harbors and channels see a lot of traffic, but the ships are of a specific class. Furthermore, the variability of the buoys from region to region and within regions is difficult and expensive to sample. Thus the amount and quality of available data is severely lacking. Furthermore very few publicly available maritime datasets exist In this work, we present a novel approach that detects possible “poor” training samples and automatically re-annotates them, based on the current state of the object detector. We show the applicability of our approach on real-life maritime data and show that the poor annotation quality of the datasets used can be mitigated. We show performance gain with respect to a baseline approach is proportional to the amount of poorly annotated data in the dataset. When 25% of the data is poor we achieve a 5.5%, 13.7%, and 8.0% increase in performance on 3 separate datasets, compared to a baseline model. With 50% noise we reach 58.5%, 18.7% and 94.2% increase respectively. Our approach also allows for the iterative improvement of a given dataset by providing a set of pseudo-annotations to replace the current incorrect ones.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Mobile robots should possess accurate self-localization capabilities in order to be successfully deployed in their environment. A solution to this challenge may be derived from visual odometry (VO), ...which is responsible for estimating the robot's pose by analysing a sequence of images. The present paper proposes an accurate, computationally-efficient VO algorithm relying solely on stereo vision images as inputs. The contribution of this work is twofold. Firstly, it suggests a non-iterative outlier detection technique capable of efficiently discarding the outliers of matched features. Secondly, it introduces a hierarchical motion estimation approach that produces refinements to the global position and orientation for each successive step. Moreover, for each subordinate module of the proposed VO algorithm, custom non-iterative solutions have been adopted. The accuracy of the proposed system has been evaluated and compared with competent VO methods along DGPS-assessed benchmark routes. Experimental results of relevance to rough terrain routes, including both simulated and real outdoors data, exhibit remarkable accuracy, with positioning errors lower than 2%.
Many robot learning algorithms depend on a model of the robot's forward dynamics for simulating potential trajectories and ultimately learning a required task. In this paper, we present a data-driven ...reservoir computing approach and apply it for learning forward dynamics models. Our proposed machine learning algorithm exploits the concepts of dynamic reservoir, self-organized learning and Bayesian inference. We have evaluated our approach on datasets gathered from two industrial robotic manipulators and compared it on both step-by-step and multi-step trajectory prediction scenarios with state-of-the-art algorithms. The evaluation considers the algorithms' convergence and prediction performance on joint and operational space for varying prediction horizons, as well as computational time. Results show that the proposed algorithm performs better than the state-of-the-art, converges fast and can achieve accurate predictions over longer horizons, which makes it a reliable, data-efficient approach for learning forward models.
In certain cases analytical derivation of physics-based models of robots is difficult or even impossible. A potential workaround is the approximation of robot models from sensor data-streams ...employing machine learning approaches. In this paper, the inverse dynamics models are learned by employing a novel real-time deep learning algorithm. The algorithm exploits the methods of self-organized learning, reservoir computing and Bayesian inference. It is evaluated and compared to other state of the art algorithms in terms of generalization ability, convergence and adaptability using five datasets gathered from four robots. Results show that the proposed algorithm can adapt to real-time changes of the inverse dynamics model significantly better than the other state of the art algorithms.
•Generation of agricultural pre-trained weights with self-supervised contrastive learning SwAV for three different agricultural datasets.•The learned pre-trained weights are transferred to ...classification as well as segmentation downstream tasks.•We show that domain-specific pre-trained weights increase the label efficiency of classification tasks significantly.
Agriculture emerges as a prominent application domain for advanced computer vision algorithms. As much as deep learning approaches can help solve problems such as plant detection, they rely on the availability of large amounts of annotated images for training. However, relevant agricultural datasets are scarce and at the same time, generic well-established image datasets such as ImageNet do not necessarily capture the characteristics of agricultural environments. This observation has motivated us to explore the applicability of self-supervised contrastive learning on agricultural images. Our approach considers numerous non-annotated agricultural images, which are easy to obtain, and uses them to pre-train deep neural networks. We then require only a limited number of annotated images to fine-tune those networks in a supervised training manner for relevant downstream tasks, such as plant classification or segmentation. To the best of our knowledge, contrastive self-supervised learning has not been explored before in the area of agricultural images. Our results reveal that it outperforms conventional deep learning approaches in classification downstream tasks, especially for small amounts of available annotated training images where up to 14% increase of average top-1 classification accuracy has been observed. Furthermore, the computational cost for generating data-specific pre-trained weights is fairly low, allowing one to generate easily new pre-trained weights for any custom model architecture or task.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
PDF
