Cables are complex, high-dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and incremental ...deformations, or various mechanical fixtures such as clamps, pins, or rings. We are interested in manipulating freely moving cables, in real time, with a pair of robotic grippers, and with no added mechanical constraints. The main contribution of this paper is a perception and control framework that moves in that direction, and uses real-time tactile feedback to accomplish the task of following a dangling cable. The approach relies on a vision-based tactile sensor, GelSight, that estimates the pose of the cable in the grip, and the friction forces during cable sliding. We achieve the behavior by combining two tactile-based controllers: (1) cable grip controller, where a PD controller combined with a leaky integrator regulates the gripping force to maintain the frictional sliding forces close to a suitable value; and (2) cable pose controller, where an linear–quadratic regulator controller based on a learned linear model of the cable sliding dynamics keeps the cable centered and aligned on the fingertips to prevent the cable from falling from the grip. This behavior is possible with the use of reactive gripper fitted with GelSight-based high-resolution tactile sensors. The robot can follow 1 m of cable in random configurations within two to three hand regrasps, adapting to cables of different materials and thicknesses. We demonstrate a robot grasping a headphone cable, sliding the fingers to the jack connector, and inserting it. To the best of the authors’ knowledge, this is the first implementation of real-time cable following without the aid of mechanical fixtures. Videos are available at http://gelsight.csail.mit.edu/cable/
Deformable Linear Objects (DLOs) such as cables, wires, ropes, and elastic tubes are numerously present both in domestic and industrial environments. Unfortunately, robotic systems handling DLOs are ...rare and have limited capabilities due to the challenging nature of perceiving them. Hence, we propose a novel approach named RT-DLO for real-time instance segmentation of DLOs. First, the DLOs are semantically segmented from the background. Afterward, a novel method to separate the DLO instances is applied. It employs the generation of a graph representation of the scene given the semantic mask where the graph nodes are sampled from the DLOs center-lines whereas the graph edges are selected based on topological reasoning. RT-DLO is experimentally evaluated against both DLO-specific and general-purpose instance segmentation deep learning approaches, achieving overall better performances in terms of accuracy and inference time.
•The algorithm could segment individual, overlapping, and self-occluding nanowires.•The length, thickness, and perimeter of instances are calculated.•A histogram for length distribution is ...provided.•The accuracy for length measurement was 99 %.•Overall processing time for a 27-megapixel image was around 15 s.
Nanoparticles in microscopy images are usually analyzed qualitatively or manually and there is a need for autonomous quantitative analysis of these objects. In this paper, we present a physics-based computational model for accurate segmentation and geometrical analysis of one-dimensional deformable overlapping objects from microscopy images. This model, named Nano1D, has four steps of preprocessing, segmentation, separating overlapped objects and geometrical measurements. The model is tested on SEM images of Ag and Au nanowire taken from different microscopes, and thermally fragmented Ag nanowires transformed into nanoparticles with different lengths, diameters, and population densities. It successfully segments and analyzes their geometrical characteristics including lengths and average diameter. The function of the algorithm is not undermined by the size, number, density, orientation and overlapping of objects in images. The main strength of the model is shown to be its ability to segment and analyze overlapping objects successfully with more than 99 % accuracy, while current machine learning and computational models suffer from inaccuracy and inability to segment overlapping objects. Benefiting from a graphical user interface, Nano1D can analyze 1D nanoparticles including nanowires, nanotubes, nanorods in addition to other 1D features of microstructures like microcracks, dislocations etc.
Display omitted
Robotic manipulation of deformable linear objects (DLOs) is important in many applications such as the assembly of deformable wire harnesses and cables in manufacturing. Despite some relevant work in ...modeling, few studies have studied the comprehensive dynamic modeling and precise automatic control of DLOs using robots due to their high degrees of freedom and high flexibility. To fill this gap, the precise single-arm and dual-arm robot manipulation control of DLOs is studied in this article. First, a more comprehensive dynamic model of DLOs is established based on a discrete elastic rod model, which takes into account the twisting deformation of DLOs. The collisions, contacts, and frictions between the DLOs and the plane, as well as the kinematic constraints of both ends of the DLOs, are also considered in the model. Second, practical dynamic control schemes of robots are proposed to realize the precise control of DLOs for both the single-arm control and dual-arm control. For validations, we built an experimental platform using an ABB Yumi robot to implement and validate the proposed approaches in addition to simulations. Finally, various DLO manipulation tasks are conducted and the results for both single-arm and dual-arm manipulations validate the proposed modeling and control approaches.
The robotic manipulation of deformable linear objects (DLOs) has broad application prospects in many fields. However, a key issue is to obtain the exact deformation models (i.e., how robot motion ...affects DLO deformation), which are hard to theoretically calculate and vary among different DLOs. Thus, the shape control of DLOs is challenging, especially for large deformation control that requires global and more accurate models. In this article, we propose a coupled offline and online data-driven method for efficiently learning a global deformation model, allowing for both accurate modeling through offline learning and further updating for new DLOs via online adaptation. Specifically, the model approximated by a neural network is first trained offline on random data, then seamlessly migrated to the online phase, and further updated online during actual manipulation. Several strategies are introduced to improve the model's efficiency and generalization ability. We propose a convex-optimization-based controller and analyze the system's stability using the Lyapunov method. Detailed simulations and real-world experiments demonstrate that our method can efficiently and precisely estimate the deformation model and achieve the large deformation control of untrained DLOs in 2-D and 3-D dual-arm manipulation tasks better than the existing methods. It accomplishes all 24 tasks with different desired shapes on different DLOs in the real world, using only simulation data for the offline learning.
Real-time reactive manipulation of deformable linear objects is a challenging task that requires robots to quickly and adaptively respond to changes in the object’s deformed shape that result from ...external forces. In this paper, a novel approach is proposed for real-time reactive deformable linear object manipulation in the context of human–robot collaboration. The proposed approach combines a topological latent representation and a fixed-time sliding mode controller to enable seamless interaction between humans and robots. The introduced topological control model offers a framework for controlling the dynamic shape of deformable objects. By leveraging the topological representation, our approach captures the connectivity and structure of the objects’ shapes within a latent space. This enables improved generalization and performance when handling complex deformable shapes. A fixed-time sliding mode controller ensures that the object is manipulated in real-time, while also ensuring that it remains accurate and stable during the manipulation process. To validate our proposed framework, we first conduct motor-robot experiments to simulate fixed human interaction processes, enabling straightforward comparisons with other approaches. We then follow up with human–robot experiments to demonstrate the effectiveness of our approach.
•A novel real-time human–robot collaboration framework for deformable object manipulation.•An efficient topology-aware latent encoding to support deformable object representation.•A latent space-based controller to support real-time human–robot collaboration for deformable object manipulation.•A detailed experimental validation of the proposed human–robot collaboration framework for deformable object manipulation.
Designing the layouts and simulating the assembly of cables are based on flexible cable modeling technology. In this paper, we review the methods used to model deformable cable-like objects. At ...present, the physical models of one-dimensional cable-like flexible objects are mostly elastic. There are different types of models, which we can classify as mass–spring, multi-body, elastic rod, dynamic spline, finite element models, and so forth. There are a number of significant issues, such as how to couple these models in an appropriate way, take more physical behaviors into account, and develop a more sophisticated/comprehensive model. Some researchers have also studied how to model plastic cables, thin viscous threads, and branched cables; however, those issues are far from fully resolved and need to be studied further. Furthermore, we believe that in future research it will be important to model the cross-sections of cables with different or deformable shapes and complex internal structures, and consider the influence of temperature and alternating stress.
•The physical models of deformable cable-like linear objects are summarized.•Some more complex issues are introduced and discussed.•Some opinions on the future research directions are put forward.
Recent advances in deep learning have dramatically improved the performance of instance segmentation, which is the task of predicting object area in images. However, depending on the shape of the ...target object, precise detection may still be difficult. For example, linear objects such as Wires still pose challenges for accurate instance segmentation due to their unique characteristics about shape. Therefore, we propose an instance segmentation method to accurately detect linear objects. The proposed method focuses on the characteristics of linear objects: continuity and irregularity of shape. We attempt to accurately detect linear objects by using Smooth Loss, which evaluates continuity, and Edge Enhance Loss, which focuses on the correctness of contours. In addition, we propose an evaluation metrics using the distance between contours to evaluate the accuracy of contour prediction. The proposed instance segmentation method improves by around 12% the average performance of contour prediction on the iShape dataset.
PDF
