This research develops a novel teleoperation for robot manipulators based on augmented reality. The proposed interface is equipped with full capabilities in order to replace the classical teach ...pendant of the robot for carrying out teleoperation tasks. The proposed interface is based on an augmented reality headset for projecting computer-generated graphics onto the real environment and a gamepad to interact with the computer-generated graphics and provide robot commands. In order to demonstrate the benefits of the proposed method, several usability tests were conducted using a 6R industrial robot manipulator in order to compare the proposed interface and the conventional teach pendant interface for teleoperation tasks. In particular, the results of these usability tests show that the proposed approach is more intuitive, ergonomic, and easy to use. Furthermore, the comparison results also show that the proposed method clearly improves the velocity of the teleoperation task, regardless of the user’s previous experience in robotics and augmented reality technology.
Full text
Available for:
DOBA, EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, IZUM, KILJ, KISLJ, MFDPS, NLZOH, NUK, OBVAL, OILJ, PILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, SIK, UILJ, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
The automatic detection of small defects (of up to 0.2mm in diameter) on car body surfaces following the painting process is currently one of the greatest issues facing quality control in the ...automotive industry. Although several systems have been developed during the last decade to provide a solution to this problem, these, to the best of our knowledge, have been focused solely on flat surfaces and have been unable to inspect other parts of the surfaces, namely style lines, edges and corners as well as deep concavities. This paper introduces a novel approach using deflectometry- and vision-based technologies in order to overcome this problem and ensure that the whole area is inspected. Moreover, since our approach, together with the system used, computes defects in less than 15s, it satisfies cycle time production requirements (usually of around 30s per car). Hence, a two-step algorithm is presented here: in the first step, a new pre-processing step (image fusion algorithm) is introduced to enhance the contrast between pixels with a low level of intensity (indicating the presence of defects) and those with a high level of intensity (indicating the absence of defects); for the second step, we present a novel post-processing step with an image background extraction approach based on a local directional blurring method and a modified image contrast enhancement, which enables detection of defects in the entire illuminated area. In addition, the post-processing step is processed several times using a multi-level structure, with computed image backgrounds of different resolution. In doing so, it is possible to detect larger defects, given that each level identifies defects of different sizes. Experimental results presented in this paper are obtained from the industrial automatic quality control system QEyeTunnel employed in the production line at the Mercedes-Benz factory in Vitoria, Spain. A complete analysis of the algorithm performance will be shown here, together with several tests proving the robustness and reliability of our proposal.
•New algorithms for defects (of up to 0.2mm in diameter) detection on specular surfaces are presented.•Defects are detected not only in areas with smooth slope changes but in edges, corners and deep concavities.•The Detection performance, robustness and reliability are verified through experimental studies.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Contact driven tasks, such as surface conditioning operations (wiping, polishing, sanding, etc.), are difficult to program in advance to be performed autonomously by a robotic system, specially when ...the objects involved are moving. In many applications, human-robot physical interaction can be used for the teaching, specially in learning from demonstrations frameworks, but this solution is not always available. Robot teleoperation is very useful when user and robot cannot share the same workspace due to hazardous environments, inaccessible locations, or because of ergonomic issues. In this sense, this article introduces a novel dual-arm teleoperation architecture with haptic and visual feedback to enhance the operator immersion in surface treatment tasks. Two task-based assistance systems are also proposed to control each robotic manipulator individually. To validate the remote assisted control, some usability tests have been carried out using Baxter, a dual-arm collaborative robot. After analysing several benchmark metrics, the results show that the proposed assistance method helps to reduce the task duration and improves the overall performance of the teleoperation.
•We develop an advanced robot teleoperation and control system to conduct surface treatments.•The robot system provides strength and accuracy and the user provides flexibility.•The remote ...teleoperation provides security and comfort to the user.•Augmented virtuality and haptic feedback are used by the user to teleoperate the robot.•The experimental results of the method are compared to the manual performance of an expert.
There are some industrial tasks that are still mainly performed manually by human workers due to their complexity, which is the case of surface treatment operations (such as sanding, deburring, finishing, grinding, polishing, etc.) used to repair defects. This work develops an advanced teleoperation and control system for industrial robots in order to assist the human operator to perform the mentioned tasks. On the one hand, the controlled robotic system provides strength and accuracy, holding the tool, keeping the right tool orientation and guaranteeing a smooth approach to the workpiece. On the other hand, the advanced teleoperation provides security and comfort to the user when performing the task. In particular, the proposed teleoperation uses augmented virtuality (i.e., a virtual world that includes non-modeled real-world data) and haptic feedback to provide the user an immersive virtual experience when remotely teleoperating the tool of the robot system to treat arbitrary regions of the workpiece surface. The method is illustrated with a car body surface treatment operation, although it can be easily extended to other surface treatment applications or even to other industrial tasks where the human operator may benefit from robotic assistance. The effectiveness of the proposed approach is shown with several experiments using a 6R robotic arm. Moreover, a comparison of the performance obtained manually by an expert and that obtained with the proposed method has also been conducted in order to show the suitability of the proposed approach.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
This work proposes a new interface for the teleoperation of mobile robots based on virtual reality that allows a natural and intuitive interaction and cooperation between the human and the robot, ...which is useful for many situations, such as inspection tasks, the mapping of complex environments, etc. Contrary to previous works, the proposed interface does not seek the realism of the virtual environment but provides all the minimum necessary elements that allow the user to carry out the teleoperation task in a more natural and intuitive way. The teleoperation is carried out in such a way that the human user and the mobile robot cooperate in a synergistic way to properly accomplish the task: the user guides the robot through the environment in order to benefit from the intelligence and adaptability of the human, whereas the robot is able to automatically avoid collisions with the objects in the environment in order to benefit from its fast response. The latter is carried out using the well-known potential field-based navigation method. The efficacy of the proposed method is demonstrated through experimentation with the Turtlebot3 Burger mobile robot in both simulation and real-world scenarios. In addition, usability and presence questionnaires were also conducted with users of different ages and backgrounds to demonstrate the benefits of the proposed approach. In particular, the results of these questionnaires show that the proposed virtual reality based interface is intuitive, ergonomic and easy to use.
High dexterity is required in tasks in which there is contact between objects, such as surface conditioning (wiping, polishing, scuffing, sanding, etc.), specially when the location of the objects ...involved is unknown or highly inaccurate because they are moving, like a car body in automotive industry lines. These applications require the human adaptability and the robot accuracy. However, sharing the same workspace is not possible in most cases due to safety issues. Hence, a multi-modal teleoperation system combining haptics and an inertial motion capture system is introduced in this work. The human operator gets the sense of touch thanks to haptic feedback, whereas using the motion capture device allows more naturalistic movements. Visual feedback assistance is also introduced to enhance immersion. A Baxter dual-arm robot is used to offer more flexibility and manoeuvrability, allowing to perform two independent operations simultaneously. Several tests have been carried out to assess the proposed system. As it is shown by the experimental results, the task duration is reduced and the overall performance improves thanks to the proposed teleoperation method.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
Teleoperation of bimanual robots is being used to carry out complex tasks such as surgeries in medicine. Despite the technological advances, current interfaces are not natural to the users, who spend ...long periods of time in learning how to use these interfaces. In order to mitigate this issue, this work proposes a novel augmented reality-based interface for teleoperating bimanual robots. The proposed interface is more natural to the user and reduces the interface learning process. A full description of the proposed interface is detailed in the paper, whereas its effectiveness is shown experimentally using two industrial robot manipulators. Moreover, the drawbacks and limitations of the classic teleoperation interface using joysticks are analyzed in order to highlight the benefits of the proposed augmented reality-based interface approach.
A MATLAB/SIMULINK software simulation model (structure and component blocks) has been constructed in order to view and analyze the potential of the PSD (Position Sensitive Detector) array concept ...technology before it is further expanded or developed. This simulation allows changing most of its parameters, such as the number of elements in the PSD array, the direction of vision, the viewing/scanning angle, the object rotation, translation, sample/scan/simulation time, etc. In addition, results show for the first time the possibility of scanning an object in 3D when using an a-Si:H thin film 128 PSD array sensor and hardware/software system. Moreover, this sensor technology is able to perform these scans and render 3D objects at high speeds and high resolutions when using a sheet-of-light laser within a triangulation platform. As shown by the simulation, a substantial enhancement in 3D object profile image quality and realism can be achieved by increasing the number of elements of the PSD array sensor as well as by achieving an optimal position response from the sensor since clearly the definition of the 3D object profile depends on the correct and accurate position response of each detector as well as on the size of the PSD array.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
This article develops a human-robot cooperation to carry out treatments such as sanding, polishing, etc. on the surface of a known rigid object. For this purpose, a vision system is considered to get ...the object location to ensure not only the perpendicularity of the robot tool to the object surface but also a smooth approach of the tool to the surface. In order to add flexibility, the proposal includes the simultaneous combination of automatic and manual modes of operation. Thus, the human user can guide the robot tool to treat arbitrary areas (manual mode) and, when the operator releases the tool, the robot goes into the automatic mode to treat prior established areas. The method uses a task prioritization framework and three types of controllers: an admittance controller for the tool guidance; a hybrid controller to modify the tool orientation and, in the automatic mode, the tool position; and a sliding mode controller to limit the velocity at which the tool approaches the object surface. The applicability and efficacy of the proposed method is demonstrated experimentally using a conventional 6R robot arm.
This research introduces a
new optimality criterion
for motion planning of wheeled mobile robots based on a cost index that assesses the nearness to singularity of forward and inverse kinematic ...models. Slip motions, infinite estimation error and impossible control actions are
avoided
escaping from singularities. In addition, high amplification of wheel velocity errors and high wheel velocity values are also avoided by moving far from the singularity. The proposed cost index can be used directly to
complement
path-planning and motion-planning techniques (e.g. tree graphs, roadmaps, etc.) in order to
select
the optimal collision-free path or trajectory among several possible solutions. To illustrate the applications of the proposed approach, an
industrial forklift
, equivalent to a tricycle-like mobile robot, is considered in a simulated environment. In particular, several results are validated for the proposed optimality criterion, which are extensively
compared
to those obtained with other classical optimality criteria, such as shortest-path, time-optimal and minimum-energy.
Full text
Available for:
EMUNI, FZAB, GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UL, UM, UPUK, VKSCE, ZAGLJ
PDF
