The design of robots that interact autonomously with the environment and exhibit complex behaviours is an open challenge that can benefit from understanding what makes living beings fit to act in the ...world. Neuromorphic engineering studies neural computational principles to develop technologies that can provide a computing substrate for building compact and low-power processing systems. We discuss why endowing robots with neuromorphic technologies - from perception to motor control - represents a promising approach for the creation of robots which can seamlessly integrate in society. We present initial attempts in this direction, highlight open challenges, and propose actions required to overcome current limitations.
Flexible organic transistors transmit force signals encoded for direct input to human neurons
The sense of touch is the ability to perceive consistency, texture, and shape of objects that we ...manipulate, and the forces we exchange with them. Touch is a source of information that we effortlessly decode to smoothly and naturally grasp and manipulate objects, maintain our posture while walking, or avoid stumbling into obstacles, allowing us to plan, adapt, and correct actions in an ever-changing external world. As such, artificial devices, such as robots or prostheses, that aim to accomplish similar tasks must possess artificial tactile-sensing systems. On page 998 of this issue, Kim
et al.
(
1
) report on a “neuromorphic” tactile sensory system based on organic, flexible, electronic circuits that can measure the force applied on the sensing regions. The encoding of the signal is similar to that used by human nerves that are sensitive to tactile stimuli (mechanoreceptors), so the device outputs can substitute for them and communicate with other nerves (e.g., residual nerve fibers of amputees or motor neurons). The proposed system exploits organic electronics that allow for three-dimensional printing of flexible structures that conform to large curved surfaces, as required for placing sensors on robots (
2
) and prostheses.
Sensory information processing in robot skins currently rely on a centralized approach where signal transduction (on the body) is separated from centralized computation and decision-making, requiring ...the transfer of large amounts of data from periphery to central processors, at the cost of wiring, latency, fault tolerance and robustness. We envision a decentralized approach where intelligence is embedded in the sensing nodes, using a unique neuromorphic methodology to extract relevant information in robotic skins. Here we specifically address pain perception and the association of nociception with tactile perception to trigger the escape reflex in a sensorized robotic arm. The proposed system comprises self-healable materials and memtransistors as enabling technologies for the implementation of neuromorphic nociceptors, spiking local associative learning and communication. Configuring memtransistors as gated-threshold and -memristive switches, the demonstrated system features in-memory edge computing with minimal hardware circuitry and wiring, and enhanced fault tolerance and robustness.
Event camera (EC) emerges as a bio-inspired sensor which can be an alternative or complementary vision modality with the benefits of energy efficiency, high dynamic range, and high temporal ...resolution coupled with activity dependent sparse sensing. In this study we investigate with ECs the problem of face pose alignment, which is an essential pre-processing stage for facial processing pipelines. EC-based alignment can unlock all these benefits in facial applications, especially where motion and dynamics carry the most relevant information due to the temporal change event sensing. We specifically aim at efficient processing by developing a coarse alignment method to handle large pose variations in facial applications. For this purpose, we have prepared by multiple human annotations a dataset of extreme head rotations with varying motion intensity. We propose a motion detection based alignment approach in order to generate activity dependent pose-events that prevents unnecessary computations in the absence of pose change. The alignment is realized by cascaded regression of extremely randomized trees. Since EC sensors perform temporal differentiation, we characterize the performance of the alignment in terms of different levels of head movement speeds and face localization uncertainty ranges as well as face resolution and predictor complexity. Our method obtained 2.7% alignment failure on average, whereas annotator disagreement was 1%. The promising coarse alignment performance on EC sensor data together with a comprehensive analysis demonstrate the potential of ECs in facial applications.
To interact with its environment, a robot working in 3D space needs to organise its visual input in terms of objects or their perceptual precursors, proto-objects. Among other visual cues, depth is a ...submodality used to direct attention to visual features and objects. Current depth-based proto-object attention models have been implemented for standard RGB-D cameras that produce synchronous frames. In contrast, event cameras are neuromorphic sensors that loosely mimic the function of the human retina by asynchronously encoding per-pixel brightness changes at very high temporal resolution, thereby providing advantages like high dynamic range, efficiency (thanks to their high degree of signal compression), and low latency. We propose a bio-inspired bottom-up attention model that exploits event-driven sensing to generate depth-based saliency maps that allow a robot to interact with complex visual input. We use event-cameras mounted in the eyes of the iCub humanoid robot to directly extract edge, disparity and motion information. Real-world experiments demonstrate that our system robustly selects salient objects near the robot in the presence of clutter and dynamic scene changes, for the benefit of downstream applications like object segmentation, tracking and robot interaction with external objects.
Abstract
Modern computation based on von Neumann architecture is now a mature cutting-edge science. In the von Neumann architecture, processing and memory units are implemented as separate blocks ...interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 10
18
calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges for each research area. We hope that this roadmap will be a useful resource by providing a concise yet comprehensive introduction to readers outside this field, for those who are just entering the field, as well as providing future perspectives for those who are well established in the neuromorphic computing community.
We propose a neuromorphic framework to process the activity of human spinal motor neurons for movement intention recognition. This framework is integrated into a non-invasive interface that decodes ...the activity of motor neurons innervating intrinsic and extrinsic hand muscles. One of the main limitations of current neural interfaces is that machine learning models cannot exploit the efficiency of the spike encoding operated by the nervous system. Spiking-based pattern recognition would detect the spatio-temporal sparse activity of a neuronal pool and lead to adaptive and compact implementations, eventually running locally in embedded systems. Emergent Spiking Neural Networks (SNN) have not yet been used for processing the activity of in-vivo human neurons. Here we developed a convolutional SNN to process a total of 467 spinal motor neurons whose activity was identified in 5 participants while executing 10 hand movements. The classification accuracy approached 0.95 ± 0.14 for both isometric and non-isometric contractions. These results show for the first time the potential of highly accurate motion intent detection by combining non-invasive neural interfaces and SNN.
IntroductionVisual applications in robotics must meet strict requirements for power efficiency, low latency, and data processing capacity. Despite the remarkable performance achievements of ...traditional computer vision methods, they struggle to generalise effectively and often rely on vast datasets, increasing data processing and transfer. The proposed system leverages bioinspired visual attention mechanisms to process only relevant parts of the scene, further exploring event-based sensing and neuromorphic computing via Spiking Neural Networks (SNNs).AimsThis scientific challenge aims to connect bioinspired hardware with biologically plausible algorithms, thereby showcasing the potential of spike-based implementations for online robotics visual applications.MethodsThe bioinspired saliency-based visual attention model processes events from event-driven cameras on the humanoid robot iCub, running on SpiNNaker neuromorphic hardware. Intensity, disparity, and motion are the bottom-up feature extraction channels competing for scene representation. These cues feed into a biologically plausible saliency-based proto-object model based on Gestalt perceptual grouping theories to detect only relevant scene parts. The model produces saliency maps with salient areas representing regions potentially containing objects, called ‘proto-objects’.ResultsThe online system accurately generates saliency maps in ~16ms detecting salient proto-objects and disregarding clutter. The system has been qualitatively and quantitatively validated, achieving comparable results to the frame-based implementation, in online simple office scenarios, as well as when compared against the ground truth fixation maps from real human subjects (NUS3D dataset).ConclusionThis project is the first significant step towards more complex real-world robotic applications for vision, where bioinspiration sets the basis for fast, power-efficient online robotic applications and innovative computer vision approaches.
Event cameras measure scene changes with high temporal resolutions, making them well-suited for visual motion estimation. The activation of pixels results in an asynchronous stream of digital data ...(events), which rolls continuously over time without the discrete temporal boundaries typical of frame-based cameras (where a data packet or frame is emitted at a fixed temporal rate). As such, it is not trivial to define
how to group/accumulate events in a way that is sufficient for computation. The suitable number of events can greatly vary for different environments, motion patterns, and tasks. In this paper, we use neural networks for rotational motion estimation as a scenario to investigate the appropriate selection of event batches to populate input tensors. Our results show that batch selection has a large impact on the results: training should be performed on a wide variety of different batches, regardless of the batch selection method; a simple fixed-time window is a good choice for inference with respect to fixed-count batches, and it also demonstrates comparable performance to more complex methods. Our initial hypothesis that a minimal amount of events is required to estimate motion (as in contrast maximization) is not valid when estimating motion with a neural network.
PDF
