Music information retrieval (MIR) is developing these years rapidly. As the fundamental MIR tasks, automatic music transcription (AMT) and expressive analysis (EA) are gaining momentum in both ...Western and non-European music. However, the annotated datasets for non-Eurogenic instruments remain scarce in terms of quantity and feature diversity so that general evaluations and data-driven models on various tasks cannot be well explored. As one of the most popular traditional plucked string instruments in Asia, which is barely studied in the MIR community, pipa has lots of distinctive national and local characteristics including the fake nails, intrinsic pitch shift, rubato, as well as a high diversity of sophisticated playing techniques that greatly enhance the music expressiveness. Our work aims to systematically clarify a creation procedure of a pipa dataset with audio, musical notation and multiview video modalities for traditional Chinese solos. The use of 4-track string vibration signals captured by optical sensors paves a path for high quality annotations. Furthermore, a transcription and Expressiveness Annotation System (TEAS) was transparently implemented to ensure the scalability of dataset. Three expressive analysis approaches in this system were newly proposed and evaluated in this paper. Finally, two AMT models were investigated and a series of the existing and emerging MIR tasks enabled by this dataset were enumerated for the future exploration.
Humpback whales (
) use vocalizations during diverse social interactions or activities such as foraging or mating. Unlike songs produced only by males, social calls are produced by all types of ...individuals (adult males and females, juveniles and calves). Several studies have described social calls in the humpback whale's breeding and the feeding grounds and from different geographic areas. We aimed to investigate for the first time the vocal repertoire of humpback whale mother-calf groups during the breeding season off Sainte Marie island, Madagascar, South Western Indian Ocean using data collected in 2013, 2014, 2016, and 2017. We recorded social calls using Acousonde tags deployed on the mother or the calf in mother-calf groups. A total of 21 deployments were analyzed. We visually and aurally identified 30 social call types and classified them into five categories: low, medium, high-frequency sounds, amplitude-modulated sounds, and pulsed sounds. The aural-visual classifications have been validated using random forest (RF) analyses. Low-frequency sounds constituted 46% of all social calls, mid-frequency 35%, and high frequency 10%. Amplitude-modulated sounds constituted 8% of all vocalizations, and pulsed sounds constituted 1%. While some social call types seemed specific to our study area, others presented similarities with social calls described in other geographic areas, on breeding and foraging grounds, and during migrating routes. Among the call types described in this study, nine call types were also found in humpback whale songs recorded in the same region. The 30 call types highlight the diversity of the social calls recorded in mother-calf groups and thus the importance of acoustic interactions in the relationships between the mother and her calf and between the mother-calf pair and escorts.
Getting maternal milk through nursing is vital for all newborn mammals. Despite its importance, nursing has been poorly documented in humpback whales (
). Nursing is difficult to observe underwater ...without disturbing the whales and is usually impossible to observe from a ship. We attempted to observe nursing from the calf's perspective by placing CATS cam tags on three humpback whale calves in the Sainte Marie channel, Madagascar, Indian Ocean, during the breeding seasons. CATS cam tags are animal-borne multi-sensor tags equipped with a video camera, a hydrophone, and several auxiliary sensors (including a 3-axis accelerometer, a 3-axis magnetometer, and a depth sensor). The use of multi-sensor tags minimized potential disturbance from human presence. A total of 10.52 h of video recordings were collected with the corresponding auxiliary data. Video recordings were manually analyzed and correlated with the auxiliary data, allowing us to extract different kinematic features including the depth rate, speed, Fluke Stroke Rate (FSR), Overall Body Dynamic Acceleration (ODBA), pitch, roll, and roll rate. We found that suckling events lasted 18.8 ± 8.8 s on average (
= 34) and were performed mostly during dives. Suckling events represented 1.7% of the total observation time. During suckling, the calves were visually estimated to be at a 30-45° pitch angle relative to the midline of their mother's body and were always observed rolling either to the right or to the left. In our auxiliary dataset, we confirmed that suckling behavior was primarily characterized by a high average absolute roll and additionally we also found that it was likely characterized by a high average FSR and a low average speed. Kinematic features were used for supervised machine learning in order to subsequently detect suckling behavior automatically. Our study is a proof of method on which future investigations can build upon. It opens new opportunities for further investigation of suckling behavior in humpback whales and the baleen whale species.
We describe an art–science project called “Feral Interactions—The Answer of the Humpback Whale” inspired by humpback whale songs and interactions between individuals based on mutual influences, ...learning process, or ranking in the dominance hierarchy. The aim was to build new sounds that can be used to initiate acoustic interactions with these whales, not in a one-way direction, as playbacks do, but in real interspecies exchanges. Thus, we investigated how the humpback whales generate sounds in order to better understand their abilities and limits. By carefully listening to their emitted vocalizations, we also describe their acoustic features and temporal structure, in a scientific way and also with a musical approach as it is done with
musique concrète
, in order to specify the types and the morphologies of whale sounds. The idea is to highlight the most precise information to generate our own sounds that will be suggested to the whales. Based on the approach developed in
musique concrète
, similarities with the sounds produced by bassoon were identified and then were processed to become “concrete sound elements.” This analysis also brought us to design a new music interface that allows us to create adapted musical phrases in real-time. With this approach, interactions will be possible in both directions, from and to whales.
Vocal communication is widespread in animals, with vocal repertoires of varying complexity. The social complexity hypothesis predicts that species may need high vocal complexity to deal with complex ...social organization (e.g. have a variety of different interindividual relations). We quantified the vocal complexity of two geographically distant captive colonies of rooks, a corvid species with complex social organization and cognitive performances, but understudied vocal abilities. We quantified the diversity and gradation of their repertoire, as well as the inter-individual similarity at the vocal unit level. We found that males produced call units with lower diversity and gradation than females, while song units did not differ between sexes. Surprisingly, while females produced highly similar call repertoires, even between colonies, each individual male produced almost completely different call repertoires from any other individual. These findings question the way male rooks communicate with their social partners. We suggest that each male may actively seek to remain vocally distinct, which could be an asset in their frequently changing social environment. We conclude that inter-individual similarity, an understudied aspect of vocal repertoires, should also be considered as a measure of vocal complexity.
The study of cetacean vocalizations is usually based on spectrogram analysis. The feature extraction is obtained from 2D methods like the
edge detection
algorithm. Difficulties appear when ...signal-to-noise ratios are weak or when more than one vocalization is simultaneously emitted. This is the case for acoustic observations in a natural environment and especially for the killer whales which swim in groups. To resolve this problem, we propose the use of the Hilbert-Huang transform. First, we illustrate how few modes (5) are satisfactory for the analysis of these calls. Then, we detail our approach which consists of combining the modes for extracting the time-varying frequencies of the vocalizations. This combination takes advantage of one of the empirical mode decomposition properties which is that the successive IMFs represent the original data broken down into frequency components from highest to lowest frequency. To evaluate the performance, our method is first applied on the simulated chirp signals. This approach allows us to link one chirp to one mode. Then we apply it on real signals emitted by killer whales. The results confirm that this method is a favorable alternative for the automatic extraction of killer whale vocalizations.
Following a production-based approach, this paper deals with the acoustic behavior of humpback whales. This approach investigates various physical factors, which are either internal (e.g., ...physiological mechanisms) or external (e.g., environmental constraints) to the respiratory tractus of the whale, for their implications in sound production. This paper aims to describe a functional scenario of this tractus for the generation of vocal sounds. To do so, a division of this tractus into three different configurations is proposed, based on the air recirculation process which determines air sources and laryngeal valves. Then, assuming a vocal function (in sound generation or modification) for several specific anatomical components, an acoustic characterization of each of these configurations is proposed to link different spectral features, namely, fundamental frequencies and formant structures, to specific vocal production mechanisms. A discussion around the question of whether the whale is able to fully exploit the acoustic potential of its respiratory tractus is eventually provided.
PDF
