Dataset search: a survey Chapman, Adriane; Simperl, Elena; Koesten, Laura ...
The VLDB journal,
2020/1, Volume:
29, Issue:
1
Journal Article
Open access
Generating value from data requires the ability to find, access and make sense of datasets. There are many efforts underway to encourage data sharing and reuse, from scientific publishers asking ...authors to submit data alongside manuscripts to data marketplaces, open data portals and data communities. Google recently beta-released a search service for datasets, which allows users to discover data stored in various online repositories via keyword queries. These developments foreshadow an emerging research field around dataset search or retrieval that broadly encompasses frameworks, methods and tools that help match a user data need against a collection of datasets. Here, we survey the state of the art of research and commercial systems and discuss what makes dataset search a field in its own right, with unique challenges and open questions. We look at approaches and implementations from related areas dataset search is drawing upon, including information retrieval, databases, entity-centric and tabular search in order to identify possible paths to tackle these questions as well as immediate next steps that will take the field forward.
The massive increase in classroom video data enables the possibility of utilizing artificial intelligence technology to automatically recognize, detect and caption students’ behaviors. This is ...beneficial for related research, e.g., pedagogy and educational psychology. However, the lack of a dataset specifically designed for students’ classroom behaviors may block these potential studies. This paper presents a comprehensive dataset that can be employed for recognizing, detecting, and captioning students’ behaviors in a classroom. We collected videos of 128 classes in different disciplines and in 11 classrooms. Specifically, the constructed dataset consists of a detection part, recognition part, and captioning part. The detection part includes a temporal detection data module with 4542 samples and an action detection data module with 3343 samples, whereas the recognition part contains 4276 samples and the captioning part contains 4296 samples. Moreover, the students’ behaviors are spontaneous in real classes, rendering the dataset representative and realistic. We analyze the special characteristics of the classroom scene and the technical difficulties for each module (task), which are verified by experiments. Due to the particularity of classrooms, our datasets proposes increasing the requirements of existing methods. Moreover, we provide a baseline for each task module in the dataset and make a comparison with the current mainstream datasets. The results show that our dataset is viable and reliable. Additionally, we present a thorough performance analysis of each baseline model to provide a comprehensive comparison for models using our presented dataset. The dataset and code are available to download online:
https://github.com/BNU-Wu/Student-Class-Behavior-Dataset/tree/master
.
Cooperative machine learning has many applications, such as data annotation, where an initial model trained with partially labeled data is used to predict labels for unseen data continuously. ...Predicted labels with a low confidence value are manually revised to allow the model to be retrained with the predicted and revised data. In this paper, we propose an alternative to this approach: an initial training process called Deep Unsupervised Active Learning. Using the proposed training scheme, a classification model can incrementally acquire new knowledge during the testing phase without manual guidance or correction of decision making. The training process consists of two stages: the first stage of supervised training using a classification model, and an unsupervised active learning stage during the test phase. The labels predicted during the test phase, with high confidence, are continuously used to extend the knowledge base of the model. To optimize the proposed method, the model must have a high initial recognition rate. To this end, we exploited the Visual Geometric Group (VGG16) pre-trained model applied to three datasets: Mathematical Image Analysis (AMI), University of Science and Technology Beijing (USTB2), and Annotated Web Ears (AWE). This approach achieved impressive performance that shows a significant improvement in the recognition rate of the USTB2 dataset by coloring its images using a Generative Adversarial Network (GAN). The obtained performances are interesting compared to the current methods: the recognition rates are 100.00%, 98.33%, and 51.25% for the USTB2, AMI, and AWE datasets, respectively.
First-person captioning is significant because it provides veracious descriptions of egocentric scenes in a unique perspective. Also, there is a need to caption the scene, a.k.a. life-logging, for ...patients, travellers, and emergency responders in an egocentric narrative. Ego-captioning is indeed non-trivial since (1) Ego-images can be noisy due to motion and angles; (2) Describing a scene in a first-person narrative involves drastically different semantics; (3) Empirical implications have to be made on top of visual appearance because the cameraperson is often outside the field of view. We note we humans make good sense out of casual footage thanks to our contextual awareness in judging when and where the event unfolds, and whom the cameraperson is interacting with. This inspires the infusion of such “contexts” for situation-aware captioning. We create EgoCap which contains 2.1K ego-images, over 10K ego-captions, and 6.3K contextual labels, to close the gap of lacking ego-captioning datasets. We propose EgoFormer, a dual-encoder transformer-based network which fuses both contextual and visual features. The context encoder is pre-trained on ImageNet before fine tuning with context classification tasks. Similar to visual attention, we exploit stacked multi-head attention layers in the captioning decoder to reinforce attention to the context features. The EgoFormer has realized state-of-the-art performance on EgoCap achieving a CIDEr score of 125.52. The EgoCap dataset and EgoFormer are publicly available at https://github.com/zdai257/EgoCap-EgoFormer.
•First quantitative study of image captioning in an egocentric perspective.•Introduction of transformer-based context fusion architecture.•EgoCap, a sizable first-person image caption dataset, is released.
High precision precipitation products are the basis of precipitation-related research. Based on 27 global climate models (GCMs) in the Coupled Model Intercomparison Project phase 6 (CMIP6), we ...designed eight schemes for comprehensively using the empirical quantile mapping (EQM) method and data ensemble method to conduct precipitation bias correction; then, we selected the scheme with the highest accuracy as the final bias correction scheme. Using the selected bias correction scheme, we created a monthly precipitation dataset with a 1° spatial resolution, which spans the historical period of 1961–2014 and the future period of 2015–2099 under three shared socioeconomic pathway (SSP) scenarios: SSP126, SSP245, and SSP585. The corrected precipitation data were validated using the CN05.1 grid precipitation dataset from the China Meteorological Data Sharing Network and were compared with the ERA5 precipitation data from the European Centre for Medium-Range Weather Forecasts. The dataset was also utilized for future prediction of alternating drought and flood events in China. The results show that this best bias correction scheme is the first to integrate precipitation simulation data from 27 GCMs using the random forest (RF) model and then the EQM method to further correct the integrated precipitation data. The corrected precipitation data are better than the original GCM precipitation data in terms of both the monthly precipitation and extreme precipitation. From the perspective of the monthly precipitation, the difference between the ERA5 and RF-EQM is small, but the extreme precipitation of the RF-EQM clearly outperforms the ERA5 extreme precipitation. For the annual maximum (minimum) monthly precipitation, the correlation coefficient, the RMSD (standardized), and the STD (standardized) between the ERA5 and CN05.1 are 0.925 (0.743), 0.474 (1.223), and 1.207 (1.765), respectively; the correlation coefficient, the RMSD (standardized), and the STD (standardized) between the RF-EQM and CN05.1 are 0.947 (0.735), 0.337 (0.837), and 0.849 (1.226), respectively. The occurrence frequency of DF (an abrupt change from drought to flood) events is continuously increasing in all scenarios, with the highest frequency observed under the SSP585 scenario. The increase in FD (an abrupt change from flood to drought) event frequency is not pronounced. This study expands the method for bias correction of meteorological data and provides a reference for other climate parameters and precipitation bias correction in other regions.
•This study proposes RF-EQM, a bias correction method that utilizes random forest and empirical quantile mapping.•Using 27 global climate models, this study created a high-precision monthly precipitation dataset spanning 1961–2099.•The new precipitation dataset encompasses three socioeconomic scenarios pathway (SSP126, SSP245, and SSP585).•A continuous increase in abrupt transitions from drought to flood events in China is predicted, especially under SSP585.
This paper is a comprehensive survey of datasets for surgical tool detection and related surgical data science and machine learning techniques and algorithms. The survey offers a high level ...perspective of current research in this area, analyses the taxonomy of approaches adopted by researchers using surgical tool datasets, and addresses key areas of research, such as the datasets used, evaluation metrics applied and deep learning techniques utilised. Our presentation and taxonomy provides a framework that facilitates greater understanding of current work, and highlights the challenges and opportunities for further innovative and useful research.
Recognizing cross-subject emotions based on brain imaging data, e.g., EEG, has always been difficult due to the poor generalizability of features across subjects. Thus, systematically exploring the ...ability of different EEG features to identify emotional information across subjects is crucial. Prior related work has explored this question based only on one or two kinds of features, and different findings and conclusions have been presented. In this work, we aim at a more comprehensive investigation on this question with a wider range of feature types, including 18 kinds of linear and non-linear EEG features. The effectiveness of these features was examined on two publicly accessible datasets, namely, the dataset for emotion analysis using physiological signals (DEAP) and the SJTU emotion EEG dataset (SEED). We adopted the support vector machine (SVM) approach and the "leave-one-subject-out" verification strategy to evaluate recognition performance. Using automatic feature selection methods, the highest mean recognition accuracy of 59.06% (AUC = 0.605) on the DEAP dataset and of 83.33% (AUC = 0.904) on the SEED dataset were reached. Furthermore, using manually operated feature selection on the SEED dataset, we explored the importance of different EEG features in cross-subject emotion recognition from multiple perspectives, including different channels, brain regions, rhythms, and feature types. For example, we found that the Hjorth parameter of mobility in the beta rhythm achieved the best mean recognition accuracy compared to the other features. Through a pilot correlation analysis, we further examined the highly correlated features, for a better understanding of the implications hidden in those features that allow for differentiating cross-subject emotions. Various remarkable observations have been made. The results of this paper validate the possibility of exploring robust EEG features in cross-subject emotion recognition.
There are numerous channels available such as social media, blogs, websites, etc., through which people can easily access the news. It is due to the availability of these platforms that the ...dissemination of fake news has become easier. Anyone using these platforms can create and share fake news content based on personal or professional motives. To address the issue of detecting fake news, numerous studies based on supervised and unsupervised learning methods have been proposed. However, all those studies do suffer from a certain limitation of poor accuracy. The reason for poor accuracy can be attributed due to several reasons such as the poor selection of features, inefficient tuning of parameters, imbalanced datasets, etc. In this article, we have proposed an ensemble classification model for detection of the fake news that has achieved a better accuracy compared to the state-of-the-art. The proposed model extracts important features from the fake news datasets, and the extracted features are then classified using the ensemble model comprising of three popular machine learning models namely, Decision Tree, Random Forest and Extra Tree Classifier. We achieved a training and testing accuracy of 99.8% and 44.15% respectively on the Liar dataset. For the ISOT dataset, we achieved the training and testing accuracy of 100%.
•Extraction of the most significant features influencing classification of fake news.•Using an Ensemble model to achieve optimial accuracy to classify fake news datasets.•Reduction of training time of the ensemble classifier.
Understanding the spatial and temporal occurrences of extreme rainfall events and their changes is crucial for reducing the risk associated with extreme events, especially under anthropogenic ...warming. This study presents a comparative analysis of twelve gridded rainfall datasets in representing the spatial and temporal variation of extreme rainfall events and trends in the recent past (1983–2015) in India. The selected datasets fall into four categories: gauge-based (APHRODITE, CPC, GPCC, REGEN), satellite-derived (CHIRPS, PERSIANN-CDR), reanalysis (ERA5, MERRA-2, PGF, and JRA-55), and product merged from these three types (WFDEI and MSWEP). For comparison, a gauge-based, high resolution (0.25° × 0.25°) daily gridded rainfall dataset, prepared by the India Meteorological Department (IMD) is used as a reference dataset. Eleven extreme climate indices, defined by the World Meteorological Organization's Expert Team on Climate Change Detection and Indices (ETCCDI) and the IMD are used to represent the magnitude, frequency, and duration of extreme rainfall events in India. The relative performance of these datasets in capturing extreme rainfall events is evaluated by computing model evaluation matrices i.e. correlation coefficient (CC) and Percentage Bias (PBIAS). Trends in extreme rainfall indices and their magnitudes are calculated using Mann-Kendell and Sen's slope methods respectively, to compare with the IMD dataset. Our study finds large uncertainties in representing extreme rainfall events where in most cases the datasets underestimate higher extreme events compared to the IMD data. GPCC and MSWEP better capture the magnitude, duration, and frequency of extreme rainfall events in India and are comparable to the pattern of the IMD dataset. CHIRPS and ERA5 perform better than other satellite and reanalysis rainfall datasets. All these selected datasets consistently underestimate extreme rainfall events over the northern Himalayan region. The gridded datasets are unable to capture the spatial pattern of observed trends in extreme rainfall indices in India when compared to the IMD dataset.
•Extreme rainfall characteristics over India studied using gauge, satellite, reanalysis, and merged gridded datasets.•Elevan ETCCDI and IMD indices were used to study extreme rainfall events over India•GPCC, CHIRPS, ERA5, and MSWEP better performed in gauge, satellite, reanalysis, and merged datasets respectively.•Datasets are incongruous in representing the observed trends in extreme rainfall indices over India.
PDF
