Dense motion estimations obtained from optical flow techniques play a significant role in many image processing and computer vision tasks. Remarkable progress has been made in both theory and its ...application in practice. In this paper, we provide a systematic review of recent optical flow techniques with a focus on the variational method and approaches based on Convolutional Neural Networks (CNNs). These two categories have led to state-of-the-art performance. We discuss recent modifications and extensions of the original model, and highlight remaining challenges. For the first time, we provide an overview of recent CNN-based optical flow methods and discuss their potential and current limitations.
•Introducing optical flow: the basic concepts, the characteristics of the variational and CNN-based techniques, and the evaluation measures.•Discussing developments of the variational method, analyzing the challenges and illustrating the corresponding treating strategies of it.•Describing the conception of the CNN-based technique, and give a detailed discussion of the issues of this technique.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set ...for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Recent developments in techniques for modeling, digitizing and visualizing 3D shapes has led to an explosion in the number of available 3D models on the Internet and in domain-specific databases. ...This has led to the development of 3D shape retrieval systems that, given a query object, retrieve similar 3D objects. For visualization, 3D shapes are often represented as a surface, in particular polygonal meshes, for example in VRML format. Often these models contain holes, intersecting polygons, are not manifold, and do not enclose a volume unambiguously. On the contrary, 3D volume models, such as solid models produced by CAD systems, or voxels models, enclose a volume properly. This paper surveys the literature on methods for content based 3D retrieval, taking into account the applicability to surface models as well as to volume models. The methods are evaluated with respect to several requirements of content based 3D shape retrieval, such as: (1) shape representation requirements, (2) properties of dissimilarity measures, (3) efficiency, (4) discrimination abilities, (5) ability to perform partial matching, (6) robustness, and (7) necessity of pose normalization. Finally, the advantages and limitations of the several approaches in content based 3D shape retrieval are discussed.
Full text
Available for:
CEKLJ, EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
Cryo-electron tomography provides 3D images of macromolecules in their cellular context. To detect macromolecules in tomograms, template matching (TM) is often used, which uses 3D models that are ...often reliable for substantial parts of the macromolecules. However, the extent of rotational searches in particle detection has not been investigated due to computational limitations. Here, we provide a GPU implementation of TM as part of the PyTOM software package, which drastically speeds up the orientational search and allows for sampling beyond the Crowther criterion within a feasible timeframe. We quantify the improvements in sensitivity and false-discovery rate for the examples of ribosome identification and detection. Sampling at the Crowther criterion, which was effectively impossible with CPU implementations due to the extensive computation times, allows for automated extraction with high sensitivity. Consequently, we also show that an extensive angular sample renders 3D TM sensitive to the local alignment of tilt series and damage induced by focused ion beam milling. With this new release of PyTOM, we focused on integration with other software packages that support more refined subtomogram-averaging workflows. The automated classification of ribosomes by TM with appropriate angular sampling on locally corrected tomograms has a sufficiently low false-discovery rate, allowing for it to be directly used for high-resolution averaging and adequate sensitivity to reveal polysome organization.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
In recent years, serious energy games (SEGs) garnered increasing attention as an innovative and effective approach to tackling energy-related challenges. This review delves into the multifaceted ...landscape of SEG, specifically focusing on their wide-ranging applications in various contexts. The study investigates potential enhancements in user engagement achieved through integrating social connections, personalization, and data integration. Among the main challenges identified, previous studies overlooked the full potential of serious games in addressing emerging needs in energy systems, opting for oversimplified approaches. Further, these studies exhibit limited scalability and constrained generalizability, which poses challenges in applying their findings to larger energy systems and diverse scenarios. By incorporating lessons learned from prior experiences, this review aims to propel the development of SEG toward more innovative and impactful directions. It is firmly believed that positive behavior changes among individuals can be effectively encouraged by using SEG.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
6.
Multi-Dataset, Multitask Learning of Egocentric Vision Tasks Kapidis, Georgios; Poppe, Ronald; Veltkamp, Remco C.
IEEE transactions on pattern analysis and machine intelligence,
2023-June-1, 2023-Jun, 2023-6-1, 20230601, Volume:
45, Issue:
6
Journal Article
Peer reviewed
Open access
For egocentric vision tasks such as action recognition, there is a relative scarcity of labeled data. This increases the risk of overfitting during training. In this paper, we address this issue by ...introducing a multitask learning scheme that employs related tasks as well as related datasets in the training process. Related tasks are indicative of the performed action, such as the presence of objects and the position of the hands. By including related tasks as additional outputs to be optimized, action recognition performance typically increases because the network focuses on relevant aspects in the video. Still, the training data is limited to a single dataset because the set of action labels usually differs across datasets. To mitigate this issue, we extend the multitask paradigm to include datasets with different label sets. During training, we effectively mix batches with samples from multiple datasets. Our experiments on egocentric action recognition in the EPIC-Kitchens, EGTEA Gaze+, ADL and Charades-EGO datasets demonstrate the improvements of our approach over single-dataset baselines. On EGTEA we surpass the current state-of-the-art by 2.47 percent. We further illustrate the cross-dataset task correlations that emerge automatically with our novel training scheme.
The Netherlands police are looking for measures to examine sentiment on social media related to protest demonstrations. While models exist to detect more subtle expressions of sentiment within ...tweets, models trained in the Dutch language are scarce. Being able to predict sentiment development during protests is relevant for parties like the Dutch government and the police to get more insight to when and where potential law enforcement is needed for public order and safety. Therefore, to analyse sentiment before, during, and after protest demonstrations, data was collected with tweets related to a Black Lives Matter protest that took place in Amsterdam during the COVID-19 pandemic. All tweets have been manually labelled by a dedicated open-source intelligence (OSINT) team within the Netherlands police following an established protocol. Both the data and the protocol are available, and interesting for researchers in natural language processing, topic detection, sentiment analysis, and protests analysis. The developed labelling tool for the labelling process is publicly available.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
•We collected ground-truth data on induced musical emotion for 400 musical excerpts.•We designed an online game with a purpose to attract a big number of participants.•We analyzed inter-rater ...agreement on emotional terms from GEMS model.•We found that mood, gender and liking or disliking the music influence induced emotion.•We suggested improvements to GEMS scale.
One of the major reasons why people find music so enjoyable is its emotional impact. Creating emotion-based playlists is a natural way of organizing music. The usability of online music streaming services could be greatly improved by developing emotion-based access methods, and automatic music emotion recognition (MER) is the most quick and feasible way of achieving it. When resorting to music for emotional regulation purposes, users are interested in the MER method to predict their induced, or felt emotion. The progress of MER in this area is impeded by the absence of publicly accessible ground-truth data on musically induced emotion. Also, there is no consensus on the question which emotional model best fits the demands of the users and can provide an unambiguous linguistic framework to describe musical emotions. In this paper we address these problems by creating a sizeable publicly available dataset of 400 musical excerpts from four genres annotated with induced emotion. We collected the data using an online “game with a purpose” Emotify, which attracted a big and varied sample of participants. We employed a nine item domain-specific emotional model GEMS (Geneva Emotional Music Scale). In this paper we analyze the collected data and report agreement of participants on different categories of GEMS. We also analyze influence of extra-musical factors on induced emotion (gender, mood, music preferences). We suggest that modifications in GEMS model are necessary.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
One of the main principles of Deep Convolutional Neural Networks (CNNs) is the extraction of useful features through a hierarchy of kernels operations. The kernels are not explicitly tailored to ...address specific target classes but are rather optimized as general feature extractors. Distinction between classes is typically left until the very last fully-connected layers. Consequently, variances between classes that are relatively similar are treated the same way as variations between classes that exhibit great dissimilarities. In order to directly address this problem, we introduce Class Regularization, a novel method that can regularize feature map activations based on the classes of the examples used. Essentially, we amplify or suppress activations based on an educated guess of the given class. We can apply this step to each minibatch of activation maps, at different depths in the network. We demonstrate that this improves feature search during training, leading to systematic improvement gains on the Kinetics, UCF-101, and HMDB-51 datasets. Moreover, Class Regularization establishes an explicit correlation between features and class, which makes it a perfect tool to visualize class-specific features at various network depths.
IntroductionYoung people (aged 10–25 years) with chronic diseases are vulnerable to have reduced social participation and quality of life. It is important to empower young people to engage in their ...chronic diseases self-management. In comparison with traditional face-to-face care, interventions delivered through the internet and related technologies (eHealth) are less stigmatising and more accessible. Gamified eHealth self-management interventions may be particularly promising for young people. This systematic review aims at identifying (1) the game mechanics that have been implemented in eHealth interventions to support young people’s self-management of their chronic (somatic or psychiatric) diseases, (2) the investigators’ rationale for implementing such game mechanics and, if possible, (3) the effects of these interventions.Methods and analysisThe Preferred Reporting Items for Systematic reviews and Meta-Analysis statement guidelines will be followed. A systematic search of the literature will be conducted in Embase, Psycinfo and Web of Science from inception until 30 August 2022. Studies will be eligible if focused on (1) young people (aged 10–25 years) with chronic diseases and (2) describing gamified eHealth self-management interventions. When possible, the effects of the gamified interventions will be compared with non-gamified interventions or care-as-usual. Primary quantitative, qualitative or mixed-method studies written in English will be included. Two independent reviewers will (1) select studies, (2) extract and summarise the implemented game mechanics as well as the characteristics of the intervention and study, (3) evaluate their methodological quality and (4) synthesise the evidence. The reviewers will reach a consensus through discussion, and if required, a third researcher will be consulted.Ethics and disseminationAs systematic reviews use publicly available data, no formal ethical review and approval are needed. Findings will be published in peer-reviewed journals, presented at conferences and communicated to relevant stakeholders including patient organisations via the eHealth Junior Consortium.PROSPERO registration numberCRD42021293037.
PDF
