•First comprehensive annotated datasets for computer vision tasks in plant phenotyping.•Publicly available data and evaluation criteria for eight challenging tasks.•Tasks include fine-grained ...categorization of age, developmental stage, and cultivars.•Example test cases and results on plant and leaf-wise segmentation and leaf counting.
In this paper we present a collection of benchmark datasets for the development and evaluation of computer vision and machine learning algorithms in the context of plant phenotyping. We provide annotated imaging data and suggest suitable evaluation criteria for plant/leaf segmentation, detection, tracking as well as classification and regression problems. The Figure symbolically depicts the data available together with ground truth segmentations and further annotations and metadata. Display omitted
Image-based approaches to plant phenotyping are gaining momentum providing fertile ground for several interesting vision tasks where fine-grained categorization is necessary, such as leaf segmentation among a variety of cultivars, and cultivar (or mutant) identification. However, benchmark data focusing on typical imaging situations and vision tasks are still lacking, making it difficult to compare existing methodologies. This paper describes a collection of benchmark datasets of raw and annotated top-view color images of rosette plants. We briefly describe plant material, imaging setup and procedures for different experiments: one with various cultivars of Arabidopsis and one with tobacco undergoing different treatments. We proceed to define a set of computer vision and classification tasks and provide accompanying datasets and annotations based on our raw data. We describe the annotation process performed by experts and discuss appropriate evaluation criteria. We also offer exemplary use cases and results on some tasks obtained with parts of these data. We hope with the release of this rigorous dataset collection to invigorate the development of algorithms in the context of plant phenotyping but also provide new interesting datasets for the general computer vision community to experiment on. Data are publicly available at http://www.plant-phenotyping.org/datasets.
A successful high-throughput image-based phenotyping system starts with the imaging approach itself. The choices are to image many plants simultaneously or one plant at a time, requiring movable ...systems to bring the plant to the camera or vice versa. These systems add cost but have the benefit of isolating the object of interest. In turn, this simplifies its processing, for example facilitating object segmentation, in other words the image analysis process isolating the plant from background. Here, Tsaftaris et al emphasize the role that image analysis and processing have in the phenotyping pipeline, both in analyzing phenotyping data and when providing effective feature extraction to be used by machine learning.
•Simulation reveals influences of PSF calibration procedure on signal deconvolution.•Deconvolving measured PSFs before deconvolving signals improves performance.•Classic Wiener filtering together ...with sharpened PSF yields best results.•Proposed deconvolution approach reduces error in SIF estimation by factor 3.8
We propose a signal deconvolution procedure for imaging spectrometer data, where a measured point spread function (PSF) is deconvolved itself before being used for deconvolution of the signal. We evaluate the effectiveness of our procedure for improvement of the spatio-spectral signal, as well as our target application, i.e. estimation of sun-induced fluorescence (SIF). Imaging spectrometers are well established instruments for remote sensing. When used for scientific purposes these instruments are usually calibrated on a regular basis. In our case the point spread function of the optics is measured in an elaborate procedure with a tunable monochromator point light source. PSFs are measured at different pixel positions of the imaging sensor, i.e. at different spatio-spectral locations, and averaged in order to get an as accurate PSF as possible. We investigate error sources in this calibration process by simulating the procedure in silico. Averaging as well as the spectral and spatial width of the point source introduce some smoothness in the measured PSF. We propose corrective measures, i.e. deconvolution of the PSF itself and median instead of mean averaging, leading to a set of sharper PSFs. We test the performance of these PSFs in deconvolving simulated as well as real hyperspectral images. For deconvolution we test a set of well-known, off the shelf deconvolution algorithms. Quantitatively in terms of PSNR (Peak Signal to Noise Ratio) a combination of Wiener filtering and sharpened PSFs yields strongest improvements, while using Wiener filtering with non-sharpened PSFs even deteriorates the signal. Comparing deconvolution results of the simulated data with results of real data reveals, that visually very similar effects can be observed. This well supports the assumption, that our findings are also valid for real spatio-spectral data. Surprisingly, the choice of PSF, sharpened or not, is of little effect for SIF estimation with the iFLD algorithm in the O2A band. Quantitatively we find that deconvolution reduces the overall error of SIF by a factor of 3.8, when using Wiener filtering instead of the currently used 1 iteration of vanCittert's method. For SIF estimation in the O2B band we observe a totally different behavior, where all deconvolution methods yield unreliable results with mostly well above 200% relative error and high standard deviations. In the discussion we can only speculate on possible reasons for this unreliability. As conclusion we therefore propose to use the O2A band for SIF estimation together with classic Wiener filtering for deconvolution of spatio-spectral data.
In 2014 plant phenotyping research was not benefiting from the machine learning (ML) revolution because appropriate data were lacking. We report the success of the first open-access dataset suitable ...for ML in image-based plant phenotyping suitable for machine learning, fuelling a true interdisciplinary symbiosis, increased awareness, and steep performance improvements on key phenotyping tasks.
The enormous diversity of seed traits is an intriguing feature and critical for the overwhelming success of higher plants. In particular, seed mass is generally regarded to be key for seedling ...development but is mostly approximated by using scanning methods delivering only two-dimensional data, often termed seed size. However, three-dimensional traits, such as the volume or mass of single seeds, are very rarely determined in routine measurements. Here, we introduce a device named phenoSeeder, which enables the handling and phenotyping of individual seeds of very different sizes. The system consists of a pick-and-place robot and a modular setup of sensors that can be versatilely extended. Basic biometric traits detected for individual seeds are two-dimensional data from projections, three-dimensional data from volumetric measures, and mass, from which seed density is also calculated. Each seed is tracked by an identifier and, after phenotyping, can be planted, sorted, or individually stored for further evaluation or processing (e.g. in routine seed-to-plant tracking pipelines). By investigating seeds of Arabidopsis (Arabidopsis thaliana), rapeseed (Brassica napus), and barley (Hordeum vulgare), we observed that, even for apparently round-shaped seeds of rapeseed, correlations between the projected area and the mass of seeds were much weaker than between volume and mass. This indicates that simple projections may not deliver good proxies for seed mass. Although throughput is limited, we expect that automated seed phenotyping on a single-seed basis can contribute valuable information for applications in a wide range of wild or crop species, including seed classification, seed sorting, and assessment of seed quality.
Image-based plant phenotyping is a growing application area of computer vision in agriculture. A key task is the segmentation of all individual leaves in images. Here we focus on the most common ...rosette model plants, Arabidopsis and young tobacco. Although leaves do share appearance and shape characteristics, the presence of occlusions and variability in leaf shape and pose, as well as imaging conditions, render this problem challenging. The aim of this paper is to compare several leaf segmentation solutions on a unique and first-of-its-kind dataset containing images from typical phenotyping experiments. In particular, we report and discuss methods and findings of a collection of submissions for the first Leaf Segmentation Challenge of the Computer Vision Problems in Plant Phenotyping workshop in 2014. Four methods are presented: three segment leaves by processing the distance transform in an unsupervised fashion, and the other via optimal template selection and Chamfer matching. Overall, we find that although separating plant from background can be accomplished with satisfactory accuracy (
>
90 % Dice score), individual leaf segmentation and counting remain challenging when leaves overlap. Additionally, accuracy is lower for younger leaves. We find also that variability in datasets does affect outcomes. Our findings motivate further investigations and development of specialized algorithms for this particular application, and that challenges of this form are ideally suited for advancing the state of the art. Data are publicly available (online at
http://www.plant-phenotyping.org/datasets
) to support future challenges beyond segmentation within this application domain.
Plants constantly adapt their leaf orientation in response to fluctuations in the environment, to maintain radiation use efficiency in the face of varying intensity and incidence direction of ...sunlight. Various methods exist for measuring structural canopy parameters such as leaf angle distribution. However, direct methods tend to be labour-intensive, while indirect methods usually give statistical information on stand level rather than on individual leaves. We present an area-based, binocular stereo system composed of commercially available components that allows three-dimensional reconstruction of small- to medium-sized canopies on the level of single leaves under field conditions. Spatial orientation of single leaves is computed with automated processes using modern, well-established stereo matching and segmentation techniques, which were adapted for the properties of plant canopies, providing high spatial and temporal resolution (angle measurements with an accuracy of approx. ±5° and a maximum sampling rate of three frames per second). The applicability of our approach is demonstrated in three case studies: (1) the dihedral leaflet angle of an individual soybean was tracked to monitor nocturnal and daytime leaf movement showing different frequencies and amplitudes; (2) drought stress was diagnosed in soybean by quantifying changes in the zenith leaflet angle distribution; and (3) the diurnal course of the zenith leaf angle distribution of a closed soybean canopy was measured.
Growth Signatures of Rosette Plants from Time-Lapse Video Dellen, Babette; Scharr, Hanno; Torras, Carme
IEEE/ACM transactions on computational biology and bioinformatics,
11/2015, Letnik:
12, Številka:
6
Journal Article, Publication
Recenzirano
Odprti dostop
Plant growth is a dynamic process, and the precise course of events during early plant development is of major interest for plant research. In this work, we investigate the growth of rosette plants ...by processing time-lapse videos of growing plants, where we use Nicotiana tabacum (tobacco) as a model plant. In each frame of the video sequences, potential leaves are detected using a leaf-shape model. These detections are prone to errors due to the complex shape of plants and their changing appearance in the image, depending on leaf movement, leaf growth, and illumination conditions. To cope with this problem, we employ a novel graph-based tracking algorithm which can bridge gaps in the sequence by linking leaf detections across a range of neighboring frames. We use the overlap of fitted leaf models as a pairwise similarity measure, and forbid graph edges that would link leaf detections within a single frame. We tested the method on a set of tobacco-plant growth sequences, and could track the first leaves of the plant, including partially or temporarily occluded ones, along complete sequences, demonstrating the applicability of the method to automatic plant growth analysis. All seedlings displayed approximately the same growth behavior, and a characteristic growth signature was found.
We describe a method for 3D reconstruction of plant seed surfaces, focusing on small seeds with diameters as small as 200 μm. The method considers robotized systems allowing single seed handling in ...order to rotate a single seed in front of a camera. Even though such systems feature high position repeatability, at sub-millimeter object scales, camera pose variations have to be compensated. We do this by robustly estimating the tool center point from each acquired image. 3D reconstruction can then be performed by a simple shape-from-silhouette approach. In experiments we investigate runtimes, theoretically achievable accuracy, experimentally achieved accuracy, and show as a proof of principle that the proposed method is well sufficient for 3D seed phenotyping purposes.
BACKGROUND: Three-dimensional canopies form complex architectures with temporally and spatially changing leaf orientations. Variations in canopy structure are linked to canopy function and they occur ...within the scope of genetic variability as well as a reaction to environmental factors like light, water and nutrient supply, and stress. An important key measure to characterize these structural properties is the leaf angle distribution, which in turn requires knowledge on the 3-dimensional single leaf surface. Despite a large number of 3-d sensors and methods only a few systems are applicable for fast and routine measurements in plants and natural canopies. A suitable approach is stereo imaging, which combines depth and color information that allows for easy segmentation of green leaf material and the extraction of plant traits, such as leaf angle distribution. RESULTS: We developed a software package, which provides tools for the quantification of leaf surface properties within natural canopies via 3-d reconstruction from stereo images. Our approach includes a semi-automatic selection process of single leaves and different modes of surface characterization via polygon smoothing or surface model fitting. Based on the resulting surface meshes leaf angle statistics are computed on the whole-leaf level or from local derivations. We include a case study to demonstrate the functionality of our software. 48 images of small sugar beet populations (4 varieties) have been analyzed on the base of their leaf angle distribution in order to investigate seasonal, genotypic and fertilization effects on leaf angle distributions. We could show that leaf angle distributions change during the course of the season with all varieties having a comparable development. Additionally, different varieties had different leaf angle orientation that could be separated in principle component analysis. In contrast nitrogen treatment had no effect on leaf angles. CONCLUSIONS: We show that a stereo imaging setup together with the appropriate image processing tools is capable of retrieving the geometric leaf surface properties of plants and canopies. Our software package provides whole-leaf statistics but also a local estimation of leaf angles, which may have great potential to better understand and quantify structural canopy traits for guided breeding and optimized crop management.
PDF
