Data Analysis WorkbeNch (DAWN) Basham, Mark; Filik, Jacob; Wharmby, Michael T. ...
Journal of synchrotron radiation,
20/May , Volume:
22, Issue:
3
Journal Article
Peer reviewed
Open access
Synchrotron light source facilities worldwide generate terabytes of data in numerous incompatible data formats from a wide range of experiment types. The Data Analysis WorkbeNch (DAWN) was developed ...to address the challenge of providing a single visualization and analysis platform for data from any synchrotron experiment (including single‐crystal and powder diffraction, tomography and spectroscopy), whilst also being sufficiently extensible for new specific use case analysis environments to be incorporated (e.g. ARPES, PEEM). In this work, the history and current state of DAWN are presented, with two case studies to demonstrate specific functionality. The first is an example of a data processing and reduction problem using the generic tools, whilst the second shows how these tools can be targeted to a specific scientific area.
Full text
Available for:
FZAB, GIS, IJS, IZUM, KILJ, NLZOH, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBMB, UL, UM, UPUK
Tomographic datasets collected at synchrotrons are becoming very large and complex, and, therefore, need to be managed efficiently. Raw images may have high pixel counts, and each pixel can be ...multidimensional and associated with additional data such as those derived from spectroscopy. In time-resolved studies, hundreds of tomographic datasets can be collected in sequence, yielding terabytes of data. Users of tomographic beamlines are drawn from various scientific disciplines, and many are keen to use tomographic reconstruction software that does not require a deep understanding of reconstruction principles. We have developed Savu, a reconstruction pipeline that enables users to rapidly reconstruct data to consistently create high-quality results. Savu is designed to work in an 'orthogonal' fashion, meaning that data can be converted between projection and sinogram space throughout the processing workflow as required. The Savu pipeline is modular and allows processing strategies to be optimized for users' purposes. In addition to the reconstruction algorithms themselves, it can include modules for identification of experimental problems, artefact correction, general image processing and data quality assessment. Savu is open source, open licensed and 'facility-independent': it can run on standard cluster infrastructure at any institution.
Full text
Available for:
BFBNIB, NMLJ, NUK, PNG, SAZU, UL, UM, UPUK
Veteran professors synthesize their combined 60+ years of expertise at primarily undergraduate, teaching-focused universities into easy-to-follow advice for graduate students and current faculty ...seeking to build thriving careers at similar institutions. Writing in a friendly tone that includes their personal reflections, the authors guide readers through the entire career trajectory: finding and applying for positions, developing essential knowledge and skills over the course of one's career, seeking tenure and promotions, and continuing to thrive in the mid- to late-career stages while preparing for retirement. The authors offer detailed insights for becoming a successful academic who can meet all the expectations of a teaching-focused institution. They explain how to develop core teaching competencies; choose advising philosophies for mentoring individual students, groups, and clubs; perform high-quality faculty service; and achieve scholarly, creative, and research goals--all while managing a high teaching load. Strategies for obtaining scarce yet crucial resources--time, money, and mentors--are also provided.
I12 is the Joint Engineering, Environmental and Processing (JEEP) beamline, constructed during Phase II of the Diamond Light Source. I12 is located on a short (5 m) straight section of the Diamond ...storage ring and uses a 4.2 T superconducting wiggler to provide polychromatic and monochromatic X‐rays in the energy range 50–150 keV. The beam energy enables good penetration through large or dense samples, combined with a large beam size (1 mrad horizontally × 0.3 mrad vertically). The beam characteristics permit the study of materials and processes inside environmental chambers without unacceptable attenuation of the beam and without the need to use sample sizes which are atypically small for the process under study. X‐ray techniques available to users are radiography, tomography, energy‐dispersive diffraction, monochromatic and white‐beam two‐dimensional diffraction/scattering and small‐angle X‐ray scattering. Since commencing operations in November 2009, I12 has established a broad user community in materials science and processing, chemical processing, biomedical engineering, civil engineering, environmental science, palaeontology and physics.
Full text
Available for:
FZAB, GIS, IJS, IZUM, KILJ, NLZOH, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBMB, UL, UM, UPUK
Recently, several convolutional neural networks have been proposed not only for 2D images, but also for 3D and 4D volume segmentation. Nevertheless, due to the large data size of the latter, ...acquiring a sufficient amount of training annotations is much more strenuous than in 2D images. For 4D time-series tomograms, this is usually handled by segmenting the constituent tomograms independently through time with 3D convolutional neural networks. Inter-volume information is therefore not utilized, potentially leading to temporal incoherence. In this paper, we attempt to resolve this by proposing two hidden Markov model variants that refine 4D segmentation labels made by 3D convolutional neural networks working on each time point. Our models utilize not only inter-volume information, but also the prediction confidence generated by the 3D segmentation convolutional neural networks themselves. To the best of our knowledge, this is the first attempt to refine 4D segmentations made by 3D convolutional neural networks using hidden Markov models. During experiments we test our models, qualitatively, quantitatively and behaviourally, using prespecified segmentations. We demonstrate in the domain of time series tomograms which are typically undersampled to allow more frequent capture; a particularly challenging problem. Finally, our dataset and code is publicly available.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
The challenge of processing big data effectively and efficiently is crucial for many synchrotron facilities which can collect up to several petabytes of data annually. At Diamond Light Source, the ...tomographic data is reconstructed with Python-based software Savu which utilises Message Passing Interface protocols to efficiently reconstruct parallel beam geometry data. When projection data is undersampled and/or noisy, regularised iterative reconstruction methods can provide a better reconstruction quality than direct methods. The iterative methods, however, require significantly more computational resources than direct methods and their usability is impeded by the choice of additional hyper-parameters. Notably, the use of 2D regularised iterative methods for reconstruction of 3D objects results in inconsistent (saw-shaped) features in a perpendicular to slicing orientation. Due to large data sizes, fully 3D regularised model-based iterative reconstruction is problematic or impossible in practice due to high memory requirements and long processing times. In this work, we demonstrate a practical solution which enables an approximated full 3D regularised iterative reconstruction running in parallel on a computing cluster. This modification delivers an equivalent to exact 3D reconstruction quality of data volumes with a high computational efficiency.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Serial focussed ion beam scanning electron microscopy (FIB/SEM) enables imaging and assessment of subcellular structures on the mesoscale (10 nm to 10 µm). When applied to vitrified samples, serial ...FIB/SEM is also a means to target specific structures in cells and tissues while maintaining constituents' hydration shells for in situ structural biology downstream. However, the application of serial FIB/SEM imaging of non-stained cryogenic biological samples is limited due to low contrast, curtaining, and charging artefacts. We address these challenges using a cryogenic plasma FIB/SEM. We evaluated the choice of plasma ion source and imaging regimes to produce high-quality SEM images of a range of different biological samples. Using an automated workflow we produced three-dimensional volumes of bacteria, human cells, and tissue, and calculated estimates for their resolution, typically achieving 20-50 nm. Additionally, a tag-free localisation tool for regions of interest is needed to drive the application of in situ structural biology towards tissue. The combination of serial FIB/SEM with plasma-based ion sources promises a framework for targeting specific features in bulk-frozen samples (>100 µm) to produce lamellae for cryogenic electron tomography.
In cryo-electron tomography (cryo-ET) of biological samples, the quality of tomographic reconstructions can vary depending on the transmission electron microscope (TEM) instrument and data ...acquisition parameters. In this paper, we present Parakeet, a 'digital twin' software pipeline for the assessment of the impact of various TEM experiment parameters on the quality of three-dimensional tomographic reconstructions. The Parakeet digital twin is a digital model that can be used to optimize the performance and utilization of a physical instrument to enable
optimization of sample geometries, data acquisition schemes and instrument parameters. The digital twin performs virtual sample generation, TEM image simulation, and tilt series reconstruction and analysis within a convenient software framework. As well as being able to produce physically realistic simulated cryo-ET datasets to aid the development of tomographic reconstruction and subtomogram averaging programs, Parakeet aims to enable convenient assessment of the effects of different microscope parameters and data acquisition parameters on reconstruction quality. To illustrate the use of the software, we present the example of a quantitative analysis of missing wedge artefacts on simulated planar and cylindrical biological samples and discuss how data collection parameters can be modified for cylindrical samples where a full 180° tilt range might be measured.
Methane (CH4) hydrate dissociation and CH4 release are potential geohazards currently investigated using X-ray computed tomography (XCT). Image segmentation is an important data processing step for ...this type of research. However, it is often time consuming, computing resource-intensive, operator-dependent, and tailored for each XCT dataset due to differences in greyscale contrast. In this paper, an investigation is carried out using U-Nets, a class of Convolutional Neural Network, to segment synchrotron XCT images of CH4-bearing sand during hydrate formation, and extract porosity and CH4 gas saturation. Three U-Net deployments previously untried for this task are assessed: (1) a bespoke 3D hierarchical method, (2) a 2D multi-label, multi-axis method and (3) RootPainter, a 2D U-Net application with interactive corrections. U-Nets are trained using small, targeted hand-annotated datasets to reduce operator time. It was found that the segmentation accuracy of all three methods surpass mainstream watershed and thresholding techniques. Accuracy slightly reduces in low-contrast data, which affects volume fraction measurements, but errors are small compared with gravimetric methods. Moreover, U-Net models trained on low-contrast images can be used to segment higher-contrast datasets, without further training. This demonstrates model portability, which can expedite the segmentation of large datasets over short timespans.
As sample preparation and imaging techniques have expanded and improved to include a variety of options for larger sized and numbers of samples, the bottleneck in volumetric imaging is now data ...analysis. Annotation and segmentation are both common, yet difficult, data analysis tasks which are required to bring meaning to the volumetric data. The SuRVoS application has been updated and redesigned to provide access to both manual and machine learning-based segmentation and annotation techniques, including support for crowd sourced data. Combining adjacent, similar voxels (supervoxels) provides a mechanism for speeding up segmentation both in the painting of annotation and by training a segmentation model on a small amount of annotation. The support for layers allows multiple datasets to be viewed and annotated together which, for example, enables the use of correlative data (e.g. crowd-sourced annotations or secondary imaging techniques) to guide segmentation. The ability to work with larger data on high-performance servers with GPUs has been added through a client-server architecture and the Pytorch-based image processing and segmentation server is flexible and extensible, and allows the implementation of deep learning-based segmentation modules. The client side has been built around Napari allowing integration of SuRVoS into an ecosystem for open-source image analysis while the server side has been built with cloud computing and extensibility through plugins in mind. Together these improvements to SuRVoS provide a platform for accelerating the annotation and segmentation of volumetric and correlative imaging data across modalities and scales.
PDF
