A description is given of a single-particle X-ray imaging reconstruction and simulation package using the expand–maximize–compress algorithm, named
Dragonfly
.
Single-particle imaging (SPI) with ...X-ray free-electron lasers has the potential to change fundamentally how biomacromolecules are imaged. The structure would be derived from millions of diffraction patterns, each from a different copy of the macromolecule before it is torn apart by radiation damage. The challenges posed by the resultant data stream are staggering: millions of incomplete, noisy and un-oriented patterns have to be computationally assembled into a three-dimensional intensity map and then phase reconstructed. In this paper, the
Dragonfly
software package is described, based on a parallel implementation of the expand–maximize–compress reconstruction algorithm that is well suited for this task. Auxiliary modules to simulate SPI data streams are also included to assess the feasibility of proposed SPI experiments at the Linac Coherent Light Source, Stanford, California, USA.
This article describes the software package
OnDA
: online data analysis and feedback for serial X-ray imaging.
This article describes a free and open-source data analysis utility designed for fast ...online feedback during serial X-ray diffraction and scattering experiments:
OnDA
(online data analysis). Three complete real-time monitors for common types of serial X-ray imaging experiments are presented. These monitors are capable of providing the essential information required for quick decision making in the face of extreme rates of data collection. In addition, a set of modules, functions and algorithms that allow developers to modify the provided monitors or develop new ones are provided. The emphasis here is on simple, modular and scalable code that is based on open-source libraries and protocols.
OnDA
monitors have already proven to be invaluable tools in several experiments, especially for scoring and monitoring of diffraction data during serial crystallography experiments at both free-electron laser and synchrotron facilities. It is felt that in the future the kind of fast feedback that
OnDA
monitors provide will help researchers to deal with the expected very high throughput data flow at next-generation facilities such as the European X-ray free-electron laser.
An integration optimization, triage and analysis tool (
IOTA
) is presented, which uses a grid-search approach to maximize the success of indexing and integrating serial X-ray free-electron laser ...diffraction images.
IOTA
also includes several useful tools for on-site diffraction data processing.
Serial femtosecond crystallography (SFX) uses an X-ray free-electron laser to extract diffraction data from crystals not amenable to conventional X-ray light sources owing to their small size or radiation sensitivity. However, a limitation of SFX is the high variability of the diffraction images that are obtained. As a result, it is often difficult to determine optimal indexing and integration parameters for the individual diffraction images. Presented here is a software package, called
IOTA
, which uses a grid-search technique to determine optimal spot-finding parameters that can in turn affect the success of indexing and the quality of integration on an image-by-image basis. Integration results can be filtered using
a priori
information about the Bravais lattice and unit-cell dimensions and analyzed for unit-cell isomorphism, facilitating an improvement in subsequent data-processing steps.
Here
cppxfel
, a software package for integration and post-refinement of serial femtosecond crystallography data, is released.
As serial femtosecond crystallography expands towards a variety of ...delivery methods, including chip-based methods, and smaller collected data sets, the requirement to optimize the data analysis to produce maximum structure quality is becoming increasingly pressing. Here
cppxfel
, a software package primarily written in C++, which showcases several data analysis techniques, is released. This software package presently indexes images using DIALS (diffraction integration for advanced light sources) and performs an initial orientation matrix refinement, followed by post-refinement of individual images against a reference data set.
Cppxfel
is released with the hope that the unique and useful elements of this package can be repurposed for existing software packages. However, as released, it produces high-quality crystal structures and is therefore likely to be also useful to experienced users of X-ray free-electron laser (XFEL) software who wish to maximize the information extracted from a limited number of XFEL images.
Accurate computational modeling allows the use of software as a first approach to some petrological problems that typically require experimentation, but most programs have not yet been fully tested ...for accuracy with lunar or Martian melt compositions. The programs pMELTS, MAGPOX, and Perple_X stand out for phase equilibrium modeling, as their calibrations include experiments of lunar compositions or have precise thermodynamic constraints for similar compositions. A set of lunar mare basalts, picritic glasses, and basaltic Martian compositions with known experimentally determined multiple saturation point (MSP) conditions were used here for phase equilibrium modeling. The accuracy of each program was tested through the determination of MSPs on the liquidus of the selected compositions. This point in pressure–temperature space can be considered as a direct proxy of the stable phases and the equilibrium conditions during partial melting of mantle sources. We identify a trend in experimental data between MSP temperature and MgO, CaO, and SiO2 concentrations, and similar trends are found in model results. However, only Perple_X is able to closely match the experimental data, despite the fact it does not accurately model ilmenite saturation for high‐Ti lunar basalts. We find that pMELTS miscalculates olivine saturation for MgO‐rich compositions and MAGPOX systematically underestimates MSP pressure and temperatures and can only be used when olivine is the liquidus phase. For modeling lunar or Martian basalt compositions, Perple_X can be used for optimal results, although no software is yet capable of bypassing the need to constrain MSP conditions through experimentation.
Hummingbird
is an open-source scalable Python-based software tool for real-time analysis of diffraction data with the purpose of giving users immediate feedback during their experiments.
Advances in ...X-ray detectors and increases in the brightness of X-ray sources combined with more efficient sample delivery techniques have brought about tremendous increases in the speed of data collection in diffraction experiments. Using X-ray free-electron lasers such as the Linac Coherent Light Source (LCLS), more than 100 diffraction patterns can be collected in a second. These high data rates are invaluable for flash X-ray imaging (FXI), where aerosolized samples are exposed to the X-ray beam and the resulting diffraction patterns are used to reconstruct a three-dimensional image of the sample. Such experiments require immediate feedback on the quality of the data collected to adjust or validate experimental parameters, such as aerosol injector settings, beamline geometry or sample composition. The scarcity of available beamtime at the laser facilities makes any delay extremely costly. This paper presents
Hummingbird
, an open-source scalable Python-based software tool for real-time analysis of diffraction data with the purpose of giving users immediate feedback during their experiments.
Hummingbird
provides a fast, flexible and easy-to-use framework. It has already proven to be of great value in numerous FXI experiments at the LCLS.
A data processing pipeline for SACLA was developed, based on
Cheetah
and
CrystFEL
. Real-time analysis and rapid structure solution were enabled.
A data processing pipeline for serial femtosecond ...crystallography at SACLA was developed, based on
Cheetah
Barty
et al.
(2014).
J. Appl. Cryst.
47
, 1118–1131
and
CrystFEL
White
et al.
(2016).
J. Appl. Cryst.
49
, 680–689
. The original programs were adapted for data acquisition through the SACLA API, thread and inter-node parallelization, and efficient image handling. The pipeline consists of two stages: The first, online stage can analyse all images in real time, with a latency of less than a few seconds, to provide feedback on hit rate and detector saturation. The second, offline stage converts hit images into HDF5 files and runs
CrystFEL
for indexing and integration. The size of the filtered compressed output is comparable to that of a synchrotron data set. The pipeline enables real-time feedback and rapid structure solution during beamtime.
The only how-to guide offering a unified, systemic approach to acquiring, cleaning, and managing data in REvery experienced practitioner knows that preparing data for modeling is a painstaking, ...time-consuming process. Adding to the difficulty is that most modelers learn the steps involved in cleaning and managing data piecemeal, often on the fly, or they develop their own ad hoc methods. This book helps simplify their task by providing a unified, systematic approach to acquiring, modeling, manipulating, cleaning, and maintaining data in R. Starting with the very basics, data scientists Samuel E. Buttrey and Lyn R. Whitaker walk readers through the entire process. From what data looks like and what it should look like, they progress through all the steps involved in getting data ready for modeling. They describe best practices for acquiring data from numerous sources; explore key issues in data handling, including text/regular expressions, big data, parallel processing, merging, matching, and checking for duplicates; and outline highly efficient and reliable techniques for documenting data and recordkeeping, including audit trails, getting data back out of R, and more.The only single-source guide to R data and its preparation, it describes best practices for acquiring, manipulating, cleaning, and maintaining dataBegins with the basics and walks readers through all the steps necessary to get data ready for the modeling processProvides expert guidance on how to document the processes described so that they are reproducibleWritten by seasoned professionals, it provides both introductory and advanced techniquesFeatures case studies with supporting data and R code, hosted on a companion websiteA Data Scientist's Guide to Acquiring, Cleaning and Managing Data in R is a valuable working resource/bench manual for practitioners who collect and analyze data, lab scientists and research associates of all levels of experience, and graduate-level data mining students.
Data Parallel C++ Reinders, James; Ashbaugh, Ben; Brodman, James ...
2020, 2020-11-02T00:00:00, 2020-11-02, 2021.
eBook
Learn how to accelerate C++ programs using data parallelism. This open access book enables C++ programmers to be at the forefront of this exciting and important new development that is helping to ...push computing to new levels. It is full of practical advice, detailed explanations, and code examples to illustrate key topics. Data parallelism in C++ enables access to parallel resources in a modern heterogeneous system, freeing you from being locked into any particular computing device. Now a single C++ application can use any combination of devices—including GPUs, CPUs, FPGAs and AI ASICs—that are suitable to the problems at hand.This book begins by introducing data parallelism and foundational topics for effective use of the SYCL standard from the Khronos Group and Data Parallel C++ (DPC++), the open source compiler used in this book. Later chapters cover advanced topics including error handling, hardware-specific programming, communication and synchronization, and memory model considerations.Data Parallel C++ provides you with everything needed to use SYCL for programming heterogeneous systems.What You'll LearnAccelerate C++ programs using data-parallel programmingTarget multiple device types (e.g. CPU, GPU, FPGA)Use SYCL and SYCL compilers Connect with computing’s heterogeneous future via Intel’s oneAPI initiativeWho This Book Is ForThose new data-parallel programming and computer programmers interested in data-parallel programming using C++.
Recent developments in CrystFEL 1 White, Thomas A.; Mariani, Valerio; Brehm, Wolfgang ...
Journal of applied crystallography,
03/2016, Volume:
49, Issue:
Pt 2
Journal Article
Peer reviewed
Open access
Developments in the
CrystFEL
software suite, for processing diffraction data from ‘serial crystallography’ experiments, are described.
CrystFEL
is a suite of programs for processing data from ‘serial ...crystallography’ experiments, which are usually performed using X-ray free-electron lasers (FELs) but also increasingly with other X-ray sources. The
CrystFEL
software suite has been under development since 2009, just before the first hard FEL experiments were performed, and has been significantly updated and improved since then. This article describes the most important improvements which have been made to
CrystFEL
since the first release version. These changes include the addition of new programs to the suite, the ability to resolve ‘indexing ambiguities’ and several ways to improve the quality of the integrated data by more accurately modelling the underlying diffraction physics.
PDF
