In operational numerical weather prediction (NWP), forecast verification against analysis from the same experiment is part of the standard evaluation practice. This “own‐analysis” verification is ...beneficial in providing complete and uniform spatial coverage but is known to suffer from overestimated skill scores when applied to short‐range forecasts due to the inevitable positive correlation between the forecast and analysis errors. To alleviate this problem, a new approach termed “twin‐analysis” verification is proposed. In this approach, the forecasts are verified against “twin analyses” that are produced by running an independent cycle using the same NWP system as the one used to produce the forecasts, but initializing the cycle using an independent first guess at the very beginning. In this way the error correlation between the forecasts and analyses is reduced, if not completely removed, while preserving the statistical properties of the analyses, allowing for a clearer interpretation of verification. This note reports the results of comparison between “twin‐analysis” and “own‐analysis” verification scores obtained for the Japan Meteorological Agency's global NWP system. It is reported that the two scores disagree up to 48 hr or even longer depending on the regions and elements, which highlights the reliability limit of “own‐analysis” verification. The proposed method does not completely remove the error correlation issue and also requires high computational costs; a practical use of the proposed approach in an operational environment is also discussed.
Schematic illustration of the design of the experiment.
Verification of the expert’s opinion Shcherbakovskyi, M. H.; Dementiev, M. V.
Вісник Харківського національного університету внутрішніх справ,
09/2022, Letnik:
98, Številka:
3
Journal Article
Recenzirano
Odprti dostop
The expert’s opinion must be checked and evaluated by the subject of the evidence in accordance with the procedural law. The analysis of the literature shows that scientists mainly focus on ...assessment problems, neglecting the issue of verifying the expert’s opinion. This is due to the fact that processual scientists perceive verification as a type of collection, research, and evaluation of evidence in the theory of evidence. It has been proven that the verification of evidence in general and the expert’s opinion in particular differ from other elements of the proof process in terms of objects, purpose, and methods of implementation.
The objects of examination of the expert’s opinion are doubtful circumstances regarding certain aspects of the conducted examination and contradictions between the expert's conclusions and other evidence. The purpose of checking the expert’s opinion is to confirm or eliminate, to refute doubts about the admissibility and reliability of the expert’s opinion and contradictions of the results of the examination with the circumstances already established in the criminal proceedings. The subject of the inspection coincides with the assessment and consists in determining the admissibility and reliability of the expert’s conclusion as a source of evidence and factual data established by expert research. Verification of the expert’s opinion is carried out by conducting procedural and non-procedural measures aimed at obtaining new data.
Procedural measures include: appointment of a new examination to resolve the same questions from the study of the same objects under inspection; interrogation of the expert and other participants in the criminal proceedings in order to establish the circumstances relating to the subject of the examination, as well as information on the peculiarities of the origin, existence, exploitation, seizure, storage of objects of examination; demand for objects and documents that can establish the factual data necessary in the case; receiving oral consultations and written explanations from a specialist (non-procedural measures provide indicative information through consultations with specialists and receiving a review of the expert’s opinion from them); familiarization of the initiator of the appointment of expertise with the scientific and methodical literature for comparison of the conducted expert research in accordance with the requirements existing in the expert field.
Generate-and-validate (G&V) automated program repair (APR) techniques have been extensively studied during the past decade. Meanwhile, such techniques can be extremely time-consuming due to the ...manipulation of program code to fabricate a large number of patches and also the repeated test executions on patches to identify potential fixes. PraPR, a recent G&V APR technique, reduces such costs by modifying program code directly at the level of compiled JVM bytecode with on-the-fly patch validation, which directly allows multiple bytecode patches to be tested within the same JVM process. However, PraPR is limited due to its unique bytecode-repair design, and is basically unsound/imprecise as it assumes that patch executions do not change global JVM state and affect later patch executions on the same JVM process. In this paper, we propose a unified patch validation framework, named UniAPR, to perform the first empirical study of on-the-fly patch validation for state-of-the-art source-code-level APR techniques widely studied in the literature; furthermore, UniAPR addresses the imprecise patch validation issue by resetting the JVM global state via runtime bytecode transformation. We have implemented UniAPR as a publicly available fully automated Maven Plugin. Our study demonstrates for the first time that on-the-fly patch validation can often speed up state-of-the-art source-code-level APR by over an order of magnitude, enabling all existing APR techniques to explore a larger search space to fix more bugs in the near future. Furthermore, our study shows the first empirical evidence that vanilla on-the-fly patch validation can be imprecise/unsound, while UniAPR with JVM reset is able to mitigate such issues with negligible overhead.
Abstract The Rapid Refresh (RAP), an hourly updated assimilation and model forecast system, replaced the Rapid Update Cycle (RUC) as an operational regional analysis and forecast system among the ...suite of models at the NOAA/National Centers for Environmental Prediction (NCEP) in 2012. The need for an effective hourly updated assimilation and modeling system for the United States for situational awareness and related decision-making has continued to increase for various applications including aviation (and transportation in general), severe weather, and energy. The RAP is distinct from the previous RUC in three primary aspects: a larger geographical domain (covering North America), use of the community-based Advanced Research version of the Weather Research and Forecasting (WRF) Model (ARW) replacing the RUC forecast model, and use of the Gridpoint Statistical Interpolation analysis system (GSI) instead of the RUC three-dimensional variational data assimilation (3DVar). As part of the RAP development, modifications have been made to the community ARW model (especially in model physics) and GSI assimilation systems, some based on previous model and assimilation design innovations developed initially with the RUC. Upper-air comparison is included for forecast verification against both rawinsondes and aircraft reports, the latter allowing hourly verification. In general, the RAP produces superior forecasts to those from the RUC, and its skill has continued to increase from 2012 up to RAP version 3 as of 2015. In addition, the RAP can improve on persistence forecasts for the 1–3-h forecast range for surface, upper-air, and ceiling forecasts.
Calysto Babic, Domagoj; Hu, Alan J.
International Conference on Software Engineering 2008,
01/2008, Letnik:
2008, Številka:
24
Conference Proceeding, Journal Article
Automatically detecting bugs in programs has been a long-held goal in software engineering. Many techniques exist, trading-off varying levels of automation, thoroughness of coverage of program ...behavior, precision of analysis, and scalability to large code bases. This paper presents the Calysto static checker, which achieves an unprecedented combination of precision and scalability in a completely automatic extended static checker. Calysto is interprocedurally path-sensitive, fully context-sensitive, and bit-accurate in modeling data operations --- comparable coverage and precision to very expensive formal analyses --- yet scales comparably to the leading, less precise, static-analysis-based tool for similar properties. Using Calysto, we have discovered dozens of bugs, completely automatically, in hundreds of thousands of lines of production, open-source applications, with a very low rate of false error reports. This paper presents the design decisions, algorithms, and optimizations behind Calysto's performance.
In-memory processing is becoming a popular method to alleviate the memory bottleneck of the von Neumann computing model. With the goal of improving both latency and energy cost associated with such ...in-memory processing, emerging non-volatile memory technologies, such as Spintronic magnetic memory, are of particular interest as they can provide a near-SRAM read/write performance and eliminate nearly all static energy without experiencing any endurance limitations. Spintronic Racetrack Memory (RM) further addresses density concerns of spin-transfer torque memory (STT-MRAM). Moreover, it has recently been demonstrated that portions of RM nanowires can function as a polymorphic gate, which can be leveraged to implement multi-operand bulk bitwise operations. With more complex control, they can also be leveraged to build arithmetic integer and floating point processing in memory (PIM) primitives. This paper proposes SPIMulator, a Spintronic PIM sim ulator that can simulate the storage and PIM architecture of executing PIM commands in Racetrack memory. SPIMulator functionally models the polymorphic gate properties recently proposed for Racetrack memory, which allows transverse access that determines the number of ‘1’s in a segment of each Racetrack nanowire. From this simulation, SPIMulator can report real-time performance statistics such as cycle count and energy. Thus, SPIMulator simulates the multi-operand bit-wise logic operations recently proposed and can be easily extended to implement new PIM operations as they are developed. Due to the functional nature of SPIMulator, it can serve as a programming environment that allows development of PIM-based codes for verification of new acceleration algorithms. We demonstrate the value of SPIMulator through the modeling and estimations of performance and energy consumption of a variety of example applications, including the Advanced Encryption Standard (AES) for encryption primarily based on logical and look-up operations; multiplication of matrices, a frequent requirement in scientific, signal processing, and machine learning algorithms; and bitmap indices a common search table employed for database lookups.
We present a modular approach to automatic complexity analysis of integer programs. Based on a novel alternation between finding symbolic time bounds for program parts and using these to infer bounds ...on the absolute values of program variables, we can restrict each analysis step to a small part of the program while maintaining a high level of precision. The bounds computed by our method are polynomial or exponential expressions that depend on the absolute values of input parameters. We show how to extend our approach to arbitrary cost measures, allowing the use of our technique to find upper bounds for other expended resources, such as network requests or memory consumption. Our contributions are implemented in the open-source tool KoAT, and extensive experiments show the performance and power of our implementation in comparison with other tools.
Certifying the safety or robustness of neural networks against input uncertainties and adversarial attacks is an emerging challenge in the area of safe machine learning and control. To provide such a ...guarantee, one must be able to bound the output of neural networks when their input changes within a bounded set. In this article, we propose a semidefinite programming (SDP) framework to address this problem for feed-forward neural networks with general activation functions and input uncertainty sets. Our main idea is to abstract various properties of activation functions (e.g., monotonicity, bounded slope, bounded values, and repetition across layers) with the formalism of quadratic constraints. We then analyze the safety properties of the abstracted network via the S -procedure and SDP. Our framework spans the tradeoff between conservatism and computational efficiency and applies to problems beyond safety verification. We evaluate the performance of our approach via numerical problem instances of various sizes.
Test-based automated program repair has been a prolific field of research in software engineering in the last decade. Many approaches have indeed been proposed, which leverage test suites as a weak, ...but affordable, approximation to program specifications. Although the literature regularly sets new records on the number of benchmark bugs that can be fixed, several studies increasingly raise concerns about the limitations and biases of state-of-the-art approaches. For example, the correctness of generated patches has been questioned in a number of studies, while other researchers pointed out that evaluation schemes may be misleading with respect to the processing of fault localization results. Nevertheless, there is little work addressing the efficiency of patch generation, with regard to the practicality of program repair. In this paper, we fill this gap in the literature, by providing an extensive review on the efficiency of test suite based program repair. Our objective is to assess the number of generated patch candidates, since this information is correlated to (1) the strategy to traverse the search space efficiently in order to select sensical repair attempts, (2) the strategy to minimize the test effort for identifying a plausible patch, (3) as well as the strategy to prioritize the generation of a correct patch. To that end, we perform a large-scale empirical study on the efficiency, in terms of quantity of generated patch candidates of the 16 open-source repair tools for Java programs. The experiments are carefully conducted under the same fault localization configurations to limit biases. Eventually, among other findings, we note that: (1) many irrelevant patch candidates are generated by changing wrong code locations; (2) however, if the search space is carefully triaged, fault localization noise has little impact on patch generation efficiency; (3) yet, current template-based repair systems, which are known to be most effective in fixing a large number of bugs, are actually least efficient as they tend to generate majoritarily irrelevant patch candidates.
PDF
