Software engineering is decision intensive. Evidence‐based software engineering is suggested for decision‐making concerning the use of methods and technologies when developing software. Software ...development often includes the reuse of software assets, for example, open‐source components. Which components to use have implications on the quality of the software (e.g., maintainability). Thus, research is needed to support decision‐making for composite software. This paper presents a roadmap for research required to support evidence‐based decision‐making for choosing and integrating assets in composite software systems. The roadmap is developed as an output from a 5‐year project in the area, including researchers from three different organizations. The roadmap is developed in an iterative process and is based on (1) systematic literature reviews of the area; (2) investigations of the state of practice, including a case survey and a survey; and (3) development and evaluation of solutions for asset identification and selection. The research activities resulted in identifying 11 areas in need of research. The areas are grouped into two categories: areas enabling evidence‐based decision‐making and those related to supporting the decision‐making. The roadmap outlines research needs in these 11 areas. The research challenges and research directions presented in this roadmap are key areas for further research to support evidence‐based decision‐making for composite software.
The paper presents a research roadmap for evidence‐based decision‐making for choosing and integrating assets in composite software systems. Eleven areas in need of research were identified in two categories. The first category outlines areas enabling evidence‐based decision‐making; the second category describes areas related to supporting decision‐making.
For several years, the software engineering research community used eye trackers to study program comprehension, bug localization, pair programming, and other software engineering tasks. Eye trackers ...provide researchers with insights on software engineers’ cognitive processes, data that can augment those acquired through other means, such as on-line surveys and questionnaires. While there are many ways to take advantage of eye trackers, advancing their use requires defining standards for experimental design, execution, and reporting. We begin by presenting the foundations of eye tracking to provide context and perspective. Based on previous surveys of eye tracking for programming and software engineering tasks and our collective, extensive experience with eye trackers, we discuss
when
and
why
researchers should use eye trackers as well as
how
they should use them. We compile a list of typical use cases—real and anticipated—of eye trackers, as well as metrics, visualizations, and statistical analyses to analyze and report eye-tracking data. We also discuss the pragmatics of eye tracking studies. Finally, we offer lessons learned about using eye trackers to study software engineering tasks. This paper is intended to be a one-stop resource for researchers interested in designing, executing, and reporting eye tracking studies of software engineering tasks.
Agile software development represents a major departure from traditional, plan-based approaches to software engineering. A systematic review of empirical studies of agile software development up to ...and including 2005 was conducted. The search strategy identified 1996 studies, of which 36 were identified as empirical studies. The studies were grouped into four themes: introduction and adoption, human and social factors, perceptions on agile methods, and comparative studies. The review investigates what is currently known about the benefits and limitations of, and the strength of evidence for, agile methods. Implications for research and practice are presented. The main implication for research is a need for more and better empirical studies of agile software development within a common research agenda. For the industrial readership, the review provides a map of findings, according to topic, that can be compared for relevance to their own settings and situations.
This open access book provides information how to choose and collect the appropriate metrics for a software project in an organization. There are several kinds of metrics, based on the analysis of ...source code and developed for different programming paradigms such as structured programming and object-oriented programming (OOP). This way, the book follows three main objectives: (i) to identify existing and easily-collectible measures, if possible in the early phases of software development, for predicting and modeling both the traditional attributes of software systems and attributes specifically related to their efficient use of resources, and to create new metrics for such purposes; (ii) to describe ways to collect these measures during the entire lifecycle of a system, using minimally-invasive monitoring of design-time processes, and consolidate them into conceptual frameworks able to support model building by using a variety of approaches, including statistics, data mining and computational intelligence; and (iii) to present models and tools to support design time evolution of systems based on design-time measures and to empirically validate them. The book provides researchers and advanced professionals with methods for understanding the full implications of alternative choices and their relative attractiveness in terms of enhancing system resilience. It also explores the simultaneous use of multiple models that reflect different system interpretations or stakeholder perspectives.
There have been many changes in statistical theory in the past 30 years, including increased evidence that non-robust methods may fail to detect important results. The statistical advice available to ...software engineering researchers needs to be updated to address these issues. This paper aims both to explain the new results in the area of robust analysis methods and to provide a large-scale worked example of the new methods. We summarise the results of analyses of the Type 1 error efficiency and power of standard parametric and non-parametric statistical tests when applied to non-normal data sets. We identify parametric and non-parametric methods that are robust to non-normality. We present an analysis of a large-scale software engineering experiment to illustrate their use. We illustrate the use of kernel density plots, and parametric and non-parametric methods using four different software engineering data sets. We explain why the methods are necessary and the rationale for selecting a specific analysis. We suggest using kernel density plots rather than box plots to visualise data distributions. For parametric analysis, we recommend trimmed means, which can support reliable tests of the differences between the central location of two or more samples. When the distribution of the data differs among groups, or we have ordinal scale data, we recommend non-parametric methods such as Cliff’s
δ
or a robust rank-based ANOVA-like method.
Recent years have seen an increasing attention to social aspects of software engineering, including studies of emotions and sentiments experienced and expressed by the software developers. Most of ...these studies reuse existing sentiment analysis tools such as
SentiStrength
and
NLTK
. However, these tools have been trained on product reviews and movie reviews and, therefore, their results might not be applicable in the software engineering domain. In this paper we study whether the sentiment analysis tools agree with the sentiment recognized by human evaluators (as reported in an earlier study) as well as with each other. Furthermore, we evaluate the impact of the choice of a sentiment analysis tool on software engineering studies by conducting a simple study of differences in issue resolution times for positive, negative and neutral texts. We repeat the study for seven datasets (issue trackers and
Stack Overflow
questions) and different sentiment analysis tools and observe that the disagreement between the tools can lead to diverging conclusions. Finally, we perform two replications of previously published studies and observe that the results of those studies cannot be confirmed when a different sentiment analysis tool is used.
Modeling is requiring increasingly larger efforts while becoming indispensable given the complexity of the problems we are solving. Modelers face high cognitive load to understand a multitude of ...complex abstractions and their relationships. There is an urgent need to better support tool builders to ultimately provide modelers with intelligent modeling assistance that learns from previous modeling experiences, automatically derives modeling knowledge, and provides context-aware assistance. However, current intelligent modeling assistants (IMAs) lack adaptability and flexibility for tool builders, and do not facilitate understanding the differences and commonalities of IMAs for modelers. Such a patchwork of limited IMAs is a lost opportunity to provide modelers with better support for the creative and rigorous aspects of software engineering. In this expert voice, we present a conceptual reference framework (RF-IMA) and its properties to identify the foundations for intelligent modeling assistance. For tool builders, RF-IMA aims to help build IMAs more systematically. For modelers, RF-IMA aims to facilitate comprehension, comparison, and integration of IMAs, and ultimately to provide more intelligent support. We envision a momentum in the modeling community that leads to the implementation of RF-IMA and consequently future IMAs. We identify open challenges that need to be addressed to realize the opportunities provided by intelligent modeling assistance.
•Software development has been characterized by disconnects between activities.•We identify a roadmap of related continuous activities that we call Continuous *.•We propose a research agenda for ...continuous software engineering.
Throughout its short history, software development has been characterized by harmful disconnects between important activities such as planning, development and implementation. The problem is further exacerbated by the episodic and infrequent performance of activities such as planning, testing, integration and releases. Several emerging phenomena reflect attempts to address these problems. For example, Continuous Integration is a practice which has emerged to eliminate discontinuities between development and deployment. In a similar vein, the recent emphasis on DevOps recognizes that the integration between software development and its operational deployment needs to be a continuous one. We argue a similar continuity is required between business strategy and development, BizDev being the term we coin for this. These disconnects are even more problematic given the need for reliability and resilience in the complex and data-intensive systems being developed today. We identify a number of continuous activities which together we label as ‘Continuous *’ (i.e. Continuous Star) which we present as part of an overall roadmap for Continuous Software engineering. We argue for a continuous (but not necessarily rapid) software engineering delivery pipeline. We conclude the paper with a research agenda.
In recent years, the number of smartphone users has increased dramatically. These users download millions of apps and use them for various services. Due to the significant demand for mobile apps, ...developers often seek faster development methods and more effective tools and techniques to generate these apps. Many of these apps are location-based apps in which users receive services based on their geographical location. In this paper, we propose a model-driven approach for the automatic generation of Android location-based mobile apps. Our framework, called ALBA, consists of a domain-specific modeling language, a modeling tool, and a plugin which includes model to code transformations. The modeling tool enables a novice designer to model a location-based app. The model is validated against the predefined constraints and the editor prevents creating invalid models. The designer uses the plugin to generate the Android code of the app. The evaluation of our work is two fold. First, to evaluate the generalizability of the ALBA framework, we conducted an experiment which includes the generation of four industrial location-based apps. Second, to evaluate the usability and quality of both the framework and the generated apps, we conducted a case study consists of three experiments. The results of the evaluation are promising both in terms of the applicability of the framework and the quality of the generated apps.
Context
Start-up companies have become an important supplier of innovation and software-intensive products. The flexibility and reactiveness of start-ups enables fast development and launch of ...innovative products. However, a majority of software start-up companies fail before achieving any success. Among other factors, poor software engineering could be a significant contributor to the challenges experienced by start-ups. However, the state-of-practice of software engineering in start-ups, as well as the utilization of state-of-the-art is largely an unexplored area.
Objective
In this study we investigate how software engineering is applied in start-up context with a focus to identify key knowledge areas and opportunities for further research.
Method
We perform a multi-vocal exploratory study of 88 start-up experience reports. We develop a custom taxonomy to categorize the reported software engineering practices and their interrelation with business aspects, and apply qualitative data analysis to explore influences and dependencies between the knowledge areas.
Results
We identify the most frequently reported software engineering (requirements engineering, software design and quality) and business aspect (vision and strategy development) knowledge areas, and illustrate their relationships. We also present a summary of how relevant software engineering knowledge areas are implemented in start-ups and identify potentially useful practices for adoption in start-ups.
Conclusions
The results enable a more focused research on engineering practices in start-ups. We conclude that most engineering challenges in start-ups stem from inadequacies in requirements engineering. Many promising practices to address specific engineering challenges exists, however more research on adaptation of established practices, and validation of new start-up specific practices is needed.
PDF
