To Örebro University

Software Cost Estimation (SCE) is a critical phase in software development projects. However, due to the growing complexity of the software itself, a common problem in building software cost models is that the available datasets contain lots of missing categorical data. The purpose of this chapter is to show how a framework of statistical, computational, and visualization techniques can be used to evaluate and compare the effect of missing data techniques on the accuracy of cost estimation models. Hence, the authors use five missing data techniques: Multinomial Logistic Regression, Listwise Deletion, Mean Imputation, Expectation Maximization, and Regression Imputation. The evaluation and the comparisons are conducted using Regression Error Characteristic curves, which provide visual comparison of different prediction models, and Regression Error Operating Curves, which examine predictive power of models with respect to under- or over-estimation.

Software Cost Estimation is a critical phase in the development of a software project, and over the years has become an emerging research area. A common problem in building software cost models is that the available datasets contain projects with lots of missing categorical data. The purpose of this chapter is to show how a combination of modern statistical and computational techniques can be used to compare the effect of missing data techniques on the accuracy of cost estimation. Specifically, a recently proposed missing data technique, the multinomial logistic regression, is evaluated and compared with four older methods: listwise deletion, mean imputation, expectation maximization and regression imputation with respect to their effect on the prediction accuracy of a least squares regression cost model. The evaluation is based on various expressions of the prediction error and the comparisons are conducted using statistical tests, resampling techniques and a visualization tool, the regression error characteristic curves.

Context: Software quality issues are commonly reported when offshoring software development. Value-based software engineering addresses this by ensuring key stakeholders have a common understanding of quality.

Objective: This work seeks to understand the levels of alignment between key stakeholder groups within a company on the priority given to aspects of software quality developed as part of an offshoring relationship. Furthermore, the study aims to identify factors impacting the levels of alignment identified.

Method: Three case studies were conducted, with representatives of key stakeholder groups ranking aspects of software quality in a hierarchical cumulative exercise. The results are analysed using Spearman rank correlation coefficients and inertia. The results were discussed with the groups to gain a deeper understanding of the issues impacting alignment.

Results: Various levels of alignment were found between the various groups. The reasons for misalignment were found to include cultural factors, control of quality in the development process, short-term versus long-term orientations, understanding of cost-benefits of quality improvements, communication and coordination.

Conclusions: The factors that negatively affect alignment can vary greatly between different cases. The work emphasises the need for greater support to align company internal success-critical stakeholder groups in their understanding of quality when offshoring software development.

Background: Software quality issues are commonly reported when off shoring software development. Value-based software engineering addresses this by ensuring key stakeholders have a common understanding of quality.

Aim: This work seeks to understand the levels of alignment between key stakeholders on aspects of software quality for two products developed as part of an offshore in sourcing arrangement. The study further aims to explain the levels of alignment identified.

Method: Representatives of key stakeholder groups for both products ranked aspects of software quality. The results were discussed with the groups to gain a deeper understanding.

Results: Low levels of alignment were found between the groups studied. This is associated with insufficiently defined quality requirements, a culture that does not question management and conflicting temporal reflections on the product's quality.

Conclusion: The work emphasizes the need for greater support to align success-critical stakeholder groups in their understanding of quality when off shoring software development

Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them fromexternal sources. In CBSE, several different sourcing options are available: 1) developing software in-house, 2) outsourcing development, 3) buying commercial-off-the-shelf software, and 4) integrating open source software components. Unfortunately, there is little available research on howorganizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection. Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire. Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality. We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.

Context: Component-based software engineering (CBSE) is a common approach to develop and evolve contemporary software systems. When evolving a system based on components, make-or-buy decisions are frequent, i.e., whether to develop components internally or to acquire them from external sources. In CBSE, several different sourcing options are available: (1) developing software in-house, (2) outsourcing development, (3) buying commercial-off-the-shelf software, and (4) integrating open source software components.

Objective: Unfortunately, there is little available research on how organizations select component sourcing options (CSO) in industry practice. In this work, we seek to contribute empirical evidence to CSO selection.

Method: We conduct a cross-domain survey on CSO selection in industry, implemented as an online questionnaire.

Results: Based on 188 responses, we find that most organizations consider multiple CSOs during software evolution, and that the CSO decisions in industry are dominated by expert judgment. When choosing between candidate components, functional suitability acts as an initial filter, then reliability is the most important quality.

Conclusion: We stress that future solution-oriented work on decision support has to account for the dominance of expert judgment in industry. Moreover, we identify considerable variation in CSO decision processes in industry. Finally, we encourage software development organizations to reflect on their decision processes when choosing whether to make or buy components, and we recommend using our survey for a first benchmarking.

Software Cost Estimation (SCE) is a critical phase in software development projects. A common problem in building software cost models is that the available datasets contain projects with lots of missing categorical data. There are several techniques for handling missing data in the context of SCE. The purpose of this paper is to show a state-of-art statistical and visualization approach of evaluating and comparing the effect of missing data on the accuracy of cost estimation models. Five missing data techniques were used: Multinomial Logistic Regression, Listwise Deletion, Mean Imputation, Expectation Maximization and Regression Imputation and compared with respect to their effect on the prediction accuracy of a least squares regression cost model. The evaluation is based on various expressions of the prediction error. The comparisons are conducted using statistical tests, resampling techniques and visualization tools like the Regression Error Characteristic curves.

Component-based software engineering is a common approach to develop and evolve contemporary software systems where different component sourcing options are available: 1)Software developed internally (in-house), 2)Software developed outsourced, 3)Commercial of the shelf software, and 4) Open Source Software.

However, there is little available research on what attributes of a component are the most important ones when selecting new components. The object of the present study is to investigate what matters the most to industry practitioners during component selection. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using Compositional Data Analysis. The descriptive results showed that Cost was clearly considered the most important attribute during the component selection. Other important attributes for the practitioners were: Support of the component, Longevity prediction, and Level of off-the-shelf fit to product. Next, an exploratory analysis was conducted based on the practitioners' inherent characteristics. Nonparametric tests and biplots were used. It seems that smaller organizations and more immature products focus on different attributes than bigger organizations and mature products which focus more on Cost.

Efficient quality management of software projects requires knowledge of how various groups of stakeholders involved in software development prioritize the product and project goals. Agreements or disagreements among members of a team may originate from inherent groupings, depending on various professional or other characteristics. These agreements are not easily detected by conventional practices (discussions, meetings, etc.) since the natural language expressions are often obscuring, subjective, and prone to misunderstandings. It is therefore essential to have objective tools that can measure the alignment among the members of a team; especially critical for the software development is the degree of alignment with respect to the prioritization goals of the software product. The paper proposes an experience-based framework of statistical and graphical techniques for the systematic study of prioritization alignment, such as hierarchical cluster analysis, analysis of cluster composition, correlation analysis, and closest agreement-directed graph. This framework can provide a thorough and global picture of a team's prioritization perspective and can potentially aid managerial decisions regarding team composition and leadership. The framework is applied and illustrated in a study related to global software development where 65 individuals in different roles, geographic locations and professional relationships with a company, prioritize 24 goals from individual perception of the actual situation and for an ideal situation.

The development of a software product in an organization involves various groups of stakeholders who may prioritize the qualities of the product differently. This paper presents an empirical study of 65 individuals in different roles and in different locations, including on shoring, outsourcing and off shoring, prioritizing 24 software quality aspects. Hierarchical cluster analysis is applied to the prioritization data, separately for the situation today and the ideal situation, and the composition of the clusters, regarding the distribution of the inherent groupings within each of them, is analyzed. The analysis results in observing that the roles are not that important in the clustering. However, compositions of clusters regarding the onshore-offshore relationships are significantly different, showing that the offshore participants have stronger tendency to cluster together. In conclusion, stakeholders seem to form clusters of aligned understanding of priorities according to personal and cultural views rather than their roles in software development.

Cumulative Voting (CV), also known as Hundred-Point Method, is a simple and straightforward technique, used in various prioritization studies in software engineering. Multiple stakeholders (users, developers, consultants, marketing representatives or customers) are asked to prioritize issues concerning requirements, process improvements or change management in a ratio scale. The data obtained from such studies contain useful information regarding correlations of issues and trends of the respondents towards them. However, the multivariate and constrained nature of data requires particular statistical analysis. In this paper we propose a statistical framework; the multivariate Compositional Data Analysis (CoDA) for analyzing data obtained from CV prioritization studies. Certain methodologies for studying the correlation structure of variables are applied to a dataset concerning impact analysis issues prioritized by software professionals under different perspectives. These involve filling of zeros, transformation using the geometric mean, principle component analysis on the transformed variables and graphical representation by biplots and ternary plots.

Background: Employee turnover in GSD is an extremely important issue, especially in Western companies offshoring to emerging nations. 

Aims: In this case study we investigated an offshore vendor company and in particular whether the employees’ retention is related with their experience. Moreover, we studied whether we can identify a threshold associated with the employees’ tendency to leave the particular company. 

Method: We used a case study, applied and presented descriptive statistics, contingency tables, results from Chi-Square test of association and post hoc tests. 

Results: The emerging results showed that employee retention and company experience are associated. In particular, almost 90% of the employees are leaving the company within the first year, where the percentage within the second year is 50-50%. Thus, there is an indication that the 2 years’ time is the retention threshold for the investigated offshore vendor company. 

Conclusions: The results are preliminary and lead us to the need for building a prediction model which should include more inherent characteristics of the projects to aid the companies avoiding massive turnover waves.

Transition from an existing IT system to modern Web technologies provides multiple benefits to an organization and its customers. Such a transition in a large organization involves various groups of stakeholders who may prioritize differently the requirements of the software under development. In our case study, the organization is a leading domestic company in the field of electricity power. The existing online system supports the customer service along with the technical activities and has more than 1,500 registered users, while simultaneous access can be reached by 300 users. The paper presents an empirical study where 51 employees in different roles prioritize 18 software requirements using hierarchical cumulative voting. The goal of this study is to test significant differences in prioritization between groups of stakeholders. Statistical methods involving data transformation, ANOVA and Discriminant Analysis were applied to data. The results showed significant differences between roles of the stakeholders in certain requirements.

Simulation is a powerful instrument in the education process that can help students experience a reality context and understand complex concepts required to accomplish practitioners’ tasks. The present study aims to investigate the software engineering students’ perception about the usefulness of the Planning Poker technique in relation to their understanding of the relative estimation concept. We conducted a simulation exercise where students first estimated tasks applying the concepts of relative estimation based on the concepts explained in the lecture, and then to estimate tasks applying the Agile Planning Poker technique. To investigate the students’ perception, we used a survey at the end of each exercise. The preliminary results did not show statistical significance on the students’ confidence to estimate relatively the user stories. However, from the students’ comments and feedback, there are indications that students are more confident in using Agile Planning Poker when they are asked to estimate user stories. The study will be replicated in the near future to a different group of students with a different background, to have a better understanding and also identify possible flaws of the exercise.

Context: In software project management, the distribution of resources to various project activities is one of the most challenging problems since it affects team productivity, product quality and project constraints related to budget and scheduling.

Objective: The study aims to (a) reveal the high complexity of modelling the effort usage proportion in different phases as well as the divergence from various rules-of-thumb in related literature, and (b) present a systematic data analysis framework, able to offer better interpretations and visualisation of the effort distributed in specific phases.

Method: The basis for the proposed multivariate statistical framework is Compositional Data Analysis, a methodology appropriate for proportions, along with other methods like the deviation from rules-ofthumb, the cluster analysis and the analysis of variance. The effort allocations to phases, as reported in around 1500 software projects of the ISBSG R11 repository, were transformed to vectors of proportions of the total effort and were analysed with respect to prime project attributes.

Results: The proposed statistical framework was able to detect high dispersion among data, distribution inequality and various interesting correlations and trends, groupings and outliers, especially with respect to other categorical and continuous project attributes. Only a very small number of projects were found close to the rules-of-thumb from the related literature. Significant differences in the proportion of effort spent in different phrases for different types of projects were found.

Conclusion: There is no simple model for the effort allocated to phases of software projects. The data from previous projects can provide valuable information regarding the distribution of the effort for various types of projects, through analysis with multivariate statistical methodologies. The proposed statistical framework is generic and can be easily applied in a similar sense to any dataset containing effort allocation to phases.

One of the most significant problems faced by project managers is to effectively distribute the project resources and effort among the various project activities. Most importantly, project success depends on how well, or how balanced, the work effort is distributed among the project phases. This paper aims to obtain useful information regarding the correlation of the composition of effort attributed in phases for around 1,500 software projects of the ISBSG R11 database based on a promising statistical method called Compositional Data Analysis (CoDA). The motivation for applying this analysis is the observation that certain types of project data (effort distributions and attributes) do not relate in a direct way but present a spurious correlation. Effort distribution is compared to the project life-cycle activities, organization type, language type, function points and other prime project attributes. The findings are beneficial for building a basis for software cost estimation and improving future empirical software studies.

Component-based software engineering is a common approach in the development and evolution of contemporary software systems. Different component sourcing options are available, such as: (1) Software developed internally (in-house), (2) Software developed outsourced, (3) Commercial off-the-shelf software, and (4) Open-Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The objective of this study is to investigate what matters the most to industry practitioners when they decide to select a component. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using compositional data analysis. The results of this exploratory analysis showed that cost was clearly considered to be the most important attribute for component selection. Other important attributes for the practitioners were: support of the component, longevity prediction, and level of off-the-shelf fit to product. Moreover, several practitioners still consider in-house software development to be the sole option when adding or replacing a component. On the other hand, there is a trend to complement it with other component sourcing options and, apart from cost, different attributes factor into their decision. Furthermore, in our analysis, nonparametric tests and biplots were used to further investigate the practitioners’ inherent characteristics. It seems that smaller and larger organizations have different views on what attributes are the most important, and the most surprising finding is their contrasting views on the cost attribute: larger organizations with mature products are considerably more cost aware.

Background: Requirements engineering is of a prin- cipal importance when starting a new project. However, the number of the requirements involved in a single project can reach up to thousands. Controlling and assuring the quality of natural language requirements (NLRs), in these quantities, is challenging.

Aims: In a field study, we investigated with the Swedish Transportation Agency (STA) to what extent the characteristics of requirements had an influence on change requests and budget changes in the project.

Method: We choose the following models to characterize system requirements formulated in natural language: Concern- based Model of Requirements (CMR), Requirements Abstrac- tions Model (RAM) and Software-Hardware model (SHM). The classification of the NLRs was conducted by the three authors. The robust statistical measure Fleiss’ Kappa was used to verify the reliability of the results. We used descriptive statistics, contingency tables, results from the Chi-Square test of association along with post hoc tests. Finally, a multivariate statistical technique, Correspondence analysis was used in order to provide a means of displaying a set of requirements in two-dimensional graphical form.

Results: The results showed that software requirements are associated with less budget cost than hardware requirements. Moreover, software requirements tend to stay open for a longer period indicating that they are ”harder” to handle. Finally, the more discussion or interaction on a change request can lower the actual estimated change request cost.

Conclusions: The results lead us to a need to further investigate the reasons why the software requirements are treated differently from the hardware requirements, interview the project managers, understand better the way those requirements are formulated and propose effective ways of Software management.

Requirements specification is a core activity in the requirements engineering phase of a software development project. Researchers have contributed extensively to the field of requirements specification, but the extent to which their proposals have been adopted in practice remains unclear. We gathered evidence about the state of practice in requirements specification by focussing on the artefacts used in this activity, the application of templates or guidelines, how requirements are structured in the specification document, what tools practitioners use to specify requirements, and what challenges they face. We conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies. We recorded the interviews and analysed these recordings, primarily by using qualitative methods. Natural language constitutes the main specification artefact but is usually accompanied by some other type of instrument. Most requirements specifications use templates or guidelines, although they seldom follow any fixed standard. Requirements are always structured in the document according to the main functionalities of the system or to project areas or system parts. Different types of tools, including MS Office tools, are used, either individually or combined, in the compilation of requirements specifications. We also note that challenges related to the use of natural language (dealing with ambiguity, inconsistency, and incompleteness) are the most frequent challenges that practitioners face in the compilation of requirements specifications. These findings are contextualized in terms of demographic factors related to the individual interviewees, the organization they are affiliated with, and the project they selected to discuss during our interviews. A number of our findings have been previously reported in related studies. These findings show that, in spite of the large number of notations, models and tools proposed from academia for improving requirements specification, practitioners still mainly rely on plain natural language and general-purpose tool support. We expect more empirical studies in this area in order to better understand the reason of this low adoption of research results.

Context: Software start-ups are emerging as suppliers of innovation and software-intensive products. However, traditional software engineering practices are not evaluated in the context, nor adopted to goals and challenges of start-ups. As a result, there is insufficient support for software engineering in the start-up context.

Objective: We aim to collect data related to engineering goals, challenges, and practices in start-up companies to ascertain trends and patterns characterizing engineering work in start-ups. Such data allows researchers to understand better how goals and challenges are related to practices. This understanding can then inform future studies aimed at designing solutions addressing those goals and challenges. Besides, these trends and patterns can be useful for practitioners to make more informed decisions in their engineering practice.

Method: We use a case survey method to gather first-hand, in-depth experiences from a large sample of software start-ups. We use open coding and cross-case analysis to describe and identify patterns, and corroborate the findings with statistical analysis.

Results: We analyze 84 start-up cases and identify 16 goals, 9 challenges, and 16 engineering practices that are common among startups. We have mapped these goals, challenges, and practices to start-up life-cycle stages (inception, stabilization, growth, and maturity). Thus, creating the progression model guiding software engineering efforts in start-ups.

Conclusions: We conclude that start-ups to a large extent face the same challenges and use the same practices as established companies. However, the primary software engineering challenge in start-ups is to evolve multiple process areas at once, with a little margin for serious errors.

Context: Software start-ups are young companies aiming to build and market software-intensive products fast with little resources. Aiming to accelerate time-to-market, start-ups often opt for ad-hoc engineering practices, make shortcuts in product engineering, and accumulate technical debt.

Objective: In this paper we explore to what extent precedents, dimensions and outcomes associated with technical debt are prevalent in start-ups.

Method: We apply a case survey method to identify aspects of technical debt and contextual information characterizing the engineering context in start-ups.

Results: By analyzing responses from 86 start-up cases we found that start-ups accumulate most technical debt in the testing dimension, despite attempts to automate testing. Furthermore, we found that start-up team size and experience is a leading precedent for accumulating technical debt: larger teams face more challenges in keeping the debt under control.

Conclusions: This study highlights the necessity to monitor levels of technical debt and to preemptively introduce practices to keep the debt under control. Adding more people to an already difficult to maintain product could amplify other precedents, such as resource shortages, communication issues and negatively affect decisions pertaining to the use of good engineering practices.

Context: Software start-ups have shown their ability to develop and launch innovative software products and services. Small, motivated teams and uncertain project scope makes start-ups good candidates for adopting Agile practices.

Objective: We explore how start-ups use Agile practices and what effects can be associated with the use of those practices.

Method: We use a case survey to analyze 84 start-up cases and 56 Agile practices. We apply statistical methods to test for statistically significant associations between the use of Agile practices, team, and product factors.

Results: Our results suggest that development of the backlog, use of version control, code refactoring, and development of user stories are the most frequently reported practices. We identify 22 associations between the use of Agile practices, team, and product factors. The use of Agile practices is associated with effects on source code and overall product quality. A teams' positive or negative attitude towards best engineering practices is a significant indicator for either adoption or rejection of certain Agile practices. To explore the relationships in our findings, we set forth a number of propositions that can be investigated in future research.

Conclusions: We conclude that start-ups use Agile practices, however without following any specific methodology. We identify the opportunity for more fine-grained studies into the adoption and effects of individual Agile practices. Start-up practitioners could benefit from Agile practices in terms of better overall quality, tighter control over team performance, and resource utilization.

Background: Teaching students to critically appraise scientific literature is an important goal for a postgraduate research methods course.

Objective: To investigate the application of checklists for assessing the scientific rigor of empirical studies support students in reviewing case study research and experiments.

Methods: We employed an experimental design where 76 students (in pairs) used two checklists to evaluate two papers (reporting a case study and an experiment) each. We compared the students' assessments against ratings from more senior researchers. We also collected data on students' perception of using the checklists.

Results: The consistency of students' ratings and the accuracy when compared to ratings from seniors varied. A factor seemed to be that the clearer the reporting, the easier it is for students to judge the quality of studies. Students perceived checklist items related to data analysis as difficult to assess.

Conclusion: As expected, this study reinforces the needs for clear reporting, as it is important that authors write to enable synthesis and quality assessment. With clearer reporting, the novices performed well in assessing the quality of the empirical work, which supports its continued use in the course as means for introducing scientific reviews.

Agile software development has become increasingly popular over the past years. Agile methods are perceived to address challenges caused by the rapid change in the market while reducing time to market and development. As the popularity of Agile method is growing, the need for Agile adoption guidance also increases. Over the past years, Agile Maturity Models (AMMs) have been proposed to offer guidelines in Agile adoption. Agile maturity models (AMMs) suggest that Agile practices are introduced in a specific order. However, a number of evaluations of AMMs show that they are not properly validated and not suited for use in industry. Current evaluations of AMMs are based on certain criteria and not based on empirical studies. There are no studies that evaluate AMMs' suggestion to introduce Agile in certain order against industry practice. The relevance of the AMMs in the industry is not yet examined.

In this study, we conducted a literature review to identify the order of Agile practice introduction mentioned in AMMs. We then compared the AMMs' suggestions to the strategies used by practitioners, which we elicited from a survey and a series of interviews.

The literature survey revealed 12 AMMs which provide explicit mappings of Agile practices to maturity levels. These mappings show that the AMMs are not in agreement pertaining to the which practice is to be included in which maturity levels. The AMMs do not provide clear information nor rationale why certain practices need to be introduced in a specific maturity level. Comparison of the AMMs suggestions and the empirical study revealed that the guidance suggested by AMMs are not aligned with industry practice. The AMMs take a simplistic view of introducing Agile practices in a pre-determined order without consideration of the different contexts in industry. Our study indicates that practitioners iteratively add, remove, and modify the set of Agile practices they use to match their needs and contexts. In addition, the AMMs do not consider the potential perceived trade-off of adding Agile practices, such as team member discomfort, conflicts with other teams, and recurring bugs.

Currently, AMMs do not provide sufficient information to guide Agile adoption in the industry. Associating Agile maturity with sets of Agile practices is inadequate and too simplified. Our results suggest that there might be no universal strategy for Agile adoption that works better than others.

Context: Agile maturity models (AMMs) have been proposed to provide guidance for adopting Agile practices. Evaluations of AMMs indicatethat they might not be suitable for industry use. One issue is that AMMs have mainly been evaluated against pre-defined sets of criteria, instead of industry practice.

Objectives: The objectives of this study are to: (1) compare current AMMs regarding their guidance for Agile adoption, (2) investigate the strategies for Agile adoption used by practitioners, and (3) investigate similarities and differences between (1) and (2).

Methods: We conducted a literature survey that included grey literature to identify strategies proposed by the AMMs. We also conducted a survey and 11 interviews to identify the strategies used by practitioners to introduce Agile practices. This study combines quantitative and qualitative analysis.

Results: From the literature survey we found 26 AMMs, whereof 12 provide explicit mappings of Agile practices to maturity levels. These mappings showed little agreement in when practices should be introduced. Based on 40 survey responses we identified three high-level strategies for introducing Agile practices: big-bang, incremental, and complex strategies. The survey andinterviews revealed that the guidance suggested by AMMs are not aligned well with industry practice and that Agile practices might already be in place before an organization starts a transition to Agile.

Conclusion: In their current form, AMMs do not provide sufficient information to guide Agile adoption in industry. Our results suggest that there might be no universal strategy for Agile adoption that works better than others.

Context: Agile maturity models (AMMs) suggest that agile practices are introduced in a certain order. However, whether the order of agile practice introduction as suggested in the AMMs is relevant in industry has not been evaluated in an empirical study.

Objectives: In this study, we want to investigate: (1) order of agile practice introduction mentioned in AMMs, (2) order of introducing agile practices in industry, and (3) similarities and differences between (1) and (2).

Methods: We conducted a literature survey to identify strategies proposed by the AMMs. We then compared the AMMs' suggestions to the strategies used by practitioners, which we elicited from a survey and a series of interviews from an earlier study.

Results: The literature survey revealed 12 AMMs which provide explicit mappings of agile practices to maturity levels. These mappings showed little agreement on when practices should be introduced. Comparison of the AMMs' suggestions and the empirical study revealed that the guidance suggested by AMMs are not aligned with industry practice.

Conclusion: Currently, AMMs do not provide sufficient information to guide agile adoption in industry. Our results suggest that there might be no universal strategy for agile adoption that works better than others.

Requirements engineering remains a discipline that is faced with a large number of challenges, including the implementation of a requirements elicitation process in industry. Although several proposals have been suggested by researchers and academics, little is known of the practices that are actually followed in industry. Our objective is to investigate the state-of-practice with respect to requirements elicitation, by closely examining practitioners' current practices. To this aim, we focus on the techniques that are used in industry, the roles that requirements elicitation involves, and the challenges that the requirements elicitation process is faced with. As method, we conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies, and we recorded the interviews and analyzed these recordings by using quantitative and qualitative methods. Several results emerged from the studies. Group interaction techniques, including meetings and workshops, are the most popular type of elicitation techniques that are employed by the practitioners, except in the case of small projects. Additionally, practitioners tend to use a variety of elicitation techniques in each project. We noted that customers are frequently involved in the elicitation process, except in the case of market-driven organizations. Technical staff (for example, developers and architects) are more frequently involved in the elicitation process compared to the involvement of business or strategic staff. Finally, we identified a number of challenges with respect to stakeholders. These challenges include difficulties in understanding and prioritizing their needs. Further, it was noted that requirements instability (i.e., caused by changing needs or priorities) was a predominant challenge. These observations need to be interpreted in the context of the study. We conclude that the relevant observations regarding the survey participants' experiences should be of interest to the industry; experiences that should be analyzed in the practitioners' context. Researchers may find evidence for the use of academic results in practice, thereby inspiring future theoretical work, as well as further empirical studies in the same area.

In this paper we present the results of a comparison on students’ perception of group workshops that traditionally have been conducted on campus, but due to the covid-19 pandemic had to be conducted online. The workshops studied in this paper are learning activities where students in groups of 4-5 collaborate to solve programming problems without the help of computers. The course context is an introductory programming course at the information systems department in a Swedish university. Under “normal” campus-based circumstances, course evaluations have shown these workshops to be the most appreciated and engaging elements in a flipped pedagogy, active learning-based course. The aim of the study is to investigate if student perception of the workshops and their outcome differs when the activity no longer is conducted on campus and face-to-face. A survey targeting different aspects of student perception of the workshops was conducted before and after the change from campus to online. The analysis shows that there is a statistically significant difference in regards to the student’s ability to achieve the course goals and the outcome of the workshops. However, there does not appear to be any difference in the student’s perception to get help and feedback from the teachers. Overall, the results show that the students are less pleased with the workshops and with their performance in the online workshops compared to when they are campus-based.

With the increasing popularity of agile development and team-oriented practices, bottom-up coordination structures have found their ways into software companies, first changing the small companies and now revolutionizing large-scale development projects and programs. One of the ways to enable bottom-up coordination is cultivation of communities of practice. Existing research has demonstrated that successful implementation of communities of practice depends on organizational support, mutual engagement and regular interaction. Engagement is said to increase, when a community creates value for the organization and individual community members, while increased engagement is further associated with the ability to create more value. However, little is known about how to ensure member engagement in large-scale environments covering many sites and thousands of developers. In this article, we report our findings from studying member engagement in large-scale distributed communities of practice at Spotify called guilds. We report the perceived value guilds provide on individual and organizational level, and discuss what hinders and what stimulates mutual engagement and value creation across time and space.

Staffing software projects with engineers from inexpensive locations has become commonplace. However, distributed development remains practically challenging because of recurring problems, e.g., decreased productivity, low quality, and high, unforeseen costs. Although it is often overlooked, one of the main underlying reasons for these challenges is high employee turnover. This might be especially noticeable in developing countries with strong economic growth such as India. This article examines turnover of Indian software engineers and introduces strategies to address it.

Visualizing CI's role in software quality attribute evaluation.

QUALITY ATTRIBUTES OF software systems, also known as system qualities, such as performance, security, and scalability, continue to grow in importance in industrial practice. The evaluation of quality attributes is critical to software development since optimizing a software system's core attributes can provide marketing advantage and set a product apart from its competitors. Many existing studies of unsuccessful development projects report that lack of quality attribute evaluation is often a contributing factor of project failure. Therefore, continuous quality attribute evaluation, throughout the development process, is needed to ensure customers' expectations and demands are met.

ContextNon-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software. Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation. Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of Continuous Integration (CI). However, studies on how companies manage automated NFR testing through CI are limited.ObjectiveThis study examines how automated NFR testing can be enabled and supported using CI environments in software development companies.MethodWe performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners.ResultsMaintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments. Testing practices, quality metrics, and challenges for measuring NFRs were reported.ConclusionsThis study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice.

Context

Continuous integration (CI) is a practice that aims to continuously verify quality aspects of a software intensive system both for functional and non-functional requirements (NFRs). Functional requirements are the inputs of development and can be tested in isolation, utilising either manual or automated tests. In contrast, some NFRs are difficult to test without functionality, for NFRs are often aspects of functionality and express quality aspects. Lacking this testability attribute makes NFR testing complicated and, therefore, underrepresented in industrial practice. However, the emergence of CI has radically affected software development and created new avenues for software quality evaluation and quality information acquisition. Research has, consequently, been devoted to the utilisation of this additional information for more efficient and effective NFR verification.

Objective

We aim to identify the state-of-the-art of utilising the CI environment for NFR testing, hereinafter referred to as CI-NFR testing.

Method

Through rigorous selection, from an initial set of 747 papers, we identified 47 papers that describe how NFRs are tested in a CI environment. Evidence-based analysis, through coding, is performed on the identified papers in this SLR.

Results

Firstly, ten CI approaches are described by the papers selected, each describing different tools and nine different NFRs where reported to be tested. Secondly, although possible, CI-NFR testing is associated with eight challenges that adversely affect its adoption. Thirdly, the identified CI-NFR testing processes are tool-driven, but there is a lack of NFR testing tools that can be used in the CI environment. Finally, we proposed a CI framework for NFRs testing.

Conclusion

A synthesised CI framework is proposed for testing various NFRs, and associated CI tools are also mapped. This contribution is valuable as results of the study also show that CI-NFR testing can help improve the quality of NFR testing in practices.