To Örebro University

The quality of the test suites and the constituent test cases significantly impacts confidence in software testing. While research has identified several quality attributes of test cases and test suites, there is a need for a better understanding of their relative importance in practice. We investigate practitioners' perceptions regarding the relative importance of quality attributes of test cases and test suites and the challenges that they face in ensuring the perceived important quality attributes. To capture the practitioners' perceptions, we conducted an industrial survey using a questionnaire based on the quality attributes identified in an extensive literature review. We used a sampling strategy that leverages LinkedIn to draw a large and heterogeneous sample of professionals with experience in software testing. We collected 354 responses from practitioners with a wide range of experience (from less than one year to 42 years of experience). We found that the majority of practitioners rated Fault Detection, Usability, Maintainability, Reliability, and Coverage to be the most important quality attributes. Resource Efficiency, Reusability, and Simplicity received the most divergent opinions, which, according to our analysis, depend on the software-testing contexts. Also, we identified common challenges that apply to the important attributes, namely inadequate definition, lack of useful metrics, lack of an established review process, and lack of external support. The findings point out where practitioners actually need further support with respect to achieving high-quality test cases and test suites under different software testing contexts. Hence, the findings can serve as a guideline for academic researchers when looking for research directions on the topic. Furthermore, the findings can be used to encourage companies to provide more support to practitioners to achieve high-quality test cases and test suites.

Visualizing CI's role in software quality attribute evaluation.

QUALITY ATTRIBUTES OF software systems, also known as system qualities, such as performance, security, and scalability, continue to grow in importance in industrial practice. The evaluation of quality attributes is critical to software development since optimizing a software system's core attributes can provide marketing advantage and set a product apart from its competitors. Many existing studies of unsuccessful development projects report that lack of quality attribute evaluation is often a contributing factor of project failure. Therefore, continuous quality attribute evaluation, throughout the development process, is needed to ensure customers' expectations and demands are met.

ContextNon-functional requirements (NFRs) (also referred to as system qualities) are essential for developing high-quality software. Notwithstanding its importance, NFR testing remains challenging, especially in terms of automation. Compared to manual verification, automated testing shows the potential to improve the efficiency and effectiveness of quality assurance, especially in the context of Continuous Integration (CI). However, studies on how companies manage automated NFR testing through CI are limited.ObjectiveThis study examines how automated NFR testing can be enabled and supported using CI environments in software development companies.MethodWe performed a multi-case study at four companies by conducting 22 semi-structured interviews with industrial practitioners.ResultsMaintainability, reliability, performance, security and scalability, were found to be evaluated with automated tests in CI environments. Testing practices, quality metrics, and challenges for measuring NFRs were reported.ConclusionsThis study presents an empirically derived model that shows how data produced by CI environments can be used for evaluation and monitoring of implemented NFR quality. Additionally, the manuscript presents explicit metrics, CI components, tools, and challenges that shall be considered while performing NFR testing in practice.

Requirements specification is a core activity in the requirements engineering phase of a software development project. Researchers have contributed extensively to the field of requirements specification, but the extent to which their proposals have been adopted in practice remains unclear. We gathered evidence about the state of practice in requirements specification by focussing on the artefacts used in this activity, the application of templates or guidelines, how requirements are structured in the specification document, what tools practitioners use to specify requirements, and what challenges they face. We conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies. We recorded the interviews and analysed these recordings, primarily by using qualitative methods. Natural language constitutes the main specification artefact but is usually accompanied by some other type of instrument. Most requirements specifications use templates or guidelines, although they seldom follow any fixed standard. Requirements are always structured in the document according to the main functionalities of the system or to project areas or system parts. Different types of tools, including MS Office tools, are used, either individually or combined, in the compilation of requirements specifications. We also note that challenges related to the use of natural language (dealing with ambiguity, inconsistency, and incompleteness) are the most frequent challenges that practitioners face in the compilation of requirements specifications. These findings are contextualized in terms of demographic factors related to the individual interviewees, the organization they are affiliated with, and the project they selected to discuss during our interviews. A number of our findings have been previously reported in related studies. These findings show that, in spite of the large number of notations, models and tools proposed from academia for improving requirements specification, practitioners still mainly rely on plain natural language and general-purpose tool support. We expect more empirical studies in this area in order to better understand the reason of this low adoption of research results.

Context: Software start-ups are emerging as suppliers of innovation and software-intensive products. However, traditional software engineering practices are not evaluated in the context, nor adopted to goals and challenges of start-ups. As a result, there is insufficient support for software engineering in the start-up context.

Objective: We aim to collect data related to engineering goals, challenges, and practices in start-up companies to ascertain trends and patterns characterizing engineering work in start-ups. Such data allows researchers to understand better how goals and challenges are related to practices. This understanding can then inform future studies aimed at designing solutions addressing those goals and challenges. Besides, these trends and patterns can be useful for practitioners to make more informed decisions in their engineering practice.

Method: We use a case survey method to gather first-hand, in-depth experiences from a large sample of software start-ups. We use open coding and cross-case analysis to describe and identify patterns, and corroborate the findings with statistical analysis.

Results: We analyze 84 start-up cases and identify 16 goals, 9 challenges, and 16 engineering practices that are common among startups. We have mapped these goals, challenges, and practices to start-up life-cycle stages (inception, stabilization, growth, and maturity). Thus, creating the progression model guiding software engineering efforts in start-ups.

Conclusions: We conclude that start-ups to a large extent face the same challenges and use the same practices as established companies. However, the primary software engineering challenge in start-ups is to evolve multiple process areas at once, with a little margin for serious errors.

Requirements engineering remains a discipline that is faced with a large number of challenges, including the implementation of a requirements elicitation process in industry. Although several proposals have been suggested by researchers and academics, little is known of the practices that are actually followed in industry. Our objective is to investigate the state-of-practice with respect to requirements elicitation, by closely examining practitioners' current practices. To this aim, we focus on the techniques that are used in industry, the roles that requirements elicitation involves, and the challenges that the requirements elicitation process is faced with. As method, we conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies, and we recorded the interviews and analyzed these recordings by using quantitative and qualitative methods. Several results emerged from the studies. Group interaction techniques, including meetings and workshops, are the most popular type of elicitation techniques that are employed by the practitioners, except in the case of small projects. Additionally, practitioners tend to use a variety of elicitation techniques in each project. We noted that customers are frequently involved in the elicitation process, except in the case of market-driven organizations. Technical staff (for example, developers and architects) are more frequently involved in the elicitation process compared to the involvement of business or strategic staff. Finally, we identified a number of challenges with respect to stakeholders. These challenges include difficulties in understanding and prioritizing their needs. Further, it was noted that requirements instability (i.e., caused by changing needs or priorities) was a predominant challenge. These observations need to be interpreted in the context of the study. We conclude that the relevant observations regarding the survey participants' experiences should be of interest to the industry; experiences that should be analyzed in the practitioners' context. Researchers may find evidence for the use of academic results in practice, thereby inspiring future theoretical work, as well as further empirical studies in the same area.

Context: Software start-ups have shown their ability to develop and launch innovative software products and services. Small, motivated teams and uncertain project scope makes start-ups good candidates for adopting Agile practices.

Objective: We explore how start-ups use Agile practices and what effects can be associated with the use of those practices.

Method: We use a case survey to analyze 84 start-up cases and 56 Agile practices. We apply statistical methods to test for statistically significant associations between the use of Agile practices, team, and product factors.

Results: Our results suggest that development of the backlog, use of version control, code refactoring, and development of user stories are the most frequently reported practices. We identify 22 associations between the use of Agile practices, team, and product factors. The use of Agile practices is associated with effects on source code and overall product quality. A teams' positive or negative attitude towards best engineering practices is a significant indicator for either adoption or rejection of certain Agile practices. To explore the relationships in our findings, we set forth a number of propositions that can be investigated in future research.

Conclusions: We conclude that start-ups use Agile practices, however without following any specific methodology. We identify the opportunity for more fine-grained studies into the adoption and effects of individual Agile practices. Start-up practitioners could benefit from Agile practices in terms of better overall quality, tighter control over team performance, and resource utilization.

Component-based software engineering is a common approach in the development and evolution of contemporary software systems. Different component sourcing options are available, such as: (1) Software developed internally (in-house), (2) Software developed outsourced, (3) Commercial off-the-shelf software, and (4) Open-Source Software. However, there is little available research on what attributes of a component are the most important ones when selecting new components. The objective of this study is to investigate what matters the most to industry practitioners when they decide to select a component. We conducted a cross-domain anonymous survey with industry practitioners involved in component selection. First, the practitioners selected the most important attributes from a list. Next, they prioritized their selection using the Hundred-Dollar ($100) test. We analyzed the results using compositional data analysis. The results of this exploratory analysis showed that cost was clearly considered to be the most important attribute for component selection. Other important attributes for the practitioners were: support of the component, longevity prediction, and level of off-the-shelf fit to product. Moreover, several practitioners still consider in-house software development to be the sole option when adding or replacing a component. On the other hand, there is a trend to complement it with other component sourcing options and, apart from cost, different attributes factor into their decision. Furthermore, in our analysis, nonparametric tests and biplots were used to further investigate the practitioners’ inherent characteristics. It seems that smaller and larger organizations have different views on what attributes are the most important, and the most surprising finding is their contrasting views on the cost attribute: larger organizations with mature products are considerably more cost aware.

With the increasing popularity of agile development and team-oriented practices, bottom-up coordination structures have found their ways into software companies, first changing the small companies and now revolutionizing large-scale development projects and programs. One of the ways to enable bottom-up coordination is cultivation of communities of practice. Existing research has demonstrated that successful implementation of communities of practice depends on organizational support, mutual engagement and regular interaction. Engagement is said to increase, when a community creates value for the organization and individual community members, while increased engagement is further associated with the ability to create more value. However, little is known about how to ensure member engagement in large-scale environments covering many sites and thousands of developers. In this article, we report our findings from studying member engagement in large-scale distributed communities of practice at Spotify called guilds. We report the perceived value guilds provide on individual and organizational level, and discuss what hinders and what stimulates mutual engagement and value creation across time and space.

In this paper we present the results of a comparison on students’ perception of group workshops that traditionally have been conducted on campus, but due to the covid-19 pandemic had to be conducted online. The workshops studied in this paper are learning activities where students in groups of 4-5 collaborate to solve programming problems without the help of computers. The course context is an introductory programming course at the information systems department in a Swedish university. Under “normal” campus-based circumstances, course evaluations have shown these workshops to be the most appreciated and engaging elements in a flipped pedagogy, active learning-based course. The aim of the study is to investigate if student perception of the workshops and their outcome differs when the activity no longer is conducted on campus and face-to-face. A survey targeting different aspects of student perception of the workshops was conducted before and after the change from campus to online. The analysis shows that there is a statistically significant difference in regards to the student’s ability to achieve the course goals and the outcome of the workshops. However, there does not appear to be any difference in the student’s perception to get help and feedback from the teachers. Overall, the results show that the students are less pleased with the workshops and with their performance in the online workshops compared to when they are campus-based.