To Örebro University

Classifying natural language requirements (NLRs) is challenging, especially with large volumes. Research shows that Large Language Models can assist by categorizing NLRs into functional requirements (FR) and non-functional requirements (NFRs). However, Generative Pretrained Transformer (GPT) models are not typically favored for this task due to concerns about consistency. This paper investigates the consistency when a GPT model classifies NLRs into FRs and NFRs using a zero-shot learning approach. Results show that ChatGPT-4o performs better for FRs, a temperature parameter set to 1 yields the highest consistency, while NFR classification improves with higher temperatures.

Classifying natural language requirements (NLRs) plays a crucial role in software engineering, helping us distinguish between functional and non-functional requirements. While large language models offer automation potential, we should address concerns about their consistency, meaning their ability to produce the same results over time. In this work, we share experiences from experimenting with how well GPT-4o and LLAMA3.3-70B classify NLRs using a zero-shot learning approach. Moreover, we explore how the temperature parameter influences classification performance and consistency for these models. Our results show that large language models like GPT-4o and LLAMA3.3- 70B can support automated NLRs classification. GPT-4o performs well in identifying functional requirements, with the highest consistency occurring at a temperature setting of one. Additionally, non-functional requirements classification improves at higher temperatures, indicating a trade-off between determinism and adaptability. LLAMA3.3-70B is more consistent than GPT-4o, and its classification accuracy varies less depending on temperature adjustments.

Software development (SD) projects are complex endeavors, and organizations employ software development methods (SDMs) both to add structure and to propel their projects. Nowadays, agile SDMs are the most used methods. Still, inefficient and ineffective use of SDMs is a challenge, and existing research shows cases of SD teams failing in their use of SDMs due to deviations made. Of course, methods should not be used in rigorous and textbook-like ways. Among practitioners, some of these deviations are referred to as cargo cult behavior; without, however, providing much of theoretical depth or any analytical tools to identify and characterize such behaviors. Although existing research has investigated SDM deviations, they have not been investigated as cargo cult behavior.

Against this backdrop, the aim of this thesis is to understand the challenges in succeeding with SDM use by developing a theory called SDM cargo cult theory (SDMCCT), to identify, analyze, and describe the cargo cult phenomenon in agile SDM use. The journey of this thesis takes its starting point in practitioners’ use of the buzzwords “cargo cult” when referring to flawed SDM use. By returning to the field of social anthropology and its studies of cargo cult, a definition of SDM cargo cult and an analytical framework are iteratively crafted through a longitudinal ethnographic study to constitute the SDMCCT. The research approach has been abductive and the SDMCCT builds on social action theory and work motivation theory.

The ethnographical study took place at an international industrial manufacturing company in Sweden that is using agile SDMs. This study includes three years of data collection, which includes observations, interviews, and gathering of business documents. The ethnographic study focused on three SD teams and their daily work using agile SDMs. The analysis using the analytical framework includes four agile SDM practices: daily scrum meeting, sprint demo, continuous integration, and visualization. In total, the analysis uncovered 36 deviations in the SD teams’ use of these practices, structured into 30 SDM cargo cult categories. In addition, this study shows that the framework is applicable to analyze and characterize effective SDM use as well, although it is not its main focus. The framework can be used by researchers to make similar analysis of cargo cult situations in other organizations, and the catalogue of cargo cult situations can serve as background knowledge for other organizations to study and improve their SD teams’ use of agile SDM practices.

When adopting and using a Software Development Method (SDM), it is important to stay true to the philosophy of the method; otherwise, software developers might execute activities that do not lead to the intended outcomes. Currently, no overview of SDM research addresses software developers’ reasoning behind adopting and using SDMs. Accordingly, this paper aims to survey existing SDM research to scrutinize the current knowledge base on software developers’ type of reasoning behind SDM adoption and use. We executed a systematic literature review and analyzed existing research using two steps. First, we classified papers based on what type of reasoning was addressed regarding SDM adoption and use: rational, irrational, and non-rational. Second, we made a thematic synthesis across these three types of reasoning to provide a more detailed characterization of the existing research. We elicited 28 studies addressing software developers’ reasoning and identified five research themes. Building on these themes, we framed four future research directions with four broad research questions, which can be used as a basis for future research.

Misinterpretations and faulty use of Software Development Method (SDM) practices and principles are identified pitfalls in Software Development (SD). Previous research indicates cases with method adoption and use failures; one reason could be the SDM Cargo Cult (CC) behavior, where SD organizations claim to be agile but not doing agile. Previous research has suggested the SDM CC framework as an analytical tool. The aim of this paper is to refine the SDM CC framework and empirically test this version of the framework. We use data from an ethnographical study on three SD teams’ Daily Scrum Meetings (DSM). The empirical material was collected through observations, interviews, and the organization’s business documents. We uncovered twelve CC situations in the SD teams’ use of the DSM practice, structured into seven categories of SDM deviations: bringing irrelevant information, canceling meetings, disturbing the team, receiving unclear information, bringing new requirements, problemsolving, and task distribution.

Organizations today adopt agile information systems development methods (ISDM), but many do not succeed with the adoption process and in achieving desired results. Systems developers sometimes fail in efficient use of ISDM, often due to a lack of understanding the fundamental intentions of the chosen method. In many cases organizations simply imitate the behavior of others without really understanding why. This conceptual paper defines this phenomenon as an ISDM cargo cult behavior and proposes an analytical framework to identify such situations. The concept of cargo cults originally comes from the field of social anthropology and has been used to explain irrational, ritualistic imitation of certain behavior. By defining and introducing the concept in the field of information systems development we provide a potential diagnostic tool to improve the understanding of one of the reasons why ISDM adoption sometimes fail.

Information Systems Development (ISD) organizations have been trying to adopt new approaches by transforming from one Information Systems Development Method (ISDM) to another, with a hope of reaching new strategies in their ISD process. This change is due to achieve faster deliveries, increase quality and respond to change iteratively. Previous research indicates cases with method deviations and method adoption failure. Reasons for this could be due to Information Systems Development Method (ISDM) Cargo Cult Behavior. This paper aims to, as a first attempt, test the ISDM Cargo cult type situation framework, by using data collected throughout observations and interviews in an ISD organization using agile methods. This study was able to identify six out of eight social actions that demonstrate an existing ISDM cargo cult behavior in an ISD team. This implies the possibility of using the framework to carry out such analysis.

Organizations today adopt agile information systems development methods (ISDM), but many do not succeed with the adoption process and in achieving desired results. Systems developers sometimes fail in efficient use of ISDM, often due to a lack of understanding the fundamental intentions of the chosen method. In many cases organizations simply imitate the behavior of others without really understanding why. This conceptual paper defines this phenomenon as an ISDM cargo cult behavior and proposes an analytical framework to identify such situations. The concept of cargo cults originally comes from the field of social anthropology and has been usedto explain irrational, ritualistic imitation of certain behavior. By defining and introducing the concept in the field of information systems development we provide a diagnostic tool to better understand one of the reasons why ISDM adoption sometimes fail.