To Örebro University

Context: Despite the widespread adoption of agile software development methods (ASDMs) today, many organizations struggle with effective implementation. One reason for this is that some organizations claim to use an ASDM without fully understanding its core principles, or they adhere to old practices while professing to follow a contemporary software development method. This phenomenon is sometimes referred to by practitioners as “cargo cult” (CC) behavior. However, simply labeling something as CC lacks analytical depth.

Objective: This paper aims to conceptualize and validate an analytical tool for diagnosing CC and non-CC behavior in software development teams’ use of ASDMs.

Method: This study uses a longitudinal ethnographic approach to conceptualize and validate the analytical tool by analyzing four agile practices used by a global industrial manufacturing company.

Results: The analytical tool features eight stereotypes—three representing non-CC behaviors and five representing CC behaviors—designed to aid in the analysis of ASDM usage. The tool draws on Social Action Theory and Work Motivation Theory to capture and interpret the CC phenomenon in ASDM use. Using the stereotypes, 36 actions were categorized as CC behavior deviating from documented ASDM practices, and 23 actions as non-CC behavior because they aligned with the documented ASDM and reflected agile goals and values. The tool thus can help both researchers and practitioners gain a deeper understanding of ASDM use in organizations.

Conclusion: This study advances understanding of ASDM use by moving beyond the simplistic use of the term “cargo cult”. The developed tool enables structured identification and classification of CC behaviors. The stereotypes provide a way of classifying recurring software development actions against the intended ASDM, allowing the identification of specific types of CC behaviors. The analytical tool enables managers to gain deeper insights into the underlying reasons for deviations, thereby supporting more grounded and effective agile practices within organizations.

In today's digital age, protecting information assets is critical to maintain organizations’ digital sovereignty. Yet existing research offers limited guidance on creating effective, actionable advice in information security policies (ISPs) that instructs employees on how to carry out their tasks and contribute to protecting information assets. Addressing this gap, the aim of this paper is to propose a definition of actionable advice. A clear definition can aid in designing ISPs and enhance communication with employees, guiding them in the expected behavior to protect the organization’s information assets. The research question guiding this work is: how can actionable advice be defined in information security policies? To achieve this aim, the definition is informed by a literature review and analysis of 47 ISPs from public agencies in Sweden. The proposed definition of actionable advice is: a demarcated part of an ISP, that instructs someone on a task to execute or not to execute regarding information security, and, in case of execution, how to carry out the task. The definition of actionable advice provides researchers with a starting point to understand this term, helping advancing future studies on ISPs. This work also has practical implications for ISP developers, offering guidance on writing pieces of actionable advice that are concrete and directly applicable in employees' daily tasks to protect their organizations.

Classifying natural language requirements (NLRs) is challenging, especially with large volumes. Research shows that Large Language Models can assist by categorizing NLRs into functional requirements (FR) and non-functional requirements (NFRs). However, Generative Pretrained Transformer (GPT) models are not typically favored for this task due to concerns about consistency. This paper investigates the consistency when a GPT model classifies NLRs into FRs and NFRs using a zero-shot learning approach. Results show that ChatGPT-4o performs better for FRs, a temperature parameter set to 1 yields the highest consistency, while NFR classification improves with higher temperatures.

Over the years, numerous studies on employee compliance with information security policies (ISPs) have been conducted, contributing valuable insights to enhance information security in organisations. However, our literature review reveals that few ISP compliance studies adopt a longitudinal approach. It is well known that cross-sectional research often provides limited insight into how constructs such as ISP compliance evolve over time. While researchers have called for more longitudinal ISP compliance studies, there is little guidance on how to conduct them. To address this gap, we propose a set of seven guidelines to support both quantitative and qualitative longitudinal ISP compliance research.

Classifying natural language requirements (NLRs) plays a crucial role in software engineering, helping us distinguish between functional and non-functional requirements. While large language models offer automation potential, we should address concerns about their consistency, meaning their ability to produce the same results over time. In this work, we share experiences from experimenting with how well GPT-4o and LLAMA3.3-70B classify NLRs using a zero-shot learning approach. Moreover, we explore how the temperature parameter influences classification performance and consistency for these models. Our results show that large language models like GPT-4o and LLAMA3.3- 70B can support automated NLRs classification. GPT-4o performs well in identifying functional requirements, with the highest consistency occurring at a temperature setting of one. Additionally, non-functional requirements classification improves at higher temperatures, indicating a trade-off between determinism and adaptability. LLAMA3.3-70B is more consistent than GPT-4o, and its classification accuracy varies less depending on temperature adjustments.

Employees compliance with information security policies (ISPs) depends on communicating clear and comprehensible content. However, existing research has shown that many ISPs are of poor communicative quality. Large language models (LLMs) could enhance ISPs if finetuned on high-quality data, but to do such fine-tuning requires a conceptual model for classifying the data and evaluating the resulting text. Therefore, as a step in this direction, the aim of this paper is to develop a conceptual model of ISPs using speech act theory as a theoretical lens. We use conceptual modelling and document analysis to develop the model and use selected parts from the SEQUAL framework to evaluate the model. Analysing 600 ISP statements from ten British National Health Service ISPs, we present a class diagram containing 19 classes, six of which address ISP statement quality as speech acts. The SEQUAL evaluation points to potential areas for improving the model’s semantic, empirical, physical and deontic qualities before using it to fine-tune LLMs to improve ISP content.

Designing accessible and relevant information security policies (ISPs) that support employees is crucial for improving organisations' information security. When employees are required to deal with cumbersome ISPs, there is a risk of reduced motivation towards information security, and employees' not following the rules in ISPs has been reported as a persistent issue. Existing research has suggested adopting a tailored approach to ISPs in order to enhance their relevance to employees. Tailoring is difficult and time consuming and information security managers lack information security management systems software (ISMSS) that can assist with this tailoring task. In this paper, we develop a design theory for ISMSS to support information security managers in tailoring ISPs to different employees. To achieve this, we employ design science research, drawing on prior studies concerning the tailoring of systems development methods. We evaluate the design theory through an expository instantiation, POLCO, and with information security managers, demonstrating both proof-of-concept and proof-of-value.

Purpose: This paper aims to investigate how congruent keywords are used in information security policies (ISPs) to pinpoint and guide clear actionable advice and suggest a metric for measuring the quality of keyword use in ISPs.

Design/methodology/approach: A qualitative content analysis of 15 ISPs from public agencies in Sweden was conducted with the aid of Orange Data Mining Software. The authors extracted 890 sentences from these ISPs that included one or more of the analyzed keywords. These sentences were analyzed using the new metric - keyword loss of specificity - to assess to what extent the selected keywords were used for pinpointing and guiding actionable advice. Thus, the authors classified the extracted sentences as either actionable advice or other information, depending on the type of information conveyed.

Findings: The results show a significant keyword loss of specificity in relation to pieces of actionable advice in ISPs provided by Swedish public agencies. About two-thirds of the sentences in which the analyzed keywords were used focused on information other than actionable advice. Such dual use of keywords reduces the possibility of pinpointing and communicating clear, actionable advice.

Research limitations/implications: The suggested metric provides a means to assess the quality of how keywords are used in ISPs for different purposes. The results show that more research is needed on how keywords are used in ISPs.

Practical implications: The authors recommended that ISP designers exercise caution when using keywords in ISPs and maintain coherency in their use of keywords. ISP designers can use the suggested metrics to assess the quality of actionable advice in their ISPs.

Originality/value: The keyword loss of specificity metric adds to the few quantitative metrics available to assess ISP quality. To the best of the authors' knowledge, applying this metric is a first attempt to measure the quality of actionable advice in ISPs.