To Örebro University

Employee compliance with Information Security Policies (ISPs) depends on communicating clear and comprehensible content. However, existing research has shown that many ISPs are of poor communicative quality. Large Language Models (LLMs) could enhance ISPs if fine-tuned on high-quality data, but to do such fine-tuning requires a conceptual model for classifying the data and evaluating the resulting text. Therefore, as a step in this direction, the aim of this article is to develop a conceptual model of ISPs using Speech Act Theory as a theoretical lens to enable assessments of the communicative quality of ISPs. We used conceptual modeling and document analysis to develop the modelbased on 600 ISP statements from ten British National Health Service ISPs. We used selected parts from the SEQUAL framework to evaluate the model. The evaluation pointed to potential areas for improving the model’s semantic, empirical, physical, and deontic qualities. By incorporating these improvements, the final class diagram contains 21 classes, six of which address ISP statement quality as speech acts.

Purpose: Although existing literature has advanced the understanding of information security policy (ISP) management, it has not examined how artificial intelligence (AI) can support ISP activities across management phases. Moreover, no study has yet mapped the empirical domains studied. The purpose of this paper is to systematically map existing ISP management research to assess to what extent AI has been addressed or used.

Design/methodology/approach: This study follows the five-step scoping review framework proposed by Arksey and O'Malley (2005): identifying the research questions, finding relevant studies, selecting studies, charting the data and reporting the results.

Findings: The review reveals that very few ISP management papers address or use AI. These few papers focused mostly on operational ISPs and addressed different ISP phases and empirical domains. Most existing work focuses on construction or compliance, while no studies have addressed the technical level. Research methods dominated by experiments, with a notable absence of organizational fieldwork. Research on ethical aspects such as fairness, transparency, accountability and data sensitivity is rare in this area.

Research limitations/implications: Given the limited research in this area, there are significant opportunities to explore AI in ISP management and to use AI in studying ISP management. The authors suggest a research agenda divided into three-time horizons: short, medium and long term.

Originality/value: This paper provides the first scoping review of AI in ISP management research, offering a systematic mapping of ISP management phases, ISP levels, research methods and empirical domains. It identifies research gaps, thereby guiding future research.

Employees need clear guidance in order to contribute to their organ-isation’s digital sovereignty. Information security policies (ISPs) should provide employees with this guidance, but existing research show that this is not always the case. Large language models (LLMs) could enhance ISPs if finetuned on high-quality data, but such fine-tuning requires a robust way to distinguish between statements in ISPs. Therefore, and as a first step in this direction, the aim of this paper is to investigate the consistency when a Generative pre-trained transformer-like model classifies information security policy (ISP) statements into speech acts using a zero-shot learning (ZSL) approach. We investigate the consistency of LLAMA3.3-70B when applying ZSL to classify 600 selected ISP statements into speech act classes derived from Speech Act Theory. The findings show that LLAMA3.3-70B achieves high similarity to human classification and demonstrates strong consistency across multiple runs per different temperature settings. Furthermore, changes in the temper-ature setting had a minimal effect, which means that the model produces stable clas-sifications. We conclude that LLAMA3.3-70B is a useful starting point for continued work on improving ISP content using LLMs. 

Purpose: The purpose of this paper is to analyse and compare how congruent keywords are used to convey actionable advice in UK and Swedish information security policies (ISPs).

Design/methodology/approach: The authors conducted a qualitative content analysis of 30 ISPs from higher education institutions, 15 from each country. To support analysis, the authors developed the ISP Keyword Analyzer software, which extracted 2,314 sentences containing analysed keywords. Each sentence was classified as actionable advice or other information. For comparison, the authors used the Keyword Loss of Specificity and Total Keyword Loss of Specificity metrics to measure how congruently keywords were used within each ISP.

Findings: The authors found two main patterns in keyword use. First, certain keywords are more prominent in ISPs from one country than the other, indicating differing importance across languages and national contexts. Second, the congruence in how keywords are used to convey actionable advice also varies. Swedish ISPs use keywords more congruently to guide employees towards action, whereas UK ISPs more often use them to convey other information than actionable advice.

Research limitations/implications: Differences in how ISPs are formulated across countries suggest that previous studies on ISP content and design are contextually bound.

Practical implications: The identified differences highlight how language and national context influence the clarity of ISPs, with implications for organisations operating in multilingual or international environments.

Originality/value: To the best of the authors' knowledge, this is the first study to compare how keywords are used in ISPs in different countries to express actionable advice.

Context: Despite the widespread adoption of agile software development methods (ASDMs) today, many organizations struggle with effective implementation. One reason for this is that some organizations claim to use an ASDM without fully understanding its core principles, or they adhere to old practices while professing to follow a contemporary software development method. This phenomenon is sometimes referred to by practitioners as “cargo cult” (CC) behavior. However, simply labeling something as CC lacks analytical depth.

Objective: This paper aims to conceptualize and validate an analytical tool for diagnosing CC and non-CC behavior in software development teams’ use of ASDMs.

Method: This study uses a longitudinal ethnographic approach to conceptualize and validate the analytical tool by analyzing four agile practices used by a global industrial manufacturing company.

Results: The analytical tool features eight stereotypes—three representing non-CC behaviors and five representing CC behaviors—designed to aid in the analysis of ASDM usage. The tool draws on Social Action Theory and Work Motivation Theory to capture and interpret the CC phenomenon in ASDM use. Using the stereotypes, 36 actions were categorized as CC behavior deviating from documented ASDM practices, and 23 actions as non-CC behavior because they aligned with the documented ASDM and reflected agile goals and values. The tool thus can help both researchers and practitioners gain a deeper understanding of ASDM use in organizations.

Conclusion: This study advances understanding of ASDM use by moving beyond the simplistic use of the term “cargo cult”. The developed tool enables structured identification and classification of CC behaviors. The stereotypes provide a way of classifying recurring software development actions against the intended ASDM, allowing the identification of specific types of CC behaviors. The analytical tool enables managers to gain deeper insights into the underlying reasons for deviations, thereby supporting more grounded and effective agile practices within organizations.

In today's digital age, protecting information assets is critical to maintain organizations’ digital sovereignty. Yet existing research offers limited guidance on creating effective, actionable advice in information security policies (ISPs) that instructs employees on how to carry out their tasks and contribute to protecting information assets. Addressing this gap, the aim of this paper is to propose a definition of actionable advice. A clear definition can aid in designing ISPs and enhance communication with employees, guiding them in the expected behavior to protect the organization’s information assets. The research question guiding this work is: how can actionable advice be defined in information security policies? To achieve this aim, the definition is informed by a literature review and analysis of 47 ISPs from public agencies in Sweden. The proposed definition of actionable advice is: a demarcated part of an ISP, that instructs someone on a task to execute or not to execute regarding information security, and, in case of execution, how to carry out the task. The definition of actionable advice provides researchers with a starting point to understand this term, helping advancing future studies on ISPs. This work also has practical implications for ISP developers, offering guidance on writing pieces of actionable advice that are concrete and directly applicable in employees' daily tasks to protect their organizations.

Classifying natural language requirements (NLRs) is challenging, especially with large volumes. Research shows that Large Language Models can assist by categorizing NLRs into functional requirements (FR) and non-functional requirements (NFRs). However, Generative Pretrained Transformer (GPT) models are not typically favored for this task due to concerns about consistency. This paper investigates the consistency when a GPT model classifies NLRs into FRs and NFRs using a zero-shot learning approach. Results show that ChatGPT-4o performs better for FRs, a temperature parameter set to 1 yields the highest consistency, while NFR classification improves with higher temperatures.

Over the years, numerous studies on employee compliance with information security policies (ISPs) have been conducted, contributing valuable insights to enhance information security in organisations. However, our literature review reveals that few ISP compliance studies adopt a longitudinal approach. It is well known that cross-sectional research often provides limited insight into how constructs such as ISP compliance evolve over time. While researchers have called for more longitudinal ISP compliance studies, there is little guidance on how to conduct them. To address this gap, we propose a set of seven guidelines to support both quantitative and qualitative longitudinal ISP compliance research.

Classifying natural language requirements (NLRs) plays a crucial role in software engineering, helping us distinguish between functional and non-functional requirements. While large language models offer automation potential, we should address concerns about their consistency, meaning their ability to produce the same results over time. In this work, we share experiences from experimenting with how well GPT-4o and LLAMA3.3-70B classify NLRs using a zero-shot learning approach. Moreover, we explore how the temperature parameter influences classification performance and consistency for these models. Our results show that large language models like GPT-4o and LLAMA3.3- 70B can support automated NLRs classification. GPT-4o performs well in identifying functional requirements, with the highest consistency occurring at a temperature setting of one. Additionally, non-functional requirements classification improves at higher temperatures, indicating a trade-off between determinism and adaptability. LLAMA3.3-70B is more consistent than GPT-4o, and its classification accuracy varies less depending on temperature adjustments.