To Örebro University

Background: Numerous Web-based interventions have been implemented to promote health and health-related behaviors in persons with chronic conditions. Using randomized controlled trials to evaluate such interventions creates a range of challenges, which in turn can influence the study outcome. Applying a critical perspective when evaluating Web-based health interventions is important.

Objective: The objective of this study was to critically analyze and discuss the challenges of conducting a Web-based health intervention as a randomized controlled trial.

Method: The MODIAB-Web study was critically examined using an exploratory case study methodology and the framework for analysis offered through the Persuasive Systems Design model. Focus was on technology, study design, and Web-based support usage, with special focus on the forum for peer support. Descriptive statistics and qualitative content analysis were used.

Results: The persuasive content and technological elements in the design of the randomized controlled trial included all four categories of the Persuasive Systems Design model, but not all design principles were implemented. The study duration was extended to a period of four and a half years. Of 81 active participants in the intervention group, a maximum of 36 women were simultaneously active. User adherence varied greatly with a median of 91 individual log-ins. The forum for peer support was used by 63 participants. Although only about one-third of the participants interacted in the forum, there was a fairly rich exchange of experiences and advice between them. Thus, adherence in terms of social interactions was negatively affected by limited active participation due to prolonged recruitment process and randomization effects. Lessons learned from this critical analysis are that technology and study design matter and might mutually influence each other. In Web-based interventions, the use of design theories enables utilization of the full potential of technology and promotes adherence. The randomization element in a randomized controlled trial design can become a barrier to achieving a critical mass of user interactions in Web-based interventions, especially when social support is included. For extended study periods, the technology used may need to be adapted in line with newly available technical options to avoid the risk of becoming outdated in the user realm, which in turn might jeopardize study validity in terms of randomized controlled trial designs. Conclusions: On the basis of lessons learned in this randomized controlled trial, we give recommendations to consider when designing and evaluating Web-based health interventions.