Abstract Background: Complex interventions encompassing several interconnecting and interacting components can be challenging to evaluate. Examining the underlying trial processes while an intervention is being tested can assist in explaining why an intervention was effective (or not). This paper describes a process evaluation of a pan-European cluster randomised controlled trial, OptiBIRTH (undertaken in Ireland, Italy and Germany), that successfully used both quantitative and qualitative methods to enhance understanding of the underlying trial mechanisms and their effect on the trial outcome. Methods: We carried out a mixed methods process evaluation. Quantitative and qualitative data were collected from observation of the implementation of the intervention in practice to determine whether it was delivered according to the original protocol. Data were examined to assess the delivery of the various components of the intervention and the receipt of the intervention by key stakeholders (pregnant women, midwives, obstetricians). Using ethnography, an exploration of perceived experiences from a range of recipients was conducted to understand the perspective of both those delivering and those receiving the intervention. Results: Engagement by stakeholders with the different components of the intervention varied from minimal intensity of women’s engagement with antenatal classes, to moderate intensity of engagement with online resources, to high intensity of clinicians’ exposure to the education sessions provided. The ethnography determined that, although the overall culture in the intervention site did not change, smaller, more individual cultural changes were observed. The fidelity of the delivery of the intervention scored average quality marks of 80% and above on repeat assessments. Conclusion: Nesting a process evaluation within the trial enabled the observation of the mode of action of the intervention in its practice context and ensured that the intervention was delivered with a good level of consistency. Implementation problems were identified as they arose and were addressed accordingly. When dealing with a complex intervention, collecting and analysing both quantitative and qualitative data, as we did, can greatly enhance the process evaluation. Trial registration: Current Controlled Trials Register, ISRCTN10612254. Registered on 3 April 2013.
Reference text: References 1. Medical Research Council (MRC). Developing and evaluating complex interventions: new guidance. London: MRC; 2006. https://www.mrc.ac.uk/ documents/pdf/complex-interventions-guidance/. Accessed 14 Dec 2016. 2. Oakley A, Strange V, Bonell C, Allen E, Stephenson J, RIPPLE Study Team. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6. 3. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. doi:https://doi.org/10. 1136/bmj.a1655. 4. Hoffmann T, Glasziou P, Boutron I, Milne R, Perera R, Moher D, Altman D, Barbour V, Macdonald H, Johnston M, Lamb S, Dixon-Woods M, McCulloch P, Wyatt J, Chan A, Michie S. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. 5. Institute of Obstetricians and Gynaecologists, Royal College of Physicians of Ireland; and Clinical Strategy and Programmes Directorate; 2011. http:// www.hse.ie/eng/services/publications/Clinical-Strategy-and-Programmes/ Delivery-after-Previous-Caesarean-Section.pdf. Accessed 29 Oct 2017. 6. Ministero della Salute. Sistema Nazionale Linee Guida (SNLG): il taglio cesareo: una scelta appropriata econsapevole. 2012. 7. Clarke M, Savage G, Smith V, Daly D, Devane D, Gross M, et al. Improving the organisation of maternal health service delivery and optimising childbirth by increasing vaginal birth after caesarean section through enhanced women-centred care (OptiBIRTH trial): study protocol for a randomised controlled trial (ISRCTN10612254). Trials. 2015;16:542.
8. Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56:119–27. 9. Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6:134–47. 10. Grant A, Treweek S, Dreischulte T, Foy R, Gutherie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15. 11. National Institute for Health and Care Excellence (NICE). Behaviour change: general approaches. Public Health Guidance [PH6]. London: NICE; Oct 2007. https://www.nice.org.uk/guidance/ph6. Accessed 29 Oct 2017. 12. Lundgren I, Begely C, Gross M, Bondas T. ‘Groping through the fog’: VBAC (Vaginal birth after Casearean section) – a meta synthesis about women’s experiences. BMC Pregnancy Childbirth. 2012;12:85. 13. Lundgren I, Smith V, Nilsson C, Vehvilinen-Julkunen K, Nicoletti J, Devane D, Bernloehr A, van Limbeek E, Lalor J, Begley C. Clinician-centred interventions to increase vaginal birth after caesarean section (VBAC): a systematic review. BMC Pregnancy Childbirth. 2015;15:16. doi:https://doi.org/ 10.1186/s12884-015-0441-3. 14. Nilsson C, Lundgren I, Smith V, Vehvilinen-Julkunen K, Nicoletti J, Devane D, et al. Women-centred interventions to increase vaginal birth after caesarean section (VBAC): a systematic review. Midwifery. 2015;31:657–63. 15. Lundgren I, van Limbeek E, Vehvilainen-Julkunen K, et al. Clinicians’ views of factors of importance for improving the rate of VBAC (vaginal birth after caesarean section): a qualitative study from countries with high VBAC rates. BMC Pregnancy Childbirth. 2015;15:196. doi:https://doi.org/10.1186/s12884015-0629-6. 16. Lundgren I, Healy P, Carroll M, Begley C, Matterne A, Gross M, Grylka-Baeschlin S, Nicoletti J, Morano S, Nilsson C, Lalor J. Clinicians’ views of factors of importance for improving the rate of VBAC (vaginal birth after caesarean section): a study from countries with low VBAC rates. BMC Pregnancy Childbirth. 2016;16:350. doi:https://doi.org/10.1186/s12884-016-1144-0. 17. Nilsson C, Limbeek E, Vehvilainen-Julkunen K, et al. Vaginal birth after caesarean—views of women from countries with high VBAC rates. Qual Health Res. 2017;27:325–40. doi:https://doi.org/10.1177/1049732315612041.18,28. 18. Nilsson C, Lalor J, Begley C, Carroll M, Gross MM, Grylka-Baeschlin S, Lundgren I, Matterne A, Morano S, Nicoletti J, Healy P. Vaginal birth after caesarean: views of women from countries with low VBAC rates. Women Birth. 2017;30:481–90. doi:https://doi.org/10.1016/j.wombi.2017.04.009. 19. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71 Suppl 1:S52–63. 20. Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity criteria: development, measurement, and validation. Am J Eval. 2003;24:315–40. 21. Maguire R. “Trying for a VBAC”: an ethnography of cultural change within a randomised trial aimed at increasing vaginal birth after caesarean section: the OptiBIRTH study. Doctoral thesis, University of Dublin, Trinity College Dublin, Ireland; 2016. 22. Steckler A, Linnan L, editors. Process evaluation for public health interventions and research. San Francisco: Jossey-Bass; 2002. 23. Organisation for Economic Co-operation and Development (OECD). Fertility rates (indicator). Paris: OECD; 2014. doi:https://doi.org/10.1787/8272fb01-en. https://data.oecd.org/pop/fertility-rates.htm. Accessed on 3 Feb 2017. 24. Smyth R, Jacoby A, Altman D, Gamble C, Williamson P. The natural history of conducting and reporting clinical trials: interviews with trialists. Trials. 2015;16:16. 25. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004;328:1561–3. 26. Durlak J, DuPre E. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–50. 27. Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13:95. 28. Santacroce S, Maccarelli L, Grey M. Intervention fidelity. Nurs Res. 2004;53:63–6. 29. National Perinatal Epidemiology Unit (NPEU). Delivered with care: a national survey of women’s experience of maternity care. Oxford, UK: NPEU; 2010.
- Process evaluation
- Complex intervention
- Randomised controlled trial