Critical Appraisal Of Occupational Environmental Medicine Research Study

Journal Occupational Environmental Medicine

Research Article and its Main Purpose

Discuss About The Journal Occupational Environmental Medicine.

In epidemiological evidence, study of causal association between exposure and outcome is necessary to determine whether the results or the conclusion of the study is valid or not. Critical evaluation of research process and methodology in research evidence is essential to determine the application of the interventions in real practice setting. To enhance the reliability of the work, evaluation of internal and external validity is important. The quality of an epidemiology study is understood from author’s consideration to reduce bias through careful design and data analysis process (Boccia et al., 2007). The main purpose of this essay is to critically appraise a research study by Strijk et al. (2013) and determine the overall quality of the study by appraising the internal and external validity of the research article. The application of the research outcome in target setting is also explained by the analysing the effectiveness of research in explaining about causal association between the exposure and the outcome.  

The study by Strijk et al. (2013) aimed to evaluate the effect of a worksite lifestyle intervention on vitality, engagement, productivity and sick leave. The main rationale for research in this area is the changes in the age of the general population because of reduction in birth rates and increased life expectancy. Due to labour shortage and aging workforce, sustaining workers for a longer period will be difficult. With the increase in age, the physical activity of workers may decline. Long shift hours can contribute to poor health status of employees and the possibility of early retirement because of chronic disease (Magnavita, 2017). Hence, to address the issue of poor occupational health in aging workers, the study aimed to address the issue by maintaining the vitality of worker. Vitality is one of the dimensions of work engagement that reflect well-being, mental resilience and perseverance of workers. To address challenges in aging workforce, Strijk et al. (2013) implemented an intervention on vitality and work engagement to improve lifestyle behaviour of older workers and prolong their work participation.

Strijk et al., (2013) used randomized controlled trial as a research design to evaluate the effect of worksite lifestyle intervention on workers. Workers aged above 45 years were taken from two academic hospitals and the intervention group received vitality exercise program (VEP) with free fruits and three visits to a personal vitality coach. The main features of the VEP was one weekly guided yoga group session for relaxation purpose and one weekly guided aerobic workout group session to improve aerobic fitness and muscle strength of workers. The visit to the personal coach mainly focused on addressing psychological issues in participants by goal setting and problem solving strategies. The primary outcome of interest for the study was vitality and the secondary outcomes included work engagement, productivity and sick leave. The data collected related to outcomes for intervention and control group showed no difference in vitality, productivity and sick leave. However, favourable outcomes were found in the group who complied with yoga sessions. Their work-related vitality increased significantly after the intervention. This is consistent with the study by Adhia, Nagendra & Mahadevan (2010) which showed positive effect of yoga on goal orientation and job satisfaction.

Research Design and Primary and Secondary Outcomes

Strijk et al., (2013) explained the reason for poor effect of the intervention by the recruitment of a healthy group of workers and the possibility of bias in research findings. The intervention was based on the [email protected] intervention and it was concluded that the intervention cannot be used as a tool to improve worker’s engagement in work, vitality and productivity. The strategies recommended for future research included identify factors to improve vitality and work engagement and evaluate effect of yoga intervention on work related outcomes.

Strijk et al. (2013) implemented worksite lifestyle intervention on hospital workers, however positive impact of the intervention (exposure) on vitality, work engagement and productivity (outcome) was not seen. Such outcome can be attributed to non-causal association between exposure and outcome such as the influence of certain biases in research methodology and chance variation on study findings. Three types of internal factors in research studies such as chance (random error), bias (systematic errors) and confounding factors can influence study results (Schmidt & Hunter, 2014).

The evaluation of research methodology used in the study shows effect of non-causal association by review of considerations used to promote the validity of the research work. The first limitation that affected the internal validity of the study includes poor data collection methods. This can be said because data related to outcomes were collected from participants by means of two questionnaires. Two separate questionnaire tools were used to assess the vitality outcome and the WHO Health Productivity Questionnaire was used to assess the productivity outcomes. The biasness in data collection method is that all focused on obtaining self-reported feedback from participant regarding vitality and engagement. This increased the risk of bias in outcomes. Combination of both self-reported data and observed data would have given appropriate idea regarding the effectiveness of exposure on the outcome (Strijk et al. 2013). Hence, this discussion indicates influence of information bias on study outcome. The measurements for exposure were not adequately done. This could explain poor association between exposure and outcome.

The internal validity of the study would have increased if the researchers took the approach to measure work related behaviour such as dedication and engagement in work by means of observation by third party. Similar approach was taken by Ishii, Shibata and Oka, (2018) as the researchers took the approach to measure work engagement by evidence based work engagement tools. In addition, questionnaires were also used to collect data on productivity. This approach helped to reduce information bias and examine relationship between work engagement and sedentary behaviour. Taking such consideration in collecting data would have improved the internal validity of the study by Strijk et al. (2013) too.

Effectiveness of Intervention on Workers

Another factor that reflects poor validity of the study includes attrition bias or loss-to-follow-up after the intervention. Follow-up data is important to evaluate long-term impact of exposure or intervention on the outcome. However, for the research by Strijk et al. (2013), selection bias was evident because of loss-to-follow up rate after 12 months. Follow-up for primary and secondary outcomes were done for 6 months and 12 months. The purpose for follow-up after 6 months was to assess short term effect of the intervention and the purpose of 12-months follow-up was to evaluate sustainability of effects in the long term. The loss-to-follow-up rate was higher for 12 months (N=117) compared to 6 months (N=74). Such kind of biases reduces the credibility and internal validity of research studies. In epidemiological studies, reducing loss to follow-up is essential to estimate the effectiveness of the intervention and evaluate clinical outcome (Shepherd et al., 2013). For the research article, loss-to-follow-up was a major limitation. However, Strijk et al. (2013) gave appropriate justification to ensure that selection bias had no impact on study outcome. This was explained by no dissimilarities in outcome between completers and non-completers for all outcome measures. This suggested influence of other factors on the study results.

Another factor that can explain non-causal association in study findings is the presence of bias due to confounding. This occurs when an unfamiliar association arise due to the failure to fully adjust for factors related to exposure and outcome (Pannucci & Wilkins, 2010). Influence of such variable is low in the study. This can be said because during the data analysis phase, all steps were taken to check for potential confounders. No difference in control and intervention group was found due to confounding factors. As the study used statistical method of analysis, there is also a possibility of chance variation in the study. However, the review of the article suggests little difference in predicted value and actual value of the variable. Hence, it can be concluded that information bias is the main factor that has influenced the validity of the work (Strijk et al., 2013). Identification of this form of biases can help to improve methodological quality of studies done in future regarding the same topic. Any form of chance and confounding can be eliminated through proper study design and taking rigorous steps to exclude bias as  an alternative explanation for association between outcome and exposure.

Internal Validity of Research Study

Causal association between exposure and outcome can be understood by proper discussion on the mechanism by which exposure influences outcome of research (Bhopal, 2016). The impact of exposure (worksite intervention) on overall group was not found. This is understood from no difference in outcome (such as vitality, work engagement, productivity and sick leave) between the intervention and control group. No difference was found for vitality outcome with time. However, the use of randomized controlled study design helped to establish causal association in those groups who adhered to the yoga intervention. By the focus on finding out the relationship between yoga and workout group compliance, causal association between work related vitality and compliance to yoga was accomplished. The more was the yoga compliance, the more was the work related vitality of workers. The effect was found to be larger for high compliance with the yoga program (Strijk et al., 2013). It also indicates dose-response relationship as the vitality outcome increased with increase in exposure (to yoga intervention). Hence, strong association between compliance and vitality suggest causal association between exposure and outcome.

The causal association between yoga and vitality is consistent with other research work. The study by Hartfiel et al. (2011) showed causal association between yoga and well-being. Resilience is also one of the components of work engagement and the causal association between yoga (exposure) and improvement of well-being and resilience to stress (outcome) was proved by the research. By means of a six week yoga intervention, the study showed positive effect of yoga on elation, energy and confidence of employees. To give strong evidence regarding causal association between exposure and outcome, it is necessary to illustrate the special mechanism behind it.

Strijk et al., (2013) explained the reason behind lack of association by commenting on the study population and degree of implementation of the intervention. The study was done with a healthy group of workers. However, to understand difference between outcome in control and intervention group, conducting research with workers experiencing chronic disease or occupational stress would have been useful. With respect to the degree of implementation, the authors gave the justification expected dose were delivered, however compliance rate and attendance rate were not up to the estimation. Hence, it can be interpreted that causal association would have been identified, if the maximum compliance rate was achieved. However, this is not an effective explanation about the possibility to get causal association because Hartfiel et al. (2011) has proved that even a short program of yoga is effective in improving emotional well-being and resilience. The study gave positive comments regarding the use of yoga as a tool to promote health and well-being of workers. However, as the same suggestion has not been given by Strijk et al. (2013), it is necessary to explore the mechanism behind positive outcome in employees after considering ways to improve and sustain participation in yoga programs. To improve the reliability of causal association, appropriate mechanism behind the causal effect should also be explained.

External Validity of Research Study

The external validity of a research can be determined by the generalizability of research findings and its application in relevant population group. The source population for this study includes aging workforce and the population taken in the study included workers above the age of 45 years from academic hospitals. The population of interest is aging workforce because ageing affects physical capacity, health status and health engagement of workers. As the research aimed to improve vitality of ageing workers, it was necessary to recruit a sample group which were in middle to old age and recruit combination of health and unhealthy workers to understand the effect of the intervention. However, the limitation of the study by Strijk et al. (2013) was that it recruited healthy participants which had no risk of adverse health effect when becoming active. The sample does not reflect a population of interest because aging population is likely to suffer from chronic illness or occupation stress. However, no such group has been taken thus limiting the applicability of findings to source population. Taking two groups such as old and young workforce group would also have helped to estimate the effect of the intervention.

Another important factor for external validity of a research study is that it should have clear research outcome and a research methodology that can be easily applied (Silverman, 2016). However, poor outcome related to the intervention and limitations in research methodology affected the applicability of research findings. For example, the main aim of the research by Strijk et al. (2013) was to evaluate effectiveness of vitality intervention on work engagement, productivity and sick leave. However, favourable outcomes were not achieved for the intervention group. The primary and secondary outcome related to work engagement were almost similar in both intervention and the control group. Negative outcome can also be attributed to inappropriate data collection method and poor rate of loss-to-follow-up. This limited evaluation of the exposed group and complete effect of intervention could not be done. Hence, due to this limitation, the study findings cannot be applied in source population. The study gave indications of positive outcome only for high compliance yoga group. To confirm the positive effects of frequency and dose of yoga on worker’s engagement, there is a need for larger research to increase the external validity of research.

As worksite intervention has been found to have no effect on participants, there is also a need to review other intervention that can help to improve the work engagement parameters of aging workers. Cloostermans et al. (2015) also suggest role of other interventions for ageing workers on work ability, productivity and retirement. For example, it showed the effect of exercise programmes, workplace programmes and individual programs on workers engagement. However, insufficient evidence have been found for these interventions. More work on these interventions can also be done to support evidence-based decision making to develop a healthy workforce for ageing workers. This would maximize the likelihood of application of research findings to target setting. On the whole, it can be said that the research is a low quality evidence which cannot be applied to resource population

Influence of Chance, Bias, and Confounding Factors

Conclusion:

The essay summarized the quality of study by Strijk et al. (2013) by the critical appraisal of research studies investigating the impact of an exposure on outcome. By the use of checklist related to finding bias and confounders in research, the internal validity of the study was assessed. The internal validity was low because of high loss-to-follow up cases and presence of information bias. Poor causal association between exposure and outcome also limits the applicability of the findings. To ensure that the exposure can induce positive outcome in future, it is essential to increase the methodological quality of the study in future and increase rigor in research design and data analysis process.

Reference:

Adhia, H., Nagendra, H. R., & Mahadevan, B. (2010). Impact of yoga way of life on organizational performance. International journal of yoga, 3(2), 55.

Bhopal, R. S. (2016). Concepts of epidemiology: integrating the ideas, theories, principles, and methods of epidemiology. Oxford University Press.

Boccia, S., La Torre, G., Persiani, R., D’Ugo, D., van Duijn, C. M., & Ricciardi, G. (2007). A critical appraisal of epidemiological studies comes from basic knowledge: a reader’s guide to assess potential for biases. World Journal of Emergency Surgery, 2, 7. https://doi.org/10.1186/1749-7922-2-7

Cloostermans, L., Bekkers, M. B., Uiters, E., & Proper, K. I. (2015). The effectiveness of interventions for ageing workers on (early) retirement, work ability and productivity: a systematic review. International archives of occupational and environmental health, 88(5), 521-532.

Hartfiel, N., Havenhand, J., Khalsa, S. B., Clarke, G., & Krayer, A. (2011). The effectiveness of yoga for the improvement of well-being and resilience to stress in the workplace. Scandinavian journal of work, environment & health, 70-76.

Ishii, K., Shibata, A. and Oka, K., 2018. Work Engagement, Productivity, and Self-reported Work-related Sedentary Behavior among Japanese Adults: A Cross-sectional Study. Journal of occupational and environmental medicine, 60(4), pp.e173-e177.

Magnavita, N. (2017). Productive aging, work engagement and participation of older workers. A triadic approach to health and safety in the workplace. Epidemiology, Biostatistics and Public Health, 14(2).

Pannucci, C. J., & Wilkins, E. G. (2010). Identifying and avoiding bias in research. Plastic and reconstructive surgery, 126(2), 619.

Schmidt, F. L., & Hunter, J. E. (2014). Methods of meta-analysis: Correcting error and bias in research findings. Sage publications.

Shepherd, B. E., Blevins, M., Vaz, L. M. E., Moon, T. D., Kipp, A. M., José, E., … Vermund, S. H. (2013). Impact of Definitions of Loss to Follow-up on Estimates of Retention, Disease Progression, and Mortality: Application to an HIV Program in Mozambique. American Journal of Epidemiology, 178(5), 819–828.

Silverman, D. (Ed.). (2016). Qualitative research. Sage.

Strijk, J. E., Proper, K. I., van Mechelen, W., & van der Beek, A. J. (2013). Effectiveness of a worksite lifestyle intervention on vitality, work engagement, productivity, and sick leave: results of a randomized controlled trial. Scandinavian journal of work, environment & health, 66-75.