Tele-prevention: How Do We Evaluate Prevention Services Provided Through Electronic Technology?I cannot speak for others, but I am beginning to suffer from “Zoom fatigue.” Since March of this year, most of my interactions with others have been through electronic meetings and social media. This adjustment has been common for many as we deal with a global pandemic. Some of the adjustments and adaptations that have been made in the healthcare and prevention fields are likely to become permanent. For example, telehealth is now being used by numerous healthcare providers and there are some definite advantages. Providing healthcare in rural areas has always been a challenge and technology can provide improved access. There are also some challenges. It is important to make sure that patients have access to electronic devices for example. There are also challenges with the quality of the connection services as some areas do not have access to reliable internet. Ethical challenges include confidentiality and how data is stored and who has access. That said, telehealth and tele mental health will continue beyond the pandemic due to reductions in cost and an increase in access to services.
Prevention providers are adjusting the way they provide services during these challenging times. The Mountain Plains PTTC recently sponsored a two-part webinar series entitled Supporting Prevention Efforts Through Virtual Settings Two-Part Webinar Series: Understanding How to Leverage Telehealth Technology for Prevention Services (Fischer, 2020). Delivering services using electronic technology may have some advantages. In a world where technology and social media impact the opinions and behaviors of many people, especially our youth and young adults, adapting and expanding prevention efforts to be provided through electronic technology are important. Many of our youth have grown up with computers, electronic tablets, and smart phones. This will provide both implementation and evaluation challenges. For example, most of the evidence-based prevention programs are delivered in a face-to-face format. Even those that provide instruction through videos or others forms of electronic technology have the challenge of providing the services in a format that engages the participant. Also, technology can be used in shaping norms that influence behaviors, including substance use and other problem behaviors.
The Strategic Prevention Framework (SPF) has been the guiding framework for prevention programs for almost twenty years. As prevention continues to use electronic technology to provide services, there are some issues that need to be considered at each stage of the model. I am going to suggest some of these challenges at each stage that need consideration. This list will in no way be exhaustive but will hopefully be a starting point for those that are considering expanding their use of technology in prevention efforts.
The first stage of the SPF is Assessment. Many prevention providers have been utilizing technology to gather assessment data for some time but the need for collecting data electronically has increased in recent months. There are informed consent issues that need to be addressed as well as how the data will be stored and who will have access. Anonymity is expected in most survey efforts and consent forms may need Human Subjects Committee Approval by an Institutional Review Board (IRB). There may also be additional software and platform costs.
The second stage is Capacity Building. Some of the capacity issues that need to be addressed in the planning of prevention efforts include access to technology and quality of the intervention connection of those that will receive services. Some studies that have investigated the use of smart phones in early intervention efforts with selected and indicated populations indicated that over ninety-five percent of the participants had access to adequate technology (Kazem, et.al., 2017). This may not be true in all communities and should be part of the capacity assessment. Also, there is an inequity in availability of reliable high-speed internet in some communities. This can be a considerable challenge in rural areas.
In the Planning stage it is important to connect risk factors that are identified in the Assessment stage with evidence-based programs, policies, and strategies. Choosing prevention programs or approaches that can be delivered reasonably through technology will be important. If the program has not been implemented using electronic technology, then adaptations will need to be considered. These adaptations will need to be addressed in the evaluation plan. It is also important to consider the strength and reach of the prevention intervention to influence change in the risk and protective factors that are the focus of the program. It is likely that both face-to-face and electronic technology will be used in comprehensive prevention programs in the future.
As the Implementation process begins it will be important to ensure that the infrastructure is in place to deliver and monitor interventions provided through technology. Process evaluation efforts are needed at this stage to make sure that services are delivered as intended and are reaching the appropriate audience. It is also important to gather data on how the services are being received and the satisfaction level of participants. Some examples of data collection efforts include surveys, focus groups, and individual interviews. Also, measuring participant engagement such as time spent on different sites and number of people visiting are standard measures that should be employed.
The last stage of the SPF is Evaluation. I have discussed some of the process evaluation efforts in the previous paragraph. It is also vital to measure the change in participants’ knowledge, opinions, and behaviors as the program or effort is being implemented. This is usually done using self-report surveys and questionnaires. For some efforts, an important measure may be looking at the level of engagement in relation to the outcomes. Again, it is important to consider informed consent, confidentiality, and data access issues when using electronic technology. Some of the usual concerns about protecting personal identity when using a traditional pre-test post-test design must be addressed. This will be a continual challenge for some time. Kaplan (2020) provides several additional recommendations on evaluating telehealth programs that may be adapted to tele-prevention.
Sustainability and Cultural Competence are important considerations at every stage of the SPF. As has been mentioned above, there are some communities where access to technology is limited. It may also be true that some sub-groups do not use technology or have the technical expertise to access some services. Programs are more likely to sustain if they meet the needs of those served. As technology is increasingly used to communicate, it is vital that prevention providers are exploring innovative ways to provide services and change social norms. This will be a continual challenge for some time.
Effective prevention efforts require flexibility in delivery systems and adaptability to environmental changes. The recent pandemic has caused many disciplines in health and health promotion efforts to change the mode of delivery. These program adaptations will need to be evaluated continually for effectiveness.
References:
Fischer, A.J. (2020). Understanding how to leverage telehealth technology for prevention services. Retrieved from https://pttcnetwork.org/sites/pttc/files/2020-04/Part%201%20Website.pdf.
Kazem, D.M., Borsari, B., Levine, M.J., Li, S., Lameberson, K.A., & Matta, L.A. (2017). A Systematic Review of the mHealth Interventions to Prevent Alcohol and Substance Abuse. Journal of Health Communication, 22: 413–432.
Kaplan B (2020), Revisting health information, technology, ethical, legal, and social issues and evaluation: Telehealth/Medicine and COVID-19, International Journal of Medical Informatics, doi: https://doi.org/10.1016/j.ijmedinf.2020.104239.
The National Quality Forum (2017). Creating a Framework to Support Measure Development for Telehealth. This report is funded by the Department of Health and Human Services under contract HHSM-500- 2012-00009I, Task Order HHSM-500-T0022
Hayden D. Center, Jr., PhD
