If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Introduction: During the COVID-19 pandemic, support services for children and youth quickly shifted to virtual means. To continue delivering essential, trauma-informed, specialized services, the center transitioned to providing most services by phone/video conference.
Method: A quality improvement project using survey methods was conducted to determine if virtual delivery was timely and satisfactory for inpatient and outpatient care.
Results: Findings indicated services were timely. Caregivers appreciated the support, felt a personal connection with staff, and confirmed services met their goals and expectations. However, challenges faced by staff included engaging the child/youth by phone/video, loss of collaboration with colleagues, and concerns about fulfilling their role through virtual means.
Discussion: Understanding stakeholder experiences illuminated the path of quality improvement during this major shift in service delivery. Benefits were shown for a blended model of in-person and virtual services on the basis of clinical judgment and the unique needs of clients and families in considering future service model options.
). To reduce the spread of the virus, countries enacted measures including physical distancing, masks, and lockdowns with businesses and schools closing (
found that 54% of parents were very/extremely concerned about loneliness and social isolation in their children/youth during the pandemic. Physical distancing from support networks and loss of structured activities can lead to worsening behaviors and mental health symptoms (
Many children/youth with complex needs worldwide receiving residential services were initially sent home to reduce the spread of the virus, with some caregivers feeling unprepared and unsupported (
). There was a 60% reduction in mental health beds for children/youth in Toronto and surrounding areas, with the need for intensive services continuing during this time (
). Virtual services offer many potential benefits, including health care access in rural and remote areas; reduced wait times; shorter, more efficient virtual appointments; and cost-effective service delivery (
). Virtual services are not new (e.g., telehealth, teletherapy, telemedicine), but barriers that once existed were quickly resolved in response to COVID-19 (e.g., access to technology, privacy issues;
This evaluation of caregiver and clinician experiences with virtual services took place at a center that delivers trauma-informed, highly specialized, targeted intervention services for children/youth in Ontario, Canada, with complex needs, including developmental disabilities, autism, and severe behavioral and emotional challenges. The center remained open during provincial lockdowns as its interdisciplinary teams provided essential outpatient and inpatient services. At the start of the pandemic, laptop computers were procured with virtual private network access to enable most staff to work from home to minimize disruption in service delivery. Services to outpatient clients continued with appointments offered by phone and video. The inpatient units initially transitioned most clients back to their homes, then gradually had clients return once pandemic guidelines allowed greater residence numbers. Intensive support by phone and video was provided to high-needs clients referred for inpatient services, both as interim support during the wait for inpatient admission and to potentially avoid readmission. As a quality assurance initiative, an evaluation was implemented to determine whether timely and satisfactory virtual services could continue to be offered after the pandemic restrictions were lifted.
METHODS
Participants
Direct care clinical staff were surveyed each month anonymously and asked to consider the past 2 weeks of service delivery, yielding 245 responses (nonunique individuals, meaning all staff had the opportunity to respond each month): 65 in June, 47 in July, 38 in August, 44 in September, 29 in October, and 22 in November.
Caregivers of clients from infancy to 18 years old were sent an invitation to respond anonymously to a survey online after each virtual appointment, yielding 155 responses (potentially nonunique individuals, meaning caregivers with multiple virtual appointments in the 6 months may have been sent the survey multiple times). Similar surveys were sent to clients (with the capacity of at least grade 2) and community partners. Because of low response rates, these results are not presented.
This evaluation was exempt from human subject ethics review in accordance with the standards for quality assurance outlined by the Western University Research Ethics Board. Participants were informed that completing the surveys was voluntary and their decision would not affect receiving any future services. Survey completion implied consent to participate in the evaluation.
Measures
Brief surveys were created with items developed through a review of key domains of virtual care in the published literature and refined by staff with expertise in survey design, research, and direct care across disciplines and service types (outpatient and inpatient). Survey domains included timeliness of services, experience with equipment and technology, privacy and safety, rapport and personal connection, and meeting clinical goals. The caregiver survey contained 23 items, and the staff survey contained 29 items. Most items presented three response options: agree, unsure, and disagree. The surveys also contained open-text options for caregiver and clinician comments. Surveys were completed through Survey Monkey from June through September 2020 and Microsoft Forms between October and November 2020.
RESULTS
The staff group had roughly equal representation from inpatient (34%) and outpatient teams (39%) from all disciplines. Staff who worked with both team types comprised 16% of the respondents, and 11% chose not to disclose their team. Across all months, 82% of staff reported delivering services from home.
For 15% of caregivers, it was their first appointment with the clinician, whereas 85% were follow-up appointments. The number for whom this was their first experience with virtual services was unknown. Caregivers reported meeting with the center's occupational therapists (20%), psychiatrists (19%), parent-infant therapists (17%), community behavioral consultants (16%), physiotherapists (14%), social workers (9%), pediatricians (8%), board certified behavior analysts (7%), speech and language pathologists (5%), nurses (3%), case management coordinators (2%), and childcare workers (1%) from inpatient and outpatient teams.
This evaluation found value in virtual services and provided the center's leadership team with timely stakeholder feedback to help guide decisions during the pandemic and into the future. In general, caregivers reported more positive experiences than clinical staff in the domains surveyed. Key findings are summarized below.
Timeliness of Services
Half of the caregivers (54%) reported that they felt the wait time was shorter than it would have been for an in-person appointment, and 26% of staff felt the wait time was shorter. Comparing operational data from April to October 2020 with the same timeframe before the pandemic in 2019, the average wait time for the first service decreased by 3.2 weeks. At the same time, there was a decrease in referrals (an average of 23 fewer outpatient clients and eight fewer inpatient clients per month) and an increase in service duration by an average of 12 weeks during the pandemic. The drop in referrals appeared to be consistent with the experience of many countries including Canada, in which there was a reduction in the use of health care services by children/youth during the pandemic (
). Regarding increased service duration, staff comments indicated that they served clients longer because they felt it often took longer to achieve clinical goals through remote means, and there were challenges discharging clients because many community agencies were unavailable for clients to transition to for continuity of care because of decreased capacity and closures during the pandemic.
Equipment and Technology
Staff mainly used phone, Microsoft Teams (Microsoft, Redmond, WA), and Ontario Telemedicine Network (Ontario Telemedicine Network, Toronto, Canada) to connect with families. Microsoft Teams and Ontario Telemedicine Network are applications that provide videoconferencing through media devices such as smartphones or computers. A greater proportion of caregivers (87%) than staff (69%) reported a positive experience overall with the technology working well (i.e., sound and/or video). Despite some technical challenges, most of the time, families and staff understood each other, with 94% of caregivers and 81% of staff reporting there were no communication difficulties. However, staff and caregivers’ comments expressed a loss from a lack of nonverbal cues (e.g., eye contact, body language). Staff comments further indicated communication difficulties with clients/families with English as a second language and difficulty connecting with those without access to phone or video (e.g., Amish and Mennonite families and clients/caregivers with severe complex challenges who were unable to use technology). Caregivers without technology would not have been surveyed, as they would not have received virtual appointments. Non-English-speaking caregivers may not be represented in the data, although interpreter services were available for the survey.
Privacy and Safety
Almost all caregivers (89%) and staff (81%) reported having a quiet space for virtual appointments, but only just over half of the caregivers (60%) and staff (58%) reported being free of distractions during virtual sessions.Figure 1 shows that although most staff reported being comfortable providing virtual services, very few were comfortable doing so from home. Staff commented on physical discomfort as a challenge (e.g., screen fatigue, poor ergonomics, not having adequate workspace at home) and lacking access to resources when working from home (e.g., printing materials, giving material resources to clients/families). However, also shown in Figure 1, most staff maintained work and home life balance by working from home, reiterating the greatest number of positive comments expressing quality work-home balance.
Figure 1Clinical staff opinions on providing virtual services from home.
As shown in Figure 2, most caregivers felt it was private and safe during their appointments. In contrast, there was greater uneasiness among staff regarding privacy over the phone/video and whether it was private and safe to talk on their client's end of the call.
Figure 2Caregivers’ and staff's feelings regarding privacy and safety talking on phone/video.
As shown in Figure 3, half of the caregivers reported their child/youth was engaged with the clinician during the virtual session. Staff perception differed in that only a minority felt successful at engaging their young clients. Caregivers and staff were mainly uncertain when asked if their children/youth would rather meet by phone/video than in person (caregivers: 21% agree, 47% unsure, 32% disagree; staff: 9% agree, 62% unsure, 29% disagree). Engaging the child/youth was challenging in many staff and caregiver comments. Staff comments indicated greater difficulty building rapport with new clients and caregivers over phone/video, and there were noted challenges having difficult conversations and in-depth treatment sessions (e.g., reading/responding to body language and expressing empathy was reported to be challenging by phone/video).
Figure 3Caregivers’ and staff's opinions on engaging children/youth by phone/video
Personal connection with clients/families was a substantial loss felt by staff and echoed by caregivers in many comments. However, when caregivers were asked if they felt they connected with the clinician personally, 79% agreed, 13% were unsure, and only 9% disagreed, indicating that rapport can be developed during virtual sessions (
). Next steps for investigation could focus on the impact that virtual work has on therapeutic alliance between clients/caregivers and clinicians, the challenges of building rapport through phone/video, and the impact on meeting clinical goals. This evaluation suggests it is possible to cultivate clinical rapport through virtual technology, but it may be of a different nature or quality than would be established in person.
The impact on interdisciplinary teamwork and staff's connection with colleagues was of equal importance. As shown in Figure 4, connection with colleagues appeared to be impacted more negatively with outpatient than inpatient teams. Staff comments echoed strong feelings of loss of personal connection with colleagues and that teams have found new ways to stay connected while apart, but with more effort required.
Figure 4Staff responses to the item: “My teamwork and communication with colleagues has not been impacted negatively by being offsite”.
Most caregivers felt that the clinician understood their concerns (94%), felt the virtual services met their goals and expectations (80%), and were confident that they would be able to follow the clinical recommendations (87%). In contrast, only 48% of staff said they could meet clinical goals using virtual services, and only 35% felt they could perform all aspects of their role virtually. The greatest number of challenges noted in staff and caregiver comments focused on a desire for clinicians to observe and/or help clients directly. Challenges were noted regarding completing comprehensive assessments requiring direct observation, testing (i.e., IQ assessment), and physical examination.
The greatest number of positive comments by caregivers were related to virtual services being efficient, effective, and helping to manage stress. However, as shown in Figure 5, only a third of caregivers and very few staff reported that virtual services are just as effective as in-person. Regarding virtual services for the future, staff and caregiver comments favor a blended model with a desire for some services to return in person. Staff comments suggested that virtual services were beneficial for some types of meetings (e.g., when travel would be burdensome) and maintaining personal connection and treatment engagement with preexisting clients and families over time. Although caregivers expressed a loss of direct contact with clinicians, their expressions of gratitude for the continued support during the pandemic were immense.
Figure 5Caregivers and staff responses to the item: “I believe virtual services are just as effective as an in-person appointment”.
). However, there are some considerations for the future when determining which service model to use for postpandemic. For example, challenges related to distractions in homes, less privacy, and technical difficulties remain ubiquitous (
). Clinicians found challenges in fulfilling all aspects of their role, as virtual appointments do not allow clients to be directly observed, evaluated, or physically examined (
). There were challenges engaging children/youth by phone/video, which may be intensified for younger children and those with delays and disabilities, leaving clinicians to rely solely on caregivers to model strategies and manage problem behaviors (
). In addition, clinicians expressed less satisfaction than when helping clients directly and a loss of close collaboration with colleagues. The positive aspects of virtual services must also be acknowledged, specifically that timeliness of services was maintained, and that caregivers felt their goals were met, they experienced a personal connection with clinicians, and they appreciated the support during difficult times.
Limitations
This evaluation has several limitations to consider. First, the evaluation was launched in June 2020, a few months after the transition to virtual services took place urgently in March. The shift to virtual services brought many early challenges that may have been overcome by the time of the surveys. Second, there is the potential for biased responses. For example, perhaps the most engaged caregivers, those with the least technical difficulties, and/or positive overall experiences completed the surveys. Third, some family types were not represented, for example, families without technology, non-English speaking, and caregivers who chose to forego virtual services until in-person appointments were available. Finally, although this evaluation was informative for advising the center's leadership team, it was not designed as a research study with methodological rigor. Future research investigations could include a control group with in-person services to compare with virtual experiences.
Conclusions
Clinicians and caregivers indicated being open to a blended model of in-person and virtual services when pandemic restrictions are lifted on the basis of clinical judgment and the unique needs of children/youth and families. Although more investigation is warranted, particularly on the quality of therapeutic rapport and clinical outcomes associated with virtual care, this quality assurance initiative suggests in-person services may be most beneficial for initial appointments to establish rapport, clients/families with language and/or technology barriers, younger children and those whose severe complex needs are such that engaging through technology is not possible (e.g., developmental delays), and assessments requiring in-person observation, testing, and physical examination. Virtual appointments could be efficient for some meeting types, reducing travel burden, providing interim support to families and clients, and maintaining engagement in the treatment process.
The authors wish to thank the families and staff who shared their experiences with virtual services to enable informed decision making at the center.
References
Chanchlani N.
Buchanan F.
Gill P.J.
Addressing the indirect effects of COVID-19 on the health of children and young people.
00289-9&id=gr1.jpg){kind=link}
00289-9&id=gr2.jpg){kind=link}
00289-9&id=gr3.jpg){kind=link}
00289-9&id=gr4.jpg){kind=link}
00289-9&id=gr5.jpg){kind=link}