If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Correspondence: Brett Enneking, PsyD, HSPP, Department of Pediatrics, Indiana University School of Medicine, Ste. 2120, 1002 Wishard Blvd., Indianapolis, IN 46202
We report caregiver agreement to attend telehealth neurodevelopmental consultation during COVID-19, demographic differences in agreement, reasons families declined, and clinical access metrics before and during COVID-19.
Method
Data were gathered from telehealth referrals and consultations from April to July 2020. Schedulers documented agreement status and reasons for the decline. Wait time, lag time, and missed appointment rates were calculated to measure access.
Results
Ninety-one percent agreed to attend telehealth consultation; 55% of those who declined preferred in-person services. There were no demographic differences between those who accepted, declined, or did not respond. The median wait time from referral to appointment was 60 days. Missed appointment rates were consistent with prepandemic rates.
Discussion
Findings support literature suggesting patients are agreeable to telehealth. They diverged from evidence suggesting telehealth reduces missed appointments. Overall, results indicate telehealth is an acceptable alternative; however, further telehealth innovation is needed to address existing disparities.
Before the COVID-19 pandemic, there was limited evidence to suggest that telehealth had the potential to increase access to needed services and decrease burdens for families of children with neurodevelopmental disabilities (
). Furthermore, small studies have demonstrated the feasibility, acceptability, and effectiveness of diagnostic and behavior intervention services delivered through telehealth (
Brief Report: Preliminary feasibility of the TEDI: A novel parent-administered telehealth assessment for autism spectrum disorder symptoms in the first year of life.
Journal of Autism and Developmental Disorders.2020; 50: 3432-3439
) because of barriers including inconsistent reimbursement for services, limited access to the necessary technology, and the need for clinician training on telehealth technologies (
). When COVID-19 made traditional diagnostic methods unsafe and inaccessible, social distancing requirements necessitated a rapid transition to telehealth for clinics worldwide.
Since the onset of COVID-19, there has been a significant increase in evidence supporting telehealth models for neurodevelopmental evaluation (
). However, there continues to be a need for additional data on the feasibility and impact of service access.
Evidence from primary care and other subspecialties suggests that the use of telehealth may reduce the rate of patients who do not arrive for or call to cancel their scheduled appointment (
found that use of telehealth reduced missed clinic visits for child and adult mental health services and general community health services for those in urban areas. Differences in show rates by race and ethnicity were consistent regardless of whether the appointment was completed in person or through telehealth (
). Furthermore, there is mixed evidence regarding the efficacy and uptake of telehealth for reducing access barriers across race, ethnicity, and socioeconomic groups in the primary care setting (
Although information regarding change in access metrics for patients with neurodevelopmental disabilities is limited, some evidence suggests that telehealth evaluation for children with ASD may decrease overall service costs and eliminate travel time (
). Furthermore, research on the impact of telehealth on waitlists or wait times, from referral to appointment, suggests that telehealth is associated with decreased numbers of patients waiting for subspecialty care (
). For children with neurodevelopmental disabilities and their families, access to telehealth consultation services was associated with a reduction in the number of referrals made from rural areas to the tertiary care center (
COVID-19 has offered a unique opportunity to examine the utility of telehealth for serving children with neurodevelopmental disabilities and their families. However, gaps in the literature remain regarding who agrees to engage in telehealth consultation services, how demographic factors might impact acceptance, and how telehealth impacts metrics of service access. As such, the overall objective of this study was to examine caregiver agreement to engage in telehealth neurodevelopmental consultation during COVID-19 when in-person clinical services were suspended at a large outpatient neurodevelopmental evaluation clinic. In addition, we report on demographic differences between those who agree, decline, and are lost to follow-up after referral (i.e., did not respond to contact), reasons for which families declined a telehealth appointment, and clinical access metrics (i.e., wait time and rates of missed appointments) before and during the COVID-19 pandemic. Although research examining the feasibility and accuracy of telehealth neurodevelopmental consultation and evaluation is promising, understanding the reasons why caregivers accept or decline telehealth, how sociodemographic factors impact acceptability, and how telehealth impacts clinical access metrics has the potential to inform how telehealth service delivery models can improve care for children and families as well as the clinicians and institutions who serve them.
METHODS
Setting
Data were collected from an outpatient neurodevelopmental evaluation clinic within an academic medical center in the Midwestern United States. The interdisciplinary clinic is staffed by nine psychologists, two developmental-behavioral pediatricians, two speech/language pathologists, two social workers, and several affiliated trainees. Services are provided to patients ranging in age from 1 to 20 years with referral questions including ASD, developmental delay, speech-language delay, and behavioral concerns. The clinic serves patients from multiple neighboring Midwestern states, resulting in families traditionally traveling up to several hours for an appointment. Typical clinic volume before COVID-19 included approximately 4,500 referrals per year, with approximately 8,000 appointments (new and follow-up) offered. Sixty-six percent of patients seen in the clinic before COVID-19 used Medicaid insurance as their primary health plan.
All patients referred to the clinic undergo an initial diagnostic consultation appointment. Initial appointment duration typically ranges between 90 and 120 min and is determined by the necessity for multiple disciplines. Interdisciplinary appointments are 120 min long and include a developmental-behavioral pediatrician and psychologist. A diagnostic consultation with a single discipline is scheduled for 90 min. During this consultation, providers conduct a clinical interview and developmental history with the patient's caregivers, obtain behavioral observations, and sometimes use a formal symptom inventory or observational screening tool. Usually, providers spend approximately 60–75 min completing a face-to-face interview and any symptom inventories, followed by 10–15 min synthesizing data, identifying resources and recommendations, and culminating with 15–45 min in face-to-face counseling with families. Typical “next steps” include further in-clinic evaluations or consultations (i.e., interdisciplinary or discipline-specific), treatment (provided by our clinic or an outside agency), and/or support via local resources and problem-specific recommendations.
Telehealth Transition
Before COVID-19, the clinic had piloted telehealth for follow-up care navigation services on a very limited basis. When stay-at-home orders were initiated in mid-March 2020, fewer than 10 telehealth visits had been conducted. The demand for continuity of care provided safely and socially distanced required a rapid transformation of clinic processes. Clinical services well-suited for a rapid transition to telehealth (e.g., initial diagnostic consultations) were prioritized while efforts were made to overcome barriers to conducting other services virtually. Telehealth appointment times remained the same for in-person services (i.e., 90 min for single discipline and 120 min for interdisciplinary). In lieu of traditional measures, providers identified and obtained training on alternative telehealth assessment measures (e.g., Tele-ASD-Peds;
Corona, L., Hine, J., Nicholson, A., Stone, C., Swanson, A., Wade, J., . . . Warren, Z. (2020). TELE-ASD-PEDS: A telemedicine-based ASD consultation tool for toddlers and young children. Retrieved from: https://vkc.vumc.org/vkc/triad/tele-asd-peds.
) used to supplement parent interviews were fully transitioned to remote electronic completion using publisher-provided platforms (e.g., Pearson Q-Global). Following the appointment, a written summary and recommendations were sent to families by secure e-mail, a final consultation report was provided to caregivers and the referring physician, and the opportunity for care navigation support from social work was provided as needed.
Administrative process
Clinic staff and providers developed detailed educational guides, support tools, and processes for three primary groups: (1) providers, (2) staff members, and (3) families of referred children. Providers were trained on the use of the telehealth platform, administration of assessments (
), telehealth billing guidelines, virtual provision of documentation to the families, and compliance with state and national clinical guidelines. The administrative staff was trained on scheduling procedures and scripts to support families accessing telehealth consultations. A technology support hotline was established to assist families before and during the consultation if needed. In addition, providers and administrative staff collaborated to ensure families received timely clinical documentation following the diagnostic consultation.
Patient experience
All patients referred to the clinic for consultation of developmental and/or behavioral concerns were eligible for a telehealth appointment during the study period. During the initial scheduling phone contact, the administrative team offered a telehealth consultation and informed the caregivers of the referred child that it was unknown when in-person appointments would resume. Those who declined telehealth were informed of their placement on a waitlist for an in-person consultation. For caregivers agreeing to be scheduled for a telehealth appointment, administrative staff verified that families had the required technology, including access to a mobile device with a camera (i.e., smartphone, tablet, or computer), the ability to download the application needed for the conferencing platform (i.e., Zoom Health), and appropriate Internet connection. Verbal and written instructions on accessing the virtual platform were provided and guidance on preparing for their diagnostic consultation appointment.
Telehealth Data Collection and Analysis
This project was part of a larger research study on the implementation of telehealth for neurodevelopmental consultation and was approved by the university institutional review board. Data were collected for patients contacted to schedule a telehealth appointment between April and July 2020. Clinical schedulers used verbal scripts to contact the patient's caregiver to determine whether they were agreeable to a telehealth visit and their preferred platform (e.g., phone or virtual). Based on the conversation, data were gathered regarding whether the family agreed to the telehealth appointment, declined the telehealth appointment, and any reasons for declining were collected. Reasons for declining were categorized into the following: (1) do not have appropriate technology (i.e., video, Internet/data, reliable phone connection), (2) have other prioritized needs (i.e., food, safety), (3) concern about payment or insurance reimbursement, (4) prefer in-person appointment and would rather wait until this is available, or (5) other. If the scheduling team could not reach the family after two phone calls 48 hr apart and the family did not return the phone call during the study period, the patient was considered “lost to follow up.” Schedulers documented this information during phone calls through an online, secure data collection survey. In addition to this data, the patient's electronic medical records were reviewed to obtain child demographic information, including sex, date of birth, referral date and concern, and insurance information.
Two measures of access were collected for the current study: (1) wait time (i.e., defined as the time from receipt of referral to appointment completion) and (2) lag time (i.e., defined as the time between when the patient is scheduled and when the appointment is completed). Because our hospital system collects lag time data, the research team was able to compare this metric between pre- and post-COVID-19. Data used to calculate both wait time and lag time were gathered from the electronic medical record. Missed appointments were defined as families who did not arrive for their scheduled appointment, called to cancel, or did not respond to a phone call from the provider to resolve any technological concerns.
Study data were collected and managed using REDCap (Research Electronic Data Capture), a secure, web-based software platform to support data capture for research studies (
Research Electronic Data Capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support.
Journal of Biomedical Informatics.2009; 42: 377-381
). Data analysis was performed with SAS version 9.4 (SAS Institute, Inc., Cary, NC). Descriptive statistics for continuous variables are presented as means and standard deviations, and descriptive statistics for categorical variables are presented as frequencies and percentages. Between-group analyses for continuous variables were conducted using a one-way analysis of variance, and between-group analyses for categorical variables were conducted using χ2 or Fisher's exact tests. The relationship between wait/lag time and demographic variables was examined using generalized linear mixed effect models with random effects to account for correlation within providers. A similar method was used to examine differences in demographic variables between those who attended their consultations and those who did not. Of note, for these analyses, children who were scheduled for a phone visit (n = 4) or required the use of an interpreter (n = 8) were excluded.
RESULTS
Patient Demographics
Participants included 430 patients referred to our clinic and contacted for consultation scheduling between April–July 2020. Patients were, on average, 71 months of age (range 18–211 months). Sixty-nine percent (n = 296) of patients were male. Patient race/ethnicity was only available on a subset (n = 124) of the sample because of incomplete information in the patient's medical record. Of those for which data was available, 80% (n = 99) identified as White. Other races included Black or African American (11%, n = 14), more than one race (5%, n = 6), Asian (3%, n = 4), and other (1%, n = 1). Eighty-eight percent (n = 109) of patients identified as non-Hispanic/Latinx. Most common referral questions included ASD (50%; n = 213), developmental delay (18%; n = 78), and behavior problems (12%; n = 50). Other referral questions (20%; n = 89) included language/communication delay, attention deficit hyperactivity disorder, learning difficulties, and anxiety. Most patients (57%; n = 246) used Medicaid as their primary health insurance plan. For those patients for whom details regarding household income were available (n = 124), 35% reported an income less than $40,000, 29% reported between $40,000 and $74,999, and 29% reported $75,000 or more. See Table 1 for a detailed summary of demographic and referral information. For details regarding patient flow through the study, see Figure 1.
Table 1Demographics of children identified for telehealth consultation (n = 430)
Of the 430 children identified for telehealth consultation, 21% (n = 91) of families were lost to follow-up (i.e., did not respond to contact). Of those who did respond to contact (n = 339; 79%), most families (92%; n = 310) agreed to be scheduled for a telehealth consultation. Ninety-nine percent (n = 308) preferred to have a video visit over a phone consultation.
Eleven percent (n = 29) declined to participate in a telehealth appointment. Of those who declined to participate in a telehealth consultation (n = 29; 11%), 55% (n = 16) reported a preference for an in-person consultation, 17% (n = 5) did not have the appropriate technology, and 31% (n = 9) declined for other reasons. Other reasons included prioritizing other family needs, being no longer interested in appointments, and receiving services elsewhere. Notably, no families endorsed hesitancy around payment or insurance reimbursement.
Additional statistical analysis was conducted to determine whether there were differences in agreement to engage in telehealth based upon demographic factors. Results indicated that there were no differences in acceptance on the basis of child's age (F2, 427 = 0.67; p = .51), child's sex (c2 [2, n = 429] = 1.60, p = .45), primary referral question (ASD; developmental delay; other; c2 [4, n = 430] = 3.74, p = .44), or insurance type (Medicaid; private; none; c2 [4, n = 429] = 3.81, p = .47).
Access Metrics
Wait time (i.e., how long families waited for a telehealth appointment) was calculated by determining the number of days from referral to consultation. The median wait time was 59.5 days. There were no significant associations between wait time and child age (F1, 276 = 0.02, p = .89), sex (F1, 275 = 0.04, p = .85), referral question (F2, 275 = 0.60, p = .55), or insurance type (F2, 275 = 2.23, p = .11).
Lag time was calculated by determining the days between a scheduling phone call and initial appointment completion. Between May and July 2020 (i.e., during COVID-19), the median lag time was 6 days, compared with the median lag time between May and July 2019 of 14–21 days (i.e., pre-COVID-19). There were no significant associations between lag time and child age (F1, 275 = 0.06, p = .81), sex (F1, 274 = 1.83, p = .18), referral question (F2, 274 = 0.38, p = .69), or insurance type (F2, 274 = 0.24, p = .78).
Missed Appointments
Of scheduled consultations, 84% (n = 251) of families completed the appointment as scheduled. Sixteen percent of families did not attend or call to cancel the telehealth appointment. There were no differences in show rate by child age (F1, 282 = 0.80, p = .37), sex (F1, 281 = 0.02, p = .89), primary referral question (F2, 281 = 0.03, p = .97), or insurance type (F2, 281 = 2.81, p = .06). When compared with hospital system data before the COVID-19 pandemic, the telehealth appointment missed appointment rate is similar to the missed appointment rate of 16% to 20% reported by our institution from May–July 2019.
DISCUSSION
Over the last 2 years, research on the use of telehealth across health care has exploded in response to the social distancing requirements necessitated by the COVID-19 pandemic. To address a gap in the literature for children with neurodevelopmental disabilities and their families, our objective was to describe rates of caregiver agreement to use telehealth, reasons for declining, and explore demographic differences between those who agree, decline, and are lost to follow-up. In addition, we explored metrics of clinical access (i.e., wait time and missed appointments) before and during the COVID-19 pandemic. Our findings suggest that most caregivers (i.e., 92%) agree to participate in telehealth consultation services. There were no demographic differences between those who accepted telehealth, declined telehealth, or were lost to follow-up. Lag time, or time between initial scheduling call and appointment, decreased during the pandemic, but no difference in show rate was observed.
As noted previously, most caregivers (92%) agreed to participate in telehealth consultation services. This is in accordance with prior literature, which suggests that caregivers perceive telehealth consultation and evaluation as an acceptable alternative to in-person services (
Brief Report: Preliminary feasibility of the TEDI: A novel parent-administered telehealth assessment for autism spectrum disorder symptoms in the first year of life.
Journal of Autism and Developmental Disorders.2020; 50: 3432-3439
). Eleven percent of caregivers declined to participate in telehealth, with half indicating a preference for an in-person consultation. Other reasons endorsed for declining telehealth included lacking appropriate technology or other reasons (e.g., prioritizing other services, being no longer interested, receiving services elsewhere). In prior studies, caregivers have reported concerns regarding the quality of Internet access and the lack of direct interaction between the clinician and their child (
). No caregivers in this study endorsed concerns regarding insurance reimbursement. Others have previously suggested this may be a barrier expressed by clinicians (
; Kruse et al., 2018) rather than caregivers; it may be that caregivers have not considered the possibility that insurance would not cover telehealth services or that the relaxed regulations because of the public health emergency eliminated any concern. Within our clinic, we did not find basic demographic differences (i.e., age, sex, insurance type, referral concern) between those who accepted telehealth, declined telehealth, or were lost to follow-up. This suggests that telehealth may potentially be accepted by a broad proportion of the population.
Regarding wait time, patients waited approximately 60 days from receipt of referral to initial consultation. Wait time for pediatric subspecialty care has long been an area of concern. For developmental evaluations specifically, average estimated wait times have hovered around 5.4 months before the COVID-19 pandemic (
), which is well above the wait time for the current study. Our hospital system calculates median lag time, which uses the time from scheduling to appointment completion as an indicator of access. Our median lag time decreased from 14 to 21 days between May and July 2019 to just 6 days from May to July 2020. However, it should be noted that changes to the type and quantity of appointments offered in light of the COVID-19 emergency played a contributing role. More specifically, in the early months of the pandemic, initial consultation appointments were offered instead of longer, more comprehensive evaluations in our clinic. There were no differences in wait time or lag time based on demographics or referral questions.
Regarding show rates, of the children who were scheduled, nearly 84% of families arrived for their scheduled appointment. This was consistent with in-person show rates for the same months in 2019. Our findings diverge from previous literature in primary care and other subspecialties where telehealth was associated with a reduced rate of missed appointments (
). It is well-known that caregivers of children with neurodevelopmental disabilities and other complex health care needs report a high level of stress and burden related to coordination of their child's care (
The impact of parenting stress: A meta-analysis of studies comparing the experience of parenting stress in parents of children with and without autism spectrum disorder.
Journal of Autism and Developmental Disorders.2013; 43: 629-642
); one hypothesized reason, why our findings may diverge from samples within primary care or other subspecialty care, could be related to the complexity of family stressors and needs of this population. In future research, it will be important to compare show rates for in-person appointments to show rates for telehealth appointments to determine if this finding persists. There were no differences in age, sex, insurance type, or referral concern between those who arrived at their appointment and those who did not. Prior research has suggested that patients and families who use Medicaid as their primary insurance plan are more likely to miss in-person appointments because of a variety of barriers (
); it may be that the ability to attend appointments by telehealth eliminated some barriers that contribute to in-person missed appointments (e.g., transportation).
Limitations and Future Directions
There are several limitations to this study. First, the data in this study comes from the initial months of the COVID-19 pandemic. Examination of these variables over time will be important as the pandemic, and its associated impacts persist. Second, we did not specifically explore caregiver or clinician perspectives on telehealth for the current study. As a result, we cannot reflect on satisfaction with the services provided. Other studies have demonstrated a high level of satisfaction from the clinician and caregiver perspective (
). Future research should explore, in greater detail, both clinician and caregiver experiences of telehealth consultation compared to in-person services for children with neurodevelopmental disabilities. A second limitation was the availability of data on patient race and ethnicity. We obtained much of our patient race and ethnicity data from the electronic medical record, in which the accuracy and completeness of demographic data are limited.
Given that prior literature has revealed differences in access metrics based upon patient race and ethnicity, it will be critical in future research to have sufficient data on patient sociodemographic factors to ascertain whether similar patterns are present for this patient population. Given the emerging evidence that telehealth uptake may be lower for certain populations (e.g., ethnic/racial and socioeconomic minority, rural areas;
), it is plausible that children who did not arrive or respond to contact may have been facing unmeasured barriers, including poor Internet access or more pressing social determinants of health needs. We did not specifically investigate differences in the acceptability of telehealth on the basis of geographic location. Given that access to high-quality Internet is critical for the success of telehealth, future research should explore geographic areas where there are gaps in access to the Internet. It will be critical to determine ways to reach children from all backgrounds and regions, whether through traditional in-person services or innovative methods combining telehealth specialty appointments in the patient's local medical home.
CONCLUSIONS
Although evidence regarding the potential benefits of telehealth for the provision of neurodevelopmental evaluation (
) began mounting before 2020, the COVID-19 pandemic was the catalyst in creating rapid change for the deployment of this health care modality. Our findings support the overall acceptability of telehealth services in an outpatient neurodevelopmental consultation clinic population. However, notably, there continues to be a sizable percentage of families that are unable to be reached, decline telehealth, or do not arrive for their appointments. As the COVID-19 crisis subsides, efforts must be made to leverage the benefits of telehealth while continuing to innovate around remaining barriers and disparities.
References
Adepoju O.E.
Chae M.
Liaw W.
Angelocci T.
Millard P.
Matuk-Villazon O.
Transition to telemedicine and its impact on missed appointments in community-based clinics.
Corona, L., Hine, J., Nicholson, A., Stone, C., Swanson, A., Wade, J., . . . Warren, Z. (2020). TELE-ASD-PEDS: A telemedicine-based ASD consultation tool for toddlers and young children. Retrieved from: https://vkc.vumc.org/vkc/triad/tele-asd-peds.
Research Electronic Data Capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support.
Journal of Biomedical Informatics.2009; 42: 377-381
The impact of parenting stress: A meta-analysis of studies comparing the experience of parenting stress in parents of children with and without autism spectrum disorder.
Journal of Autism and Developmental Disorders.2013; 43: 629-642
Brief Report: Preliminary feasibility of the TEDI: A novel parent-administered telehealth assessment for autism spectrum disorder symptoms in the first year of life.
Journal of Autism and Developmental Disorders.2020; 50: 3432-3439
Brett Enneking, Assistant Professor of Clinical Pediatrics, Division of Developmental Medicine, Department of Pediatrics, Indiana University School of Medicine, Indianapolis, IN.
Tybytha Ryan, Assistant Professor of Clinical Pediatrics, Division of Developmental Medicine, Department of Pediatrics, Indiana University School of Medicine, Indianapolis, IN.
Qing, Tang, Biostatistician, Department of Biostatistics and Health Data Science, Indiana University School of Medicine, Indianapolis, IN.
Cristina James, Data Engineer, Division of Developmental Medicine, Department of Pediatrics, Indiana University School of Medicine, Indianapolis, IN.
Maura Rouse, Assistant Professor of Clinical Pediatrics, Division of Developmental Medicine, Department of Pediatrics, Indiana University School of Medicine, Indianapolis, IN.
Michelle Curtin, Associate Professor of Clinical Pediatrics, Department of Pediatrics, Wake Forest School of Medicine, Winston-Salem, NC.
Rebecca McNally Keehn, Assistant Professor of Clinical Pediatrics, Department of Pediatrics, Indiana University School of Medicine, Indianapolis, IN.
Article info
Publication history
Published online: September 26, 2022
Footnotes
This work was supported by funds from the Riley Children's Foundation and the Health Resources and Services Administration (T73MC00015-27-01; PI, Angela Tomlin). Rebecca McNally Keehn was supported in part supported by the Robert Helen Haddad Foundation Scholar Program through Riley Children's Foundation.
00241-3&id=gr1.jpg){kind=link}