Preliminary Examination of a New Mental Health Screener in a Pediatric Sample

Article Outline

• Abstract
• Method
  • Participants
  • Measure
  • Procedures
  • Data Analyses
• Results
• Discussion
• Acknowledgment
• References
• Biography
• Copyright

Abstract 

Introduction

Approximately 80% of children with mental health problems do not receive services. It has been recommended that mental health screening be conducted during pediatric visits (Huffman & Nichols, 2004).

Methods

The Primary Care Mental Health Screener (PCMHS) was designed to screen for DSM-IV disorders (APA, 1994) in children. The PCMHS was completed by 328 parents of 3- to 12-year-olds.

Results

The long-form showed adequate to excellent internal consistency across three age groups and eight subscales with one exception (depression in preschoolers). Next, data were used to shorten the screener without compromising internal consistency, resulting in a 32-item short-form.

Discussion

The long-form of the PCMHS is a promising mental health screener. The internal consistency of the proposed short-form should be examined with a separate sample. Additional research is needed to improve the reliability of the PCMHS for 3- to 5-year-olds and, in general, to examine the validity of this screener.

Key words: Mental health, screening, pediatrics, children, adolescents

It is estimated that 20% of children could benefit from mental health services (Mash & Dozois, 2003); however, approximately 80% of these children do not receive services (Kataoka, Zhang, & Wells, 2002). Levant  (2006) argued that mental and physical health care professionals should work together to integrate psychological and physical health care. Furthermore, he argued that patients would be more likely to use mental health services if such services were suggested by their primary care providers. Similarly, Bricker, Davis, and Squires (2004) pointed out that most children (75%) in the United States use medical/physical health services and, for this reason, medical professionals are in an excellent position to promote mental health screening, assessment, and treatment. Even the Surgeon General has argued that mental health problems be addressed in primary care settings (U.S. Public Health Service, 1999). Nonetheless, widespread screening of emotional, behavioral, and mental health issues is not occurring (Sices et al., 2003, Simonian, 2006).

It should be noted that research on psychological screening in primary care has focused on two areas: (a) screening for developmental delays and (b) screening for behavioral, emotional, and mental health problems. As such, there are different screeners for these distinct but related areas of concern. Many popular measures have been developed to screen for developmental delays (e.g., Denver II; Frankenburg, Dodds, Archer, Shapiro, & Bresnick, 1992), but they will not be discussed in detail because the focus of this article is on screening for behavioral, emotional, and mental health problems.

Several explanations exist for the lack of routine, formal screening for behavioral, emotional, and mental health problems. First, some authors have argued that primary care providers do not have mental health training (Burklow et al., 2001, Leaf et al., 2004, Simonian, 2006) and therefore do not feel comfortable conducting behavioral and/or mental health assessments (Sices et al., 2003). Second, it has been suggested that many providers do not believe they have time to conduct behavioral and/or mental health screening (Sices et al.; Simonian). Third, some researchers have argued that parents may be hesitant to discuss their child's mental health, especially without being prompted by a pediatric provider (Olfson, 1992; Simonian). However, Burklow and colleagues found that 87% of parents thought pediatric providers should ask about psychosocial issues. Thus, parents may want to discuss mental health issues but may be hesitant to do so if the pediatric care provider does not initiate the discussion. Furthermore, Levant (2006) argued that primary care providers often treat mental health problems, possibly without the benefit of knowledge about empirically supported diagnostic and treatment techniques. Nonetheless, time constraints and a lack of primary care provider training in mental health remain important obstacles to widespread behavioral, emotional, and mental health screening (Sices et al.).

Little research exists on mental health screeners in primary care. It has been proposed that screeners need to be short, cover a wide range of problems, not require specialty training, be easy to score and interpret, help pediatric providers make referrals, and have good psychometric properties (Glascoe, 2005, Huffman and Nichols, 2004, Shedler, 2000, Simonian, 2006). Moreover, there are advantages to having screeners correspond to the Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition (DSM-IV; American Psychiatric Association [APA], 1994) because providers will be able to identify which disorders need to be assessed further.

Simonian (2006) also argued that screeners should have clear-cut scores for determining when a child should be referred to a mental health professional. In addition, Simonian pointed out the need for screeners to be culturally sensitive and for normative data to be available for diverse populations. Many measures exist for assessing children's mental health, but few meet these criteria. Huffman and Nichols (2004) and Glascoe (2005) reviewed widely used measures for assessing children's mental health. Many of these measures have good psychometric properties; however, none meets all of the guidelines for use in primary care.

For example, the Child Behavior Checklist (Achenbach & Rescorla, 2000) and the Behavior Assessment System for Children (Reynolds & Kamphaus, 2004) are long (110 to 160 items), and the Behavior Assessment System for Children is not based on the DSM-IV. The Infant-Toddler Social and Emotional Assessment (Carter & Briggs-Gowan, 1993) is long (139 items) and is only for use with very young children. Although the Brief Infant-Toddler Social and Emotional Assessment (Briggs-Gowan & Carter, 2007) is much shorter, it also is for very young children only. Similarly, the Toddler Behavior Screening Inventory (Mouton-Simien, McCain, & Kelley, 1997) is brief (40 items) but is for use with young children only. The Conners Rating Scale (Conners, 1997) is not based on the DSM-IV and lacks breadth of coverage. The Eyberg Child Behavior Inventory (Eyberg, 1980) and Strengths and Difficulties Questionnaire (Bourdon, Goodman, Rae, Simpson, & Koretz, 2005) have some DSM-IV items but do not cover learning or autism-spectrum disorders. The Child Symptom Inventory (Gadow & Sprafkin, 1994) is based on the DSM-IV and has good breadth of coverage; however, it is fairly long (97 items). The Pediatric Symptom Checklist (Jellinek et al., 1988) is brief, but it is not based on the DSM-IV. The Vanderbilt (Wolraich, Lambert, & Doffing, 2003) is based on the DSM-IV, but it was designed primarily to assess attention deficit hyperactivity disorder and comorbid conditions. Thus, it covers externalizing disorders more thoroughly than internalizing disorders and does not cover autism-spectrum disorders.

The Primary Care Mental Health Screener (PCMHS) is a new mental health screening measure designed by the authors. The PCMHS long form has 69 items and takes approximately 7 to 12minutes to complete. It does not require extensive training or scoring, it is based on the DSM-IV, and it covers a range of ages and mental health problems. The current study was designed to examine the reliability of the PCMHS in a pediatric setting with 3- to 12-year-olds. Given that the PCMHS is based largely on the DSM-IV, internal consistency was expected to be strong. However, with regard to factor analyses, some of the subscales include items that cover multiple disorders (i.e., the learning disorder subscale includes math and reading items; the anxiety subscale includes generalized and separation anxiety items). Therefore, subscales that were designed to measure a single dimension of mental health were expected to have a single factor solution (such as inattention, hyperactivity, oppositionality); however, subscales that included items related to multiple dimensions were not expected to have a single factor solution (such as conduct problems, learning problems, and anxiety). Furthermore, the internal consistency results of the current study were used to create the PCMHS short form, which is a more appropriate length for use in pediatric settings but will need to be evaluated in a separate sample.

Back to Article Outline

Method 

Participants 

Participants were 328 parents and/or caregives of children ages 3 to 12 years. Additional participant demographic information is shown in Table 1.

Table 1. Participant demographics

Measure 

The PCMHS long form has 69 items and measures inattention; hyperactivity; oppositionality; anxiety; depression; and conduct, learning, and pervasive developmental problems. The distribution of items across subscales is shown in Table 2. The Flesch-Kincaid reading grade level is 8.8 (Microsoft Corporation, 2008). The inattention, hyperactivity, and oppositionality subscales directly correspond to DSM-IV symptoms for attention deficit hyperactivity disorder and oppositional defiant disorder. The conduct problems subscale consists of seven items that are based on the DSM-IV and three items that measure relational aggression (Crick & Grotpeter, 1995). Some of the more severe DSM-IV conduct disorder symptoms were not included (i.e., using a weapon; forcing sexual activity; breaking and entering; and running away). These items were not included because this is a screening instrument, and it was expected that children who exhibit these more severe behaviors also would exhibit some of the less severe items that were included.

Table 2. Contents of the Primary Care Mental Health Screener long and short forms

The learning problems subscale includes six items to screen for learning disorders and two items to screen for developmental delays. Because the DSM-IV does not provide symptom criteria that are adaptable for a checklist, these items were adapted from Willcutt, Boada, Riddle, and Pennington (2008). The anxiety subscale consists of four items that address DSM-IV generalized anxiety disorder and four items that address DSM-IV separation anxiety disorder. The depression subscale consists of six items that address DSM-IV major depressive disorder, two items that address suicidality and were adapted from Willcutt et al., and one item that measures low self-esteem. The pervasive developmental problems subscale consists of items that address DSM-IV autism and/or Asperger's disorder. Two of the eight items were adapted from Willcutt et al.

Parents were instructed to “Check the column that best describes your child in comparison to other children the same age.” Parents also were told, “Some items may not be relevant for younger children” and were informed that shaded items were optional for parents of 3- to 5-year-olds. Parents chose from five forced-choice answers (never, rarely, sometimes, often, and very often). Parents of 3- to 12-year-olds seen for well-child visits were asked to complete the screener. Screeners also were used for other types of office visits (sick visit or medication consultation) if the parent or pediatric provider had concerns about the child's mental health. First, parents completed the PCMHS long form. Next, they received a letter explaining the research study, and lastly, they completed the research consent if they chose to do so.

Pediatricians were given guidelines for clinical interpretation of the screener. First, they were given a handout describing the items that corresponded to specific symptom dimensions. Second, they were told to look for symptoms endorsed by the parent as occurring “often” or “very often.” If a parent endorsed two or more items as occurring “often” or “very often,” the pediatric providers were instructed to ask some follow-up questions, including: (a) Would you please give me examples of these behaviors? (b) Do you think this child displays these behaviors more often than would be typical for a child of the same age? (c) Do these behaviors cause problems for this child at home, at school, with family, or with peers? and (d) Has another significant adult in this child's life expressed concern about these problems? Pediatricians were instructed to use this additional information to make a clinical decision about whether the family should be referred to a mental health provider. Specifically, they were instructed to refer children who showed symptoms resulting in functional impairment. They were not instructed to refer children who were displaying two or more symptoms with no evidence of functional impairment. Thus, the decision to refer was primarily based on the pediatric provider's assessment of the degree of functional impairment and not on a strict cut-off score on the screener because the psychometric properties of this instrument and cut-off scores have yet to be established. A list of mental health providers in the community was given to the pediatric providers and all parents who completed the screener.

Procedures 

Procedures for recruiting participants and the study method were approved by the institutional review board at the university where the research was conducted, and these procedures were in compliance with the ethical standards of the APA. Parents of 328 children completed the PCMHS and provided informed consent during a visit to a private pediatric clinic in a small Southwestern town. Screeners were distributed by receptionists during well-child visits and at other times when the pediatric provider thought it would be helpful (i.e., when the provider or parent had specific concerns about the child's mental health). The measure was used as part of standard care for well-child visits for children ages 3 to 12 years. Although the psychometric properties of the screener have not been established, given that the screener was based on the DSM-IV, the providers were comfortable using it as part of standard care. Previously, they had not been using any mental health screener, and they believed that a screener based on the DSM-IV was superior to no screener at all. In addition, they were instructed to discuss level of impairment with parents before making referrals for mental health services. Thus, they reported feeling comfortable using their clinical judgment when discussing the results of the screener with parents. Screeners were collected either by a receptionist in the waiting room or by a nurse in the examination room. After parents completed the screener, they were given the option to consent to share the screener for research purposes. It is unknown what percentage of parents who completed the screener agreed to participate by signing the consent form. The office staff believed that it was too cumbersome to keep a record of this percentage for research purposes.

Data Analyses 

First, principal components analyses were conducted to examine whether subscale items produced coherent factors. Given that there is a high degree of comorbidity among the constructs represented by the subscales and that this screener includes a limited number of items from numerous sets of DSM-IV diagnostic criteria, it was not expected that the items would neatly cluster into the established diagnostic subscales if a screener-wide factor analysis was conducted. Therefore, principal components analyses were conducted separately for each group of subscale items rather than with all screener items. It was expected that items from each subscale would load onto a single factor in subscale-specific factor analyses. Second, internal consistency was calculated to further examine subscale reliability. As a measure of internal consistency, Cronbach's α was calculated. Values of .69 or lower are referred to as “inadequate,” .70 to .79 are referred to as “adequate,” .80 to .89 are referred to as “good,” and .90 or higher are referred to as “excellent” (Charter, 2003, Henson, 2001). Finally, data from the 69-item PCMHS long form were used to determine which items could be eliminated to create a PCMHS short form, as providers and office staff commented that the PCMHS was too long. To create a screener that could be completed in 3 to 6minutes rather than 7 to 12minutes, the goal was to shorten the screener by approximately 50%. Therefore, four items were identified for retention for each symptom dimension, while maintaining internal consistency at or above .80 for the total sample. Finally, principal components analyses were calculated for the short form subscales. In addition, internal consistency was calculated by sex and age for the short form subscales.

Back to Article Outline

Results 

Principal components analysis was conducted for each of the eight subscales. First, for the inattention subscale, the nine items were entered into an analysis set to extract factors with eigenvalues greater than 1.00. This analysis resulted in a single factor solution with an eigenvalue of 6.41, accounting for 71.2% of the variance. These nine items had factor loadings ranging from .79 to .87. Similarly, for the hyperactivity subscale, the nine items were entered into the analysis, and a single factor solution resulted. This factor had an eigenvalue of 6.04, accounting for 67.1% of the variance. The nine hyperactivity items had factor loadings ranging from .65 to .86. Next, for the oppositionality subscale, the eight items resulted in a single factor solution with an eigenvalue of 5.79, accounting for 72.4% of the variance. The eight oppositionality items had factor loadings ranging from .81 to .87.

Next, for the conduct problems subscale, the 10 items were entered into an analysis with the same settings as the previous principal components analysis. This analysis resulted in two factors with eigenvalues greater than 1.00. However, an inspection of the scree plot yielded only one interpretable factor. Specifically, the first factor had an eigenvalue of 5.39 and accounted for 53.9% of the variance. Although the second factor had an eigenvalue greater than 1.00, this value was much more similar in magnitude to the remaining trivial factors than it was to the first factor. Thus, the factor analysis was re-run and a single factor solution was forced. For the single-factor solution, the 10 conduct problems items had loadings ranging from .55 to .87. Similarly, for the learning problems subscale, eight items were entered into the analysis, which resulted in two factors with eigenvalues greater than 1.00. Again, an inspection of the scree plot suggested only one interpretable factor. Specifically, the first factor had an eigenvalue of 4.82 and accounted for 60.3% of the variance. Again, the second factor had an eigenvalue greater than 1.00, but this value was similar in magnitude to the remaining trivial factors. Thus, the factor analysis was re-run and a single factor solution was forced. For the single-factor solution, the eight learning problems items had loadings ranging from .55 to .88.

Next, for the anxiety subscale, the eight items were entered into the analysis, which resulted in two factors with eigenvalues greater than 1.00. This time, however, the examination of the scree plot suggested that a two-factor solution may be tenable. The first factor had an eigenvalue of 3.98 and accounted for 49.8% of the variance; the second factor had an eigenvalue of 1.34 and accounted for 16.7% of the variance. To determine whether two interpretable factors had been produced, the analysis was re-run with a varimax rotation. The varimax rotation produced a two-factor solution with six items loading on the first factor (factor loadings ranged from .52 to .82) and four items loading on the second factor (factor loadings ranged from .50 to .90). It should be noted that two of the items had strong cross-loadings. This finding is not surprising given that the first factor appeared to measure generalized anxiety disorder and the second factor appeared to measure separation anxiety disorder, and comorbidity is often found between these two disorders in children (Mash & Wolfe, 2009).

Next, for the depression subscale, the nine items were entered into a principal components analysis. This analysis resulted in two factors with eigenvalues greater than 1.00. However, an inspection of the scree plot yielded only one interpretable factor. Specifically, the first factor had an eigenvalue of 4.44 and accounted for 49.3% of the variance. Again, the second factor had an eigenvalue greater than 1.00, but this value was similar in magnitude to the remaining trivial factors. Thus, the factor analysis was re-run and a single factor solution was forced. For the single-factor solution, the eight depression items had loadings ranging from .57 to .80. Finally, for the pervasive developmental disorder (PDD) subscale, the eight items resulted in a single factor solution with an eigenvalue of 4.25, accounting for 53.1% of the variance. The eight PDD items had factor loadings ranging from .59 to .81.

Internal consistency was good to excellent for all subscales when the total sample was analyzed (Table 3). Also, internal consistency was calculated separately by sex, and α values remained good to excellent for boys and girls. Lastly, internal consistency was calculated for each age group (Table 3). Across all three age groups, α values were adequate to excellent with one exception. For preschoolers, internal consistency for the depression subscale was inadequate (α=.67).

Table 3. Coefficient α values for long and short forms of the Primary Care Mental Health Screener for the total sample and by age group

For the total sample, ns for α were 303 to 325; for preschoolers, ns for α were 122 to 135; for school-aged children, ns were 110 to 120; for pre-adolescents, ns were 70 to 73.

aExcellent internal consistency.

bGood internal consistency.

cAdequate internal consistency.

dInadequate internal consistency.

The resulting PCMHS short form has 32 items (Table 2), and internal consistency remained good to excellent for all subscales when the total sample was analyzed (Table 3). It must be emphasized that this short form was created from the current data and preliminary analyses based on the current data will be presented. The short form will need to be cross-validated on a separate sample before conclusions are drawn about its usefulness.

For the short form, principal components analysis was conducted for each of the eight subscales. First, for the inattention subscale, the four items were entered into an analysis set to extract factors with eigenvalues greater than 1.00. This analysis resulted in a single factor solution with an eigenvalue of 3.17, accounting for 79.3% of the variance. These four items had factor loadings ranging from 0.86 to 0.92. For the hyperactivity subscale, four items were entered, and this analysis resulted in a single-factor solution with an eigenvalue of 3.21, accounting for 80.2% of the variance. These four items had factor loadings ranging from 0.88 to 0.90. For the oppositionality subscale, four items were entered and resulted in a single factor solution with an eigenvalue of 3.25 accounting for 81.2% of the variance. These four items had factor loadings ranging from 0.88 to 0.92. For the conduct problems subscale, four items were entered and resulted in a single-factor solution with an eigenvalue of 2.85, accounting for 71.3% of the variance. These four items had factor loadings ranging from 0.79 to 0.89.

For the learning problems subscale, four items were entered and resulted in a single-factor solution with an eigenvalue of 3.27, accounting for 81.8% of the variance; items had factor loadings ranging from 0.79 to 0.95. For the anxiety subscale, four items were entered and resulted in a single-factor solution with an eigenvalue of 2.60, accounting for 65.0% of the variance; items had factor loadings ranging from 0.72 to 0.91. For the depression subscale, four items were entered and resulted in a single-factor solution with an eigenvalue of 2.14, accounting for 53.5% of the variance; items had factor loadings ranging from 0.64 to 0.80. For the pervasive developmental problems subscale, four items were entered and resulted in a single-factor solution with an eigenvalue of 1.88, accounting for 47.1% of the variance; items had factor loadings ranging from 0.56 to 0.79.

For the short form, internal consistency was calculated separately by sex, and the resulting α values remained adequate to excellent for boys and girls. Alpha values are limited by the number of items in a subscale; thus, internal consistency values for the short form are not expected to be as high as for the long form. Next, internal consistency was calculated separately for each age group (Table 3). For pre-adolescents, internal consistency was good to excellent for all subscales. For school-aged children, internal consistency was good to excellent for most subscales; however, internal consistency for anxiety (α=.77) and depression (α=.72) dropped to adequate. For preschoolers, internal consistency remained good to excellent for most subscales. However, for depression, internal consistency remained inadequate (α=.55), and for conduct problems (α=.63) and pervasive developmental problems (α=.64), internal consistency dropped from adequate to inadequate.

Back to Article Outline

Discussion 

This study examined internal consistency of the PCMHS long form in 3- to 12-year-olds. Although preliminary analyses suggest that the PCMHS is a promising mental health screener for 6- to 12-year-olds in pediatric settings, additional analyses (test-retest reliability, inter-rater reliability, sensitivity, specificity, concurrent validity, and predictive validity) with larger and more diverse samples are needed. Both the original PCMHS and the short form cover a wide range of mental health problems that affect children. Furthermore, the PCMHS does not require extensive training to use or interpret. The results of the current study show that the PCMHS long form has adequate to excellent internal consistency reliability across two genders, three age groups, and eight subscales with a few exceptions for preschoolers. The internal consistency of the short form, which is of a more practical length, needs to be validated with another sample.

It is notable that both forms resulted in inadequate internal consistency for measuring depression in preschoolers, and the short form resulted in inadequate internal consistency for measuring conduct problems and pervasive developmental problems in preschoolers. Thus, the utility of either the original PCMHS or the short form with preschoolers may not be acceptable. However, it is possible that internal consistency will improve with a larger sample size. Although this sample may appear reasonably large, we expect only about 20% of children to show mental health problems of any type (Mash & Dozois, 2003). Thus, we would expect about 66 children in our entire sample, and about 27 preschoolers based on our age distribution, to show significant problems. Furthermore, given the prevalence rates cited in the DSM-IV, there are likely more preschoolers with inattention, hyperactivity, or oppositionality problems than with depression, conduct problems, or pervasive developmental problems (APA, 1994). Specifically, for the conduct disorder, depression, and PDD subscales, reliability may have been poor in preschoolers because the number of symptoms endorsed was very low. In the current study, levels of symptomology were statistically significantly lower in preschoolers than in the two older age groups for seven out of eight subscales; the hyperactivity subscale was the exception. Not surprisingly, internal consistency values on seven out of eight subscales were substantially lower in preschoolers than in the older age groups; the anxiety subscale was the exception. Therefore, internal consistency may improve with a larger sample. Although it is very important for the screener to be of a practical length, it also is necessary that the screener have adequate psychometric properties. Therefore, additional research examining the reliability of particular subscales in preschoolers also is necessary before the PCMHS should be promoted for use with this age group.

Future work with the PCMHS also should focus on lowering the reading level so it can be completed by parents with less than an 8th-grade education. In addition, as mentioned previously, more research is recommended to examine inter-rater reliability, test-retest reliability, diagnostic sensitivity and specificity, and concurrent and predictive validity. These additional psychometric properties of PCMHS short and long forms should be examined for each subscale across age and sex, especially because age differences in internal consistency were found in this preliminary study. Furthermore, additional research with more diverse samples in terms of race/ethnicity also is needed. With regard to diagnostic sensitivity and specificity, it is important to ensure that the PCMHS does not result in high rates of false positives or false negatives. False negatives may result in children who need services not getting them, whereas false positives may result in unnecessary worry and expense for parents. The authors currently are conducting studies to examine the test-retest reliability, diagnostic sensitivity, and specificity of the PCMHS long and short forms, as well as reliability in an ethnic minority sample.

Whether pediatric care providers choose to use the PCMHS or another instrument, they should consider mental health screening in their practices, which will increase the likelihood that children will receive needed mental health services. Primary care health providers can likely be extremely helpful in decreasing stigma and increasing the number of children who obtain mental health evaluations and services. However, the PCMHS and other screeners that have not been adequately tested in terms of psychometric properties should be used cautiously. Therefore, although there is a call for the use of mental health screeners in pediatric settings, more research is necessary to identify instruments that have adequate psychometric properties for this purpose.

Back to Article Outline

 

We thank the following persons for their contributions: Monica Armendariz, Ashley Baldwin, Chelsea Bauer, Brett Deacon, Katie Gordon, Devin Hinds, Carrie Little, William MacLean, Narina Nuñez, Sara Penning, Brittany Romero, Douglas Scambler, Brandi Shaw, David Short, Benjamin Sigel, Karen Stockwell, Maureen Sullivan, Laticia Wiggins, and Benjamin Wilkowski.

Back to Article Outline

References 

1. Achenbach TM, Rescorla LA. Manual for the ASEBA. Burlington, VT: University of Vermont; 2000;
   • View In Article
2. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
   • View In Article
3. Bourdon K, Goodman R, Rae D, Simpson G, Koretz D. Strengths and Difficulties Questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry. 2005;44:557–564
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (95 KB)
   • CrossRef
4. Bricker D, Davis MS, Squires J. Mental health screening in young children. Infants and Young Children. 2004;17:129–144
   • View In Article
5. Briggs-Gowan MJ, Carter AS. Applying the infant-toddler social and emotional assessment (ITSEA) and Brief-ITSEA in early intervention. Infant Mental Health Journal. 2007;28:564–583
   • View In Article
6. Burklow K, Vaughn L, Valerius K, Schultz J. Parental expectations regarding discussions on psychological topics during pediatric visits. Clinical Pediatrics. 2001;40:555–562
   • View In Article
   • MEDLINE
   • CrossRef
7. Carter A, Briggs-Gowan M. The infant-toddler social and emotional assessment. New Haven: Yale University; 1993;
   • View In Article
8. Charter RA. A breakdown of reliability coefficients by test type, reliability method, and implications of low reliability. Journal of General Psychology. 2003;130:290–304
   • View In Article
9. Conners CK. Conners' rating scales—Revised. New York: Multi-Health Systems; 1997;
   • View In Article
10. Crick NR, Grotpeter JK. Relational aggression, gender, and social-psychological adjustment. Child Development. 1995;66:710–722
    • View In Article
    • MEDLINE
    • CrossRef
11. Eyberg S. Eyberg Child Behavior Inventory. Journal of Clinical Child Psychology. 1980;54:587–599
    • View In Article
12. Frankenburg W, Dodds J, Archer P, Shapiro H, Bresnick B. Denver-II. Pediatrics. 1992;89:91–97
    • View In Article
    • MEDLINE
13. Gadow KD, Sprafkin J. Child symptom inventories. Stony Brook, NY: Checkmate Plus; 1994;
    • View In Article
14. Glascoe FP. Screening for developmental and behavioral problems. Mental Retardation and Developmental Disabilities Research Reviews. 2005;11:173–179
    • View In Article
    • MEDLINE
    • CrossRef
15. Henson RK. Understanding internal consistency reliability estimates. Measurement and Evaluation in Counseling and Development. 2001;34:177–189
    • View In Article
16. Huffman LC, Nichols M. Early detection of young children's mental health problems in primary care settings. In:  DelCarmen-Wiggins R,  Carter A editor. Handbook of infant, toddler, and preschool mental health assessment. New York: Oxford; 2004;p. 467–489
    • View In Article
17. Jellinek MS, Murphy JM, Robinson J, Feins A, Lamb S, Fenton T. Pediatric symptom checklist. Pediatrics. 1988;112:201–209
    • View In Article
18. Kataoka SH, Zhang L, Wells KB. Unmet need for mental health care among U.S. children. American Journal of Psychiatry. 2002;159:1548–1555
    • View In Article
19. Leaf P, Owens P, Leventhal J, Forsyth B, Vaden-Kiernan M, Epstein L, et al. Pediatricians' training and identification and management of psychosocial problems. Clinical Pediatrics. 2004;43:355–365
    • View In Article
    • MEDLINE
    • CrossRef
20. Levant RF. Making psychology a household word. American Psychologist. 2006;61:383–395
    • View In Article
21. Mash EJ, Dozois DJ. Child psychopathology. In:  Mash EJ,  Barkley RA editor. Child psychopathology (2nd ed., pp. 3–71). New York: Guilford; 2003;
    • View In Article
22. Mash EJ, Wolfe DA. Abnormal child psychology (4th ed.). New York: Wadsworth; 2009;
    • View In Article
23. Microsoft Corporation. (2008). Understanding readability scores. Retrieved April 29, 2008, from http://office.microsoft.com/en-us/word/HP101485061033.aspx#2
    • View In Article
24. Mouton-Simien P, McCain AP, Kelley ML. The development of the Toddler Behavior Screening Inventory. Journal of Abnormal Child Psychology. 1997;25:59–64
    • View In Article
    • MEDLINE
    • CrossRef
25. Olfson M. Diagnosing mental disorder in office-based pediatric practice. Developmental and Behavioral Pediatrics. 1992;13:363–365
    • View In Article
26. Reynolds C, Kamphaus R. Behavior Assessment System for Children (BASC-2). Circle Pines, MN: AGS; 2004;
    • View In Article
27. Shedler J. The Shedler Quick PsychDiagnostics Panel. In:  Maruish ME editors. Handbook of psychological assessment in primary care settings. NJ: Erlbaum; 2000;p. 277–296
    • View In Article
28. Sices L, Feudtner C, McLaughlin J, Drotar D, Williams M. How do primary care physicians identify young children with developmental delays?. Developmental and Behavioral Pediatrics. 2003;24:409–417
    • View In Article
29. Simonian SJ. Screening and identification in pediatric primary care. Behavior Modification. 2006;30:114–131
    • View In Article
    • MEDLINE
    • CrossRef
30. U.S. Public Health Service . Mental health: A report of the Surgeon General. Rockville, MD: Department of Health and Human Services; 1999;
    • View In Article
31. Willcutt EG, Boada R, Riddle MW, Pennington BF. A parent-report screening questionnaire for learning difficulties in children. Manuscript in preparation; 2008;
    • View In Article
32. Wolraich M, Lambert W, Doffing M. Psychometric properties of the Vanderbilt ADHD Diagnostic Parent Rating Scale in a referred population. Journal of Pediatric Psychology. 2003;28:559–568
    • View In Article
    • MEDLINE
    • CrossRef