parent report case study

CASE REPORT article

Case report: a case study significance of the reflective parenting for the child development.

• Department of Psychology, Plovdiv University “Paisii Hilendarski,” Plovdiv, Bulgaria

There are studies that connect the “child” in the past with the “parent” in the present through the prism of high levels of stress, guilt, anxiety. This raises the question of the experiences and internal work patterns formed in childhood and developed through parenthood at a later stage. The article (case study) presents the quality of parental capacity of a family raising a child with an autism spectrum. The abilities of parents (the emphasis is on the mother) to recognize and differentiate the mental states of their non-verbal child are discussed. An analysis of the parental representations for the child and the parent–child relationship is developed. The parameters of reflective parenting are measured. The methodology provides good opportunities for identifying deficits in two aspects: parenting and the functioning of the child itself. Without their establishment, therapy could not have a clear perspective. An integrative approach for psychological support of the child and his family is presented: psychological work with the child on the main areas of functioning, in parallel with the therapy conducted with the parents and the mother, as the main caregiver. The changes for the described period are indicated, which are related to the improvement of the parental capacity in the mother and the progress in the therapy in the child. A prognosis for ongoing therapy is given, as well as topics that have arisen in the process of diagnostic procedures.

Introduction

Attachment theory focuses on parent–child attachment and the effects this relationship has on the child's personality, interpersonal skills, and its capacity to form healthy relationships with adults. According to Bowlby (1969) , parents who are approachable and responsive allow their child to develop a sense of security, thus creating a sound basis for it to learn about the world. The capability of parents to verbalize the feelings and experiences of their child through conversations, reading stories or fairy tales, commenting on everyday situations develops the skills of mentalization in the child ( Ханчева, 2019 ).

In mature age, the ability to mentalize depends on the emotional load of the interpersonal situation. Optimal mentalization implies integration of cognitive knowledge with insights into the emotional world, which allows a man to see more clearly and achieve “emotional knowledge” ( Allen and Fonagy, 2006 ).

The processes of mentalization can be influenced by the “heritage” that is passed down through generations. In their life experience, individuals operate and make their choices not being aware that they repeat the history of their ancestors. In part, these complex relationships can be seen, felt, or anticipated. They are experienced as elusive, insensible, unnamed, or secret, and may leave traumatic traces ( Kellermann, 2001 ).

Tisseron (2011) , associates the process of transmission of traumatic experience from generation to generation with three types of symbolization of experience: affective/sensory/motor, figurative, and verbal. If the event is symbolized in just one of the modalities, the results are associated with violation of mental life. The result becomes a distortion of the parent–child relationships, of their functioning.

The main psychopathological mechanisms that are activated in the transmission of mental content between individuals from generation to generation are associated with the identification and the projective identity. In this case of transmission through generations, insensible patterns, conflicts, scenarios and roles, ideals, and perceptions of the object are identified.

Children of severely traumatic parents reproduce scenes that their parents went through, trying to understand their pain, and at the same time establish a connection with them. They maintain family ties through the integration of parenting experiences. In the meantime, the parent seeks to teach his/her child survival strategies in situations of future persecution, thus passing on his/her traumatic experience ( Baranowsky et al., 1998 ).

Wilgowicz (1999) , introduces the term “vampire complex” describing the impact of unexpressed and insensible experiences passed down from generation to generation. These traumatic experiences form the unconscious connection between the generations which interferes the natural course of the processes of separation and individuation. This complex is associated with experience of the child who in its development turns out to be “locked” in the prison of the parental traumatic experience being neither alive, nor dead, or in other words, unborn.

Krystal (1978) , describes the affective blindness of the principal caregiver as a characteristic of unprocessed traumatic experience ( Den Velde, 1998 ; Коростелева et al., 2017 ). It is associated with incomplete integration of the somatic Self into the Self.

Ammon (2000) , Hirsch (1994) describe in this context the “psychosomatic mother whose behavior is characterized by a lack of understanding of boundaries, intrusiveness, alexithymia, excessive concern for the physical functioning of her child, and at the same time “blind” to its psychic experiences.” Hope et al. (2019) report that maternal depression and complaints of psychological distress are associated with an increased risk of trauma and hospitalization for the age 3–11 years, with the highest being in the period 3–5 years. In another study, Baker et al. (2017) , reported an increased risk of burns, poisoning, and fractures in children aged 0–4 years raised by depressed mothers and/or such found in an anxious episode. Postpartum depression in the mother presupposes a high risk of burns, fractures, poisoning ( Nevriana et al., 2020 ).

The relationship between parental attitudes and child development is influenced by unconscious dynamics of the intrapsychic world of mother and father ( Tagareva, 2019 ). The ability of parents for reflexion and metacognitive monitoring allows them to recognize and regulate, to modulate, to turn into a symbolic (verbal) form the states they observe in their child. This gives an opportunity to comprehend and return in an understandable form to the child interpretation of its state based on understanding and empathy. If this capacity fails, the parent cannot give an adequate and meaningful interpretation of what is happening, because he/she himself/herself gets lost and confused in his/her own (threatening his/her integrity) experiences, and strong, meaningless, overwhelming emotions. The consequences of the lack of a “secure base” in the face of the caregiver may be associated with: low self-esteem, behavior of decompensation under stress, inability to develop and maintain friendships, trust and intimacy, pessimism toward themselves, family, society ( Matanova, 2015 ). The low level of reflexion on the trauma and the unaddressed traumatic experience as the mother's internal position, affect, and are a risk factor for, psychopathology later in the development.

In addition, parenting skills can be further tested when raising a child with Autism Spectrum Disorders in the family. Therapy for this nosology needs to include both psychological work with the child and support for the parents, especially for the mother, who in most cases limits her social roles and devotes herself only to parenthood. This is a serious argument to seek and optimize approaches in clinical practice to support the family environment in which children with neurodevelopmental disorders are raised.

Materials and Methods

This article is designed to present a case of a family with a child diagnosed with Autism Spectrum Disorder, where the non-integrated individual traumatic experience in the mother (N.) affects the quality of her reflective parenting.

The analysis aims to display the status of individual functioning and skills for reflective parenting, as well as the effectiveness of psychological intervention to revive and optimize the relationship mother-child. Although the functioning of the mother is the focus of the present study, an analysis of parenting and the father has also been applied.

The study is a pilot one and marks the start of a project lasting over time.

Diagnostic tools have been used for:

- Assessment of the development and functioning of the child according to the methodology of Matanova et al. ( Matanova and Todorova, 2013 ). The methodology includes research of cognitive, linguistic, social, emotional, and motor sphere of functioning. Based on the identified deficits, it is possible to arrange a therapeutic plan for the child.

- Self-assessment scales for the study of the quality of the parental relationship and the formed internal work patterns (of affection and romantic relationship) of N. with her parents:

° The Parental Reflective Functioning Questionnaire (PRFQ) by Luyten et al. (2017a , b ). The PRF assessment screening tool provides additional evidence of the complexity and multidimensionality of the PRF ( Luyten et al., 2009 ). It contains 18 items intended mainly for use in the study of PRF of parents with children aged 0–5 years. Three different aspects of PRF are evaluated on a 7-point Likert scale. Based on validated factor analysis, the authors identified three theoretically consistent and clinically significant factors, each of which included six items: (1) prementalization modes (PM), (2) certainty about mental states (CMS), (3) interest and curiosity about mental states (IC).

° Assessment of emotional bonding in the parent–child relationship (PBI) Gordon Parker ( Parker, 1979 ; Parker et al., 1979 ). The questionnaire consists of two scales which measure the variables “Care” and “Overcare” or “Control” by evaluating basic parenting styles through the prism of children's perception. It consists of two identical questionnaires of 25 items, one for each parent.

- Family sociogram to report its representation in the current family.

° Version of Eidemiller and Cheremisin ( Eidemiller et al., 2007 ). It is a drawing projective technique exploring several aspects: identify the position of the subject in the system of interpersonal relationships; determine the nature of communication in the family (direct or indirect). Dimensions: Number of family members who fall into the very circle; Size of the circles which mark the members; Disposition of circles (members) relative to each other (location); Distance between circles (members).

The case under study includes: demographic data of the family, anamnesis of the child (data obtained from psychological and medical research), prescribed therapy and progress, “The Time Line” ( Stanton, 1992 )—technique to retrieve significant events from the mother's history during the main stages of her development, located on the “axis of time,” data obtained from her psychological research—hers and her husband's.

N. is married with one child at 2.6 years, with suspected Autism Spectrum Disorder.

Demographic Data at First Visit

Mother (N.)—age: 36 years, education: higher, occupation: technologist.

Father (K.)—age: 39 years, education: higher, occupation: technologist.

Now, the mother is taking care of her child. Only her husband works. They live alone in a small town. The child is separated in his own room.

The child—bears his father's first name. According to parents: does not speak, does not eat independently—“He opens his mouth a little,” walks on tiptoe, does not play with other children, does not obey to commands, gets tired easily. The child attends the nursery until noon (on the recommendation of the director of the institution: “He does not eat”) and the Municipal Center for Personal Development. A social pedagogue works with him.

Data for Assessment of the Child's Development

The child was carried to full-term, born from a second, pathological pregnancy of the mother, laid in bed to avoid miscarriage in the first months. He had a protracted jaundice, which passed after a year and a half. He was not breastfed.

After a consultation with a psychologist, dysfunction was found in the following areas: Sensory: the child does not hold pelvic reservoirs, shows behavior of sensory hunger—needs intensely sensory stimuli; Motor development: with evidence of late walking, the child steps on toes; Cognitive processes: the child has not yet formed a body schema, he tends to suck the thumbs of his lower limbs; he still explores the objective world through oral modality; passivity regarding the choice of a toy if it is not in his filed vision; he does not play with his toys as intended; Emotional and social functioning: he is easily separated from the adult; the emotional expression is poorly differentiated and is played through the body by waving hands; lack of social interest; interaction is possible after prolonged sensory stimulation. Language development: he vocalizes; does not respond to his name.

During the study, the child is calm, passive. When coming into interaction, he retains his interest in the adult, but without any initiative to develop it further.

Electroencephalography was performed, in awake state and with open eyes, which displayed mixed main activity: of diffuse beta waves, and tetha waves 4.5–5 Hz, in the anterior areas: sporadically slower waves 3–4 Hz.

The child was prescribed a therapy with psychologist with live setting twice a week. The therapy with the parents was once a week. It started online prior to the beginning of the therapy with the child due to COVID-19 quarantine. Twenty sessions were held with the child, i.e., work continued for 5 weeks (with setting twice a week). The therapy includes psychological work with the child in the main areas of functioning, established as therapeutic lines of the conducted diagnostics. Ten sessions were held with the parents and the mother. Two of the sessions were held with the parents. The following were studied: their functioning through the different subsystems: marital, parental, child–parental; difficulties in raising a child with an autism spectrum. It was found that the family system organized its resource for therapy only for the child. They realized that their well-being was important for their child's development. The marital subsystem was in the background. A session was held with the father, in which his role as the Third Significant in the child's life was discussed. Seven sessions were held with the mother. In them was unfolded her personal story through early experience, child–parent relationship, main topics of growing up, intimacy, parent–child relationship with her child. The therapy is going on.

Progress of the Therapy With the Child

Decrease of sensory hunger, no tactile simulation is required to activate the child to study the objective reality; General motor skills: reduced toe walking, except in moments of agitation, he walks on a full step on a sensory path. The child jumps on tiptoe, climbing stairs is easier than getting down; Fine motor skills: improved grip (small toys, sticks, without clenching them in the fist); Cognition: recognizes himself in the mirror, experiments on dropping toys (primary circular reactions). Still uses oral inspection of some toys, beginnings of a play by designation (zone of proximal development). The active choice of toys is in progress, he explores freely the specialist room. Object constancy is formed, he seeks an object which he has played with. Lively, interesting. Emotional development: he expresses his joy by shouting and laughing, rejoices when imitated. Expresses anger. Attempts to manipulate by imitating crying. Language development: sporadically pronounces syllables, still does not respond to his name; Peculiarities: likes objects with small holes and pays lasting attention to them. He enters the oral-sadistic stage, bites toys, and gnaws some of them. Learned helplessness.

Data From Performed Psychological Studies

Child–parent relationships and internal work patterns for oneself and for the other (pbi).

The results of the self-assessment questionnaire on emotional closeness in the parent–child relationship with the mother indicate:

With reference to the relations with her mother: high results along the dimension “Overcare/Control” (24 points) and low results along the dimension “Care/Concern” (22 points). From these results it is evident that the mother in childhood is represented as emotionally cold, indifferent, and careless, and at the same time imposing control, intrusiveness, and excessive contact, infantilizing, and hindering the autonomy of N. as a child.

With reference to the relations with her father: high results along both dimensions “Overcare/Control” (32 points) and “Care/Concern” (25 points) what relates to a representation of the father's character as emotionally restrained in his behavior, but at the same time controlling, intrusive, and in attitude which is highly infantilizing and hindering the autonomy of N.

The model of adult attachment, proposed by Bartholomew and Horowitz (1991) , related through the Parker quadrant, for the emotional closeness of a child–parent shows that N. has an active negative internal work pattern for herself along the dimension of “anxiety” and is associated with vulnerability to separation, rejection, or insufficient love. The work pattern of the other is negative, associated with fear of intimacy and social avoidance, i.e., along the dimension of “avoidance.” The attachment style corresponds to style B avoidant, subcategory cowardly avoidant.

In her husband, the internal work pattern is ambivalent. The mother's character from childhood is represented as emotionally restrained and controlling, while the father's is emotionally indifferent, however encouraging autonomy.

Family Sociogram

As a child, she presented inadequate, low self-esteem, and anxiety, an experience of emotional rejection and isolation. The father was the most significant figure, he was more emotionally close. This is also observed in her relations with the maternal grandmother. The size and thickening of the circle, which it is represented with, shows high levels of intrapersonal neuroticism. There are too many figures in the circle: apart from her four-member family, mother, father, brother, and she, it also includes her maternal grandmother and her uncle, the brother of her mother, as the division is in two camps on the basis of proximity-distance: her mother, uncle, and brother are found at one end of the circle, and the other end is occupied by her, her father and grandmother.

As an adult, prior to the birth of her child, her mother was also included in the circle which is associated with a tendency to unsatisfied needs from her.

After the birth of the child, the hierarchy is maintained and there is enough space between the members of the family now.

Through the life cycle of the family and the separation/individuation, this crisis must be lived through and integrated as a new experience. The stages show that in her childhood N. did not have a sound family model, the boundaries between the parent family and the maternal family are permeable. The above configuration could be interpreted as the presence of triangulations in the family system, and as well as intergenerational ones.

Within the romantic couple, in the period of the dyad, N. presents herself and her husband in a line, as the lower part of the test field includes the figure of her mother depicted by a smaller circle. This could be interpreted with the still insufficient density of the family boundaries. Establishing family boundaries (internal and external) is an important task at this stage of family life, as well as creating an optimal balance of proximity and distance; distribution of the roles in the family; establishing the hierarchy; negotiating family rules; coordination of future life plans, as well as joint understanding and acceptance.

It is also confirmed by the results of the interpretation of the family sociogram with the father as well. As a child he presented himself with inadequate, low self-assessment, he was hierarchically placed next to the mother's figure. Prior to the birth of the child, he presented unsatisfied needs from his parental family: no separation, the boundaries between own and native family are permeable. In the present one, the experience in the reality of what is happening is available. There is no differentiation between the relationships, and dissatisfaction with them is present. The child is put in the place of unsolved contradictions.

Reflective Parenting PRFQ

In all three dimensions, the results show values above the average as IC (“interest and curiosity about mental states”) is leading-−85.5%. It is associated with intrusive hypermentalization, i.e., she is difficult to regulate and interpret her own mental states when faced with her unregulated, difficult child. As a sequence, an inadequate reaction in response to his affective signals by the mother is provoked, as well as the presence of low levels distress tolerance. In hypermentalization as a process, there is a tendency to understand or explain mental dynamics based on complex logical constructs, sometimes abstract, notional, and without pragmatic benefit. Its extreme forms are characterized by autistic, groundless fantasizing.

The possibilities for reflective parenting with the father show increased trends in the dimensions of IC (“interest and curiosity about mental states”) and CMS, which is associated with enhanced hypermentalization, as in the mother, in the cases when she does not recognize the vague mental states of her child, however, here is also a desire to understand.

In her story N. unfolds a picture of the transmission of a traumatic experience of rejection/avoidance. The experience of emotional neglect has formed a negative notion of the Self. Through her anger, she repeats the model of her mother, not realizing that her own model is possible.

N. demonstrates a personal style in which fear and anxiety constitute a centrally organizing dimension. Reported phobias are associated with behaviors of shyness, restraint, aptitude for low self-esteem, indecisiveness, uselessness, and emotional inhibition. It is difficult for her to identify anxious thoughts, as well as to connect them with their triggers from reality, to master them and to allow a “decentralized” point of view on anxious situations, what might be the birth and upbringing of a child with arrested development. Avoidance behavior is associated with a remarkably high level of distress and a low level of long-term adaptation. ( Mikulincer and Shaver, 2012 ; Lingiardi and McWilliams, 2017 ). In cognitive theory, this feature (functioning through fear and avoidance) is considered an excellent example of an early maladaptive self-assessment scheme. The theory of mentalization conceptualizes this as an implicit (automatic) mentalizing deficit. In addition, there are difficulties in understanding the mind of others ( Dimaggio et al., 2007 ; Lampe and Malhi, 2018 ). Another major deficit of mentalization is their weak affective consciousness ( Steinmair et al., 2020 ).

Mother–Child Relationship

The relationship with her child is not objective. There is no construct to include references to the related problems outlined in her child. N. includes projective identification against guild as a protective mechanism related to her wishes for the child's future. The relationship with her child is idealized, in her aspiration and strong desire for love, characteristic of her personal structure. In this case, the child serves the mother's deficits and is not perceived objectively. The projection also supports this structure in her fear of rejection. She is parenting by satisfying the child's physical needs without giving the father the opportunity to be introduced to the child's mental life. And, although the projection is central to the father, in describing the relationship with the child, their shared experiences are related to “curiosity,” “play.” The mother's fear of loss, of rejection is the result of the unprocessed mourning. It could be also thought of splitting through the non-integrated image of the early figure of attachment. Presently, she is still demonized, and the father is idealized.

In the described period the child's study of objective reality is activated. recognizable in a mirror. Demonstrates the beginning of a game as intended, expresses joy in interaction, anger. Attempts to manipulate through imitation.

Parent Couple

The possibilities for reflective parenting in both parents are associated with increased hypermentalization, and the father has a desire to understand the mental states of the child.

Married Couple

N.'s internal working models are of a cowardly avoidant style (her husband's internal working model is ambivalent). The level of adherence to therapy is low, a high level of symptom reporting, and a low level of basic confidence. Those who have a negative BPM for themselves and for the other both want and fear of intimacy in the couple. This also presupposes the future occurrence of crisis in N. married couple.

Family System

In families such as the above described, raising a child who is unable to express their own needs in a conventional way, unresolved conflicts from the beginning of their life cycle, can escalate and lead to marital dissatisfaction and dysfunction throughout the family system.

The presented integrative model of psychological support in a family raising a child with an autistic spectrum outlines a picture of improvement in two lines: in the child and in the child–parent relationship. In mother, the process of disidentification, the formation of the transmission of the object, the separation of what has been transmitted to it, allows the history of the past to be restored, therefore gives more freedom to the individual in the shaping of the individuality. Currently, the inserted traumas, even if not one's own, in the subjective experience of conflicts and fantasies, allow to integrate this experience and to turn it from destructive to structuring.

If the traumatic event is mentally processed, symbolized, and inserted in the individual memory as an experience, it receives the status of the past, of memory. It is passed on to generations not only as the content of traumatic experience but also the aptitude of its mental processing and coping with it, which affects the individual development of the child.

N.'s feedback on the therapy so far: “He showed it to us, but I, my fault, my mistake, was that I did not see it.” She finds that now is more observant.

Data Availability Statement

The datasets generated for this article are not readily available because personal data. Requests to access the datasets should be directed to Zlatomira Kostova.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin. Written informed consent was obtained from the individual(s), and minor(s)' legal guardian/next of kin, for the publication of any potentially identifiable images or data included in this article.

Author's Note

The article presents a research perspective on the possibilities of parental capacity, through the integration of different approaches to understanding human suffering in clinical psychology.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Коростелева, И. С., Ульник, Х., Кудрявцева, A. В., and Pатнер, Е. A. (2017). Трансгенерационная передача: роль трансмиссионного объекта в формировании психосоматического симптома. Журнал практической Психологии Психоанализа . Korosteleva, I. S., Ulnik, H., Kudryavtseva, A. V., and Patner, E. A. (2017). Transgenerative transmission: the role of transmission object in the formation of a psychosomatic symptom. J. Pract. Psychol. Psychoanal .

Google Scholar

Ханчева, К. (2019). Mентализация и ранни етапи на социо-емоционално развитие. Университетско издателство. Климент Охридски . Hancheva, C. (2019). Mentalization and Early Stages of Socio-Emotional Development . University press: Kliment Ohridski.

Allen, J. G., and Fonagy, P., (eds.). (2006). The Handbook of Mentalization-Based Treatment. Hoboken, NJ: John Wiley and Sons. doi: 10.1002/9780470712986

CrossRef Full Text | Google Scholar

Ammon, G. (2000). Psychosomatic Therapy . Saint Petersburg, FL: Speech.

Baker, R., Kendrick, D., Tata, L. J., and Orton, E. (2017). Association between maternal depression and anxiety episodes and rates of childhood injuries: a cohort study from England. Inj. Prev . 23, 396–402. doi: 10.1136/injuryprev-2016-042294

PubMed Abstract | CrossRef Full Text | Google Scholar

Baranowsky, A. B., Young, M., Johnson-Douglas, S., Williams-Keeler, L., and McCarrey, M. (1998). PTSD transmission: a review of secondary traumatization in Holocaust survivor families. Canad. Psychol . 39:247–256. doi: 10.1037/h0086816

Bartholomew, K., and Horowitz, L. M. (1991). Attachment styles among young adults: a test of a four-category model. J. Pers. Soc. Psychol . 61, 226–244. doi: 10.1037/0022-3514.61.2.226

Bowlby, J. (1969). Attachment and Loss . v. 3, Vol. 1. New York, NY: Basic Books.

Den Velde, W. O. (1998). “Children of Dutch war sailors and civilian resistance veterans,” in International Handbook of Multigenerational Legacies of Trauma , ed Y. Danieli (Boston, MA: Springer), 147–161. doi: 10.1007/978-1-4757-5567-1_10

Dimaggio, G., Semerari, A., Carcione, A., Nicolo, G., and Procacci, M. (2007). Psychotherapy of Personality Disorders: Metacognition, States of Mind and Interpersonal Cycles . London: Routledge. doi: 10.4324/9780203939536

Eidemiller, E. G., Alexandrova, N. V., and Justickis, V. (2007). Family Psychotherapy . Saint Petersburg, FL: Speech.

Hirsch, M. (1994). The body as a transitional object. Psychother. Psychosom . 62, 78–81. doi: 10.1159/000288907

Hope, S., Pearce, A., Chittleborough, C., Deighton, J., Maika, A., Micali, N., et al. (2019). Temporal effects of maternal psychological distress on child mental health problems at ages 3, 5, 7 and 11: analysis from the UK Millennium Cohort Study. Psychol. Med . 49, 664–674. doi: 10.1017/S0033291718001368

Kellermann, N. P. (2001). Transmission of Holocaust trauma-an integrative view. Psychiatry 64, 256–267. doi: 10.1521/psyc.64.3.256.18464

Krystal, H. (1978). Trauma and affects. Psychoanal. Study Child . 33:81–116. doi: 10.1080/00797308.1978.11822973

Lampe, L., and Malhi, G. S. (2018). Avoidant personality disorder: current insights. Psychol. Res. Behav. Manag . 11, 55–66. doi: 10.2147/PRBM.S121073

Lingiardi, V., and McWilliams, N., (eds.). (2017). Psychodynamic Diagnostic Manual: PDM-2 . New York, NY: Guilford Publications. doi: 10.4324/9780429447129-11

Luyten, P., Fonagy, P., Mayes, L., and Van Houdenhove, B. (2009). Mentalization as a multidimensional concept. Manuscript submitted for publication.

Luyten, P., Mayes, L. C., Nijssens, L., and Fonagy, P. (2017a). The parental reflective functioning questionnaire: development and preliminary validation. PLoS ONE 12:e0176218. doi: 10.1371/journal.pone.0176218

Luyten, P., Nijssens, L., Fonagy, P., and Mayes, L. C. (2017b). Parental reflective functioning: Theory, research, and clinical applications. Psychoanal. Study Child . 70, 174–199. doi: 10.1080/00797308.2016.1277901

Matanova, V. (2015). Attachment: There and Then, Here and Now . Varna: Steno.

Matanova, V., and Todorova, E. (2013). Guide for Applying a Methodology for Assessing the Educational Needs of Children and Students . Institute of Mental Health and Development.

Mikulincer, M., and Shaver, P. R. (2012). An attachment perspective on psychopathology. World Psychiatry 11, 11–15. doi: 10.1016/j.wpsyc.2012.01.003

Nevriana, A., Pierce, M., Dalman, C., Wicks, S., Hasselberg, M., Hope, H., et al. (2020). Association between maternal and paternal mental illness and risk of injuries in children and adolescents: nationwide register based cohort study in Sweden. BMJ 369:m853. doi: 10.1136/bmj.m853

Parker, G. (1979). Parental characteristics in relation to depressive disorders. Br. J. Psychiatry 134, 138–147. doi: 10.1192/bjp.134.2.138

Parker, G., Tupling, H., and Brown, L. B. (1979). A parental bonding instrument. Br. J. Med. Psychol . 52, 1–10. doi: 10.1111/j.2044-8341.1979.tb02487.x

Stanton, M. D. (1992). The time line and the “Why now?” Question: a technique and rationale for therapy, training, organization consultations and research. J. Marit. Fam. Ther . 18, 331–343. doi: 10.1111/j.1752-0606.1992.tb00947.x

Steinmair, D., Richter, F., and Loeffler-Stastka, H. (2020). Relationship between mentalizing and working conditions in health care. Int. J. Environ. Res. Public Health . 17:2420. doi: 10.3390/ijerph17072420

Tagareva, K. (2019). “Psychological readiness for motherhood in pregnant women in different social environments,” in Proceedings of the Interdisciplinary Scientific Conference of the Faculty of Pedagogy at Plovdiv University: “Man and Global Society.” (Plovdiv).

Tisseron, S. (2011). Secrets de Famille Mode D'emploi. Paris: Marabout. doi: 10.3917/puf.tisse.2011.02

CrossRef Full Text

Wilgowicz, P. (1999). Listening psychoanalytically to the Shoah half a century on. Int. J. Psychoanal . 80, 429–438. doi: 10.1516/0020757991598765

Keywords: traumatic experiences, emotional bonding, autistic spectrum disorder, family system, reflective parenting

Citation: Kostova Z (2021) Case Report: A Case Study Significance of the Reflective Parenting for the Child Development. Front. Psychol. 12:724996. doi: 10.3389/fpsyg.2021.724996

Received: 14 June 2021; Accepted: 26 July 2021; Published: 17 August 2021.

Reviewed by:

Copyright © 2021 Kostova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Zlatomira Kostova, z_kostova@uni-plovdiv.bg

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

9. Case Studies

Case studies #1-5 parent/guardian interactions, case study #1.

You receive this email from a parent:

“Dear Teacher,

I am very worried about my son starting kindergarten and getting on the bus. When we go to the store, he doesn’t stay with me and sometimes wanders off. We think he has a behavior disability, but no one has helped him. I heard the county has resources, but we haven’t found any yet. Could you walk him to the bus in the afternoons and make sure he gets on it and stays on?

Case Study #2

A parent comes to have lunch with their son, a typical practice at your elementary school. This student is on your IEP caseload and you know the parents well. At the conclusion of lunch, you walk by and say hello to the parent and conversationally ask “how are you?” The parent becomes visibly upset and begins to tell you that her son is physically acting out at home and hitting her. She begins crying. You haven’t seen this behavior from the child at school and you are shocked to hear this report from the parent.

Case Study #3

You co-teach 1st grade with a general education teacher who receives this email from a parent:

I am really worried about Jimmy’s reading. He seems really far behind his sisters when they were his age. He barely knows his sight words and forgets them all the time. I think there might be a problem. Can we schedule a conference?”

The co-teacher forwards you the email, asking if you can help her with a response. From your evaluations and observations, Jimmy is reading on grade level and demonstrates age typical academic achievement. You have not seen signs that Jimmy is behind his peers.

Case Study #4

You are a 9th grade Science teacher. You are sitting in a parent/ teacher conference with the guardian of a child in your class. You are discussing the student’s distractibility during independent work. The guardian of the student says, “You see a lot of kids, do you think my son has ADHD? Should he be on medication? Would he benefit from special education?”

Case Study #5

You are a 10th grade teacher. You send an email to a parent about their child, Josie, who qualified for special education services in 1st grade. Josie’s eligibility records indicate she was identified as having an intellectual disability. Late last week, Josie began pushing other students with her body while transitioning in the hallways. The behavior appears to be escalating. Shortly after you send your email, you receive this reply: “No thank you.” You are concerned that a potential language barrier exists, but when you consult the school records, they indicate that the parent has not asked for a translator for any meeting to date. Josie’s former teachers have expressed they had no difficulty communicating with the parent, even though the parent’s first and primary language is Spanish. Your colleagues believe this is an avoidance tactic so the parent doesn’t have to address Josie’s behaviors.

Case Studies #6-10 Collaboration with Professional Colleagues

Case study #6.

You are the special education teacher assigned to provide collaborative push-in support in a general education setting. You do not have common planning time with the general education teacher. Each day when you arrive in class, you learn about the topic of instruction at the same time as the students. When you asked the general education teacher for lesson plans to help you prepare in advance you were told, “I don’t have time to write lesson plans. I just know what I am doing every day.” You are concerned about your ability to provide accommodations and needed IEP support without prior knowledge of what is going to be taught each day.

Case Study #7

You are a general education teacher and were just informed that your class has been designated as the “inclusion class.” Approximately ⅓ of your students will have IEPs. You know that at least two of the students have significant behavioral concerns. A special education teacher or paraprofessional will be present during most of the academic instruction, but you are concerned about meeting the needs of all students and handling behaviors.

Case Study #8

One of your students frequently demonstrates disruptive and unsafe behaviors (e.g., cursing and throwing things) in the classroom. You have tried every behavioral strategy you can think of and they just don’t work to stop the behavior, so you resort to sending the student to the office. One day, as you are writing the disciplinary referral to send the student to the office, he comments, “That’s fine. When I’m hanging out in the office, Mrs. Angelo (the office manager) talks with me and gives me candy.”

Case Study #9

You are a special education teacher responsible for supervising two special education paraprofessionals. Some of your students require assistance with bathroom routines and occasionally have accidents which require adult support to clean up and change clothes. One of the paraprofessionals is unwilling to support students with these needs and is quite vocal about how the bathroom duties make her “feel sick.” The other paraprofessional shares with you in confidence that she feels it is unfair that she is always the one to handle these needs.

Case Study #10

You are a special education teacher working with a collaborative team of general education teachers and paraprofessionals. Your team has a student with significant medical needs and all team members are concerned about his safety. You asked your principal to provide training for all staff related to the students needs but were told that there is not enough funding for training.

High Leverage Practices in Special Education: Collaboration: https://highleveragepractices.org/four-areas-practice-k-12/collaboration

IRIS Module on collaboration for students with cognitive disabilities: https://iris.peabody.vanderbilt.edu/module/scd/cresource/q2/p05/

IRIS Module on Family Engagement: Collaborating with Families of Students with Disabilities: https://iris.peabody.vanderbilt.edu/module/fam/

IRIS Module on Serving Students with Visual Impairments: The Importance of Collaboration: https://iris.peabody.vanderbilt.edu/module/v03-focusplay/

Virginia Department of Education Inclusive Practices: https://www.doe.virginia.gov/programs-services/special-education/iep-instruction/inclusive-practices

A Case Study Guide to Special Education Copyright © by Jennifer Walker; Melissa C. Jenkins; and Danielle Smith. All Rights Reserved.

Share This Book

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Analyzing differences between parent- and self-report measures with a latent space approach

Dongyoung go, minjeong jeon, saebyul lee, ick hoon jin, hae-jeong park.

• Author information
• Article notes
• Copyright and License information

Competing Interests: The authors have declared that no competing interests exist.

* E-mail: [email protected] (HJP); [email protected] (IHJ)

Received 2021 Jun 30; Accepted 2022 May 19; Collection date 2022.

This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

We explore potential cross-informant discrepancies between child- and parent-report measures with an example of the Child Behavior Checklist (CBCL) and the Youth Self Report (YSR), parent- and self-report measures on children’s behavioral and emotional problems. We propose a new way of examining the parent- and child-report differences with an interaction map estimated using a Latent Space Item Response Model (LSIRM). The interaction map enables the investigation of the dependency between items, between respondents, and between items and respondents, which is not possible with the conventional approach. The LSIRM captures the differential positions of items and respondents in the latent spaces for CBCL and YSR and identifies the relationships between each respondent and item according to their dependent structures. The results suggest that the analysis of item response in the latent space using the LSIRM is beneficial in uncovering the differential structures embedded in the response data obtained from different perspectives in children and their parents. This study also argues that the differential hidden structures of children and parents’ responses should be taken together to evaluate children’s behavioral problems.

Introduction

How children think of themselves may not be consistent with how their parents think about themselves. Reportedly, parents’ evaluations of their children are often biased, sometimes superficial, and affected by their relationship with their children. Similarly, children often do not view themselves objectively [ 1 – 5 ].

Parent review and child review are frequently utilized in assessing children’s behavior or psychology [ 6 – 9 ]. Among many scales, the Child Behavior Checklist (CBCL) [ 7 ] and the Youth Self Report (YSR) are widely used scales to evaluate children’s behavior problems, which are based on parent reports (CBCL) and self (children) reports (YSR) on the same questionnaires. To evaluate children, the CBCL and YSR comprise items of eight domains (often called syndromes): Aggressive Behavior (AB), Anxious/Depressed (AD), Attention Problems (AP), Rule-Breaking Behavior (RBB), Somatic Complaints (SC), Social Problems (SP), Thought Problems (TP), and Withdrawn/Depressed (WD). These domains are often categorized into scores of internalizing (a combination of AD, WD, and SC) and externalizing problems (a combination of RBB and AB).

The large discrepancies between the CBCL and YSR data are a commonly discussed issue in the literature [ 2 , 6 , 10 – 12 ]. To study the discrepancies between parent- and self-report measures from the CBCL and YSR, two types of approaches are conventionally taken: the direct comparison and the model-based approaches. For the direct comparison, researchers typically utilized the measures that estimate the dependency between items of two different scales, using Pearson correlation or Cohen’s Kappa [ 2 , 10 , 12 , 13 ]. However, these direct comparison methods do not take into account the potential dependency among items that may differently exist in the two sets of measures on children. Consequently, comparison between CBCL and YSR based on the responses without considering intrinsic item inter-dependency is questionable. Three types of inter-dependent hidden structures may exist in the group item response data; between items and items, between respondents and respondents, and between items and respondents. These inter-dependent structures may differ between CBCL and YSR.

To consider item-item inter-dependency, factor analysis [ 7 , 14 , 15 ] has been applied to evaluate the CBCL and YSR relationship by modeling item response data with linear combinations of latent factors. Not being based on the item-response theory, those latent variables do not directly reflect the innate item or respondent characteristics embedded in the choice behaviors.

Several studies exist that defined the characteristics of items based on the item-response theory model to compare the characteristics between the two groups of respondents in the latent space [ 2 , 12 , 16 – 19 ]. Although the item-response theory model includes latent variables for items and respondents to explain group response data (see Eq 2 in the Method section), the direct item-item, respondent-respondent, or item-respondent relationships are not easily discernible. Furthermore, the previous item-response theory model may not directly evaluate each individual’s tendency toward items. Evaluating item response characteristics in each individual is critical in the practical application. Looking at individual-level discrepancies between a child-parent couple enables to generate personalized feedback on the child and parent. When exploring how a pair of child and parent views the child’s behavior differently, it would be intuitive if the items and each respondent (child and parent) have their positions (thus can be visualized) in the same latent space and if the distance between each item and respondent is explicitly defined.

In this study, to explore the multi-dimensional inter-dependent structures between items, respondents and item-respondent pairs differently embedded in the CBCL and YSR responses, we approached the problem in the latent space with a newly developed Latent Space Item Response Model (LSIRM) [ 20 ].

Latent spaces, or in other words, interaction maps, are commonly utilized to represent the relationship or dependencies between actors in various literature, such as social network analysis [ 21 , 22 ], recommendation [ 23 , 24 ]. The LSIRM assumes the inter-dependency can be modeled with ‘the distance’ in latent space in which both respondents and items have their ‘latent positions’ where the short distances in the latent space implies the strong dependency. For example, two items closer to each other in a latent space have stronger dependency than two items far apart. Two items with strong dependency indicate that respondents tend to show a similar response pattern to those items. Similarly, two respondents close to each other have strong dependencies, meaning that they tend to show a similar pattern of responding to test items (i.e., if one respondent responds positively to some items, the other respondent is likely to respond positively to those items). We will show that examining these dependency patterns in the two sets of measures can enlighten and shed new light on understanding the differences between CBCL and YSR. We will further present that the currently proposed method has an advantage in directly exploring respondent tendency toward items at the individual level, making it possible to examine different views in a child-parent pair on the child’s behaviors.

The rest of the paper is organized as follows: We begin by describing the empirical data example that we will use in this study, followed by conventional analyses of the data. We then briefly describe the LSIRM and show how this model can capture the dependency in item response data. Next, we describe our strategy for comparing children and parent reports from the CBCL and YSR with LSIRM. In the Result and Discussion sections, we present and discuss our analysis results, compared with conventional analysis methods, and the differential views about the children’s behaviors from the children’s and their patients’ perspectives.

Materials and methods

Empirical data example.

We used a dataset from the Children Mind Institute’s healthy brain network MRI database ( https://childmind.org/data-sharing-initiatives/ ) [ 25 ]. Among 1,479 children, we selected 662 children (male: 397, female: 265) who had both the CBCL and YSR scores. Their age ranged from 10–18 years (mean: 13.8 years, standard deviation: 1.97 years). The CBCL and YSR tests consist of 120 items stating identical children’s behavioral problems in both tests. Among them, 118 items are multiple-choice questions such as “Acts too young for his/her age.” Respondents are asked to choose an option among the three response categories, “Not true,” “Somewhat or sometimes true,” “Very true or often true”, which are coded 0, 1, and 2, respectively. The other two free response items such as “Please write in any problems that were not listed above” is not considered here. The CBCL and YSR items are categorized into eight syndromes. We provided each item’s syndrome membership in the S1 Table . Several items do not belong to any particular syndromes. Those include Items 6, 7, 15, 24, 44, 49, 53, 55, 59, 60, 73, 74, 77, 80, 88, 92, 93, 98, 106, 107, 108, 109, and 110.

For data analysis, we dichotomized the original responses by combining the two positive categories of “Somewhat or sometimes true,” “Very true or often true,” and contrasting it with the only negative category of “Not true.” Such data dichotomization is not uncommon in the CBCL and YSR analysis, in part because of the frequently reported low reliability of the response categories [ 26 – 29 ]. After dichotomization, the mean proportion of positive responses was 0.43 with a standard deviation (SD) of 0.24 in the YSR, while in the CBCL, it was 0.27 with an SD of 0.17.

Analysis of CBCL and YSR differences using conventional methods

To compare with the current LSIRM analysis, we applied four analysis methods commonly used in the literature to compare the CBCL and YSR: three direct comparisons using pairwise measures, i.e., Pearson correlation, Kappa coefficients, and Jaccard similarity, and one model-based approach using item factor analysis. First, we evaluated the Pearson correlations between the CBCL and YSR at the syndrome level, using the syndrome-specific sum scores. The syndrome-level sum score is the sum of binary responses to the individual items, which can be treated as continuous data. Overall, the Pearson correlation was low, ranging from 0.23 (thought problem; TP) to 0.47 (rule-breaking problem; RBB). Table 1 shows the CBCL-YSR correlations for all syndromes. The size of the correlations was similar to the reports in the literature (e.g., [ 13 ]).

Table 1. Syndrome-level correlations and Kappa coefficients between CBCL and YSR.

a For the Kappa coefficients, we computed the syndrome mean of the item-level coefficients.

We then computed the Kappa coefficients to evaluate the degree of agreement between CBCL and YSR at the item level, following the procedure used in [ 12 ]. The Kappa coefficients were also low, ranging from -0.02 to 0.49, with a mean of 0.18. Table 1 lists the mean Kappa coefficients for all syndromes. This result is also in line with the reports in the literature (e.g., [ 12 ]).

Next, we applied conventional dyadic similarity analysis at the item level by using the Jaccard similarity measure [ 30 ]. Jaccard similarity is a dyadic similarity measure that compares the positive response counts of item pairs. Specifically, Jaccard similarity for binary vectors A and B is defined as J ( A , B ) = | A ∩ B |/| A ∪ B |. For the CBCL and YSR data, we computed the Jaccard similarity between P item vectors in the CBCL and YSR, respectively, where P are the number of items; we then subtracted the CBCL Jaccard similarity from the YSR similarity matrix. Table 2 lists the item pairs with the top 12 most significant differences in Jaccard similarity. Interestingly, none of the identified items have syndrome membership.

Table 2. Twelve item-item pairs are showing the most significant differences in Jaccard similarity between the CBCL and YSR.

[] indicates syndromes that the items belong to. [X] indicates that the item is not a member of any particular syndrome. For all pairs, the YSR showed high similarity (mean 0.963), while the CBCL showed low similarity (mean 0.014).

Lastly, we applied exploratory item factor analysis to the CBCL and YSR data, using the R package ‘mirt’ [ 31 ]. In the item factor model, the probability of answering positively to item i for respondent k is defined as follows:

where d i is item intercept, α i is item-specific latent factor, and θ k is respondent-specific latent factor [ 31 ].

In this experiment, we found the optimal number of factors was four, both for the CBCL and YSR data, based on G2 goodness-of-fit statistics and their p-values [ 32 ]. The factor structure comparison table and their fit statistics are given in the Supplementary S2 Table . The factor structure, however, was different between the two datasets. To summarize, for the first two factors, the CBCL and YSR show similar loading structures: the first significant factor covers most items of the AD syndrome (internalizing syndrome), and the second important factor covers most items of the RBB syndrome, an externalizing syndrome. However, the last two factor structures are quite different between the CBCL and YSR. The third factor is loaded on attention, social and thought problems (AP, SP, and TP) syndromes in the CBCL, while in the YSR, it is loaded on SC and WD syndromes. The fourth factor is loaded on the items with no syndrome membership in the YSR, while in the CBCL, this factor is loaded on most of the AB syndrome. These results are also consistent with the literature [ 6 , 14 ].

All four analyses point to that there are significant discrepancies between the CBCL and YSR data. We will later compare and discuss the proposed latent space approach’s results compared with these conventional methods. These conventional methods work as a reference for validating the proposed algorithm and providing a rationale for applying the current model-based approach.

Latent space item response model

The LSIRM [ 20 ] has been developed as an extension of the Rasch model to alleviate the conventional assumption of conditional independence (item responses are independent given a latent trait) and of homogeneity (respondents with the same trait level have equal success probability to an item). The LSIRM introduces latent positions, w i and z k for each item i and each respondent k in a d -dimensional latent space. The probability of positive response P ( y k i = 1 ) to an item i from a respondent k is then formulated as follows:

with item coefficients β i (item easiness), respondent coefficients θ k (latent trait), and the latent positions of respondent z k and item w i ( z k , w i ∈ R d ). The latent positions of respondent z k and item w i are determined by their distances, given the respondent and item coefficients. The resulting latent space approach provides an interaction map that represents the interactions of respondents and items, and helps derive insightful diagnostic information on items as well as respondents. We will use the words latent space and interaction map simultaneously. Though the dimension of the latent space or the distance measure can be arbitrarily chosen by researcher, we stick to R 2 and euclidean distance as Jeon et al. [ 20 ] remarked because of its strength in visualization. Eq 2 explains that when a respondent is further away from an item (i.e., larger distance and weaker dependency), the probability of giving a positive response to the item decreases. When a respondent becomes closer to an item (i.e., the shorter distance, the stronger dependency), the probability of giving a positive response increases. Note that Eq 2 is slightly different from Jeon et al. [ 20 ]’s formulation in that we fix the distance weight as one. This set-up enables us to match the two interaction maps from the CBCL and YSR data.

Transitivity

Note that although the model specifies distances between respondents and items in Eq 2 , the model also captures item-item and respondent-respondent distances. Fig 1(a) and 1(b) explains how it is possible; (a) if a k -th respondent responds positively to an i -th item and at the same time responds positive to a j -th item, the latent positions of w i and w j are likely to be close to each other; (b) if an i -th item is positively answered by respondents k and l , z k and z l are likely to be close to each other in interaction map. This is a property interaction map referred to as transitivity [ 33 , 34 ].

Fig 1. The concept of the distance, position and the latent space (interaction map).

(a) and (b) visualize the triangle inequality of the distance. (a) k is a respondent, and i and j are two items. If the positions of two items i and j are close to respondent k in the interaction map, the two items are fairly close to each other, by the triangle inequality. (b) i is an item, and k and l are two respondents. If two respondents k and l are close to the item i , the two respondents k and l are also close. (c) is an example of the posterior distributions of latent positions in the CBCL interaction map, and (d) is the posterior distributions in the YSR interaction map. For visualization, only four items (w18, w40, w49, w106) and four respondents (z91, z336, z397, z610) are displayed. Green dots and lines around the item positions indicate the posterior distributions of the item positions. Purple dots and lines around the person positions indicate the posterior distributions of the person positions. Note that the interaction maps differ for parents (CBCL) and children (YSR). Note also that both item and person positions are located in the same map. The short distance between two latent positions of items (respondents) in the interaction map implies a large dependency between two items (respondents). (e) and (f) present posterior distributions of selected item coefficients (e) and respondent coefficients (f). Blue indicates the CBCL, and red indicates the YSR distributions for each coefficient. All Bayesian inferences are made through these posterior samples; thereby flexible inference such as variance or overlapped portion with other distribution can be made.

The distance between two latent positions in latent space is the measure of dependency. The short distance between i th item latent position ( w i ) and j th item latent position ( w j ) means high dependency between two items. The respondent who responds positively to item i is likely to respond positively to j th item, vice-versa. The LSIRM models the item-item, respondent-respondent, and item-respondent dependency by projecting respondents and items to the continuous latent space. Therefore, the model-based comparison of two scales from different informants using the LSIRM allows us to evaluate pairwise dependencies between items, between respondents, and between items and respondents. Additionally, comparisons in aspects of higher-order dependencies, such as clustering patterns and relationships between clusters, are available using the LSIRM.

Fig 1(c) and 1(d) illustrate the interaction maps of the CBCL and YSR for the selected respondents and items for specific example. The figures display the posterior samples of latent positions for the selected items and respondents. For the item-item comparison, Items 40 and 49 (w40, w49) are relatively close in both interaction maps. This implies that Items 40 and 49 have a high dependency in both YSR and CBCL responses. On the other hand, Items 49 and 106 (w49, w106) are apart more in the CBCL interaction map than in the YSR interaction map, which implies a difference in dependency between the YSR and CBCL responses. Note that Item 40 states “Hears sounds or voices that aren’t there”, Item 49 states “Constipated, doesn’t move bowels”, and Item 106 is about “Vandalism”. Therefore, we can identify the differences of inherent dependency between pairs of items by comparing the interaction maps of the CBCL and YSR. For example, parents tended to make different reports on vandalism and constipation, while children responded similarly to those topics.

Regarding the respondent-respondent relationship, Fig 1(c) and 1(d) also show that some respondents are close to their locations, while others are further away. For example, Respondents 610 and 397 (z610, z397) are close to each other in both interaction maps, meaning that their response patterns were similar, given their latent trait levels and item difficulty. Similarly, for the respondent-item, we can also observe that Respondent 236 (z236) is closer to Item 106 (w106) in the CBCL space than in the YSR space. This indicates that for Respondent 236, the child’s probability of positively responding to “Vandalism” was lower than her parent’s likelihood of giving positive responses.

Of note, the current model assumes some hidden tendencies in the parents’ responses toward their children. Thus, other parent respondents’ responses are needed to decompose those hidden parent-to-children tendencies as a structure, even though the parent respondents are not directly related to the other children.

To estimate the LSIRM parameters, we used a Bayesian approach, following [ 20 ]. The priors of β i , θ k , z k and w i are set to an independent normal distribution with a mean of 0 and some variances. The hyper-prior for the variance parameter of θ k ( k = 1, ⋯, N ) is set to be a conjugate inverse Gamma distribution:

Metropolis-Hasting-within-Gibbs sampler [ 35 ] was used, which generates the posterior samples of β i , θ k , z k , and w i for k = 1, ‥, N and i = 1, ‥, P . Additional details of the estimation procedure are provided in the S1 Appendix . We set the number of iterations to 30,000 and take every 5-th sample after the first 5,000 steps as a burn-in period. Convergence was satisfactory, and the posterior distributions of the example item and person parameters are presented in Fig 1(e) and 1(f) .

Due to the invariance property of the distances (invariance to rotation, reflection, and translation), multiple solutions may be available for the latent positions that produce the same distance matrix. This issue is common for models that involve latent spaces [ 21 ]. To resolve such an identifiability issue, we applied Procrustes transformation [ 36 ] as a post-processing procedure, which is a standard method to resolve latent position identifiability in the literature [ 21 , 22 ]. For [ Z ] the class of positions equivalent to Z under rotation, reflection, translation, the Procrustean transformation is Z * = argmin TZ tr( Z 0 − TZ ) T ( Z 0 − TZ ), where Z 0 is a fixed set of positions and T ranges over the set of rotations, reflections, and translations. It is known that Z * is the closest element to Z 0 in terms of the sum of squared and is unique if Z 0 Z T is nonsingular [ 37 ]. Here the target Z 0 would be the positions draw that produces maximum a posteriori and the other posterior draw of latent positions are carried out Procrustes transformation to resolve the invariance to reflections, rotations and translations. All Bayesian inferences are made through the posterior samples obtained after the post-processing.

To deal with missingness in the datasets under investigation, we extended the estimation procedure of Jeon et al. [ 20 ] with Bayesian data augmentation [ 38 ]. Specifically, we imputed the missing data with the posterior samples, where an estimated item response for missing y ^ i k was generated using the estimated parameters of the previous step of the Gibbs sampling. The imputed response values were updated in every Gibbs sampling step. After assuming missing at random (MAR), this Bayesian data augmentation produces valid imputation results [ 39 ]. Of all item and respondent pairs, the missing proportion was 0.192% in YSR and 0.259% in the CBCL. We made MAR assumption because currently there is no established method for testing the nature of missingness, and MAR is often assumed in the CBCL and YSR analysis. Further, addressing potential non-ignorable missingness is not the primary purpose of the current study. We will look into the sensitivity of this assumption in future research.

Strategies for analyzing the CBCL and YSR differences with the LSIRM

We applied a series of further analyses for model-based examinations of the CBCL-YSR differences with the discussed latent space approach.

First, we fit the separate LSIRM to the CBCL and YSR datasets and estimated two sets of item coefficients β i , respondent coefficients θ k , and their latent space positions ( z k and w i ). We evaluated the goodness of fit of the model to each dataset.

Second, we examined the differences between the item coefficients β i estimated from the CBCL and YSR analysis. We also compared item positions w i in the CBCL and YSR interaction maps and checked whether the items with a large discrepancy between CBCL and YSR in item coefficients β i show distinct interaction patterns between them.

Third, we compared the CBCL and YSR in terms of item-pair distance or item-item dependency over the interaction map. For this, we first evaluated the distance distribution of the two items i and j (i.e., the posterior distributions of the distance || w i − w j ||) in the CBCL denoted as D i j C B C L ( x ) and YSR denoted as D i j Y S R ( x ) , respectively. This posterior distribution function D i j C B C L and D i j Y S R were obtained by calculating the Euclidean distance of each posterior sample of w i and w j in CBCL and YSR, obtained by fitting the LSIRM. This distribution represents the estimated dependency between item i and item j in terms of the distribution. We then assessed the significance of the differences of each item-pair dependency by evaluating the overlapped portion of their distance distribution. The overlapped portion was defined as

where min ( D i j C B C L ( x ) , D i j Y S R ( x ) ) indicates the overlapping areas of the two distributions. R ≤ .05 indicates that the two distributions are fairly different in a statistical sense [ 40 – 42 ]. An alternative approach, the calculation of the Kullback-Leibler (KL) divergence between D i j C B C L ( x ) and D i j Y S R ( x ) , was also applied to capture the difference of item-pair dependency and its results were reported in the Supplementary S2 Appendix .

Fourth, item syndrome-level dependency is evaluated, with the syndrome positions identified with the centroid of the syndrome-specific items. Since the item syndrome is named with its semantic properties, it makes the axis of the interaction map more interpretable and comparable with intuition. To highlight the meaning of this axis, we used cosine similarity to measure the similarity of item syndrome level. The cosine similarity is measured based on the coordinates of the syndrome latent positions in the CBCL and YSR separately. The cosine similarity of two vectors a and b can be computed as

where ϕ is the angle between two vectors a and b , and || a || 2 > 0 and || b || 2 > 0 are their lengths. cos( ϕ ) ranges from -1 to 1, while -1 indicates that the two vectors point to the opposite directions, and 1 indicates the same direction. Cosine similarity of 0 means that the two vectors are orthogonal.

Fifth, we evaluated differences in the dependency between respondents with their syndrome levels. We presented that the distance between each respondent and syndrome in the interaction map could be used to find the vulnerable syndromes for each respondent. We then compared K-means clustering of all respondents based on the distance from all item syndromes and demonstrated how the interaction map approach could derive unique findings.

To evaluate the goodness of fit of the LSIRM to CBCL and YSR data, we assessed the proximity of the predicted item responses based on the estimated model to the original item responses, which is a commonly used strategy for evaluating the prediction accuracy of binary classification [ 43 – 45 ]. As evaluation criteria, we used sensitivity, specificity, and overall accuracy, which are reported as reliable measures [ 46 , 47 ].

The three indices are defined as Specifity = TN TN + FP , Sensitivity = TP TP + FN , Overall accuracy = TP + TN TP + TN + FP + FN , where TN is true negative, TP is true positive, FP is false positive, and FN is false negative. Table 3 shows the result. All values are higher than 0.70, except for the sensitivity for the CBCL. This suggests that overall, the LSIRM showed satisfactory fit to both the CBCL and YSR data in terms of prediction.

Table 3. Three goodness-of-fit measures of the LSIRM to the CBCL and YSR data.

Item coefficients and positions.

Fig 2(a) shows the relationship between the CBCL and YSR in the estimated item coefficients β ^ i . For most test items, the two sets of β ^ i are similar with the correlation of r = 0.83. However, a group of items does not follow this general pattern (marked in blue in the scatter plot). For those items, β ^ Y S R were higher than β ^ C B C L , meaning that the children than their parents more easily endorsed them. The specific contents of those items were listed in Table 4 . Those items appear to address behaviors related to sexual or physiological problems. That is, parents tended to believe that their children did not have sexual and physiological problems, even when the children themselves acknowledged such problems. Further analysis of these items with the interaction map can lead to more detailed interpretation.

Fig 2. Item-wise comparison between the CBCL and YSR.

(a) Comparison between the CBCL and YSR item coefficients β ^ i is displayed. The red line is the linear regression line between the CBCL and YSR estimates. The dotted line is y = x . Numbers indicate the outliers that deviate largely from the linear trend. For most test items, the two sets of β ^ i are similar with the correlation of r = 0.83. However, the indicated items do not follow this general pattern. (b) An integrated interaction map for the CBCL and YSR item positions is presented. Blue and red dots represent the item positions identified in the CBCL and YSR data, respectively. Larger dots with number labels are the outlier items identified in (a). Note that the distance between dots indicates the degree of association of the two items in the latent space. For the close items, if respondents respond positively to an item, they are more likely to respond positively to the other item.

Table 4. Items that differ between the CBCL and YSR in item-wise coefficients β ^ i or item easiness (a tendency of the positive answer).

The item easinesses of the YSR for these items are greater than the CBCL ( β ^ C B C L < β ^ Y S R ).

Fig 2(b) shows interaction maps with item latent positions w i obtained from CBCL and YSR analysis with respective LSIRMs. For visual comparisons, the two estimated interaction maps were matched and integrated using Procrustes matching. Blue dots indicate the positions of test items from the CBCL space, and red dots indicate the positions of test items in the YSR space. Respondents’ positions are not presented in Fig 2(b) to focus on item position comparisons.

The distributions of item positions were generally similar between CBCL and YSR spaces, but there were some notable exceptions. The aforementioned items with dissimilar item coefficients (items in Fig 2(a) and Table 4 ) were placed in different regions of the CBCL and YSR interaction maps. See items marked with numbers in Fig 2(b) . These items were closely located to each other in YSR (marked with a dotted red oval), which was not the case in the CBCL data.

Differences in item-item dependency

We evaluated item-pair distance differences between the CBCL and YSR data based on D i j C B C L and D i j Y S R values. For most item pairs, the difference was statistically significant ( p < 0.01) based on the Kolmogorov-Smirnov test. Table 5 lists the top 12 item pairs with the largest CBCL-YSR differences.

Table 5. Top 12 item pairs that show the largest differences between the CBCL and YSR in terms of D i j C B C L and D i j Y S R .

AP, SP and TP etc. within the brackets indicate the syndrome that each item belongs.

Fig 3(a) illustrates how different the item-pair distances were between the CBCL and YSR data for the top four pairs. Interestingly, these items did not match the items identified from the item coefficient ( β i ) difference analysis ( Table 4 ). In addition, they did not match the items identified with the Jaccard similarity analysis ( Table 2 and Fig 3(b) ). Fig 3(b) shows the Jaccard similarity between the CBCL and YSR data. The items identified in conventional direct comparison using Jaccard similarity were not coherently located with Fig 3(a) in the interaction map. This implies that our analysis of item-pair dependency does not offer the same kind of information as the analysis of the item coefficients and the conventional similarity analysis.

(a) Four item-item pairs with the largest differences in the distance (see Table 5 ). Double-side arrows are drawn to show the distances between two pairs (items 40 and 01; items 36 and 12) in the CBCL (blue) and YSR data (red). In addition to the position of each item, the distance between the two items’ positions should be noted. (b) Four item-item pairs with the largest difference in the Jaccard similarity between the CBCL and YSR data. This implies that our analysis of item-pair dependency does not offer the same kind of information as the analysis of the item coefficient and the conventional similarity analysis.

With item-pair differences, two groups of pairs can be distinguished in Table 5 . In one group, those pairs were believed to have similar behaviors from the children’s perspective, but not so much to the parents’ view (Pairs 1–9). In the other group, the item pairs showed the opposite patterns: while they were perceived similar behaviors to the parents’ eyes, they were believed to have different problems to the children (Pairs 10–12).

For example, to the children, “Act too young” and “Impulsive” (Pair 1) were similar behaviors, while it was not the case to the parents. The parents consider their children’s behaviors of “Complains of loneliness” and “Stores up too many things he/she doesn’t need” (Pair 11) are highly relevant, but children consider them irrelevant. Most of these items are the members of attention, social and thought problems (AP, SP, and TP) syndromes, indicating that those syndromes’ behaviors were likely to be evaluated and reported differently by different informants, i.e., parents and children in the current context.

We assessed the overall differences in item-item dependency between the CBCL and YSR data by evaluating the overlap of the D i j C B C L and D i j Y S R distributions. Fig 4 summarizes above Fig 3 with respect to item-item dependency categorized by syndromes. The distance of each item pair (i.e., expectation of the probability distribution D ij ) is displayed in Fig 4(a) and 4(b) for the CBCL and YSR, respectively. The low value of distance means that their latent position was located closely and had a strong interaction. Fig 4(c) summarizes the overlapped portion for all pairs of items between CBCL and YSR. The item pairs with overlapped portion less than 0.05 are marked with bold edges. Of all the item pairs, about 9% of the pairs had overlapped portions of less than 0.05. The alternative approach using KL-divergence ( S2 Appendix ) suggests that pairs of items with a large discrepancy in KL-divergence between CBCL and YSR are positioned differently in their interaction maps.

Fig 4. Item-item distance heatmaps of the CBCL and YSR, and an overlapped portion heatmap between the CBCL and YSR.

(a) displays the distance heatmap between pairs of item latent positions in the CBCL ( D i j C B C L ) and (b) displays that in the YSR ( D i j Y S R ). (c) shows the heatmap of the overlapped portion between D i j C B C L and D i j Y S R distributions for all item pairs. The item pair with a large overlapped portion means the perceived relation of that item pair is similar in both informants, while the item pair with a small overlapped portion means their relationship is perceived as different. The item pair with a large overlapped portion is colored with red, and the item pair with a small overlapped portion is colored with blue. The item pairs with overlapped portions less than 0.05 are marked with bold edges. About 9% of the pairs had overlapping portions of less than 0.05, and most of these pairs exist between different item syndromes (off-diagonal blocks), related to attention (AP), social (SP), and thought (TP) problems.

Syndrome-level distances

Fig 5(a) shows the syndromes’ latent positions (identified as the centroid of the syndrome item members) from the CBCL and YSR spaces. In both spaces, externalizing syndromes (RBB, AB) are close to each other, and internalizing syndromes (AB, WD, and SC) are also close to each other, while externalizing and internalizing syndromes are far apart from each other. Fig 5(a) also shows that syndromes tend to be closer in their positions in the YSR space than in the CBCL space, meaning that the syndromes were seen more similar each other to the children’s perspective than to the parents’ view.

Fig 5. (a) Interaction map of the syndrome positions identified with CBCL (blue) and YSR (red).

Recall that syndromes are denoted with their abbreviation, Aggressive Behavior (AB), Anxious/Depressed (AD), Attention Problems (AP), Rule-Breaking Behavior (RBB), Somatic Complaints (SC), Social Problems (SP), Thought Problems (TP), and Withdrawn/Depressed (WD). These domains are often further categorized into internalizing (INT, a combination of AD, WD, and SC) and externalizing problems (EXT, a combination of RBB and AB). (b) and (c) are the heatmaps of the cosine similarity matrices of the item syndrome positions in the CBCL and YSR. (d) and (e) show the heatmaps of the correlation matrices of the raw syndrome scores in the CBCL and YSR. The identified dependency is roughly similar. However, the inter-correlations between externalizing syndromes (RBB, AB) and between internalizing syndromes (AD, WD, SC) were not outstanding in (d) and (e) compared to (b) and (c).

We measured cosine similarity between syndrome positions identified in the CBCL and YSR data. The cosine similarity is measured based on the coordinates of the syndrome latent positions in CBCL and YSR separately. Fig 5(b) and 5(c) display the heatmaps of cosine similarity measures for CBCL and YSR, and they suggest that the attention, social and thought problems (AP, SP, and TP) syndromes show different similarity patterns between CBCL and YSR. In particular, the characteristics of AP are distinct from other syndromes in the CBCL space. This result is consistent with the findings based on item-item dependency patterns.

In contrast, a simple inter-syndrome correlation analysis based on syndrome-level raw scores presented in Fig 5(d) and 5(e) did not identify inter-and intra-correlations between the syndromes. For example, the inter-correlations between externalizing syndromes (RBB, AB) and between internalizing syndromes (AD, WD, SC) were not outstanding.

Respondent distances to syndromes

Individual respondents’ distances to the behaviors or the syndromes in the interaction map are useful as they are the basis to draw diagnostic conclusions about the respondents. To illustrate differences and similarities in the respondents’ distances to the syndromes, we selected three respondents as an example (s16, s102, and s250) in Fig 6(a) and 6(b) . Respondent s16 was closest to the syndromes RBB and AB in both the CBCL and YSR spaces, indicating that she/he were more likely to show the behaviors of the RBB and AB syndrome than other behaviors, and such a diagnosis of s16 would be similar based on the CBCL, and YSR reports. On the other hand, respondent s102 was located somewhat differently in the two spaces. This student was close to RBB and apart from AP in the CBCL space, but she/he was closer to AP than RBB in the YSR space.

(a) and (b) display the syndrome positions and three respondents (s16, s102, s250) in the CBCL and YSR space, i.e., couples of parents (CBCL) and children (YSR), respectively. Note that both latent variables for items and latent variables for respondents are plotted in a latent coordinate. Latent positions for syndromes are presented as colored circles with solid black borders, and respondents’ latent positions are displayed as dots. Three respondents are exemplary selected based on their latent positions in the CBCL and YSR: the two respondents located in the center (s250) and in the boundary (s16) show similarity between children and parents but s102 show difference in the latent positions between children and parents. (c) and (d) show the K-means clustering results of all respondents based on the distance from all item syndromes in the CBCL and YSR spaces.

Finding vulnerable syndromes using an interaction map is different from the conventional approach, such as simply comparing the positive response counts of syndromes for each respondent, because the latent position in the LSIRM is assigned by considering the dependency between all other objects. For example, one could draw the same conclusion with Fig 6(b) using simply comparing the positive response counts for s16, the respondent who answered about 2.5 times more positively to externalizing syndromes (23 positive responses) than internalizing syndromes (9 positive responses) in the YSR. On the other hand, s16 in the CBCL answered in externalizing and internalizing syndromes quite similarly (20 and 18 positive responses, respectively.) but is still located close to the RBB and AB, the externalizing syndromes in the interaction map, Fig 6(a) . This is because the other respondents with similar response patterns to s16 in terms of each item, not discrete syndrome-level, responded more positively to externalizing syndromes. Therefore, although s16 responds similarly to both internalizing and externalizing syndromes in the CBCL, s16 might be more vulnerable to externalizing syndromes when considering other respondents with similar patterns. This means that one could end up with different conclusions about their most likely behaviors and the existence of cross-informants discrepancy depending on whether the dependency was considered or not.

We applied K-means clustering to the respondent positions to see the differences in the respondent-syndrome distances between the CBCL and YSR spaces. The results are shown in Fig 6(c) and 6(d) . Four clusters were identified, each with the CBCL and YSR respondent positions (distinguished with four colors). However, the four clusters identified from the two spaces do not necessarily match in terms of their most likely syndromes, given clusters’ different distances to the syndromes. For example, the cluster located in the upper side of the interaction map (blue) did not appear close to any particular syndrome in the YSR space ( Fig 6(d) ), while such cluster did not exist in the CBCL space ( Fig 6(c) ). This re-confirms that using either parent’s or children’s reports can lead us to a different conclusion about the children’s most likely syndromes and behavioral problems.

This paper discusses a new way of exploring the hidden structure of item responses from different informants with a latent space modeling approach, referred to as the LSIRM method. This new methodology was applied to the comparative analysis of the CBCL and YSR, which are widely-used multi-item scales on children’s behavioral and emotional problems, based on parent- and self-reports, respectively. The LSIRM analysis spotted substantial differences between the CBCL and YSR measures which were not examined with existing methods, either direct comparison approaches or model-based item factor analysis.

Direct comparison analysis using Pearson correlation, Cohen’s Kappa, or Jaccard similarity, computes the similarity between each item (or between each syndrome, with the sum of items belonging to it) pair-wisely and thereby treat each subject independently. This type of direct item-wise approach does not consider the potential dependency that may exist in other items or other respondents and cannot be used to compare the dependency between items and respondents at the same time. To figure out the item-respondent relationship, additional analysis such as linear regression or subject grouping is needed [ 2 , 12 , 18 , 19 ]. We do not disregard the possibility that some of those dependencies can be explored by extending the conventional methods. For example, the residual covariance matrix may be used to find item-item dependencies. We have found that conventionally used direct comparison methods partly capture the relationship inherent in data more obtusely but sometimes lead to different conclusions (e.g., Fig 3 ). The consistency between the results of the LSIRM and direct comparison methods reported in the previous literature and this study supports the validity of the proposed algorithm. This validity is critical for the LSIRM, which has inherent overfitting problems as a complex model. It also lays the basis for the new findings’ reliability derived from the further analysis unique to the LSIRM.

Item factor analysis [ 48 , 49 ], shown in Eq 1 , is designed to identify the latent factor structure underlying the item response data by assuming latent factors to be linear combinations of observed item responses. The item clusters identified with item factor analysis ( S2 Table ) are roughly similar to the LSIRM, showing high consistency between the CBCL and YSR in the internalizing and externalizing syndromes. That is, item factor analysis can identify clustering of items similar to the LSIRM. More specifically, the item factor analysis can capture item-by-respondent interactions to some degree in the sense that factor loadings (specific to items) are multiplied by factors (specific to respondents) (See Eq 1 ). Nevertheless, not all item-by-respondent interactions can be quantified with the item factor analysis because factor loadings are item-specific, not item-by-respondent specific. No parameters represent interactions between items and respondents in the item factor analysis model ( Eq 1 ). Furthermore, the item factor analysis cannot capture the similarity of respondents in terms of response patterns. In contrast, the LSIRM explicitly models item-by-respondent interactions in the form of distances in a metric space, which makes it possible to examine similarities among items and similarities among respondents ( Eq 2 ). As exampled in Fig 6 , the characteristics of each individual can intuitively be represented in terms of the respondent’s position and distances from items in the LSIRM. This model enables the investigation of each child and parent pair precisely. Of note, in terms of model parameter estimation, we used a Bayesian estimation with uninformative priors; thus, it is less likely that the model inversion scheme based on Bayesian or frequentist approaches may not be a significant factor in deriving the results.

The LSIRM differentiates between each item’s coefficient and item position in the latent space. The item coefficient is estimated independent of other items and individual characteristics. In contrast, the item position is estimated by considering structured distances from respondents ( Eq 2 ). The highly dependent structure of item response data differently embedded in the CBCL and YSR is first reflected in the item coefficient analysis. The high accordance exists in most item-wise coefficients between the CBCL and YSR, except for some outliers ( Fig 2(a) ). The items with a large discrepancy appear primarily related to sexual or physiological problems ( Table 4 ). Parents do not respond positively to sexual and physiological issues of their children, even though their children are aware of these problems. This is predictable and is consistent with the findings in previous studies [ 50 ], which were revealed by using simple correlation analysis of the responses. In this respect, the current approach for analyzing item-wise coefficients in the latent space does not significantly differ from the direct comparison methods such as the Jaccard similarity measure (to compare the positive response count) in identifying discrepant items between the CBCL and YSR. However, the current study further characterizes the structured relationships between those items by presenting their positions in the latent space of the CBCL and YSR. As shown in Fig 2(b) , those items are densely located on the upper boundary of the YSR, forming a clear cluster, but not in CBCL.

For items related to sexual or physiological problems with a large difference in item coefficients, β i , the positive response count is more endorsed in children than parents. Thus, those items have a more clustering tendency in the YSR. That means the respondent who positively responded to one of these outlier items also showed similar response patterns to other outlier items in the YSR data. In contrast, those patterns were not shown in the CBCL data. In other words, children think those items are highly related to each other, which is not considered the same for parents. This item dependency pattern was more evidenced in the item-item distance analysis.

In addition to providing a method to compare two scales using each item’s position, the advantage of the LSIRM-based approach is its ability to identify item-item distances in a structured way in the latent space. The item-item pairs that show differences between children and parents can be divided into two groups. One is the item pairs that children perceived as the underlying relationship while parents did not find them related to their children (Pairs 1–9 in Table 5 ). The other is item pairs that children did not find a relationship between domains in their behaviors, while parents observed a strong relationship between the two items in their children. For example, parents implicitly perceived the items in pair 1, i.e., “Act too young” and “Impulsive”, as not much relevance to children, while their children regarded the two as highly relevant. Furthermore, pairs 2–6 were related to “Trouble sleeping”. Trouble in sleeping may manifest in various syndromes, particularly in attention, such as inattentive or impulsive. However, it may be difficult for parents (as observers) to perceive the underlying connection. Meanwhile, parents consider that “Complains of loneliness” and “Stores up too many things he/she doesn’t need” are highly relevant items regarding their children, but children consider those items are irrelevant from their perspectives on themselves.

Note that the top item pairs showing the largest difference between parents and children in terms of item-item distance ( Table 5 ) belong mostly to pairs between non-extreme syndromes (attention, social and thought problems (AP, SP, and TP)). Only a few item pairs from the same syndrome showed differences between children and parents, indicating that the item-item similarity within the same syndromes was relatively consistent across children and parents. This is conspicuous for items in syndromes with extreme properties (i.e., externalizing scores, internalizing scores), as was found in previous studies that report high consistency in extreme problem scores [ 7 , 10 , 13 , 51 ].

Of note, the item-item pairs that showed distance differences between the children’s and parents’ views on the children ( Fig 3(a) ) are composed of items that do not belong to the list of items showing item-wise parameter differences between the CBCL and YSR ( Fig 2 ). Meanwhile, the conventional response-based approach using the Jaccard similarity did not provide additional information compared to item-wise analysis of the difference between the CBCL and YSR. The item pairs with the most considerable difference of the Jaccard similarity ( Fig 3(b) ) are almost combinations of the outlier items in Fig 2(a) . Therefore, the conventional dyadic comparison did not separate item-specific effects and the interactions within the CBCL or YSR.

Overlapped portion analysis of items’ latent positions between the CBCL and YSR also revealed similar results to the item-item distance analysis. Several item pairs in attention, social, and thought problems (AP, SP, and TP) have a significantly different relationship in the CBCL and YSR, especially for item pairs with one belonging to one of AP, SP, or TP and the other belonging to externalizing or internalizing scores Fig 4(c) . In addition, most of the item pairs that showed high differences between children and parents were closer in the children compared to the parents ( Fig 4 ).

Taking all results together, the current item-item dependency analysis using the LSIRM successfully specified the structural differences of children’s self-views and parents’ views on the children. Our latent space approach can reveal non-ignorable item-item dependence and identify that item-item dependence patterns and magnitude are fairly different between the CBCL and YSR data. In the CBCL-YSR context, where the test items indicate behavioral symptoms for certain syndromes, the identified difference in item dependency suggests that one could draw different conclusions, prevalent problems (or symptoms) and their relationships when data analysis was based on a single source, parent- or child- report only.

When we analyzed the latent position of the item syndrome, it shows directional order along the x-axis, i.e., the direction of ‘external’ to ‘internal’ axis ( Fig 5(a) ). This polarized property of externalizing and internalizing scores is consistent with the results of the item factor analysis [ 7 ]. Indeed, the axis of the fitted latent space does not have an interpretative meaning. However, by displaying the positions of item syndromes, one can make inferences about the space with reference to the position of each item syndrome. We found that the positions of syndromes in the YSR were generally closer than those in the CBCL. The congregation of the YSR syndromes reflects stronger interaction and a closer distance between syndromes in the YSR compared to the CBCL. This may be attributable to a tendency of positive answers in the YSR overall [ 51 , 52 ]. The calculated cosine similarity (in Fig 5(b) and 5(c) ) shows more apparent separation for items in externalizing and internalizing scores. Item syndromes belonging to externalizing or internalizing scores are generally perceived to be strongly related within the same group in both the CBCL and YSR, except for reduced association with SC (WD and SC) in the CBCL. However, for the attention, social, and thought problems (AP, SP, and TP), the perceived item relationship differs between the CBCL and YSR. This result is consistent with the findings based on the item-item distance pattern and their overlapped portion heatmap ( Fig 4 ). Note that the distance among nonextreme syndromes (the attention, social, and thought problems) differ obviously between children and parents. Thus categorization of items done in the parent’s perspective may not be sufficient to characterize children’s diverse problems and demands a new attempt to make a more specific division of questionnaires.

Because the latent position of each respondent is mapped with the items’ latent position, clustering of respondents can identify not only a group of respondents who responded positively to the same item but also a group of respondents who responded positively to items with similar characteristics. This type of clustering index can be used to determine the relationship between each respondent or the discordance between parents and their children. For example, the clustering of respondents based on the distance between the item syndrome and each respondent ( Fig 6(c) and 6(d) ) shows that a cluster of the YSR located on the upper side of the latent space did not overlap with any specific symptoms and can be characterized by positive responses exclusively to items in Fig 2(b) . This analysis revealed that the respondent-syndrome distances were not identical to some respondents in the CBCL and YSR spaces. The syndromes close to the respondent are the syndromes that the respondents (children in our context) are likely to have. Thus, this finding implies that one could draw different diagnostic conclusions about some children depending on whether children’s or parents’ reports were chosen for data analysis. These results also suggest that the conventional division of symptoms and clustering of individuals according to the dominant symptomatic problem may not necessarily apply to the division of the YSR.

Discrepancies between parent- and self-report measures have been commonly discussed, although examining and evaluating such discrepancies is an open area of research. The current study introduced a new and unique way of assessing parent- and self-report differences with a latent space approach using the LSIRM. Our empirical data analysis demonstrated that the two sets of measures from the two different informants did show much difference from the perspectives that have not been investigated with standard methods. Some of the differences that we identified have not been seen before and hold important practical implications. For example, behaviors and syndromes might show different structural relationships based on parent- or self-report measures. In addition, children might end up being diagnosed with a different set of behaviors and syndromes depending on whether parent- or self-report measure was used. Using the LSIRM, we mainly illustrated the potential discrepancies between parent- and child-report measures in terms of dependency. The identified CBCL-YSR differences from our proposed method illuminate and invite applied researchers to investigate and take a closer look at those discrepancies in other CBC-YSR datasets or other types of cross-informant measures.

In the current study, we did not examine how to resolve the identified differences, which is undoubtedly an important question, and we reserve it for future research. We primarily focused on the differences between informants in the latent space. Further analyses identifying the similarly responded items between informants, particularly items beyond internalizing and externalizing syndromes, remain as future research. Furthermore, the proposed model-based approach can be extended to a wide range of future work. First, our approach can be extended to non-binary item response data by choosing a suitable link function for non-binary item response data like in generalized linear models [ 20 , 53 ]. Second, we can incorporate explanatory variables in our model to evaluate the discrepancy between the YSR and CBCL caused by the covariate information of the children, such as age [ 54 ], intelligence score, sex [ 1 ], culture, and ethnicity [ 11 ].

Supporting information

Data availability.

The data underlying the results presented in the study are available from https://figshare.com/articles/dataset/lsrm_comparison_with_cbcl_ysr/19962830 .

Funding Statement

This research was supported by Brain Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2017M3C7A1030750), by the Yonsei University Research Fund of 2019-22-0210, and by Basic Science Research Program through the National Research Foundation of Korea (NRF 2020R1A2C1A01009881).

• 1. Barrett M, Berney T, Bhate S, Famuyiwa O, Fundudis T, Kolvin I, et al. Diagnosing Childhood Depression. Who Should be Interviewed-Parent or Child? The British Journal of Psychiatry. 1991; p. 22–27. [ PubMed ] [ Google Scholar ]
• 2. Berg-Nielsen TS, Vika A, Dahl AA. When adolescents disagree with their mothers: CBCL-YSR discrepancies related to maternal depression and adolescent self-esteem. Child: Care, Health, and Development. 2003;29:207–213. doi: 10.1046/j.1365-2214.2003.00332.x [ DOI ] [ PubMed ] [ Google Scholar ]
• 3. Loeber R, Green SM, Lahey BB, Stouthamer-Loeber M. Differences and similarities between children, mothers, and teachers as informants on disruptive child behavior. Journal of Abnormal Child Psychology. 1991;19:75–95. doi: 10.1007/BF00910566 [ DOI ] [ PubMed ] [ Google Scholar ]
• 4. Martin JL, Ford CB, Dyer-Friedman J, Tang J, Huffman LC. Patterns of agreement between parent and child ratings of emotional and behavioral problems in an outpatient clinical setting: When children endorse more problems. Journal of Developmental & Behavioral Pediatrics. 2004;25:150–155. doi: 10.1097/00004703-200406000-00002 [ DOI ] [ PubMed ] [ Google Scholar ]
• 5. Youngstrom E, Loeber R, Stouthamer-Loeber M. Patterns and correlates of agreement between parent, teacison of self-reported and parent-reported emotional and behavioral probher, and male adolescent ratings of externalizing and internalizing problems. Journal of Consulting and Clinical Psychology. 2000;68:1038. doi: 10.1037/0022-006X.68.6.1038 [ DOI ] [ PubMed ] [ Google Scholar ]
• 6. Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Department of Psychiatry, University of Vermont; 1991. Available from: https://books.google.co.kr/books?id=I5btOwAACAAJ .
• 7. Achenbach Thomas M Rescorla L. Manual for the ASEBA Preschool formsand Profiles. L.A.: Burlington, Vt.: ASEBA; 2002. [ Google Scholar ]
• 8. Sisteré ML, Massons JMD, Pérez RG, Ascaso LE. Validity of the DSM-oriented scales of the child behavior checklist and youth self-report. Psicothema. 2014; p. 364–371. [ DOI ] [ PubMed ] [ Google Scholar ]
• 9. Bordin IA, Rocha MM, Paula CS, Teixeira MCT, Achenbach TM, Rescorla LA, et al. Child Behavior Checklist (CBCL), Youth Self-Report (YSR) and Teacher’s Report Form (TRF): an overview of the development of the original and Brazilian versions. Cadernos de Saúde Pública. 2013;29:13–28. [ DOI ] [ PubMed ] [ Google Scholar ]
• 10. Lacalle M, Ezpeleta L, Doménech JM. DSM-Oriented Scales of the Child Behavior Checklist and Youth Self-Report in Clinically Referred Spanish Children. The Spanish Journal of Psychology. 2012;15:377–387. doi: 10.5209/rev_SJOP.2012.v15.n1.37344 [ DOI ] [ PubMed ] [ Google Scholar ]
• 11. Rescorla LA, Ginzburg S, Achenbach TM, Ivanova MY, Almqvist F, Begovac I, et al. Cross-informant agreement between parent-reported and adolescent self-reported problems in 25 societies. Journal of Clinical Child & Adolescent Psychology. 2013;42:262–273. doi: 10.1080/15374416.2012.717870 [ DOI ] [ PubMed ] [ Google Scholar ]
• 12. Gearing RE, Schwalbe CS, MacKenzie MJ, Brewer KB, Ibrahim RW. Assessment of adolescent mental health and behavioral problems in institutional care: discrepancies between staff-reported CBCL scores and adolescent-reported YSR scores. Adm Policy Ment Health. 2015;42(3):279–287. doi: 10.1007/s10488-014-0568-y [ DOI ] [ PubMed ] [ Google Scholar ]
• 13. Sourander A, Helstelä L, Helenius H. Parent-adolescent agreement on emotional and behavioral problems. Social psychiatry and psychiatric epidemiology. 1999;34:657–663. doi: 10.1007/s001270050189 [ DOI ] [ PubMed ] [ Google Scholar ]
• 14. Hartman CA, Hox J, Auerbach J, Erol N, Fonseca AC, Mellenbergh GJ, et al. Syndrome dimensions of the Child Behavior Checklist and the Teacher Report Form: A critical empirical evaluation. Journal of Child Psychology and Psychiatry. 1999;40:1095–1116. doi: 10.1111/1469-7610.00527 [ DOI ] [ PubMed ] [ Google Scholar ]
• 15. de Groot A, Koot HM, Verhulst FC. Cross-cultural generalizability of the Youth Self-Report and Teacher’s Report Form cross-informant syndromes. Journal of Abnormal Child Psychology. 1996;24:651–664. doi: 10.1007/BF01670105 [ DOI ] [ PubMed ] [ Google Scholar ]
• 16. Lambert MC, Essau CA, Schmitt N, Samms-Vaughan ME. Dimensionality and psychometric invariance of the Youth Self-Report Form of the Child Behavior Checklist in cross-national settings. Assessment. 2007;14,3:231–45. doi: 10.1177/1073191107302036 [ DOI ] [ PubMed ] [ Google Scholar ]
• 17. Panter A, Reeve BB. Assessing tobacco beliefs among youth using item response theory models. Drug and Alcohol Dependence. 2002;68:21–39. doi: 10.1016/S0376-8716(02)00213-2 [ DOI ] [ PubMed ] [ Google Scholar ]
• 18. Ferro MA, Boyle MH, Scott JG, Dingle K. The child behavior checklist and youth self-report in adolescents with epilepsy: testing measurement invariance of the attention and thought problems subscales. Epilepsy & Behavior. 2014;31:34–42. doi: 10.1016/j.yebeh.2013.11.009 [ DOI ] [ PubMed ] [ Google Scholar ]
• 19. Van Lieshout RJ, Boyle MH, Schmidt L, Saigal S, Ferro MA. Measurement Invariance Across Parent and Self-Ratings of Extremely Low Birth Weight Survivors and Normal Birth Weight Controls in Childhood and Adolescence on the Child Behavior Checklist and Youth Self-Report. J Pediatr Psychol. 2015;40(8):825–833. doi: 10.1093/jpepsy/jsv043 [ DOI ] [ PubMed ] [ Google Scholar ]
• 20. Jeon M, Jin IH, Schweinberger M, Baugh S. Mapping unobserved item-response interactions: A latent space item response model with interaction maps. Psychometrika. 2021;. doi: 10.1007/s11336-021-09762-5 [ DOI ] [ PubMed ] [ Google Scholar ]
• 21. Hoff P, Raftery A, Handcock MS. Latent space approaches to social network analysis. Journal of the American Statistical Association. 2002;97:1090–1098. doi: 10.1198/016214502388618906 [ DOI ] [ Google Scholar ]
• 22. Sewell DK, Chen Y. Latent space models for dynamic networks. Journal of the American Statistical Association. 2015;110:1646–1657. doi: 10.1080/01621459.2014.988214 [ DOI ] [ Google Scholar ]
• 23. Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer. 2009;42(8):30–37 doi: 10.1109/MC.2009.263 [ DOI ] [ Google Scholar ]
• 24. Rendle S. FactorizationFactorization machines. In: 2010 IEEE International conference on data mining. IEEE; 2010. p. 995–1000.
• 25. Alexander LM, Escalera J, Ai L, Andreotti C, Febre K, Mangone A, et al. An open resource for transdiagnostic research in pediatric mental health and learning disorders. Scientific Data. 2017;4:1–26. doi: 10.1038/sdata.2017.181 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 26. Pandolfi V, Magyar CI, Dill CA. An initial psychometric evaluation of the CBCL 6–18 in a sample of youth with autism spectrum disorders. Research in Autism Spectrum Disorders. 2012;6:96–108. doi: 10.1016/j.rasd.2011.03.009 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 27. Read KL, Settipani CA, Peterman J, Kendall PC, Compton S, Piacentini J, et al. Predicting anxiety diagnoses and severity with the CBCL-A: improvement relative to other CBCL scales? Journal of Psychopathology and Behavioral Assessment. 2015;37:100–111. doi: 10.1007/s10862-014-9439-9 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 28. Sood BG, Bailey BN, Covington C, Sokol RJ, Ager J, Janisse J, et al. Gender and alcohol moderate caregiver reported child behavior after prenatal cocaine. Neurotoxicology and Teratology. 2005;27:191–201. doi: 10.1016/j.ntt.2004.10.005 [ DOI ] [ PubMed ] [ Google Scholar ]
• 29. Tan TX, Dedrick RF, Marfo K. Factor structure and clinical implications of child behavior checklist/1.5–5 ratings in a sample of girls adopted from China. Journal of Pediatric Psychology. 2007;32:807–818. doi: 10.1093/jpepsy/jsm025 [ DOI ] [ PubMed ] [ Google Scholar ]
• 30. Choi S, Cha SH, Tappert C. A Survey of Binary Similarity and Distance Measures. J Syst Cybern Inf. 2009;8. [ Google Scholar ]
• 31. Chalmers RP. mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software. 2012;48:1–29. doi: 10.18637/jss.v048.i06 [ DOI ] [ Google Scholar ]
• 32. McKinley RL, Mills CN. A comparison of several goodness-of-fit statistics. Applied Psychological Measurement. 1985;9(1):49–57. doi: 10.1177/014662168500900105 [ DOI ] [ Google Scholar ]
• 33. Holland PW, Leinhardt S. Transitivity in structural models of small groups. Comparative Group Studies. 1971;2:107–124. doi: 10.1177/104649647100200201 [ DOI ] [ Google Scholar ]
• 34. Watts DJ, Strogatz SH. Collective dynamics of ‘small-world’networks. nature. 1998;393:440–442. doi: 10.1038/30918 [ DOI ] [ PubMed ] [ Google Scholar ]
• 35. Chib S, Greenberg E. Understanding the metropolis-hastings algorithm. The American Statistician. 1995;49:327–335. doi: 10.1080/00031305.1995.10476177 [ DOI ] [ Google Scholar ]
• 36. Gower JC. Generalized procrustes analysis. Psychometrik. 1975;40:33–51. doi: 10.1007/BF02291478 [ DOI ] [ Google Scholar ]
• 37. Sibson R. Studies in the robustness of multidimensional scaling: Perturbational analysis of classical scaling. Journal of the Royal Statistical Society: Series B (Methodological). 1979;41(2):217–229. [ Google Scholar ]
• 38. Tanner MA, Wong WH. The Calculation of Posterior Distributions by Data Augmentation (with discussion). Journal of the American Statistical Association. 1987;82:528–550. doi: 10.2307/2289463 [ DOI ] [ Google Scholar ]
• 39. Little RJ, Rubin DB. Statistical analysis with missing data. vol. 793. John Wiley & Sons; 2019. [ Google Scholar ]
• 40. Pastore M, Calcagnì A. Measuring distribution similarities between samples: a distribution-free overlapping index. Frontiers in Psychology. 2019;10:1089. doi: 10.3389/fpsyg.2019.01089 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 41. Queen JP, Quinn GP, Keough MJ. Experimental design and data analysis for biologists. Cambridge university press; 2002. [ Google Scholar ]
• 42. Neyman J, Pearson ES. The testing of statistical hypotheses in relation to probabilities a priori. In: Mathematical proceedings of the Cambridge philosophical society. vol. 29. Cambridge University Press; 1933. p. 492–510. [ Google Scholar ]
• 43. Yerushalmy J. Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Public Health Reports (1896-1970). 1947; p. 1432–1449. doi: 10.2307/4586294 [ DOI ] [ PubMed ] [ Google Scholar ]
• 44. Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters. 2006;27:861–874. doi: 10.1016/j.patrec.2005.10.010 [ DOI ] [ Google Scholar ]
• 45. Powers DM. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:201016061. 2020.
• 46. Warnick EM, Bracken MB, Kasl S. Screening efficiency of the Child Behavior Checklist and Strengths and Difficulties Questionnaire: A systematic review. Child and Adolescent Mental Health. 2008;13:140–147. doi: 10.1111/j.1475-3588.2007.00461.x [ DOI ] [ PubMed ] [ Google Scholar ]
• 47. Nelson EC, Hanna GL, Hudziak JJ, Botteron KN, Heath AC, Todd RD. Obsessive-compulsive scale of the child behavior checklist: specificity, sensitivity, and predictive power. Pediatrics. 2001;108:e14–e14. doi: 10.1542/peds.108.1.e14 [ DOI ] [ PubMed ] [ Google Scholar ]
• 48. Thissen DE, Wainer HE. Lawrence Erlbaum Associates Publishers; 2001.
• 49. Birnbaum AL. Some latent trait models and their use in inferring an examinee’sability. Statistical theories of mental test scores. 1968;
• 50. Pirinen T, Kolho KL, Simola P, Ashorn M, Aronen E. Parent–adolescent agreement on psychosocial symptoms and somatic complaints among adolescents with inflammatory bowel disease. Acta Paediatrica. 2012;101:433–437. doi: 10.1111/j.1651-2227.2011.02541.x [ DOI ] [ PubMed ] [ Google Scholar ]
• 51. Begovac I, Rudan V, Skočić M, Filipović O, Szirovicza L. Comparison of self-reported and parent-reported emotional and behavioral problems in adolescents from Croatia. Collegium antropologicum. 2004;28:393–401. [ PubMed ] [ Google Scholar ]
• 52. Van Der Ende J, Verhulst FC. Informant, gender and age differences in ratings of adolescent problem behaviour. European Child & Adolescent Psychiatry. 2005;14:117–126. doi: 10.1007/s00787-005-0438-y [ DOI ] [ PubMed ] [ Google Scholar ]
• 53. McCullagh P, Nelder JA. Generalized linear models. Routledge; 2019. [ Google Scholar ]
• 54. Angold A, Weissman MM, John K, Merikancas KR, Prusoff BA, Wickramaratne P, et al. Parent and child reports of depressive symptoms in children at low and high risk of depression. Journal of Child Psychology and Psychiatry. 1987;28(6):901–915. doi: 10.1111/j.1469-7610.1987.tb00678.x [ DOI ] [ PubMed ] [ Google Scholar ]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Data availability statement.

• View on publisher site
• PDF (1.8 MB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Comparison of Parent Report and Direct Assessment of Child Skills in Toddlers

Lauren e miller , m.s., kayla a perkins , b.a., yael g dai , b.a., deborah a fein , ph.d..

• Author information
• Article notes
• Copyright and License information

Corresponding Author: Lauren E. Miller, M.S., Department of Psychological Sciences, University of Connecticut, 406 Babbidge Road, Unit 1020, Storrs, CT 06269, USA; [email protected] ; Phone: (513) 884–4019; Fax: (860) 486–3680

Present Address: Rhode Island Consortium for Autism Research & Treatment, Bradley Hospital, 1011 Veteran’s Memorial Parkway, East Providence, RI 02915, USA

Issue date 2017 Sep.

There are unique challenges associated with measuring development in early childhood. Two primary sources of information are used: parent report and direct assessment. Each approach has strengths and weaknesses, particularly when used to identify and diagnose developmental delays. The present study aimed to evaluate consistency between parent report and direct assessment of child skills in toddlers with and without Autism Spectrum Disorder (ASD) across receptive language, expressive language, and fine motor domains.

109 children were evaluated at an average age of two years; data on child skills were collected via parent report and direct assessment. Children were classified into three groups (i.e., ASD, Other Developmental Disorder, or Typical Development) based on DSM-IV-TR diagnosis. Mixed design ANOVAs, with data source as a within subjects factor and diagnostic group as a between subjects factor, were used to assess agreement. Chi square tests of agreement were then used to examine correspondence at the item level.

Results suggested that parent report of language and fine motor skills did not significantly differ from direct assessment, and this finding held across diagnostic groups. Item level analyses revealed that, in most cases of significant disagreement, parents reported a skill as present, but it was not seen on direct testing.

Conclusions

Results indicate that parents are generally reliable reporters of child language and fine motor abilities in toddlerhood, even when their children have developmental disorders such as ASD. However, the fullest picture may be obtained by using both parent report and direct assessment.

Keywords: parent report, direct assessment, toddlers, child ability, autism spectrum disorder

Introduction

Early detection of developmental delays, including autism spectrum disorder (ASD), has been shown to facilitate earlier intervention and better outcomes ( Orinstein et al., 2014 ; Rogers & Vismara, 2008 ). These findings have precipitated greater focus on improving routine developmental screening and evaluation rates for at risk young children and the general population. There are, however, unique challenges associated with obtaining accurate developmental data in early childhood, as toddlers tend to respond differently across contexts ( Sachse & Von Suchodoletz, 2008 ). Currently, two primary methods are used to evaluate child development: parent report and direct assessment ( Luyster, Kadlec, Carter, & Tager-Flusberg, 2008 ; Nordahl-Hansen, Kaale, & Ulvund, 2014 ). There is limited consensus regarding which method offers the best picture of child ability, as each approach has strengths and limitations.

Parents are an important source of information regarding child skills deficits and atypical behaviors, as they are uniquely positioned to observe and interact with children across various situations ( Sasche & Von Suchodoletz, 2008 ). Thus, they may provide data regarding child development that could otherwise not be measured in a clinical setting. Parent report, too, is not subject to issues with child motivation and cooperation that frequently occur in testing situations ( Nordahl-Hansen et al., 2014 ). Furthermore, parent report measures are an increasingly attractive option for detecting developmental delays, as they are quick, easy to use, and cost effective relative to formal evaluation ( Nordahl-Hansen et al., 2014 ; Sasche & Von Suchodoletz, 2008 ). Based on these strengths, the pediatric healthcare system is moving toward greater involvement of parents in the process of identifying early developmental delays ( Feldman et al., 2005 ; Nordahl-Hansen, Kaale & Ulvund, 2013 ). Specifically, routine developmental screening via parent report is increasingly being used to identify at risk children, in accordance with American Academy of Pediatrics recommendations ( Emerson, Morrell, & Neece, 2016 ; Johnson & Myers, 2007 ).

However, concerns have been raised about the accuracy of parent report ( Ozonoff et al., 2011 ; Tomasello & Mervis, 1994 ; Zapolski & Smith, 2013 ). Although they are often keen observers of their child’s early development, parents generally lack expertise in evaluating developmental milestones, sometimes making it difficult to report reliably ( Nordahl-Hansen et al., 2014 ). Alternatively, parents may attend more to challenging or unusual behaviors, thus introducing bias into their reporting ( Zapolski & Smith, 2013 ). In addition, parents may overestimate child abilities because of a reluctance to acknowledge that their child has a delay ( Ozonoff et al., 2011 ). As parent report necessarily reflects a parent’s perception of child functioning, it is considered subjective rather than objective ( Sachse & Von Suchodoletz, 2008 ).

In contrast, standardized tests administered by a trained tester are, by definition, objective. Direct testing should thus offer an unbiased picture of child development, as each child is evaluated in a highly similar (i.e., standard) manner, and data is based on observations made by a professional with expertise and experience in assessing early development ( Nordahl-Hansen et al., 2014 ; Sachse & Von Suchodoletz, 2008 ). However, in an unfamiliar clinical setting, children tend to behave differently than they may in familiar settings. Issues of behavioral noncompliance and poor attention and motivation commonly seen in young children may influence test results, potentially limiting validity ( Nordahl-Hansen et al., 2014 ). Particularly when evaluating language skills, children may produce fewer utterances when outside of their everyday communicative activities, thus limiting the validity of formal assessment. In addition, formal developmental and diagnostic testing can be time intensive and cost ineffective ( Sasche & Von Suchodoletz, 2008 ). Furthermore, some critics suggest that standardized tests are inappropriate for children with ASD and global delays, as they often measure skills that are too advanced for the child ( Luyster et al., 2008 ). Thus, it is unclear whether direct testing, long regarded as the gold standard, best estimates child abilities in very young children, particularly those with developmental deficits.

To address concerns with both parent report and direct assessment as outlined above, efforts to determine the ideal approach to assessing child development have focused on evaluating agreement between these two primary sources of information. However, the existing literature relies on a variety of methodological approaches and definitions of ‘parent report’ and ‘direct assessment.’ Reliability of parent recall of early developmental milestones and health events, as compared to medical records, indicates that parents are generally good reporters of gross motor milestones (e.g., age at first steps) and medical outcomes (e.g., birth weight, illnesses during early infancy), but that their ability to recall early language milestones is lower ( Majnemer & Rosenblatt, 1994 ; Pless & Pless, 1995 ). Most often, though, parent report has been used to refer to parent completed screening tools or developmental checklists, which have then been compared to standardized developmental evaluation measures ( Bodnarchuk & Eaton, 2004 ; Luyster et al., 2008 ; Nordahl-Hansen et al., 2014 ; Sachse & Von Suchodoletz, 2008 ; Voigt et al., 2007 ) or, in the case of ASD symptoms, home videos ( Ozonoff et al., 2011 ). Findings suggest that parents are adequate reporters of child language ability ( Nordahl-Hansen et al., 2014 ), although agreement between parent report and direct assessment is stronger for speech production than for comprehension ( Luyster et al., 2008 ; Sachse & Von Suchodoletz, 2008 ; Tomasello & Mervis, 1994 ). This discrepancy may be due to the generally low reliability of measures of early receptive language ( Sachse & Von Suchodoletz, 2008 ). Alternatively, for children with ASD, discrepancies between parent report and direct testing may be an artifact of their difficulty generalizing language across contexts ( Nordahl-Hansen et al., 2014 ). Parents demonstrate good concordance with trained assessors when evaluating gross motor milestones ( Bodnarchuk & Eaton, 2004 ), yet limited research is available on agreement for fine motor skills in early childhood.

The present study aims to evaluate consistency between two sources of information, parent report and direct assessment, when measuring child development in three domains: receptive language, expressive language, and fine motor skills. Here, we define ‘parent report’ as information obtained from a parent during a structured interview, the Vineland Adaptive Behavior Scales, Second Edition (Vineland), which we then compare to results of direct testing using the Mullen Scales of Early Learning (Mullen). Although the basis of information about a child’s development in a parent report is different from that in direct assessment, in that the Vineland aims to assess what skills a child uses in his or her daily life and the Mullen aims to measure a child’s competence, both approaches yield similar information at a content level. A similar method of comparing parent report and direct testing of child language skills using the Vineland and Mullen was employed by Luyster et al. (2008) to study development in toddlers with ASD. We expand on this approach by investigating agreement between parent report and direct testing of early developmental outcomes in children with ASD, other developmental disorders, and typical development. The aims of the current study are threefold:

We examine consistency between Vineland and Mullen scores in the domains of receptive language, expressive language, and fine motor skills. Based on prior research showing good agreement between parent report and direct testing of language production, but somewhat weaker agreement when assessing language comprehension ( Luyster et al., 2008 ; Nordahl-Hansen et al., 2014 ; Sachse & Von Suchodoletz, 2008 ; Tomasello & Mervis, 1994 ), we expected to find a similar pattern in our data. We also hypothesized that parent report of fine motor skills would be generally consistent with standardized assessment, as similar findings exist for gross motor functioning ( Bodnarchuk & Eaton, 2004 ).

We assess the impact of child diagnosis (i.e., ASD, other developmental disorders, or typical development) on consistency between Vineland and Mullen scores. As children with ASD often have difficulty generalizing skills and behavior across contexts ( Nordahl-Hansen et al., 2014 ), we hypothesized that agreement between parent report and direct testing would be weaker for children with ASD as compared to those toddlers with other developmental delays or typical development.

We explore agreement between parent report and direct assessment at the individual item level in order to evaluate parent reporting of particular developmental skills. Although we consider these analyses exploratory, we expected to see the greatest discrepancies for items measuring receptive language skills, based on prior research findings (see Aim 1).

Participants

Participants were 109 children drawn from a larger federally funded project at the University of Connecticut. The aims of the original study focused on validating the Modified Checklist for Autism in Toddlers, Revised (M-CHAT-R), a population based screening tool used to detect ASD in young children ( Robins et al., 2014 ). All participants screened positive on the M-CHAT-R and follow-up phone interview, indicating risk for ASD, between the ages of 16 and 30 months and received a developmental evaluation at the average age of two years. Exclusion criteria included significant sensory or motor impairments (e.g., blindness, severe cerebral palsy) that would negatively impact the participant’s ability to complete testing, as well as documented receptive and expressive language functioning (i.e., Mullen scores) below a 12 month level, which resulted in insufficient item level data for comparative analyses. Participants were also excluded if they had an older sibling with an ASD diagnosis, as infant sibling research has shown that having another child with ASD impacts parent report of developmental concerns ( McMahon et al., 2007 ; Ozonoff et al., 2009 ).

Participants were classified into three groups based on DSM-IV-TR diagnosis. All diagnoses were assigned based on clinical best estimate judgment of symptoms, incorporating behavioral observation, developmental history, and testing data. The ASD group ( n = 28) was composed of children with Autistic Disorder ( n = 15) and Pervasive Developmental Disorder–Not Otherwise Specified ( n = 13). The other developmental disorder group ( n = 57) was composed of children with global developmental delay ( n = 34), developmental language disorder ( n = 22), and reactive attachment disorder ( n = 1). The typically developing group ( n = 24) was composed of children who were either typically developing ( n = 13) or who exhibited minor delays that were insufficient to qualify for a DSM-IV-TR diagnosis at the time of evaluation ( n = 11). Participant demographic characteristics are summarized in Table 1 .

Participant Demographic Characteristics

Note. ASD = autism spectrum disorder; DD = other developmental disorder; TD = typically developing.

Children who screened positive on the M-CHAT-R and follow-up phone interview were offered a free developmental and diagnostic evaluation, which was completed by a licensed clinical psychologist or developmental pediatrician and a doctoral student in clinical psychology. During the evaluation, a clinician conducted a developmental history and clinical interview with the child’s parent using the Vineland and a structured interview for the diagnosis of ASD designed by the research team. The child’s cognitive level and autism diagnostic status and severity were assessed using the Mullen and Autism Diagnostic Observation Schedule, Generic (ADOS; Lord, Risi, & Lambrecht, 2000 ), respectively. Evaluations lasted approximately three hours, including a feedback session in which diagnosis and recommendations were reviewed with the child’s parent, with a comprehensive written report to follow. All children were evaluated in their primary language; 98 children were evaluated in English, and 11 were evaluated in Spanish.

The University of Connecticut Institutional Review Board (IRB) approved this study, which was carried out in accordance with the ethical standards of the 1964 Declaration of Helsinki and its later amendments (i.e., 2000 revision). At the time of screening, parents were given an information sheet describing the larger original study. Consent for participation in the research project was indicated by completion of the M-CHAT-R, as a waiver of written consent was granted by the IRB. Written informed consent was obtained from parents at the time of evaluation, prior to inclusion in the current study.

The Vineland Adaptive Behavior Scales, Second Edition: Survey Interview Form (Vineland; Sparrow, Cicchetti, & Balla, 2005 ) is a semi-structured caregiver interview that assesses adaptive behaviors (i.e., how a child functions in his or her daily life) in the domains of socialization, communication, daily living, and motor skills. Age equivalent scores on the receptive and expressive language and fine motor subscales were used in the current study. Internal consistency on the Vineland, as measured by split half reliability, is good to excellent, ranging from .81 to .96 for the subscales used and ages tested in the current study. Inter-rater reliability is fair, averaging .70 for subscales for the normative sample aged birth to six years ( Sparrow et al., 2005 ).

The Mullen Scales of Early Learning (Mullen; Mullen, 1995 ) is a developmental assessment of cognitive, motor, and language abilities in young children. The current study used age equivalent scores on the receptive and expressive language and fine motor scales. Average estimates of internal consistency for the Mullen are satisfactory, ranging from .75 to .83 across all scales, and inter-rater reliability is considered strong, ranging from .91 to .99 ( Mullen, 1995 ).

The Autism Diagnostic Observation Schedule, Generic (ADOS; Lord et al., 2000 ) is a semi-structured observational assessment designed to measure symptoms of ASD in toddlerhood through adulthood. The ADOS includes four separate modules based on an individual’s expressive language level and chronological age. The current study used Module 1, designed for pre-verbal children and those with single words. Inter-rater reliability on the ADOS is considered good across all domains: social (.93), communication (.84), social communication (.92), and RRBs (.82) ( Lord et al., 2000 ). ADOS scores and observations were included in diagnostic decision making, but were not primary outcome variables of interest in the current study.

Participant performance on primary study measures (i.e., Vineland and Mullen) is summarized in Table 2 .

Average Vineland and Mullen Age-Equivalent Scores

All statistical analyses were conducted using IBM SPSS Statistics for Windows, Version 22.0 ( IBM Corporation, 2013 ). Primary analyses included mixed design analysis of variance (ANOVA), with data source (i.e., parent report, direct assessment) as a within subjects factor and diagnostic group (i.e., ASD, other developmental disorder, and typical development) as a between subjects factor, to examine consistency between parent report and direct assessment of child ability. Separate mixed design ANOVAs were run for each developmental domain. The decision to use a mixed design ANOVA was based on the need to compare differences between groups split on two factors: a within subjects factor in which all participants, serving as their own matched pair, were measured in two conditions (i.e., sources of information), and a between subjects factor in which participants were classified separately based on DSM-IV-TR diagnosis. This analytic approach allowed us to examine consistency between parent report and direct testing, while also allowing for investigation of the impact of child diagnosis on agreement between the Vineland and Mullen. Assumptions of normality, homogeneity of variances, and sphericity were met, and no significant outliers were identified in our sample.

Secondary analyses included Chi square tests of agreement on individual matched pairs of items from both primary study measures, to determine agreement at the level of specific developmental skills. In cases where assumptions of Chi square testing were violated due to small sample sizes (i.e., less than five cases in a contingency table cell), Fisher’s Exact test was used. In preparation for these item level analyses, corresponding items on the Vineland and Mullen measuring similar developmental skills were selected. Item scores on the Vineland were recoded to 0 (fail) or 1 (pass) by considering raw scores of 1 or 2 as passes, indicating that a child at least “sometimes” performed the measured skill, as this suggests emerging competence. Raw scores of 0 were maintained as 0 (fail), as this suggests lack of competence. Similarly, Mullen raw item scores were recoded to 0 (fail) or 1 (pass) to match scoring thresholds in corresponding Vineland items. For example, Vineland fine motor item 13, “turns book or magazine pages one by one,” corresponds to Mullen fine motor item 14, “turns pages in a book.” On this particular Mullen item, a raw score of 0 indicates that a child cannot turn pages in a book, a raw score of 1 indicates that a child turns pages in a book several at a time, and a raw score of 2 indicates that a child turns pages in a book one at a time. Thus, for this item to be consistent with the Vineland, a raw score of 0 or 1 on the Mullen would be recoded as 0 (fail), as the child cannot turn pages in a book one by one, and a raw score of 2 would be recoded as 1 (pass), as the child can turn pages one at a time. Given the small sample sizes in the ASD and typical development groups, item level analyses were conducted on the full sample instead of separately for each diagnostic group.

An alpha level of .05 was adopted for all statistical tests.

Effect of Demographics on Agreement

Difference scores were calculated by subtracting Vineland subscale age equivalent scores from corresponding Mullen scale age equivalent scores (see Tables 3 and 4 for descriptions of item content). One-way ANOVAs were then run to determine whether the language in which an evaluation was conducted (English or Spanish) had an effect on agreement. Results indicated no significant main effect of evaluation language on agreement between data sources (parent report vs. direct assessment) in the receptive language domain, F (1, 107) = .086, p = .770, η 2 < .001, expressive language domain, F (1, 107) = 3.572, p = .061, η 2 = .032, or fine motor domain, F (1, 107) = 1.907, p = .170, η 2 = .018.

Chi Square Analyses of Agreement on Item Pairs

Note. Where Chi square statistics are absent, Fisher’s Exact p -values are reported due to violations of Chi square assumptions. A significant p -value indicates significant disagreement between methods. For items using Fisher’s Exact test, an odds ratio (OR) was used as a measure of effect size, except in cases where one cell in the contingency table equaled 0.

Categorization of Evaluator Response by Item

To assess whether maternal education level (high education = some college, Bachelor’s degree, advanced degree; low education = some high school, high school diploma/test of general education development (GED), vocational/technical school) impacted agreement in each domain, a second set of one-way ANOVAs was conducted. Results indicated no significant main effect of maternal education on agreement in the receptive language domain, F (1, 106) = .083, p = .774, η 2 < .001, expressive language domain, F (1,106) = .545, p = .462, η 2 = .005, or fine motor domain, F (1, 106) = .345, p = .558, η 2 = .003.

Effect of Data Source on Domain Scores

Receptive language ability.

A main effect of data source was not significant, F (1, 106) = .765, p = .384, η p 2 = .007, suggesting that parents did not significantly differ in their ratings of child receptive language ability compared to direct testing. Additionally, no interaction effect was found between source of information and diagnostic group, F (2, 106) = .518, p = .597, η p 2 = .010. A significant main effect of diagnostic group was found in the expected direction, F (2, 106) = 18.013, p < .001, η p 2 = .254, such that scores were lower for children in the ASD group than for children in the DD ( p = .047) and TD ( p < .001) groups ( Table 2 ). Receptive language scores were also significantly lower for children in the DD group compared to those in the TD group ( p < .001).

Expressive language ability

A main effect of data source was not significant, F (1, 106) = .067, p = .796, η p 2 = .001. Furthermore, an interaction between source of information and diagnostic group was not significant, F (2, 106) = 1.044, p = .356, η p 2 = .019. Results did reveal a significant main effect of diagnostic group, F (2, 106) = 31.467, p < .001, η p 2 = .373. Expressive language scores were higher for children in the TD group than for children in the ASD ( p < .001) and DD ( p < .001) groups, but scores did not differ significantly between children in the ASD and DD groups ( p = .156) ( Table 2 ).

Fine motor ability

A main effect of data source was not significant, although results did trend in that direction, with parents reporting slightly higher fine motor abilities than seen on direct assessment, F (1, 106) = 3.880, p = .051, η p 2 = .035. No significant interaction was found between source of information and diagnostic group, F (2, 106) = .063, p = .939, η p 2 = .001. Results revealed a significant main effect of diagnostic group, F (2, 106) = 7.421, p = .001, η p 2 = .123, with fine motor scores higher for children in the TD group than for children in the ASD ( p = .001) and DD ( p = .004) groups ( Table 2 ). Scores did not differ significantly between children in the ASD and DD groups ( p = .640).

Item-Level Comparison of Agreement

To determine agreement at the item level, a series of Chi square tests of agreement were performed on individual matched item pairs across diagnostic groups ( Table 3 ). Table 4 describes the direction of item level agreement and disagreement between parents and direct assessment. Percent agreement in Table 3 is defined as the sum of “Both” and “Neither” in Table 4 .

Overall, item level analyses revealed somewhat mixed findings. Percent agreement on items assessing basic abilities (e.g., “Listens and looks”) was strong ( Table 3 ). However, there are key limitations to interpreting our kappa values: nearly all scores for easy items accrued in one quadrant of the Chi square contingency table, indicating that both parents and direct testing reported that a child could perform the skill. This contributed to misleadingly low kappa values that do not accurately reflect the strength of agreement between data sources.

For items assessing more advanced comprehension skills (e.g., following instructions), there was some disagreement between parent report and direct testing. For following one- and two-step directions, approximately equal numbers of children could perform the tasks on parent report only or testing only ( Table 4 ). However, on other items measuring more complex developmental skills on receptive language, expressive language, and fine motor domains, there was significant disagreement ( Table 3 ). For these items, parents mostly reported that the child had the skill, but it was not seen on direct testing (i.e., parents reported that the child could name three or 10 objects, recognize three or more body parts, identify pictures of named objects, turn pages in a book, stack four blocks, and use a hand-wrist twisting motion more often than seen on direct assessment) ( Table 4 ).

The current study aimed to evaluate consistency between two primary sources of information about early childhood development, parent report and direct assessment. Specifically, we examined agreement between the Vineland, a parent interview, and the Mullen, a standardized developmental measure, across the domains of receptive language, expressive language, and fine motor skills. To further understand the influence of child diagnosis on agreement, we assessed consistency between data sources in a sample of two-year-old children with ASD, other developmental disorders, and typical development. Finally, we explored consistency at the individual item level to determine if certain skills are more subject to reporter bias, or disagreement. Overall, our results suggest that parent report of child ability does not differ significantly from direct assessment, and this finding is generally stable across diagnostic groups. Taken together, these findings suggest that both parent report and direct testing are appropriate measures of child developmental functioning.

We first aimed to investigate agreement between parent report and direct assessment of child receptive language, expressive language, and fine motor skills. On ANOVA, no significant main effect of data source was found, suggesting that evaluation of overall skills in each domain did not differ based on source of information (i.e., parent report vs. direct testing). This finding is largely consistent with prior research in this area, particularly when assessing language production and gross motor functioning ( Bodnarchuk & Eaton, 2004 ; Luyster et al., 2008 ; Nordahl-Hansen et al., 2014 ; Sachse & Von Suchodoletz, 2008 ). However, contrary to our hypothesis, we did not find weaker agreement when assessing language comprehension ability. Instead, in our sample, parent report of receptive language skills was roughly equivalent to scores demonstrated on direct testing ( Table 2 ). This finding contrasts with prior evidence of relatively weak agreement between parent report and direct assessment of language understanding in young children with ASD ( Luyster et al., 2008 ; Nordahl-Hansen et al., 2014 ), as well as in late talkers and typically developing toddlers ( Sachse & Von Suchodoletz, 2008 ). It is possible that our reliance on the Vineland, a semi structured parent interview, instead of a parent report checklist (e.g., MacArthur-Bates Communicative Development Inventories, which was used in Luyster et al., 2008 , Nordahl-Hansen et al., 2014 , and Sachse & Von Suchodoletz, 2008 ) contributed to improved consistency . Even so, our data showing good agreement suggests that parents are generally reliable reporters of both child expressive and receptive language abilities.

Although no significant disagreement between parent report and direct assessment of fine motor skills was found on ANOVA, results indicated an effect of data source trending toward significance, such that parents reported slightly higher fine motor skills than seen on direct assessment. However, as shown in Table 2 , this discrepancy was quite small, with Vineland age equivalent scores only approximately one month higher than scores on the Mullen. This small inconsistency could result from parents assuming that a child can perform a developmentally age appropriate motor task without having actually observed it, or it could be a result of child unwillingness to perform these tasks in the evaluation setting due to disinterest in testing materials, frustration, or inability to comprehend testing demands. Fairly limited literature exists on agreement between parent report and direct testing of fine motor skills in early childhood, and our hypothesis in this domain was based on findings suggesting good consistency when evaluating gross motor skills. It is possible that parents attend more to whole body (i.e., gross motor) movements, such as sitting, standing, and walking, than finger and hand (i.e., fine motor) movements, as acquisition of gross motor milestones are seen as key signs of whether or not a toddler is developing typically ( Bodnarchuk & Eaton, 2004 ). This may result in less accurate parent reporting of fine motor skills. However, it is equally likely that early developmental testing of fine motor functioning is particularly subject to problems of child noncompliance, as successful testing of emerging fine motor skills depends on a child’s interest in and willingness to manipulate testing stimuli.

We then aimed to evaluate the impact of child diagnosis on consistency between parent report and direct assessment. As expected, we found that, across data sources, children in the ASD group were lower functioning than those in the typical development group, with large effect sizes, with toddlers with other developmental disorders generally falling in between ( Table 2 ). However, on ANOVA, we did not find a significant interaction between data source (i.e., parent report vs. direct testing) and diagnostic group, suggesting that a child’s diagnostic status did not impact agreement. This finding contradicted our hypothesis, as, based on the tendency of children with ASD to have difficulty generalizing skills across contexts ( Nordahl-Hansen et al., 2014 ), we predicted greater disagreement between parent report and direct testing for children with ASD. Given the low functional level (i.e., language skills developed at a 12 month age equivalence) of children in our ASD group, it is possible that they are not yet experiencing the characteristic difficulties with generalizing skills often seen in older children with ASD ( Nordahl-Hansen et al. (2014) examined four-year-olds). Overall, our findings suggest that even parents of children with developmental delays, including ASD, can report accurately on their child’s functioning, further supporting the utility of parent report measures for the identification of delays.

We finally aimed to explore agreement between data sources at the individual item level, to determine if specific developmental skills are more sensitive to reporter bias. Although these analyses were exploratory, we hypothesized that we would find greater disagreement for items measuring receptive language ability, given prior research indicating weaker agreement when assessing speech understanding ( Luyster et al., 2008 ; Nordahl-Hansen et al., 2014 ; Sachse & Von Suchodoletz, 2008 ; Tomasello & Mervis, 1994 ). Overall, item level analyses revealed somewhat mixed findings. That is, for items measuring basic skills, even those requiring comprehension of language, results suggested very good agreement, with most parents and clinicians (i.e., through direct assessment) reporting that a child could perform these skills. Yet, for slightly more challenging receptive language items, specifically those assessing a child’s ability to follow instructions, there was some disagreement, even though the majority of parents and clinicians agreed that a child could follow one-step instructions and could not follow two-step instructions. For these particular skills, the percentage of only parents or only testing indicating that a child could perform the items was roughly equivalent, indicating no systematic pattern of disagreement ( Table 4 ). A more systematic pattern of disagreement emerged for items tapping more complex skills across all three functional domains. In these cases, more parents endorsed a child’s ability to perform a skill than seen on direct testing, for almost all items.

The patterns of disagreement shown in our item level analyses may reflect underestimation of abilities by testing, overestimation by parents, or differences in child behavior across settings. That is, if a clinician documented that a child performed a skill on direct testing of that skill, then the child clearly has the skill in his or her repertoire. If, however, the child did not perform the item on direct assessment, the parent may still be accurate in reporting that the behavior occurs at home. Alternatively, this disagreement could be related to unexamined child (e.g., age, temperament) or evaluator (e.g., gender, clinician-child interaction style) characteristics. Overall, these findings reflect the challenges associated with assessing child development in the toddler age range. That is, while basic skills appear to be easy to quantify using either parent report or direct testing, accurate evaluation of more complex skills and behaviors is likely more dependent on the standardized measure used, particularly whether or not its stimuli are attractive and attention grabbing for young children, as well as a parent’s ability to observe and recollect very specific behaviors (e.g., stacking four blocks) reliably.

Limitations and Future Directions

Due to several limitations, the results of the current study should be interpreted with some caution. Most notably, due to small within groups numbers, item level analyses were conducted on the full sample, limiting our ability to explore patterns of data source agreement or disagreement at the item level within certain clinical populations, such as children with ASD. As such, our item level analyses should be considered exploratory, and future research is needed to examine specific skills that are under or over reported, as well as the influence of child diagnosis on reporting of particular developmental skills. Additionally, as noted previously, specific child and evaluator characteristics, as well as comprehensive parent demographic information (e.g., age), were not directly measured and therefore could not be controlled for in analyses. Due to the number of different clinicians involved in the present study, evaluator characteristics may play a role. Future research should examine the impact of evaluator characteristics on agreement between parent report and direct testing, as these variables may influence the way in which parents, and children, respond in the evaluation setting.

Furthermore, as participants in the current project were drawn from a larger study validating the M-CHAT-R ( Robins et al., 2014 ), our findings may not generalize to all typically developing children, particularly those without a history of any developmental concerns. Given the goals of the larger M-CHAT-R validation study, only children who screened positive on the M-CHAT-R and follow up interview, thus indicating risk for ASD or another developmental disorder, were evaluated. We did not evaluate children who screened negative on the M-CHAT-R, nor did we actively recruit a typically developing sample for comparison. In addition, because some children were excluded from analyses due to very low performance on the Mullen, the current study is not representative of all lower functioning toddlers and does not capture the full range of child abilities across the autism spectrum. Results should thus be generalized with some caution.

In addition, our measure of parent report (Vineland) is a clinician-administered parent interview, rather than a parent checklist done independently. Our findings may not generalize to parent rating scales, including developmental screening tools and child behavior checklists. Further research investigating the role of format (i.e., interview or rating scale) on patterns of disagreement between sources of information on child development, particularly for children with ASD, is indicated. Our standardized developmental measure (Mullen), too, has its limitations. Although widely used in research with young children with ASD, the Mullen was normed in the 1980s and is thus outdated compared to other early developmental tests. Its testing stimuli also reflect its age, and the measure provides only a limited assessment of each core area of development. On the whole, standardized testing only allows for a snapshot of a child at one time point, thereby limiting the ability to capture the full range of a child’s functioning. In addition, due to the current study’s focus on child capabilities, we did not evaluate agreement on ASD symptoms, which is an important area for further study.

Finally, when a parent reports that a child has not yet attained a skill, but that skill is evidenced on testing, it is clear that the parent is in error. However, when a parent reports that a child has a skill, yet the skill is not seen on direct assessment, there is no way to determine if the parent’s report is correct without employing more invasive research and evaluation techniques, such as extended home videos. We also did not systematically ascertain whether a child’s behavior and performance during the evaluation was typical for his or her behavior at home. It is important to consider data both from parent report and direct testing when evaluating child development, particularly for young children with potential delays.

Implications

To our knowledge, this is one of few studies investigating agreement between parent report and direct assessment of child ability, particularly at the item level, in a broad sample of children with ASD, other developmental disorders, and typical development. Taken together, the results of the present study suggest that parents are generally a reliable source of information regarding child language and motor abilities, as we found good overall consistency between Vineland and Mullen scores. Additionally, parent sociodemographic factors, such as level of education and language spoken, as well as child diagnostic status, do not appear to significantly impact the accuracy of reporting. Although our findings suggest that parents are generally reliable reporters of child ability in toddlerhood, given the challenges associated with assessing skills in early development, the fullest picture of child functioning may be obtained by using both sources of information. Thus, results argue for the combined utility of parent report and direct assessment in creating an accurate composite of child behavior. As such, it appears that parents can, and should, be utilized to facilitate early detection and diagnosis of developmental delays, including ASD. As the healthcare climate shifts to further incorporate parent reporting into pediatric preventative care visits, particularly through routine developmental screening, it is increasingly necessary to improve our understanding of the accuracy of parent report of child ability and, perhaps more importantly, to identify and address any barriers (i.e., parent, child, or clinician characteristics) to the utility of parent reporting.

Highlights.

Parent report of child receptive and expressive language and fine motor skills is generally consistent with direct assessment.

Child diagnosis does not influence consistency between parent report and direct testing.

When disagreement between sources exists, parents are more likely to report a skill as present than seen on direct testing.

Acknowledgments

This study was funded by grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICD) R01 HD039961 and the Maternal and Child Health Bureau (MCHB) R40 MC00270. Neither the NICD nor the MCHB had any involvement in the study design; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to submit the article for publication.

Conflicts of Interest

Deborah Fein is part owner of the M-CHAT-R, LLC, which receives royalties from companies that incorporate the M-CHAT-R into commercial products and charge for its use. Data reported in the current article is from the freely available paper version of the M-CHAT-R. Lauren Miller, Kayla Perkins, and Yael Dai declare that they have no potential or competing conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

• Bodnarchuk JL, Eaton WO. Can parent reports be trusted? Validity of daily checklists of gross motor milestone attainment. Applied Developmental Psychology. 2004;25:481–490. [ Google Scholar ]
• Emerson ND, Morrell HE, Neece C. Predictors of Age of Diagnosis for Children with Autism Spectrum Disorder: The Role of a Consistent Source of Medical Care, Race, and Condition Severity. Journal of Autism and Developmental Disorders. 2016;46(1):127–138. doi: 10.1007/s10803-015-2555-x. [ DOI ] [ PubMed ] [ Google Scholar ]
• Feldman HM, Dale PS, Campbell TF, Coburn DK, Kurs-Lasky M, Rockette HE, Paradise JL. Concurrent and predictive validity of parent reports of child language at ages 2 and 3 years. Child Development. 2005;76(4):856–868. doi: 10.1111/j.1467-8624.2005.00882.x. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• IBM Corporation. IBM SPSS statistics for Windows, version 22.0. Armonk, NY: Author; 2013. [ Google Scholar ]
• Johnson CP, Myers SM. Identification and evaluation of children with autism spectrum disorders. Pediatrics. 2007;120(5):1183–1215. doi: 10.1542/peds.2007-2361. [ DOI ] [ PubMed ] [ Google Scholar ]
• Lord C, Risi S, Lambrecht L. The autism diagnostic observation schedule – generic. A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders. 2000;30:205–223. [ PubMed ] [ Google Scholar ]
• Luyster RJ, Kadlec MB, Carter A, Tager-Flusberg H. Language assessment and development in toddlers with autism spectrum disorders. Journal of autism and developmental disorders. 2008;38(8):1426–1438. doi: 10.1007/s10803-007-0510-1. [ DOI ] [ PubMed ] [ Google Scholar ]
• Majnemer A, Rosenblatt B. Reliability of parental recall of developmental milestones. Pediatric Neurology. 1994;10:304–308. doi: 10.1016/0887-8994(94)90126-0. [ DOI ] [ PubMed ] [ Google Scholar ]
• McMahon CR, Malesa EE, Yoder PJ, Stone WL. Parents of children with autism spectrum disorders have merited concerns about their later-born infants. Research and Practice for Persons with Severe Disabilities. 2007;32(2):154–160. [ Google Scholar ]
• Mullen EM. Mullen Scales of Early Learning. Bloomington, MN: Pearson; 1995. [ Google Scholar ]
• Nordahl-Hansen A, Kaale A, Ulvund S. Inter-rater reliability of parent and preschool teacher ratings of language in children with autism. Research in Autism Spectrum Disorders. 2013;7:1391–1396. [ Google Scholar ]
• Nordahl-Hansen A, Kaale A, Ulvund SE. Language assessment in children with autism spectrum disorder: Concurrent validity between report-based assessments and direct tests. Research in Autism Spectrum Disorders. 2014;8(9):1100–1106. [ Google Scholar ]
• Orinstein AJ, Helt M, Troyb E, Tyson KE, Barton ML, Eigsti IM, Naigles L, Fein DA. Intervention for optimal outcome in children and adolescents with a history of autism. Journal of Developmental and Behavioral Pediatrics: JDBP. 2014;35(4):247–256. doi: 10.1097/DBP.0000000000000037. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ozonoff S, Iosif AM, Young GS, Hepburn S, Thompson M, Colombi C, … Rogers SJ. Onset patterns in autism: Correspondence between home video and parent report. Journal of the American Academy of Child and Adolescent Psychiatry. 2011;50(8):796–806. doi: 10.1016/j.jaac.2011.03.012. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ozonoff S, Young GS, Steinfeld MB, Hill MM, Cook I, Hutman T, … Sigman M. How early do parent concerns predict later autism diagnosis? Journal of Developmental and Behavioral Pediatrics. 2009;30:367–375. doi: 10.1097/dbp.0b013e3181ba0fcf. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pless CE, Pless IB. How well they remember: The accuracy of parent reports. Archives of Pediatric and Adolescent Medicine. 1995;149(5):553–558. doi: 10.1001/archpedi.1995.02170180083016. [ DOI ] [ PubMed ] [ Google Scholar ]
• Robins DL, Casagrande K, Barton M, Chen CA, Dumont-Mathieu T, Fein D. Validation of the Modified Checklist for Autism in Toddlers, Revised with Follow-up (M-CHAT-R/F) Pediatrics. 2014;133(1):37–45. doi: 10.1542/peds.2013-1813. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Rogers SJ, Vismara LA. Evidence-based comprehensive treatments for early autism. Journal of Clinical Child & Adolescent Psychology. 2008;37(1):8–38. doi: 10.1080/15374410701817808. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sachse S, Von Suchodoletz W. Early identification of language delay by direct language assessment or parent report? Journal of Developmental and Behavioral Pediatrics. 2008;29:34–41. doi: 10.1097/DBP.0b013e318146902a. [ DOI ] [ PubMed ] [ Google Scholar ]
• Sparrow SS, Cicchetti DV, Balla DA. Vineland adaptive behavior scales. 2. Circle Pines, MN: American Guidance Service; 2005. [ Google Scholar ]
• Tomasello M, Mervis CB. The instrument is great, but measuring comprehension is still a problem. Monographs of the Society for Research in child Development. 1994;59(5):174–179. [ Google Scholar ]
• Voigt RG, Llorente AM, Jensen CL, Kennard Fraley J, Barbaresi WJ, Heird WC. Comparison of the validity of direct pediatfigric developmental evaluation versus developmental screening by parent report. Clinical Pediatrics. 2007;46(6):523–529. doi: 10.1177/0009922806299100. [ DOI ] [ PubMed ] [ Google Scholar ]
• Zapolski TCB, Smith GT. Comparison of parent versus child-report of child impulsivity traits and prediction of outcome variables. Journal of Psychopathology and Behavioral Assessment. 2013;35:301–313. doi: 10.1007/s10862-013-9349-2. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• View on publisher site
• PDF (347.1 KB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections