An Evaluation of the Participation and Environment Measure for Children and Youth (PEM-CY)
In Brief © Wendy Coster, Mary Law, Gary Bedell, Mary Khetani, Dana Anaby, and Rachel Teplicky 2013
In the In Brief titled “The Participation and Environment Measure for Children and Youth (PEM-CY): An innovative measure for home, school and community”, we explained why we developed the PEM-CY, how we did it, and the potential benefits of using this new measure. We tested the measure with nearly 600 parents of children and youth aged 5 to 17 across Canada and the United States. This helped us understand how well the PEM-CY works as a measurement tool. This current In Brief includes more details about what we did to evaluate whether the PEM-CY provides consistent information about participation and actually measures what we want it to measure.
What was involved in the final stage of developing the PEM-CY?
In this phase of the study, parents of children with and without disabilities in both Canada and the USA completed the PEM-CY using a web-based survey. Community-based agencies (such as the YMCA) and service providers helped us recruit parents by posting flyers and sending out notices in newsletters and electronic communications. In addition, some parents who were involved in earlier phases of the study continued to participate.
Interested parents went to the study website where they were given a username and password to access the survey. Parents who provided consent went on to complete a questionnaire about themselves and their child, and then the PEM-CY. A small group of parents were invited to do the survey a second time within four weeks to see if the PEM-CY could provide consistent results over time (that is, test-retest reliability). A total of 576 parents completed the study: slightly more than 60% were from Canada; there were almost equal percentages of children with and without disabilities; and a wide range of disabilities were represented, including physical, intellectual, and emotional disabilities.
We also wanted to measure the extent to which questions we grouped into different sets measured the same thing (that is, internal consistency – a measure of reliability). We were not expecting the internal consistency to be very high because many factors could have an impact on participation in a given setting.
In addition, we assessed how the child’s age and disability related to their PEM-CY scores, and looked at the relationship between the parents’ desire for change in their child’s participation and their ratings of how supportive the environment was. This helped us determine whether the PEM-CY measured what it was supposed to measure (that is, validity).
What did we learn from this field test?
The PEM-CY is a reliable measurement tool:
- The test-retest reliability was found to be moderate to good across a 1- to 4-week time period. For example, good test-retest reliability for “environmental supportiveness” in the school setting means that parents who completed the PEM-CY two times within the four week time period provided similar ratings for the environment supportiveness questions both times they replied.
- The internal consistency was found to be moderate to good across different item groupings of the PEM-CY. For example, good internal consistency on “participation involvement” in the home setting means that all participation involvement questions in the home setting related to one another well and all questions measured the same thing to a relatively great extent
The PEM-CY measures what it is intended to measure:
- We found significant differences in the PEM-CY scores between the groups of children and youth with and without disabilities. This suggests that the PEM-CY is able to detect differences between children with and without disabilities in both participation and the impact of the environment. There were also some age differences, but they did not follow a consistent pattern.
- There was a significant negative correlation between parents’ desire for change in their child’s participation and their perceptions of how supportive the environment is in each setting. That is, greater desire for change in participation was associated with lower environmental supportiveness ratings. This suggests that parents perceived that participation and environment to go hand-in-hand.
Summary
In this phase of the study, we evaluated the PEM-CY to see how good a measurement tool it is. We found that the PEM-CY was reliable and measured what it was supposed to measure. These results support using the PEM-CY in research studies to learn more about child and youth participation and the environment factors that help or make participation harder in the home, school or community. To learn more about this work, please refer to:
Coster, C., Bedell, G., Law, M., Khetani, M. A., Teplicky, R., Liljenquist, K., Gleason, K., & Kao, Y.-C. (2011). Psychometric evaluation of the Participation and Environment Measure for Children and Youth. Developmental Medicine and Child Neurology, 53, 1030-7.
For more information on the Participation and Environment Measure for Children and Youth (PEM-CY), please contact Rachel Teplicky, the project coordinator (teplicr@mcmaster.ca)