Testing sampling bias in estimates of adolescent social competence and behavioural control

People from a lower socioeconomic background are less likely to participate in research and thus tend to be underrepresented in study samples. A potential risk of this underrepresentation is that research results do not reflect the psychosocial development of the entire population, but instead only reflect people with a higher socioeconomic status (SES). Ties Fakkel and colleagues estimated to what extent this so called ‘sampling bias’ plays a role for the two main CID outcomes measures in the six CID cohorts by weighting estimates with respect to SES.

CID includes adolescents from six Dutch developmental cohorts (i.e., Generation R, L-CID, RADAR, NTR, TRAILS, and YOUth) that geographically cover the Netherlands. Despite every effort to ensure representation of the entire Dutch population, lower SES participants were severely underrepresented in five of these cohorts. Moreover, in three of the six cohorts differences in social competence and behavioural control were observed between SES variable categories. Contrary to expectation, weighting for SES did not produce different normative estimates for social competence and behavioural control. These findings suggest that normative adolescent social competence and behavioural control is not overestimated as a result of predominantly high SES adolescents in Dutch developmental research cohorts. However, the authors recommend carefully weighing this finding against their important finding that in almost all CID cohorts lower SES groups are severely underrepresented.

Robust findings

For this study a total of 12,874 adolescents between the ages of 9 to 13 years old were analysed (50.4% girls; 91.6% Dutch or Western European). SES was measured with mother’s educational attainment, father’s educational attainment, and family income. Behavioural control was measured with the Achenbach Self-control Scale or the BAS-Drive subscale from the BIS/BAS questionnaire. Social competence was measured with the Social Problems-subscale from the Child’s Behavior Checklist or the Prosocial Behavior-subscale from the Strengths and Difficulties Questionnaire (and with a custom scale in RADAR). By testing the research question in six different developmental cohorts; with various measures of SES, social competence and behavioural control; on various test statistics; across a broad age range of late childhood and adolescence; and verified through several sensitivity analyses, the findings can be considered robust.

Important how low socioeconomic status is defined

However, it can be questioned how representative the low SES participants in the cohorts are for the low SES population. A considerable proportion of ‘low SES’ adolescents in the six cohorts have a single low SES risk factor for development (e.g., low educated mother) which might be compensated or outweighed by high SES protective factors (e.g., high educated father and high income). When applying a stricter definition of ‘low SES’ – being low SES on all SES indicators – some of the cohorts studied here might suffer from underrepresentation and undersampling more than the initial estimates suggest: on average, 22.8% of adolescents have at least one single low SES indicator, but only approximately 5% of adolescents is low SES on all measured indicators. Hence, it should be questioned whether adolescents that are considered low SES in some cohorts are actually exposed to the developmental adversities that are typically associated with socioeconomic deprivation.

Recommendations to assess socioeconomic validity

For studies on adolescent psychosocial development, the authors recommend to assess the socioeconomic validity of research samples: 1) by counting the number of SES indicators that are measured, 2) by checking for undersampling in any combination of SES indicators (e.g., lower educated parents and low income), and 3) by contrasting the SES characteristics of excluded participants to those of included participants. To understand the extent to which socioeconomic circumstances do or do not affect normative psychosocial development, it is of paramount importance to include a sufficient number of research participants from across the entire SES spectrum.

More information

Fakkel M., Peeters M., Lugtig P., Zondervan-Zwijnenburg M.A.J., Blok E., White T., van der Meulen M., Kevenaar S.T., Willemsen G.,  Bartels M., Boomsma D.I., Schmengler H., Branje S., Vollebergh W.A.M. (2020) Testing sampling bias in estimates of adolescent social competence and behavioral control Developmental cognitive neuroscience DOI: 10.1016/j.dcn.2020.100872

This paper is part of a special issue in Developmental cognitive neuroscience about the Consortium on Individual Development. For an overview of all papers go to here