Missing data are always an issue in community-based longitudinal studies, calling into question the representativeness of samples and bias in conclusions, the research has generated. This may be due to the difficulty of implementing random sampling procedures in these studies and/or the inherent difficulty in sampling hard-to-reach segments of the population being studied. In fact, the ability to accurately study hard-to-reach populations in light of potential bias created by missing data remains an open question. In this study, missing data are defined as both failure to interview potential research participants identified in the sampling frame and failure to retain enrolled research participants longitudinally. Using the sample from the Mobile Youth Survey, a multiple-cohort, longitudinal study of adolescents living in highly impoverished neighborhoods in Mobile, Alabama, we examined sample representativeness and dropout to determine whether missing data led to a nonrepresentative, and therefore, biased sample. Results indicate that even though random procedures are not strictly used to draw the sample, (a) the sample appears to be largely representative of the population that was studied, and (b) attrition is largely uncorrelated with characteristics of those who dropped out. This suggests that it is possible to study with validity hard-to reach populations in community settings.
KeywordsHard-to-Reach PopulationsMissing DataRepresentativenessCommunity-Based Research
Gorin, S., Hooper, C.A., Dyson, C. and Cabral, C. (2008) Ethical Challenges in Conducting Research with Hard to Reach Families. Child Abuse Review, 17, 275-287. https://doi.org/10.1002/car.1031
Shaghaghi, A., Bhopal, R.S. and Sheikh, A. (2011) Approaches to Recruiting “Hard-to-Reach” Populations into Research: A Review of the Literature. Health Promotion, 1, 1-9.
Freimuth, V.S. and Mettger, W. (1990) Is There a Hard-to-Reach Audience? Public Health Reports, 105, 232-238.
Lambert, E. and Wiebel, W. (1990) Introduction. In: Lambert, E. and Wiebel, W., Eds., The Collection and Interpretation of Data from Hidden Populations, National Institute on Drug Abuse Research Monograph 98, Government Printing Office, Washington DC, 1-3.
Haley, D.F., Lucas, J., Golin, C.E., Wang, J., Hughes, J.P., Emel, L., et al. (2014) Retention Strategies and Factors Associated with Missed Visits among Low Income Women at Increased Risk of HIV Acquisition in the US (HPTN 064). AIDS Patient Care and STDs, 28, 206-217. https://doi.org/10.1089/apc.2013.0366
Katz, K.S., El-Mohandes, A., Johnson, D.M., Jarrett, M., Rose, A. and Cober, M. (2001) Retention of Low Income Mothers in a Parenting Intervention Study. Journal of Community Health, 26, 203-218. https://doi.org/10.1023/A:1010373113060
Sullivan, C.M., Rumptz, M.H., Campbell, R., Eby, K.K. and Davidson, W.S. (1996) Retaining Participants in Longitudinal Community Research: A Comprehensive Protocol. The Journal of Applied Behavioral Science, 32, 262-276. https://doi.org/10.1177/0021886396323002
Reiss, F. (2013) Socioeconomic Inequalities and Mental Health Problems in Children and Adolescents: A Systematic Review. Social Science & Medicine, 90, 24-31.
Nandi, A., Glymour, M.M. and Subramanian, S.V. (2014) Associations among Socioeconomic Status, Health Behaviors, and All-Cause Mortality in the United States. Epidemiology, 25, 170-177. https://doi.org/10.1097/EDE.0000000000000038
Entwisle, D.R., Alexander, K.L. and Olson, L.S. (1994) The Gender Gap in Math: Its Possible Origins in Neighborhood Effects. American Sociological Review, 59, 822-838. https://doi.org/10.2307/2096370
Alspaugh, J.W. (1991) Out-of-School Environmental Factors and Elementary Achievement in Mathematics and Reading. Journal of Research and Development in Education, 24, 53-55.
Balli, S.J., Demo, D.H. and Wedman, J.F. (1998) Family Involvement with Children’s Homework: An Intervention in the Middle Grades. Family Relations, 47, 149-157. https://doi.org/10.2307/585619
Graham, J.W., Hofer, S.M., Donaldson, S.I., MacKinnon, D.P. and Schafer, J.L. (1997) Analysis with Missing Data in Prevention Research. In: Bryant, K.J., Windle, M. and West, S.G., Eds., The Science of Prevention: Methodological Advances from Alcohol and Substance Abuse Research, American Psychological Association, Washington DC. https://doi.org/10.1037/10222-010
Do, D.P., Frank, R. and Finch, B.K. (2012) Does SES Explain More of the Black/ White Health Gap than We Thought? Revisiting Our Approach toward Understanding Racial Disparities in Health. Social Science & Medicine, 74, 1385-1393.
Braveman, P.A., Cubbin, C., Egerter, S., Chideya, S., Marchi, K.S., Metzler, M. and Posner, S. (2005) Socioeconomic Status in Health Research: One Size Does Not Fit All. JAMA, 294, 2879-2888. https://doi.org/10.1001/jama.294.22.2879
Hatchett, B.F., Holmes, K., Duran, D.A. and Davis, C. (2000) African Americans and Research Participation: The Recruitment Process. Journal of Black Studies, 30, 664-675. https://doi.org/10.1177/002193470003000502
Kerkorian, D., Traube, D.E. and McKay, M.M. (2007) Understanding the African American Research Experience (KAARE): Implications for HIV Prevention. Social Work in Mental Health, 5, 295-312. https://doi.org/10.1300/J200v05n03_03
Pottick, K.J. and Lerman, P. (1991) Maximizing Survey Response Rates for Hard-to-Reach Inner-City Populations. Social Science Quarterly, 72, 172-180.
Merry, S.E. (1981) Urban Danger: Life in a Neighborhood of Strangers. Temple University Press, Philadelphia, PA.
Ross, C.E. and Jang, S.J. (2000) Neighborhood Disorder, Fear, and Mistrust: The Buffering Role of Social Ties with Neighbors. American Journal of Community Psychology, 28, 401-420. https://doi.org/10.1023/A:1005137713332
Yancey, A.K., Ortega, A.N. and Kumanyika, S.K. (2006) Effective Recruitment and Retention of Minority Research Participants. Annual Review of Public Health, 27, 1-28. https://doi.org/10.1146/annurev.publhealth.27.021405.102113
Sampson, R.J., Morenoff, J.D. and Gannon-Rowley, T. (2002) Assessing “Neighborhood Effects”: Social Processes and New Directions in Research. Annual Review of Sociology, 28, 443-478. https://doi.org/10.1146/annurev.soc.28.110601.141114
Peterson, R.D., Krivo, L.J. and Harris, M.A. (2000) Disadvantage and Neighborhood Violent Crime: Do Local Institutions Matter? Journal of Research in Crime and Delinquency, 37, 31-63. https://doi.org/10.1177/0022427800037001002
Veysey, B.M. and Messner, S.F. (1999) Further Testing of Social Disorganization Theory: An Elaboration of Sampson and Groves’s “Community Structure and Crime”. Journal of Research in Crime and Delinquency, 36, 156-174. https://doi.org/10.1177/0022427899036002002
Odierna, D.H. and Schmidt, L.A. (2009) The Effects of Failing to Include Hard-to-Reach Respondents in Longitudinal Surveys. American Journal of Public Health, 99, 1515-1521. https://doi.org/10.2105/AJPH.2007.111138
Weitzman, B.C., Guttmacher, S., Weinberg, S. and Kapadia, F. (2003) Low Response Rate Schools in Surveys of Adolescent Risk Taking Behaviours: Possible Biases, Possible Solutions. Journal of Epidemiology and Community Health, 57, 63-67. https://doi.org/10.1136/jech.57.1.63
National Institutes of Health: Office of Extramural Research (2001) NIH Policy and Guidelines on The Inclusion of Women and Minorities as Subjects in Clinical Research—Amended, October, 2001. http://grants.nih.gov/grants/funding/women_min/guidelines_amended_10_2001.htm
Belone, L., Lucero, J.E., Duran, B., Tafoya, G., Baker, E.A., Chan, D., et al. (2016) Community-Based Participatory Research Conceptual Model: Community Partner Consultation and Face Validity. Qualitative Health Research, 26, 117-135. https://doi.org/10.1177/1049732314557084
Hacker, K. (2013) Community-Based Participatory Research. Sage Publications, Thousand Oaks.
Oetzel, J.G., Zhou, C., Duran, B., Pearson, C., Magarati, M., Lucero, J., et al. (2015) Establishing the Psychometric Properties of Constructs in a Community-Based Participatory Research Conceptual Model. American Journal of Health Promotion, 29, e188-e202.
Chevalier, J.M. and Buckles, D. (2013) Participatory Action Research: Theory and Methods for Engaged Inquiry. Routledge, Abingdon-on-Thames.
Marcellus, L. (2004) Are We Missing Anything? Pursuing Research on Attrition. CJNR (Canadian Journal of Nursing Research), 36, 82-98.
Collins, L.M., Schafer, J.L. and Kam, C. (2001) A Comparison of Inclusive and Restrictive Strategies in Modern Missing Data Procedures. Psychological Methods, 6, 330-351. https://doi.org/10.1037/1082-989X.6.4.330
Schafer, J.L. and Graham, J.W. (2002) Missing Data: Our View of the State of the Art. Psychological Methods, 7, 147-177. https://doi.org/10.1037/1082-989X.7.2.147
Daniels, M.J. and Hogan, J.W. (2008) Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. Taylor & Francis, Boca Raton, FL.
Molenberghs, G. and Kenward, M. (2007) Missing Data in Clinical Studies. Wiley, West Sussex, England. https://doi.org/10.1002/9780470510445
Guan, Z. and Qin, J. (2017) Empirical Likelihood Method for Non-Ignorable Missing Data Problems. Lifetime Data Analysis, 23, 113-135. https://doi.org/10.1007/s10985-016-9381-0
Kim, J.K. and Yu, C.L. (2011) A Semiparametric Estimation of Mean Functionals with Nonignorable Missing Data. Journal of the American Statistics Association, 106, 157-165. https://doi.org/10.1198/jasa.2011.tm10104
Scharfstein, D.O., Rotnitzky, A. and Robins, J.M. (1999) Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models. Journal of the American Statistical Association, 94, 1096-1146. https://doi.org/10.1080/01621459.1999.10473862
Bolland, J.M., Lian, B.E. and Formichella, C.M. (2005) The Origins of Hopelessness among Inner-City African-American Adolescents. American Journal of Community Psychology, 36, 293-305. https://doi.org/10.1007/s10464-005-8627-x
King, D.W., Taft, C., King, L.A., Hammond, C. and Stone, E.R. (2006) Directionality of the Association between Social Support and Posttraumatic Stress Disorder: A Longitudinal Investigation. Journal of Applied Social Psychology, 36, 2980-2992. https://doi.org/10.1111/j.0021-9029.2006.00138.x
McAuley, E., Elavsky, S., Motl, R.W., Konopack, J.F., Hu, L. and Marquez, D.X. (2005) Physical Activity, Self-Efficacy, and Self-Esteem: Longitudinal Relationships in Older Adults. Journal of Gerontology, 60, 268-275. https://doi.org/10.1093/geronb/60.5.P268
Peugh, J.L. and Enders, C.K. (2004) Missing Data in Educational Research: A Review of Reporting Practices and Suggestions for Improvement. Review of Educational Research, 74, 525-556. https://doi.org/10.3102/00346543074004525
Rebellon, C.J., Manasse, M.E., Van Gundy, K.T. and Cohn, E.S. (2014) Rationalizing Delinquency: A Longitudinal Test of the Reciprocal Relationship between Delinquent Attitudes and Behavior. Social Psychology Quarterly, 77, 361-386. https://doi.org/10.1177/0190272514546066
Rahn, W.M. and Transue, J.E. (1998) Social Trust and Value Change: The Decline of Social Capital in American Youth, 1976-1995. Political Psychology, 19, 545-565. https://doi.org/10.1111/0162-895X.00117
U.S. Census Bureau (2012) 2000 Census. https://www.census.gov/
U.S. Census Bureau (2012) 1990 Census. https://www.census.gov/
Bolland, K.A., Bolland, J.M., Tomek, S., Devereaux, R.S., Mrug, S. and Wimberly, J.C. (2016) Trajectories of Adolescent Alcohol Use by Gender and Early Initiation Status. Youth & Society, 48, 3-32. https://doi.org/10.1177/0044118X13475639
Bolland, J.M. (2007) Mobile Youth Survey Overview. Unpublished Manuscript, University of Alabama at Birmingham, Birmingham.
U.S. Census Bureau (2015) Table 8. https://www.census.gov/topics/education/school-enrollment.html
Redford, J., Battle, D. and Bielick, S. (2016) Homeschooling in the United States: 2012 (NCES 2016-096). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington DC.
Freudenberg, N. and Ruglis, J. (2007) Reframing School Dropout as a Public Health Issue. Preventing Chronic Disease, 4, A107.
Townsend, L., Flisher, A.J. and King, G. (2007) A Systematic Review of the Relationship between High School Dropout and Substance Use. Clinical Child and Family Psychology Review, 10, 295-317. https://doi.org/10.1007/s10567-007-0023-7
Chen, D., Drabick, D.A. and Burgers, D.E. (2015) A Developmental Perspective on Peer Rejection, Deviant Peer Affiliation, and Conduct Problems among Youth. Child Psychiatry & Human Development, 46, 823-838. https://doi.org/10.1007/s10578-014-0522-y
McGee, L. and Newcomb, M.D. (1992) General Deviance Syndrome: Expanded Hierarchical Evaluations at Four Ages from Early Adolescence to Adulthood. Journal of Consulting and Clinical Psychology, 60, 766-776. https://doi.org/10.1037/0022-006X.60.5.766
Liang, K.Y. and Zeger, S.L. (1986) Longitudinal Data Analysis Using Generalized Linear Models. Biometrika, 73, 13-22. https://doi.org/10.1093/biomet/73.1.13
Diggle, P., Heagerty, P., Liang, K. and Zeger, S. (2002) Analysis of Longitudinal Data. Oxford University Press, Oxford, England.
Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences. 2nd Edition, Lawrence Erlbaum Associates, Inc., Hillsdale, NJ.
Bor, W., Najman, J., O’Callaghan, M., Williams, G.M. and Anstey, K. (2001) Aggression and the Development of Delinquent Behavior in Children. Australian Institute of Criminology Trends and Issues in Crime and Criminal Justice, 207, 1-6.
Buhi, E.R., Goodson, P. and Neilands, T.B. (2008) Out of Sight, Not Out of Mind: Strategies for Handling Missing Data. American Journal of Health Behavior, 32, 83-92. https://doi.org/10.5993/AJHB.32.1.8
Teitler, J.O., Reichman, N.E. and Sprachman, S. (2003) Costs and Benefits of Improving Response Rates for a Hard-to-Reach Population. The Public Opinion Quarterly, 67, 126-138. https://doi.org/10.1086/346011
Way, N. and Robinson, M.G. (2003) A Longitudinal Study of the Effects of Family, Friends, and School Experiences on the Psychological Adjustment of Ethnic Minority, Low-SES Adolescents. Journal of Adolescent Research, 18, 324-346. https://doi.org/10.1177/0743558403018004001
Groves, R.M. (2006) Nonresponse Rates and Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 646-675.
Groves, R.M. and Peytcheva, E. (2008) The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis. Public Opinion Quarterly, 72, 167-189. https://doi.org/10.1093/poq/nfn011
Lennon, M.C., Clark, W.A. and Joshi, H. (2016) Residential Mobility and Wellbeing: Exploring Children’s Living Situations and Their Implications for Housing Policy. Longitudinal and Lifecourse Studies, 7, 197-200. https://doi.org/10.14301/llcs.v7i3.393
Rosenblatt, P. and DeLuca, S. (2012) “We Don’t Live Outside, We Live in Here”: Neighborhood and Residential Mobility Decisions Among Low-Income Families. City & Community, 11, 254-284. https://doi.org/10.1111/j.1540-6040.2012.01413.x
Bolland, A.C. (2012) Representativeness Two Ways: An Assessment of Representativeness and Missing Data Mechanisms in a Study of an At-Risk Population. Doctoral Dissertation, The University of Alabama, Tuscaloosa.
Von Stumm, S. and Plomin, R. (2015) Socioeconomic Status and the Growth of Intelligence from Infancy through Adolescence. Intelligence, 48, 30-36.
de Winter, A.F., Visser, L., Verhulst, F.C., Vollebergh, W.A. and Reijneveld, S.A. (2016) Longitudinal Patterns and Predictors of Multiple Health Risk Behaviors among Adolescents: The TRAILS Study. Preventive Medicine, 84, 76-82. https://doi.org/10.1016/j.ypmed.2015.11.028
Evans, G.W., Li, D. and Whipple, S.S. (2013) Cumulative Risk and Child Development. Psychological Bulletin, 139, 1342-1396. https://doi.org/10.1037/a0031808
Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., et al. (2014) Reaching the Hard-to-Reach: A Systematic Review of Strategies for Improving Health and Medical Research with Socially Disadvantaged Groups. BMC Medical Research Methodology, 14, 42. https://doi.org/10.1186/1471-2288-14-42
Faugier, J. and Sargeant, M. (1997) Sampling Hard to Reach Populations. Journal of Advanced Nursing, 26, 790-797. https://doi.org/10.1046/j.1365-2648.1997.00371.x