Improving Model Specifications When Estimating Treatment Effects across Alternative Medical Interventions

Abstract

Objective: The purpose of this paper is to critique the list of independent variables commonly used in observational research and test the impact of variables for prior use and treatment history on estimates of treatment effects. Methods: Using data from the California Medicaid program, this study generated a series of OLS estimates of the effect of atypical antipsychotic medications on costs and duration of therapy to illustrate the impact of alternative model specifications on treatment effects. The first sequence of estimates consisted of six model specifications, the last of which included variables reflecting the type of episode defined according to prior treatment history and compliance. The second sequences repeated the specification of the first 6 models but were carried out separately by episode type to examine the heterogeneity of treatment effect. The second sequence of models documented the impact of additional drug history variables. Results: Estimates of the impact of atypical antipsychotic use on total costs and duration on initial drug were statistically significant in the first 6 models. Estimates changed significantly when dummy variables indicating prior use of inpatient service and nursing home care were included in the model specification. Estimated effects changed substantially when prior total cost was included in cost analysis, or when prior treatment duration was included in duration analysis. Significant variation also existed in estimated effects across episode types, and it was particularly pronounced before controlling for prior cost/duration. Conclusion: It is important to add prior measures of the outcome variable to control for unobserved bias in retrospective studies. Also, the accuracy and utility of results to clinicians can be improved significantly if analyses are performed by episode type.