Estimating Effect Size With Logistic Regression
A researcher is not able to estmate effect size (i.e., Cohen’s d) when the outcome is binary. Chinn has show that the natural log of the odds ratio, ln(OR), divided by 1.81, provides a reasonable measure of effect size
Estimating Effect Size with Chinn’s (2000) Transformation
In her article, Chinn (2000) proposed a simple conversion of the odds ratio, namely ln(odds ratio)/1.81, as a reasonable equivalent to Cohen’s d effect size measure in cases where this measure cannot readily be estimated (e.g., binary outcomes). In this presentation we will examine the accuracy of this claim through the Motor Trends data mtcars example. In this example we will estimate the regular Cohen’s d for the continuous outcome miles per gallon (mpg) and the predictor horse power (hp). We will then dichotomize the outcome variable (values lower than the median of 19.2 equal zero, else 1) and perform a similar analysis, this time using the Chinn transformation.
Example of Chinn’s (2000) Transformation on Motor Trends Data
As Chinn argues, a good approximation of the continuous effect size measure Cohen’s d may be obtained through a simple conversion method. Below is R code to perform conversion:
rm(list=ls())
data(mtcars)
mtcars$bin_mpg<-ifelse(mtcars$mpg<=19.2,0,1)
log_mod<-glm(bin_mpg~hp+vs, data=mtcars, family="binomial"(link = "logit"))
summary(log_mod)##
## Call:
## glm(formula = bin_mpg ~ hp + vs, family = binomial(link = "logit"),
## data = mtcars)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.65807 -0.29004 -0.00356 0.29418 2.35158
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 12.96514 6.34620 2.043 0.0411 *
## hp -0.08951 0.04024 -2.225 0.0261 *
## vs -2.48312 2.27299 -1.092 0.2746
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 44.236 on 31 degrees of freedom
## Residual deviance: 16.361 on 29 degrees of freedom
## AIC: 22.361
##
## Number of Fisher Scoring iterations: 7This logistic regression equation controls for the other variables. If we wanted to calculate the effect size of horse power (hp), we can use the Chinn transformation, which is the logit (ln(Odds Ratio))/1.81. Since the coefficients are already in logits, this would be the following:
rm(list=ls())
data(mtcars)
mtcars$bin_mpg<-ifelse(mtcars$mpg<=19.2,0,1)
log_mod<-glm(bin_mpg~hp+vs, data=mtcars, family="binomial"(link = "logit"))
chinn_effsz=summary(log_mod)$coefficients[2]/1.81
chinn_effse=summary(log_mod)$coefficients[5]/1.81
chinn_effsz## [1] -0.04945556chinn_effse## [1] 0.02223192#Divide the effect size by the se to get a Cohen's d-like statistic. Recall, Cohen's d
#is a standardized statistic and essentially a z-score. Since horsepower 'hp' is
#-2 standard deviations from the mean, it is a very large effect size.
cohensd<-chinn_effsz/chinn_effse
cohensd## [1] -2.224529Notice that we had to divide effect size coefficient by its standard effect to get a standardized value equivalent to Cohen’s d (recall the Cohen’s d is a standardized value). You can see if the Chinn statistic is comparable to Cohen’s d by treating mpg as continuous (which originally it was) and calculating Cohen’s d:
library(effsize)## Warning: package 'effsize' was built under R version 3.6.1rm(list=ls())
data(mtcars)
#cohen.d function is available in 'effsize' R package
cohen.d(mtcars$mpg, mtcars$hp, pooled=TRUE, paired=FALSE, na.rm=FALSE,
hedges.correction = FALSE, conf.level = 0.95, noncentral = FALSE)##
## Cohen's d
##
## d estimate: -2.601223 (large)
## 95 percent confidence interval:
## lower upper
## -3.280174 -1.922272The effect size is larger with continous variable since you are losing information when you dichotomize the variable.
References
- Chinn, S.(2000). A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in Medicine, 19: 3127-3131.