Pitfall: Logistic Regression Coefficients and Odds Ratios

Based on the independent characteristics of the observations, logistic regression models estimate the probability of a specific category of a categorical dependent variable occurring. To estimate these probabilities, the regression weights of the independent variables must undergo a transformation. As a result, logistic regression coefficients reflect the relationship between the independent variables and the logits for the respective category of the dependent variable. Analogous to linear regression, one can conclude that increasing a given independent variable by one unit corresponds to changing the logit for the occurrence of the dependent variable's specific category by β units. While this interpretation is formally correct, it is evidently not very insightful.

Logits, Odds Ratios, and Probabilities

An obvious question arises: What exactly are logits? The answer is apparently quite simple: logits are the natural logarithms of odds ratios. We can summarize this as follows: Logit = ln(Odds Ratio)

But this naturally raises the next question: What are odds ratios? In German, odds ratios are referred to as "Chancenverhältnisse" (or "Quotenverhältnisse"). In fact, odds ratios are nothing more than simple ratios of odds. In the given context, odds ratios represent the ratio of odds for the occurrence of a specific category of the dependent variable between two groups differing in their levels of an independent variable. We summarize as follows: Odds Ratio = Odds of characteristic occurring in group 1 : Odds of characteristic occurring in group 2.

Naturally, this leads to another question: What exactly are odds? Odds represent the ratio of the probability of a particular characteristic occurring relative to the probability of it not occurring within a group defined, for instance, by an independent variable. We summarize this as follows: Odds of characteristic occurring = Probability of occurrence : Probability of non-occurrence.

The probability of a particular characteristic is equivalent to the proportion of observations within a group exhibiting the respective characteristic. We summarize this as follows: Probability of characteristic = Proportion of group members with the characteristic.

An Example: Nerds, Normies, and Star Wars

To illustrate, the following calculates the logit and odds ratio of being a Star Wars fan for a group of 10 "statistics nerds" compared to a group of 10 "normies."

Manual calculation

7 of the 10 nerds are Star Wars fans 4 of the 10 normal people are Star Wars fans. It follows from this:

$ P_{ Nerds }\left( Fan \right)=\frac{ 7 }{10 }=0,7 $

$ Odds_{Nerds }\left( Fan \right)=\frac{ P_{ Nerds }\left( Fan \right) }{ P_{ Nerds }\left( kein Fan \right) }=\frac{ 0,7 }{ 1-0,7} = 2,\overline{3} $

$ P_{ Normalos }\left( Fan \right)=\frac{ 4 }{10 }=0,4 $

$ Odds_{Normalos }\left( Fan \right)=\frac{ P_{Normalos }\left( Fan \right) }{ P_{ Normalos }\left( kein Fan \right) }=\frac{ 0,4 }{ 1-0,4} = 0,\overline{6} $

$ Odds Ratio_{ Nerds : Normalos }\left( Fan \right)=\frac{ Odds_{ Nerds }\left( Fan \right) }{ Odds_{ Normalos }\left( Fan \right) }=\frac{ 2,\overline{3} }{ 0,\overline{6} }=3,5 $

$ \beta_{ Nerds:Normalos }=ln\left( Odds Ratio_{ Nerds:Normalos } \right)=1,25 $

Calculation via Logistic Regression in R

You obtain the same result by estimating a logistic regression for the given data in R and exponentiating the logit coefficient that is provided by default.

Group membership is represented by a dummy variable coded as 1 for all nerds and 0 for all normies; therefore, in this case, increasing the independent variable by one unit corresponds to switching group membership.

(Logarithmic) Ratios of Ratios

Calculating odds ratios is simple; however, odds ratios are only superficially more suitable than logistic regression coefficients for interpreting logistic models. Odds ratios are ratios of probability ratios. Just like logits, their logarithmic form, odds ratios are not intuitively understandable for most people.

Formally, it is correct to say that increasing a given independent variable by one unit corresponds to a change in the odds for the occurrence of the dependent variable's characteristic by a factor of eβ. Nevertheless, neither odds ratios nor logistic regression coefficients allow direct conclusions about the probabilities within groups or the probability ratios between contrasted groups.

Thus, statements such as "…an increase of one unit in the independent variable is associated with a change in probability by eβ or β…" are not permissible when interpreting logistic regression models. The misguided nature of such claims becomes evident when considering that very different baseline probabilities can result in identical odds ratios or logits. For example, the odds ratio from the previous example could also arise from completely different probabilities in two contrasted groups:

Conclusion

Since even formally correct interpretations of the absolute values of logits (β), as well as odds ratios (eβ), are uninformative and potentially misleading, it is recommended here to interpret only the direction of relationships implied by logits and odds ratios. An increase in an independent variable (by one unit) is associated with an increased probability of the considered category of the dependent variable when odds ratios > 1 or β > 0, and with a decreased probability when odds ratios < 1 or β < 0.

References

• Best, H., & Wolf, C. (2012). Modellvergleich und Ergebnisinterpretation in Logit-und Probit-Regressionen. KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie, 64(2), 377-395.