22 Logistische Regression
22.1 Set-up
22.2 Logistische Funktion
\[ f(z) = \frac{1}{1 + e^{-z}} \]
22.3 Alternative Formulierung der logistischen Funktion
\[\begin{align*} f(z) &= \frac{1}{1 + e^{-z}} \\ &= \frac{e^z}{e^z + e^{-z}e^z} \\ &= \frac{e^z}{e^z + e^{-z+z}} \\ &= \frac{e^z}{e^z + 1} \end{align*}\]
22.4 logistic function \(f(z,a) = \frac{1}{1+e^{-az}}\)
22.5 Logistic function to model probabilities
\[\begin{align*} p(X) = f(\beta_0 + \beta_1 X) &= \frac{e^{\beta_0+\beta_1 X}}{1 + e^{\beta_0 + \beta_1 X}} \\ &= \frac{1}{1+e^{-(\beta_0 + \beta_1 X)}} \end{align*}\]
\(p(X)\) as the probability of scoring a goal, or the risk of developing a disease, etc.
\[ P(Y = 1|X) = \frac{1}{1 + e^{-(\beta_0 + \beta_1 X)}} \]
22.6 Logistische Regression in R
22.7 Vorhersage
22.8 Chance
\[\begin{align*} \text{odds} &= \frac{P(\text{Event})}{P(\text{Not Event})} \\ &= \frac{p}{1-p} \\ \end{align*}\]
22.9 odds and the logistic model
\[\begin{align*} p(X) &= \frac{e^{\beta_0 + \beta_1X}}{1 + e^{\beta_0 + \beta_1 X}} \\ \Leftrightarrow p(X)(1 + e^{\beta_0 + \beta_1 X}) &= e^{\beta_0 + \beta_1X} \\ \Leftrightarrow p(X) + p(X)e^{\beta_0 + \beta_1 X} &= e^{\beta_0 + \beta_1X} \\ \Leftrightarrow p(X) &= e^{\beta_0 + \beta_1X} - p(X)e^{\beta_0 + \beta_1 X} \\ \Leftrightarrow p(X) &= e^{\beta_0 + \beta_1X}(1 - p(X)) \\ \Leftrightarrow \frac{p(X)}{1 - p(X)} &= e^{\beta_0 + \beta_1X} \\ \end{align*}\]
22.10 Logit function
\[ \text{logit}(x) = \text{log}\left(\frac{x}{1-x}\right) \]
22.11 log-odds or logit
\[ \text{logit}\left(\frac{p(X)}{1-p(X)} \right) = \beta_0 + \beta_1 X \]
The logistic regression model is linear for \(X\) in the log-odds. \[\begin{align*} \left(\frac{p(X)}{1-p(X)} \right) &= e^{\beta_0 + \beta_1 (X + \Delta)} = e^{\beta_0 + \beta_1X}e^{\beta_1 \Delta} \\ log\left(\frac{p(X)}{1-p(X)} \right) &= log\left(e^{\beta_0 + \beta_1X}e^{\beta_1\Delta}\right) = \beta_0 + \beta_1 X + \beta_1 \Delta \end{align*}\]
22.12 Changes in \(Y\) according to \(X\)
\(f(x) = \frac{e^{0.5+0.2x}}{1+e^{0.5+0.2x}}\)
22.13 Konfusionsmatrize
| Predicted: Positive | Predicted: Negative | |
|---|---|---|
| Actual: Positive | True Positive (TP) | False Negative (FN) |
| Actual: Negative | False Positive (FP) | True Negative (TN) |
- Accuracy: \((TP + TN) / \text{Total Samples}\)
- Precision (Positive Predictive Value): \(TP / (TP + FP)\)
- Recall (Sensitivity): \(TP / (TP + FN)\)
- F1 Score: \(\frac{2}{\frac{1}{\text{Precision}} + \frac{1}{\text{Sensitivity}}} \in [0,1]\)
22.14 Welche Eigenschaften sind wichtig für einen Klassifikationsalgorithmus?
- Discrimination
- Calibration
22.15 Discrimination
22.15.1 Definition
Discrimination refers to a model’s ability to distinguish between positive and negative cases. Discrimination assesses how well a model separates different outcome classes.
Intuition: Discrimination tells us how well the model differentiates between classes, regardless of the predicted probability values being perfectly aligned with real-world outcomes.
22.16 Kalibrierung
22.16.1 Definition:
Calibration refers to the agreement between predicted probabilities and the observed proportions of outcomes. A well-calibrated model outputs probabilities that reflect the true likelihood of an event.
Intuition: Calibration ensures that the model’s predictions are not just accurate in classification but also reliable as probability estimates.
22.17 Receiver Operator Characteristic (ROC)
\[ \text{True Positive Rate (TPR, Sensitivity)} = \frac{\text{True Positives}}{\text{True Positives + False Negatives}} \]
\[ \text{False Positive Rate (FPR)} = \frac{\text{False Positives}}{\text{False Positives + True Negatives}} \]
\[ \text{Specificity} = \frac{\text{True Negatives}}{\text{True Negatives + False Positives}} \]
\[ \text{FPR} = 1 - \text{Specificity}} \]
22.18 ROC
22.19 ROC curve
22.20 Area under the curve (AUC)
22.21 Calibration curve
22.22 Calculating a prediction curve
22.23 CalibrationCurves
22.24 Take-aways Diskriminierung vs. Kalibrierung
| Aspect | Discrimination | Calibration |
|---|---|---|
| Focus | Ability to distinguish classes | Accuracy of predicted probabilities |
| Measurement | AUC, ROC curves | Calibration curves |
| Key Question | “How well does the model rank cases?” | “Are the probabilities realistic?” |