· Deep Learning · 2 min read
📋 Prerequisites
- Basic Python knowledge
- Understanding of linear regression
🎯 What You'll Learn
- Understand what binary logistic regression is
- Learn how it differs from linear regression
- See how it is used for binary classification tasks
- Implement logistic regression using Python
Introduction
Binary logistic regression is a fundamental algorithm used for binary classification tasks, where the output is either 0 or 1, true or false, yes or no.
It is widely used in:
✅ Email spam detection (spam or not spam).
✅ Medical diagnosis (disease or no disease).
✅ Predicting customer churn (will leave or stay).
1️⃣ What is Binary Logistic Regression?
While linear regression predicts continuous values, logistic regression predicts probabilities between 0 and 1, which are then converted to class labels (0 or 1).
Equation:
[ p = \frac{1}{1 + e^{-(wx + b)}} ] where:
- ( p ) = predicted probability of the positive class (1).
- ( w ) = weight (slope).
- ( x ) = input feature.
- ( b ) = bias (intercept).
- ( e ) = Euler’s number (~2.718).
2️⃣ The Sigmoid Function
The sigmoid function is used to map the output of the linear combination ( wx + b ) to a probability between 0 and 1.
[ \sigma(z) = \frac{1}{1 + e^{-z}} ]
If:
✅ ( p \geq 0.5 ), the output class = 1.
✅ ( p < 0.5 ), the output class = 0.
3️⃣ Loss Function for Logistic Regression
Binary Cross-Entropy Loss is used: [ L = -[y \log(p) + (1 - y) \log(1 - p)] ] where:
- ( y ) = actual label (0 or 1).
- ( p ) = predicted probability.
The optimizer updates ( w ) and ( b ) to minimize this loss using gradient descent.
4️⃣ Example in Python
Using scikit-learn for a simple binary classification:
from sklearn.linear_model import LogisticRegression
import numpy as np
# Sample data
X = np.array([[1], [2], [3], [4], [5], [6]])
y = np.array([0, 0, 0, 1, 1, 1]) # Labels: 0 for low, 1 for high
# Create and train the model
model = LogisticRegression()
model.fit(X, y)
# Predict probabilities
print("Predicted probabilities:", model.predict_proba(X))
# Predict classes
print("Predicted classes:", model.predict(X))
# View learned weight and bias
print("Weight:", model.coef_)
print("Bias:", model.intercept_)5️⃣ Why Binary Logistic Regression is Important
✅ Helps in classification tasks with clear decision boundaries.
✅ Introduces concepts of probabilities and thresholds.
✅ Forms a foundation for understanding classification in neural networks.
Conclusion
Binary logistic regression is a powerful yet simple tool for binary classification and is essential for building your deep learning foundation.
What’s Next?
✅ Experiment with logistic regression on datasets like Iris (for binary subsets) or custom binary tasks.
✅ Learn how logistic regression extends to multiclass classification (softmax regression).
✅ Continue with classification using neural networks.
Join the SuperML Community to practice and share your projects while learning classification.
Happy Learning! ✅