Multiclass Logistic Regression

Introduction

Multiclass logistic regression is used when the target variable has more than two classes.

It extends binary logistic regression to handle:

✅ Image classification into multiple categories.
✅ Text classification into multiple sentiments or topics.
✅ Any task where you predict more than two categories.

1️⃣ How Does Multiclass Logistic Regression Work?

Instead of predicting a single probability (as in binary logistic regression), it predicts probabilities for each class using the softmax function.

Given classes ( c_1, c_2, …, c_k ), the softmax function computes: [ P(y = c_j | x) = \frac{e^{z_j}}{\sum_{k} e^{z_k}} ] where:

• ( z_j = w_j x + b_j ) is the linear combination for class ( j ).
• The denominator sums over all classes to normalize the probabilities.

The class with the highest probability is chosen as the predicted class.

2️⃣ Loss Function: Categorical Cross-Entropy

The loss function used for multiclass logistic regression is categorical cross-entropy: [ L = -\sum_{i} y_i \log(p_i) ] where:

• ( y_i ) = true label (one-hot encoded).
• ( p_i ) = predicted probability for class ( i ).

3️⃣ Example in Python

Using scikit-learn for multiclass classification:

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset
iris = load_iris()
X = iris.data
y = iris.target  # Three classes: 0, 1, 2

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create and train model
model = LogisticRegression(multi_class='multinomial', solver='lbfgs', max_iter=200)
model.fit(X_train, y_train)

# Predict
predictions = model.predict(X_test)

# Evaluate
print("Accuracy:", accuracy_score(y_test, predictions))

4️⃣ Why Multiclass Logistic Regression is Important

✅ Allows you to handle real-world problems with multiple classes.
✅ Introduces softmax and categorical cross-entropy, which are foundational for neural networks.
✅ Builds intuition for classification in deep learning architectures.

Conclusion

Multiclass logistic regression extends your classification skills to multiple categories, enabling you to handle more complex projects confidently.

What’s Next?

✅ Practice with datasets like Iris, MNIST (for digits), and news topic classification.
✅ Continue with neural network classifiers for multiclass problems.
✅ Join the SuperML Community to share your practice projects and clarify doubts.

Happy Learning! 🪐