Sentiment Analysis

A sentiment analysis project that classifies IMDB movie reviews as positive or negative. It includes text preprocessing, model training and evaluation, model comparison, visualizations, and a Streamlit app for single-review and CSV predictions.

Features

• Cleans review text and preserves negation terms.
• Trains a TF-IDF and Logistic Regression sentiment classifier.
• Evaluates the trained model and saves a confusion matrix.
• Compares Logistic Regression, Multinomial Naive Bayes, and Linear SVM.
• Generates sentiment charts, word clouds, and frequent-word charts.
• Provides single-review and bulk CSV predictions through Streamlit.

Setup

Create and activate a virtual environment from the project root:

python -m venv .venv

Windows PowerShell:

.\.venv\Scripts\Activate.ps1

Linux/macOS:

source .venv/bin/activate

Install the project requirements:

python -m pip install -r requirements.txt

Dataset

Download the IMDB Dataset of 50K Movie Reviews from Kaggle. Place the downloaded CSV at data/imdb.csv. It must contain review and sentiment columns.

Run the project

Run these commands from the project root after setting up the environment and placing the dataset:

python src/train.py
python src/evaluate.py
python src/compare_models.py
python src/visualize.py

Start the Streamlit app:

streamlit run app.py

Run the test suite:

pytest

Results

The core sentiment pipeline achieved 89.4% accuracy.

The following comparison was generated from a random sample of up to 20,000 reviews with an 80/20 stratified split:

Model	Accuracy	Precision	Recall	F1
Logistic Regression	87.13%	85.90%	88.86%	87.36%
Multinomial Naive Bayes	84.73%	83.33%	86.86%	85.06%
Linear SVM	85.65%	85.42%	86.01%	85.71%

Folder structure

.
├── app.py
├── data/
│   └── imdb.csv
├── models/
│   ├── sentiment_model.pkl
│   └── tfidf_vectorizer.pkl
├── outputs/
│   ├── confusion_matrix.png
│   ├── model_comparison.csv
│   ├── sentiment_pie_chart.png
│   ├── top_words_negative.png
│   ├── top_words_positive.png
│   ├── wordcloud_negative.png
│   └── wordcloud_positive.png
├── src/
│   ├── compare_models.py
│   ├── data_loader.py
│   ├── evaluate.py
│   ├── predict.py
│   ├── preprocess.py
│   ├── train.py
│   └── visualize.py
└── tests/
    ├── test_data_loader.py
    ├── test_predict.py
    └── test_preprocess.py

Limitations

The model may not understand sarcasm or understated praise, such as “not bad.”