Movie Critics Analysis

COMP 341: Practical Machine Learning · Assignment 2

Homework 2: Movie Critics

Overview

Find critics whose ratings best predict audience scores. You'll clean messy review data, handle missing values with different imputation methods, and use correlation analysis to identify the most predictive critics.

Data Files:

rotten_tomatoes_critic_reviews.csv - 1.1M reviews from critics (226 MB)
rotten_tomatoes_movies.csv - 17,712 movies with audience ratings
Location: data/hw2/ (mounted at /data/hw2 in Docker)

Key Columns:

Critics: critic_name, review_score, review_type (Fresh/Rotten), rotten_tomatoes_link
Movies: rotten_tomatoes_link, audience_rating, movie_title

Tasks

Part 0: Getting to Know the Data (11 pts)

1. Load both CSV files into DataFrames 2. Remove critics with no name (empty or NaN critic_name) 3. Determine the number of unique critics 4. Plot the distribution of movies reviewed per critic

Written Question: What do you notice about the distribution? Are most critics prolific or occasional reviewers?

Part 1: Reviewer Bias (9 pts)

Analyze whether critics tend toward positive or negative reviews: 1. Calculate fresh vs rotten review percentages per critic 2. Visualize the distribution of critic bias

Written Question: Do critics tend to be harsh or generous? Are there outliers?

Part 2: Cleaning Scores (15 pts)

The review_score column is messy - critics use different rating scales:

Fractions: 3/5, 3/4, 87/100, 4/10
Letter grades: A, A-, B+, B, C, F
Numbers: 75, 3.5
Empty values

Normalize all scores to a 0-100 scale:

Original	Normalized
`3/5`	60
`3/4`	75
`87/100`	87
`A`	95
`B+`	88
`C`	75
`F`	50
Empty/NaN	NaN (preserve)

Letter Grade Scale:

A: 95, A-: 92
B+: 88, B: 85, B-: 82
C+: 78, C: 75, C-: 72
D+: 68, D: 65, D-: 62
F: 50

Part 3: Handling Missing Values & Correlation (50 pts)

Most critics review only a small subset of movies, creating a sparse matrix. You'll try three imputation strategies:

Step 1: Create Critic-Movie Matrix

Build a pivot table where:

Rows = movies
Columns = critics (their normalized scores)
Include audience_score from movies table

Step 2: Filter to Active Critics

Keep only critics who have scored at least 500 movies. This reduces sparsity and focuses on prolific reviewers.

Step 3: Three Imputation Methods

Method A - Zeros: Fill missing scores with 0

Simple but biased (treats "not reviewed" as "terrible")

Method B - Mean: Fill with each critic's mean score

Assumes critic would give their average rating

Method C - KNN (k=5): Use similar movies to estimate missing scores

Most sophisticated, uses patterns in the data

Step 4: Calculate Correlations

For each imputation method: 1. Calculate Pearson correlation between each critic and audience_score 2. Identify the top 5 most correlated critics

Step 5: PCA Visualization

Run PCA on the imputed matrix to visualize the data in 2D.

Written Questions:

Which imputation method produces the highest correlations?
Do the same critics appear in the top 5 across methods?
What does the PCA plot reveal about critic clusters?

Part 4: Reflection (15 pts)

Written Questions: 1. If you were building a movie recommendation system, which critics would you weight most heavily? 2. What are the limitations of using correlation to identify "good" critics? 3. How might the 500-movie threshold bias your results?

Functions to Implement

# Data Loading
def load_critic_data(data_dir: str) -> pd.DataFrame:
    """Load rotten_tomatoes_critic_reviews.csv."""

def load_movie_data(data_dir: str) -> pd.DataFrame:
    """Load rotten_tomatoes_movies.csv."""

# Data Cleaning
def remove_unnamed_critics(df: pd.DataFrame) -> pd.DataFrame:
    """Remove critics with no name."""

def filter_critics_by_review_count(df: pd.DataFrame, threshold: int = 500) -> pd.DataFrame:
    """Keep critics with >= threshold scored movies."""

def clean_review_scores(df: pd.DataFrame) -> pd.DataFrame:
    """Normalize review_score to 0-100 scale."""

# Imputation
def prepare_critic_movie_matrix(critic_df, movie_df) -> pd.DataFrame:
    """Create movies x critics pivot table with audience_score."""

def impute_missing_zeros(df: pd.DataFrame) -> pd.DataFrame:
    """Fill NaN with 0."""

def impute_missing_mean(df: pd.DataFrame) -> pd.DataFrame:
    """Fill NaN with column mean."""

def impute_missing_knn(df: pd.DataFrame, k: int = 5) -> pd.DataFrame:
    """Fill NaN using KNN imputation."""

# Analysis
def calculate_critic_correlation(df: pd.DataFrame) -> pd.Series:
    """Pearson correlation between each critic and audience_score."""

def get_top_correlated_critics(correlations: pd.Series, n: int = 5) -> List[str]:
    """Top n critics by correlation."""

def run_pca(df: pd.DataFrame, n_components: int = 2) -> Tuple[np.ndarray, PCA]:
    """PCA on the imputed matrix."""

Hints

Score cleaning: Use regex or string methods to detect fractions ('/' in score)
Large data: The critic reviews file has 1.1M rows - operations may take a few seconds
Sparse matrix: After filtering to 500+ reviews, you'll have ~50-100 critics
KNN is slow: KNN imputation on sparse data can take 30+ seconds
Correlation: Use df.corrwith(df['audience_score']) for efficient calculation

Grading

Part	Points
Part 0: Data Exploration	11
Part 1: Reviewer Bias	9
Part 2: Score Cleaning	15
Part 3: Imputation & Correlation	50
Part 4: Written Reflection	15
Total	100

Extra Credit (up to 5 pts)

Propose and implement an alternative approach to identify predictive critics. Ideas:

Use only critics who reviewed the same movies as the test set
Weight by critic expertise (top_critic flag)
Time-weighted correlation (recent reviews matter more)

solution.py8.1 KB

python

"""
COMP 341 Homework 2: Movie Critics

Find critics whose ratings best predict audience scores using data cleaning,
imputation, PCA, and correlation analysis.

Functions to implement:
- load_critic_data: Load critic reviews CSV
- load_movie_data: Load movies CSV
- remove_unnamed_critics: Filter out critics with no name
- filter_critics_by_review_count: Keep critics with >= threshold scored movies
- clean_review_scores: Normalize all scores to 0-100 scale
- prepare_critic_movie_matrix: Create pivot table of critics x movies
- impute_missing_zeros: Fill NaN with 0
- impute_missing_mean: Fill NaN with column mean
- impute_missing_knn: Fill NaN using KNN imputation
- calculate_critic_correlation: Pearson correlation with audience score
- get_top_correlated_critics: Top n critics by correlation
- run_pca: Principal Component Analysis
"""

import pandas as pd
import numpy as np
from pathlib import Path
from typing import Union, List, Tuple
from sklearn.impute import KNNImputer
from sklearn.decomposition import PCA


# =============================================================================
# Data Loading Functions
# =============================================================================

def load_critic_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load rotten_tomatoes_critic_reviews.csv.

    Args:
        data_dir: Path to directory containing the CSV file

    Returns:
        DataFrame with columns: rotten_tomatoes_link, critic_name, top_critic,
        publisher_name, review_type, review_score, review_date, review_content
    """
    # TODO: Implement this function
    # Hint: Use pd.read_csv()
    # Hint: File is ~226MB with 1.1M rows
    pass


def load_movie_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load rotten_tomatoes_movies.csv.

    Args:
        data_dir: Path to directory containing the CSV file

    Returns:
        DataFrame with movie information including audience_rating
    """
    # TODO: Implement this function
    pass


# =============================================================================
# Data Cleaning Functions
# =============================================================================

def remove_unnamed_critics(df: pd.DataFrame) -> pd.DataFrame:
    """Remove critics with no name (empty string or NaN critic_name).

    Args:
        df: Critic reviews DataFrame

    Returns:
        DataFrame with unnamed critics removed
    """
    # TODO: Implement this function
    # Hint: Check for both NaN and empty strings
    pass


def filter_critics_by_review_count(
    df: pd.DataFrame,
    threshold: int = 500
) -> pd.DataFrame:
    """Filter to keep only critics who have scored at least `threshold` movies.

    Important: Count movies with actual scores (non-NaN review_score),
    not just total reviews.

    Args:
        df: Critic reviews DataFrame
        threshold: Minimum number of scored movies

    Returns:
        DataFrame containing only reviews from critics who meet the threshold
    """
    # TODO: Implement this function
    # Hint: Group by critic_name, count non-null review_score
    # Hint: Filter to critics meeting threshold, then filter original df
    pass


def clean_review_scores(df: pd.DataFrame) -> pd.DataFrame:
    """Normalize all review_score values to 0-100 scale.

    Must handle these formats:
    - Fractions: "3/5" -> 60, "3/4" -> 75, "87/100" -> 87, "4/10" -> 40
    - Letter grades: "A" -> 95, "A-" -> 92, "B+" -> 88, "B" -> 85,
                     "B-" -> 82, "C+" -> 78, "C" -> 75, "C-" -> 72,
                     "D+" -> 68, "D" -> 65, "D-" -> 62, "F" -> 50
    - Direct numbers: Already in 0-100 range, keep as-is
    - Empty/NaN: Preserve as NaN (do not convert to 0)

    Edge cases to handle:
    - Fractions where numerator > denominator (e.g., "92/10"):
      These are likely already on a 0-100 scale, so use numerator directly
      (92/10 -> 92, not 9.2*100=920)
    - Values > 100: cap at 100 (likely data entry errors)

    Args:
        df: Critic reviews DataFrame with review_score column

    Returns:
        DataFrame with normalized review_score column (float, 0-100 or NaN)
    """
    # TODO: Implement this function
    # Hint: Create a helper function to convert a single score
    # Hint: Use df['review_score'].apply() with the helper
    # Hint: Handle fractions by splitting on '/' and dividing
    # Hint: Use a dictionary for letter grade mappings
    # Hint: Be careful with edge cases like "92/10" -> should be 92, not 920
    pass


# =============================================================================
# Matrix Preparation and Imputation Functions
# =============================================================================

def prepare_critic_movie_matrix(
    critic_df: pd.DataFrame,
    movie_df: pd.DataFrame
) -> pd.DataFrame:
    """Create pivot table: movies (rows) x critics (columns) with audience_score.

    Args:
        critic_df: Cleaned critic reviews DataFrame (with normalized scores)
        movie_df: Movies DataFrame with audience_rating

    Returns:
        DataFrame where:
        - Each row is a movie (indexed by rotten_tomatoes_link)
        - Each column is a critic (their normalized score for that movie)
        - Last column is 'audience_score' from movie_df
        - Missing values are NaN (critic didn't review that movie)
    """
    # TODO: Implement this function
    # Hint: Use pd.pivot_table() with critic_df
    # Hint: Join with movie_df on rotten_tomatoes_link to get audience_rating
    # Hint: Rename audience_rating to audience_score for clarity
    pass


def impute_missing_zeros(df: pd.DataFrame) -> pd.DataFrame:
    """Fill missing values with 0.

    Args:
        df: Critic-movie matrix with NaN values

    Returns:
        DataFrame with NaN replaced by 0
    """
    # TODO: Implement this function
    # Hint: Use df.fillna(0)
    pass


def impute_missing_mean(
    df: pd.DataFrame,
    exclude_cols: List[str] = None
) -> pd.DataFrame:
    """Fill missing values with column mean (per critic).

    Args:
        df: Critic-movie matrix with NaN values
        exclude_cols: Columns to exclude from imputation (e.g., ['audience_score'])

    Returns:
        DataFrame with NaN replaced by column means
    """
    # TODO: Implement this function
    # Hint: For each column (except excluded), fill NaN with that column's mean
    # Hint: df[col].fillna(df[col].mean())
    if exclude_cols is None:
        exclude_cols = ['audience_score']
    pass


def impute_missing_knn(
    df: pd.DataFrame,
    k: int = 5,
    exclude_cols: List[str] = None
) -> pd.DataFrame:
    """Fill missing values using KNN imputation.

    Uses k nearest neighbors to estimate missing values based on
    similar movies' scores.

    Args:
        df: Critic-movie matrix with NaN values
        k: Number of neighbors for KNN

    Returns:
        DataFrame with NaN replaced by KNN-imputed values
    """
    # TODO: Implement this function
    # Hint: Use sklearn.impute.KNNImputer
    # Hint: Exclude audience_score from imputation, add back after
    # Hint: KNNImputer returns numpy array, convert back to DataFrame
    if exclude_cols is None:
        exclude_cols = ['audience_score']
    pass


# =============================================================================
# Analysis Functions
# =============================================================================

def calculate_critic_correlation(
    df: pd.DataFrame,
    target_col: str = 'audience_score'
) -> pd.Series:
    """Calculate Pearson correlation between each critic and the target.

    Args:
        df: Imputed critic-movie matrix (no NaN values)
        target_col: Column to correlate against (default: audience_score)

    Returns:
        Series with critic names as index and correlation coefficients as values
    """
    # TODO: Implement this function
    # Hint: Use df.corrwith() or calculate manually with df.corr()
    # Hint: Exclude target_col from the result
    pass


def get_top_correlated_critics(
    correlations: pd.Series,
    n: int = 5
) -> List[str]:
    """Return the top n critic names most positively correlated with audience score.

    Args:
 
... (truncated 2103 chars)

solution.py8.1 KB

python

"""
COMP 341 Homework 2: Movie Critics

Find critics whose ratings best predict audience scores using data cleaning,
imputation, PCA, and correlation analysis.

Functions to implement:
- load_critic_data: Load critic reviews CSV
- load_movie_data: Load movies CSV
- remove_unnamed_critics: Filter out critics with no name
- filter_critics_by_review_count: Keep critics with >= threshold scored movies
- clean_review_scores: Normalize all scores to 0-100 scale
- prepare_critic_movie_matrix: Create pivot table of critics x movies
- impute_missing_zeros: Fill NaN with 0
- impute_missing_mean: Fill NaN with column mean
- impute_missing_knn: Fill NaN using KNN imputation
- calculate_critic_correlation: Pearson correlation with audience score
- get_top_correlated_critics: Top n critics by correlation
- run_pca: Principal Component Analysis
"""

import pandas as pd
import numpy as np
from pathlib import Path
from typing import Union, List, Tuple
from sklearn.impute import KNNImputer
from sklearn.decomposition import PCA


# =============================================================================
# Data Loading Functions
# =============================================================================

def load_critic_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load rotten_tomatoes_critic_reviews.csv.

    Args:
        data_dir: Path to directory containing the CSV file

    Returns:
        DataFrame with columns: rotten_tomatoes_link, critic_name, top_critic,
        publisher_name, review_type, review_score, review_date, review_content
    """
    # TODO: Implement this function
    # Hint: Use pd.read_csv()
    # Hint: File is ~226MB with 1.1M rows
    pass


def load_movie_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load rotten_tomatoes_movies.csv.

    Args:
        data_dir: Path to directory containing the CSV file

    Returns:
        DataFrame with movie information including audience_rating
    """
    # TODO: Implement this function
    pass


# =============================================================================
# Data Cleaning Functions
# =============================================================================

def remove_unnamed_critics(df: pd.DataFrame) -> pd.DataFrame:
    """Remove critics with no name (empty string or NaN critic_name).

    Args:
        df: Critic reviews DataFrame

    Returns:
        DataFrame with unnamed critics removed
    """
    # TODO: Implement this function
    # Hint: Check for both NaN and empty strings
    pass


def filter_critics_by_review_count(
    df: pd.DataFrame,
    threshold: int = 500
) -> pd.DataFrame:
    """Filter to keep only critics who have scored at least `threshold` movies.

    Important: Count movies with actual scores (non-NaN review_score),
    not just total reviews.

    Args:
        df: Critic reviews DataFrame
        threshold: Minimum number of scored movies

    Returns:
        DataFrame containing only reviews from critics who meet the threshold
    """
    # TODO: Implement this function
    # Hint: Group by critic_name, count non-null review_score
    # Hint: Filter to critics meeting threshold, then filter original df
    pass


def clean_review_scores(df: pd.DataFrame) -> pd.DataFrame:
    """Normalize all review_score values to 0-100 scale.

    Must handle these formats:
    - Fractions: "3/5" -> 60, "3/4" -> 75, "87/100" -> 87, "4/10" -> 40
    - Letter grades: "A" -> 95, "A-" -> 92, "B+" -> 88, "B" -> 85,
                     "B-" -> 82, "C+" -> 78, "C" -> 75, "C-" -> 72,
                     "D+" -> 68, "D" -> 65, "D-" -> 62, "F" -> 50
    - Direct numbers: Already in 0-100 range, keep as-is
    - Empty/NaN: Preserve as NaN (do not convert to 0)

    Edge cases to handle:
    - Fractions where numerator > denominator (e.g., "92/10"):
      These are likely already on a 0-100 scale, so use numerator directly
      (92/10 -> 92, not 9.2*100=920)
    - Values > 100: cap at 100 (likely data entry errors)

    Args:
        df: Critic reviews DataFrame with review_score column

    Returns:
        DataFrame with normalized review_score column (float, 0-100 or NaN)
    """
    # TODO: Implement this function
    # Hint: Create a helper function to convert a single score
    # Hint: Use df['review_score'].apply() with the helper
    # Hint: Handle fractions by splitting on '/' and dividing
    # Hint: Use a dictionary for letter grade mappings
    # Hint: Be careful with edge cases like "92/10" -> should be 92, not 920
    pass


# =============================================================================
# Matrix Preparation and Imputation Functions
# =============================================================================

def prepare_critic_movie_matrix(
    critic_df: pd.DataFrame,
    movie_df: pd.DataFrame
) -> pd.DataFrame:
    """Create pivot table: movies (rows) x critics (columns) with audience_score.

    Args:
        critic_df: Cleaned critic reviews DataFrame (with normalized scores)
        movie_df: Movies DataFrame with audience_rating

    Returns:
        DataFrame where:
        - Each row is a movie (indexed by rotten_tomatoes_link)
        - Each column is a critic (their normalized score for that movie)
        - Last column is 'audience_score' from movie_df
        - Missing values are NaN (critic didn't review that movie)
    """
    # TODO: Implement this function
    # Hint: Use pd.pivot_table() with critic_df
    # Hint: Join with movie_df on rotten_tomatoes_link to get audience_rating
    # Hint: Rename audience_rating to audience_score for clarity
    pass


def impute_missing_zeros(df: pd.DataFrame) -> pd.DataFrame:
    """Fill missing values with 0.

    Args:
        df: Critic-movie matrix with NaN values

    Returns:
        DataFrame with NaN replaced by 0
    """
    # TODO: Implement this function
    # Hint: Use df.fillna(0)
    pass


def impute_missing_mean(
    df: pd.DataFrame,
    exclude_cols: List[str] = None
) -> pd.DataFrame:
    """Fill missing values with column mean (per critic).

    Args:
        df: Critic-movie matrix with NaN values
        exclude_cols: Columns to exclude from imputation (e.g., ['audience_score'])

    Returns:
        DataFrame with NaN replaced by column means
    """
    # TODO: Implement this function
    # Hint: For each column (except excluded), fill NaN with that column's mean
    # Hint: df[col].fillna(df[col].mean())
    if exclude_cols is None:
        exclude_cols = ['audience_score']
    pass


def impute_missing_knn(
    df: pd.DataFrame,
    k: int = 5,
    exclude_cols: List[str] = None
) -> pd.DataFrame:
    """Fill missing values using KNN imputation.

    Uses k nearest neighbors to estimate missing values based on
    similar movies' scores.

    Args:
        df: Critic-movie matrix with NaN values
        k: Number of neighbors for KNN

    Returns:
        DataFrame with NaN replaced by KNN-imputed values
    """
    # TODO: Implement this function
    # Hint: Use sklearn.impute.KNNImputer
    # Hint: Exclude audience_score from imputation, add back after
    # Hint: KNNImputer returns numpy array, convert back to DataFrame
    if exclude_cols is None:
        exclude_cols = ['audience_score']
    pass


# =============================================================================
# Analysis Functions
# =============================================================================

def calculate_critic_correlation(
    df: pd.DataFrame,
    target_col: str = 'audience_score'
) -> pd.Series:
    """Calculate Pearson correlation between each critic and the target.

    Args:
        df: Imputed critic-movie matrix (no NaN values)
        target_col: Column to correlate against (default: audience_score)

    Returns:
        Series with critic names as index and correlation coefficients as values
    """
    # TODO: Implement this function
    # Hint: Use df.corrwith() or calculate manually with df.corr()
    # Hint: Exclude target_col from the result
    pass


def get_top_correlated_critics(
    correlations: pd.Series,
    n: int = 5
) -> List[str]:
    """Return the top n critic names most positively correlated with audience score.

    Args:
 
... (truncated 2103 chars)

Rank	Model	Score	Tests	Time	Cost
1	Claude Opus 4.6	100.0%	16/16	26m 59s	$0.82
2	Claude Sonnet 4.6	72.5%	16/16	13m 15s	$0.70
3	Claude Haiku 4.5	72.5%	16/16	23m 31s	$0.29

Claude Opus 4.6

100.0%

16/16 tests27.0m$0.82

Claude Sonnet 4.6

72.5%

16/16 tests13.2m$0.70

Claude Haiku 4.5

72.5%

16/16 tests23.5m$0.29