Agent Work: Stroke Prediction

Claude Haiku 4.5 · COMP 341: Practical Machine Learning

Homework 3: Stroke Prediction (Classification)

Overview

Predict whether an individual has had a cerebrovascular accident (CVA/stroke) given a snapshot of their health and demographic information. You will train and compare three classification methods -- Decision Trees, Logistic Regression, and Linear Discriminant Analysis -- exploring how feature encoding, scaling, and regularization affect performance.

Data File:

CVA.csv - Health and demographic data for stroke prediction
Location: data/hw3/ (mounted at /data/hw3 in Docker)

Key Columns:

id - Patient identifier (non-predictive, exclude from features)
sex - Male, Female, Other (categorical)
age - Age of patient (numeric)
hypertension - 0 or 1 (binary)
heart_disease - 0 or 1 (binary)
ever_married - Yes or No (categorical)
work_type - children, Govt_job, Never_worked, Private, Self-employed (categorical)
residence_type - Rural or Urban (categorical)
avg_glucose_level - Average glucose level (numeric)
bmi - Body mass index (numeric, has missing values)
smoking_status - formerly smoked, never smoked, smokes, Unknown (categorical)
CVA - 1 if the patient had a stroke, 0 otherwise (target variable)

Tasks

Part 0: Getting Familiar with the Data (13 pts)

Explore the dataset to understand its structure, feature types, missing values, and class distribution.

1. Load CVA.csv into a DataFrame 2. Determine the number of individuals in the dataset 3. Identify the feature types: nominal, ordinal, and numeric 4. Check for missing values -- how many rows and columns are affected? 5. Count how many individuals have had a CVA vs. not 6. Visualize the distribution of features (use appropriate plots for each data type)

Written Questions (answered in writeup.md):

Q1: How many individuals in this dataset?
Q2: List the nominal features, ordinal features, and numeric features
Q3: Without additional coding -- do you think the missing values will be problematic during classification? Why or why not?
Q4: Without additional coding -- which features do you think will be the most informative? Explain your rationale.

Data Preparation

Before training any models, prepare the data:

1. Separate features and labels: Create feature matrix X (exclude CVA and non-predictive columns like id) and label vector y (the CVA column) 2. Train/test split: Use train_test_split(X, y, test_size=0.40, random_state=6) 3. Impute missing values: After the split, fill missing values using mean imputation. Compute the mean on the training set and apply to both train and test sets separately (to avoid data leakage).

Part 1: Decision Trees (30 pts)

Step 1: Numeric Features Only

Train a DecisionTreeClassifier with default parameters using only the numeric (continuous-valued) features. Evaluate accuracy on the test set.

Step 2: One-Hot Encode Categorical Features

Convert categorical features using one-hot encoding (e.g., pd.get_dummies). Train a new decision tree with default parameters on all features (numeric + one-hot encoded categorical). Evaluate accuracy on the test set.

Written Question: Q5: Did you notice any accuracy improvements after adding the categorical features? Comment on why this did or did not help.

Step 3: Scale Numeric Features

Scale all numeric features using StandardScaler. Fit the scaler on the training set and transform both train and test sets. Retrain and re-test the decision tree.

Written Question: Q6: Did you notice any accuracy improvements after scaling? Comment on why scaling did or did not help.

Step 4: Tune max_depth

Vary the max_depth parameter from 1 to 20 (inclusive). For each depth, train a decision tree and record accuracy on both the training and test sets. Plot max_depth vs. accuracy for both sets on a single figure.

Written Question: Q7: Which maximum depth parameter is the best choice?

Step 5: Feature Importances

Using the decision tree with the optimal max_depth from Step 4, extract the feature importances (model.feature_importances_). Create a DataFrame showing each feature and its importance.

Written Question: Q8: Looking at the feature importances, does this match your initial expectations? Why or why not?

Part 2: Logistic Regression (30 pts)

Step 1: Unscaled Logistic Regression

Train LogisticRegression with default parameters on the one-hot encoded (but unscaled) feature set. Evaluate accuracy on the test set.

Step 2: Scaled Logistic Regression

Train LogisticRegression on the scaled and one-hot encoded features. Evaluate accuracy on the test set.

Written Question: Q9: Does scaling have an effect on logistic regression performance? Why or why not?

Step 3: Extract Coefficients

Extract the feature coefficients from the scaled logistic regression model (model.coef_). Display them alongside feature names.

Written Question: Q10: Based on the coefficients, are the following features associated with an increased or decreased chance of CVA: living in an urban area, being married, BMI?

Step 4: Compare Regularization

Train three logistic regression models on the scaled data and extract coefficients from each: 1. L1 regularization: LogisticRegression(solver='liblinear', penalty='l1') 2. L2 regularization: LogisticRegression(solver='liblinear', penalty='l2') 3. No regularization: LogisticRegression(solver='lbfgs', penalty=None)

Compare the coefficient values across all three.

Written Questions:

Q11: Do the different regularization methods change the feature weights? If so, which features are affected?
Q12: Based on the feature weights across the different regularization schemes, would you say that regularization helps prevent overfitting in this case? Explain.

Part 3: Linear Discriminant Analysis (12 pts)

Step 1: Train LDA

Train LinearDiscriminantAnalysis with default settings on the scaled and one-hot encoded features. Evaluate accuracy on the test set.

Step 2: Extract Coefficients

Extract the LDA coefficients (model.coef_). Display them alongside feature names.

Written Questions:

Q13: Can the coefficients extracted from LDA be interpreted as feature importances? Why or why not?
Q14: Now that you have run three classification methods (decision trees, logistic regression, LDA) with various parameters, which method is best suited for the CVA prediction problem? Explain.
Q15: Which method would you use for identifying risk factors for CVAs? Explain.

General Coding Style (15 pts)

10 points for coding style: logical variable names, comments as needed, clean code
5 points for code flow: accurate results when functions are called sequentially

Functions to Implement

import pandas as pd
import numpy as np
from pathlib import Path
from typing import Union, Optional, List, Tuple, Dict
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.preprocessing import StandardScaler


def load_cva_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load CVA.csv into a DataFrame.

    Args:
        data_dir: Path to directory containing CVA.csv

    Returns:
        DataFrame with all columns from CVA.csv
    """


def prepare_features_labels(df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.Series]:
    """Separate features (X) and labels (y) from the dataset.

    Exclude the target column 'CVA' and any non-predictive columns (e.g., 'id')
    from the feature matrix.

    Args:
        df: Full CVA DataFrame

    Returns:
        Tuple of (X, y) where X is feature DataFrame and y is CVA Series
    """


def impute_missing_values(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Impute missing values using mean imputation.

    Compute the mean on X_train and apply to both X_train and X_test
    to avoid data leakage.

    Args:
        X_train: Training feature DataFrame
        X_test: Test feature DataFrame

    Returns:
        Tuple of (X_train_imputed, X_test_imputed)
    """


def train_decision_tree(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depth: Optional[int] = None
) -> Tuple[DecisionTreeClassifier, float, float]:
    """Train a DecisionTreeClassifier and evaluate accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depth: Maximum depth of tree (None for unlimited)

    Returns:
        Tuple of (fitted model, training accuracy, test accuracy)
    """


def one_hot_encode(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """One-hot encode categorical features in both train and test sets.

    Must produce consistent columns across train and test. Use pd.get_dummies
    or similar, ensuring both DataFrames end up with the same columns.

    Args:
        X_train: Training features with categorical columns
        X_test: Test features with categorical columns

    Returns:
        Tuple of (X_train_encoded, X_test_encoded) with consistent columns
    """


def scale_features(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Scale numeric features using StandardScaler.

    Fit the scaler on X_train and transform both train and test.

    Args:
        X_train: Training features
        X_test: Test features

    Returns:
        Tuple of (X_train_scaled, X_test_scaled)
    """


def find_optimal_depth(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depths: List[int] = list(range(1, 21))
) -> Dict:
    """Find the optimal max_depth by training trees at each depth.

    Args:
        X_train: Training features (scaled, one-hot encoded)
        y_train: Training labels
        X_test: Test features (scaled, one-hot encoded)
        y_test: Test labels
        max_depths: Range of max_depth values to try (default 1-20)

    Returns:
        Dict with keys:
        - 'train_accuracies': list of training accuracies for each depth
        - 'test_accuracies': list of test accuracies for each depth
        - 'best_depth': int, the max_depth with highest test accuracy
    """


def get_feature_importances(
    model: DecisionTreeClassifier,
    feature_names: List[str]
) -> pd.DataFrame:
    """Extract feature importances from a fitted decision tree.

    Args:
        model: Fitted DecisionTreeClassifier
        feature_names: List of feature names matching model's features

    Returns:
        DataFrame with columns ['feature', 'importance'],
        sorted by importance descending
    """


def train_logistic_regression(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    penalty: str = 'l2',
    solver: str = 'lbfgs',
    max_iter: int = 1000
) -> Tuple[LogisticRegression, float, float]:
    """Train a LogisticRegression model and evaluate accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        penalty: Regularization penalty ('l1', 'l2', or None)
        solver: Solver algorithm (default 'lbfgs')
        max_iter: Maximum iterations (default 1000)

    Returns:
        Tuple of (fitted model, training accuracy, test accuracy)
    """


def train_lda(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series
) -> Tuple[LinearDiscriminantAnalysis, float, float]:
    """Train an LDA model and evaluate accuracy.

    Args:
        X_train: Training features (scaled, one-hot encoded)
        y_train: Training labels
        X_test: Test features (scaled, one-hot encoded)
        y_test: Test labels

    Returns:
        Tuple of (fitted model, training accuracy, test accuracy)
    """


def compare_regularization(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series
) -> Dict[str, Tuple[LogisticRegression, float, float]]:
    """Compare L1, L2, and no regularization for logistic regression.

    Trains three models on the scaled features:
    1. L1: LogisticRegression(solver='liblinear', penalty='l1')
    2. L2: LogisticRegression(solver='liblinear', penalty='l2')
    3. None: LogisticRegression(solver='lbfgs', penalty=None)

    Args:
        X_train: Scaled training features
        y_train: Training labels
        X_test: Scaled test features
        y_test: Test labels

    Returns:
        Dict mapping regularization name to (model, train_accuracy, test_accuracy).
        Keys: 'L1', 'L2', 'none'
    """

Hints

Non-predictive columns: The id column is a patient identifier and should be excluded from features
Train/test split: Always use test_size=0.40, random_state=6 for reproducibility
Imputation order: Impute missing values *after* train/test split, computing means on training data only
One-hot encoding consistency: Use pd.get_dummies() on combined train+test, then re-split, or use handle_unknown='ignore' with OneHotEncoder to ensure consistent columns
Scaling: Fit StandardScaler on training data only, then transform both sets
Decision tree and scaling: Decision trees split on thresholds, so scaling should not affect their performance
max_depth tuning: The best depth balances train and test accuracy (look for where test accuracy peaks before overfitting begins)
Logistic regression convergence: Increase max_iter if you see convergence warnings (1000 is usually sufficient)
No regularization: Use penalty=None (Python None, not the string 'none')

Grading

Part	Points
Part 0: Data Exploration	13
Part 1: Decision Trees	30
Part 2: Logistic Regression	30
Part 3: LDA & Comparison	12
General Coding Style	15
Total	100

Extra Credit (up to 10 pts)

Using plotnine (not matplotlib or seaborn), write visualization functions:

Plot a sigmoid curve for each feature from logistic regression (up to 5 pts)
Plot the LDA classification boundary with training points (up to 5 pts)

solution.py8.1 KB

python

"""
COMP 341 Homework 3: Stroke Prediction

Predict cerebrovascular accidents (CVAs) using classification methods including
Decision Trees, Logistic Regression, and Linear Discriminant Analysis.

Functions to implement:
- load_cva_data: Load CVA.csv into a DataFrame
- prepare_features_labels: Separate features X and labels y
- impute_missing_values: Mean imputation on numeric columns with missing values
- train_decision_tree: Train a DecisionTreeClassifier and evaluate accuracy
- one_hot_encode: One-hot encode categorical features
- scale_features: StandardScaler on numeric features
- find_optimal_depth: Tune max_depth from 1-20 and find best depth
- get_feature_importances: Extract and rank feature importances from a decision tree
- train_logistic_regression: Train LogisticRegression and evaluate accuracy
- train_lda: Train LinearDiscriminantAnalysis and evaluate accuracy
- compare_regularization: Compare L1, L2, and no regularization for logistic regression
"""

import pandas as pd
import numpy as np
from pathlib import Path
from typing import Union, Dict, List, Tuple, Optional
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer


# =============================================================================
# Data Loading and Preparation
# =============================================================================

def load_cva_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load CVA.csv into a pandas DataFrame.

    Args:
        data_dir: Path to directory containing CVA.csv

    Returns:
        DataFrame with all columns from the CSV file, including health
        and demographic features along with the CVA label column
    """
    # TODO: Implement this function
    # Hint: Use pd.read_csv() to load data_dir/CVA.csv
    pass


def prepare_features_labels(
    df: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.Series]:
    """Separate features (X) and labels (y) from the DataFrame.

    The label column is 'CVA' (cerebrovascular accident indicator).
    Non-predictive columns (e.g., 'id') should be excluded from X.

    Args:
        df: Full CVA DataFrame with all columns

    Returns:
        Tuple of:
        - X: DataFrame of features (excluding CVA and non-predictive columns)
        - y: Series of labels (the CVA column)
    """
    # TODO: Implement this function
    # Hint: Drop 'CVA' and any non-predictive columns like 'id' from X
    # Hint: y should be the 'CVA' column
    pass


def impute_missing_values(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Impute missing values using mean imputation on numeric columns.

    Important: Fit the imputer on X_train only, then transform both
    X_train and X_test. This prevents data leakage from test to train.

    Args:
        X_train: Training features with potential missing values
        X_test: Test features with potential missing values

    Returns:
        Tuple of:
        - X_train with missing numeric values filled by training set means
        - X_test with missing numeric values filled by training set means
    """
    # TODO: Implement this function
    # Hint: Identify numeric columns with missing values
    # Hint: Use SimpleImputer(strategy='mean') or manual fillna
    # Hint: Fit on X_train only, transform both X_train and X_test
    pass


# =============================================================================
# Decision Tree Functions
# =============================================================================

def train_decision_tree(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depth: Optional[int] = None
) -> Tuple[DecisionTreeClassifier, float, float]:
    """Train a DecisionTreeClassifier and evaluate accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depth: Maximum depth of the tree (None for unlimited)

    Returns:
        Tuple of:
        - Fitted DecisionTreeClassifier model
        - Training accuracy (float between 0 and 1)
        - Test accuracy (float between 0 and 1)
    """
    # TODO: Implement this function
    # Hint: Create DecisionTreeClassifier with max_depth parameter
    # Hint: Fit on training data, then score on both train and test
    pass


def one_hot_encode(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """One-hot encode categorical features in the dataset.

    Categorical columns (object/string dtype) should be one-hot encoded.
    Numeric columns should be kept as-is.

    Args:
        X_train: Training features with categorical columns
        X_test: Test features with categorical columns

    Returns:
        Tuple of:
        - X_train with categorical columns replaced by one-hot encoded columns
        - X_test with categorical columns replaced by one-hot encoded columns
    """
    # TODO: Implement this function
    # Hint: Identify categorical columns (object dtype)
    # Hint: Use pd.get_dummies() or OneHotEncoder
    # Hint: Ensure X_train and X_test have the same columns after encoding
    pass


def scale_features(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Scale all features using StandardScaler.

    Fit the scaler on X_train only, then transform both X_train and X_test.

    Args:
        X_train: Training features (may include one-hot encoded columns)
        X_test: Test features (may include one-hot encoded columns)

    Returns:
        Tuple of:
        - X_train with numeric features scaled (zero mean, unit variance)
        - X_test with numeric features scaled using training set statistics
    """
    # TODO: Implement this function
    # Hint: Use StandardScaler from sklearn.preprocessing
    # Hint: Fit on X_train numeric columns only, transform both
    # Hint: Preserve DataFrame structure and column names
    pass


def find_optimal_depth(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depths: List[int] = list(range(1, 21))
) -> Dict:
    """Find the optimal max_depth for a DecisionTreeClassifier.

    Train a decision tree for each max_depth value and record train/test
    accuracy. The best depth is the one with highest test accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depths: Range of max_depth values to try (default: 1 to 20)

    Returns:
        Dict with keys:
        - 'train_accuracies': list of training accuracies for each depth
        - 'test_accuracies': list of test accuracies for each depth
        - 'best_depth': int, the max_depth with highest test accuracy
    """
    # TODO: Implement this function
    # Hint: Loop over max_depths, train a tree for each, record accuracies
    # Hint: Use train_decision_tree() or train directly
    # Hint: best_depth = max_depths[argmax(test_accuracies)]
    pass


def get_feature_importances(
    model: DecisionTreeClassifier,
    feature_names: List[str]
) -> pd.DataFrame:
    """Extract feature importances from a fitted decision tree.

    Args:
        model: Fitted DecisionTreeClassifier
        feature_names: List of feature names corresponding to model's features

    Returns:
        DataFrame with columns 'feature' and 'importance',
        sorted by importance in descending order
    """
    # TODO: Implement this function
    # Hint: Use model.feature_importances_ attribute
    # Hint: Create DataFrame and sort by importance descending
    pass


# =============================================================================
# Logistic Regression Functions
# =============================================================================

def train_logistic_regression(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    pe
... (truncated 3651 chars)

solution.py8.1 KB

python

"""
COMP 341 Homework 3: Stroke Prediction

Predict cerebrovascular accidents (CVAs) using classification methods including
Decision Trees, Logistic Regression, and Linear Discriminant Analysis.

Functions to implement:
- load_cva_data: Load CVA.csv into a DataFrame
- prepare_features_labels: Separate features X and labels y
- impute_missing_values: Mean imputation on numeric columns with missing values
- train_decision_tree: Train a DecisionTreeClassifier and evaluate accuracy
- one_hot_encode: One-hot encode categorical features
- scale_features: StandardScaler on numeric features
- find_optimal_depth: Tune max_depth from 1-20 and find best depth
- get_feature_importances: Extract and rank feature importances from a decision tree
- train_logistic_regression: Train LogisticRegression and evaluate accuracy
- train_lda: Train LinearDiscriminantAnalysis and evaluate accuracy
- compare_regularization: Compare L1, L2, and no regularization for logistic regression
"""

import pandas as pd
import numpy as np
from pathlib import Path
from typing import Union, Dict, List, Tuple, Optional
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer


# =============================================================================
# Data Loading and Preparation
# =============================================================================

def load_cva_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load CVA.csv into a pandas DataFrame.

    Args:
        data_dir: Path to directory containing CVA.csv

    Returns:
        DataFrame with all columns from the CSV file, including health
        and demographic features along with the CVA label column
    """
    # TODO: Implement this function
    # Hint: Use pd.read_csv() to load data_dir/CVA.csv
    pass


def prepare_features_labels(
    df: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.Series]:
    """Separate features (X) and labels (y) from the DataFrame.

    The label column is 'CVA' (cerebrovascular accident indicator).
    Non-predictive columns (e.g., 'id') should be excluded from X.

    Args:
        df: Full CVA DataFrame with all columns

    Returns:
        Tuple of:
        - X: DataFrame of features (excluding CVA and non-predictive columns)
        - y: Series of labels (the CVA column)
    """
    # TODO: Implement this function
    # Hint: Drop 'CVA' and any non-predictive columns like 'id' from X
    # Hint: y should be the 'CVA' column
    pass


def impute_missing_values(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Impute missing values using mean imputation on numeric columns.

    Important: Fit the imputer on X_train only, then transform both
    X_train and X_test. This prevents data leakage from test to train.

    Args:
        X_train: Training features with potential missing values
        X_test: Test features with potential missing values

    Returns:
        Tuple of:
        - X_train with missing numeric values filled by training set means
        - X_test with missing numeric values filled by training set means
    """
    # TODO: Implement this function
    # Hint: Identify numeric columns with missing values
    # Hint: Use SimpleImputer(strategy='mean') or manual fillna
    # Hint: Fit on X_train only, transform both X_train and X_test
    pass


# =============================================================================
# Decision Tree Functions
# =============================================================================

def train_decision_tree(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depth: Optional[int] = None
) -> Tuple[DecisionTreeClassifier, float, float]:
    """Train a DecisionTreeClassifier and evaluate accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depth: Maximum depth of the tree (None for unlimited)

    Returns:
        Tuple of:
        - Fitted DecisionTreeClassifier model
        - Training accuracy (float between 0 and 1)
        - Test accuracy (float between 0 and 1)
    """
    # TODO: Implement this function
    # Hint: Create DecisionTreeClassifier with max_depth parameter
    # Hint: Fit on training data, then score on both train and test
    pass


def one_hot_encode(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """One-hot encode categorical features in the dataset.

    Categorical columns (object/string dtype) should be one-hot encoded.
    Numeric columns should be kept as-is.

    Args:
        X_train: Training features with categorical columns
        X_test: Test features with categorical columns

    Returns:
        Tuple of:
        - X_train with categorical columns replaced by one-hot encoded columns
        - X_test with categorical columns replaced by one-hot encoded columns
    """
    # TODO: Implement this function
    # Hint: Identify categorical columns (object dtype)
    # Hint: Use pd.get_dummies() or OneHotEncoder
    # Hint: Ensure X_train and X_test have the same columns after encoding
    pass


def scale_features(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Scale all features using StandardScaler.

    Fit the scaler on X_train only, then transform both X_train and X_test.

    Args:
        X_train: Training features (may include one-hot encoded columns)
        X_test: Test features (may include one-hot encoded columns)

    Returns:
        Tuple of:
        - X_train with numeric features scaled (zero mean, unit variance)
        - X_test with numeric features scaled using training set statistics
    """
    # TODO: Implement this function
    # Hint: Use StandardScaler from sklearn.preprocessing
    # Hint: Fit on X_train numeric columns only, transform both
    # Hint: Preserve DataFrame structure and column names
    pass


def find_optimal_depth(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depths: List[int] = list(range(1, 21))
) -> Dict:
    """Find the optimal max_depth for a DecisionTreeClassifier.

    Train a decision tree for each max_depth value and record train/test
    accuracy. The best depth is the one with highest test accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depths: Range of max_depth values to try (default: 1 to 20)

    Returns:
        Dict with keys:
        - 'train_accuracies': list of training accuracies for each depth
        - 'test_accuracies': list of test accuracies for each depth
        - 'best_depth': int, the max_depth with highest test accuracy
    """
    # TODO: Implement this function
    # Hint: Loop over max_depths, train a tree for each, record accuracies
    # Hint: Use train_decision_tree() or train directly
    # Hint: best_depth = max_depths[argmax(test_accuracies)]
    pass


def get_feature_importances(
    model: DecisionTreeClassifier,
    feature_names: List[str]
) -> pd.DataFrame:
    """Extract feature importances from a fitted decision tree.

    Args:
        model: Fitted DecisionTreeClassifier
        feature_names: List of feature names corresponding to model's features

    Returns:
        DataFrame with columns 'feature' and 'importance',
        sorted by importance in descending order
    """
    # TODO: Implement this function
    # Hint: Use model.feature_importances_ attribute
    # Hint: Create DataFrame and sort by importance descending
    pass


# =============================================================================
# Logistic Regression Functions
# =============================================================================

def train_logistic_regression(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    pe
... (truncated 3651 chars)

solution.py12.6 KB

python

"""
COMP 341 Homework 3: Stroke Prediction

Predict cerebrovascular accidents (CVAs) using classification methods including
Decision Trees, Logistic Regression, and Linear Discriminant Analysis.

Functions to implement:
- load_cva_data: Load CVA.csv into a DataFrame
- prepare_features_labels: Separate features X and labels y
- impute_missing_values: Mean imputation on numeric columns with missing values
- train_decision_tree: Train a DecisionTreeClassifier and evaluate accuracy
- one_hot_encode: One-hot encode categorical features
- scale_features: StandardScaler on numeric features
- find_optimal_depth: Tune max_depth from 1-20 and find best depth
- get_feature_importances: Extract and rank feature importances from a decision tree
- train_logistic_regression: Train LogisticRegression and evaluate accuracy
- train_lda: Train LinearDiscriminantAnalysis and evaluate accuracy
- compare_regularization: Compare L1, L2, and no regularization for logistic regression
"""

import pandas as pd
import numpy as np
from pathlib import Path
from typing import Union, Dict, List, Tuple, Optional
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer


# =============================================================================
# Data Loading and Preparation
# =============================================================================

def load_cva_data(data_dir: Union[str, Path]) -> pd.DataFrame:
    """Load CVA.csv into a pandas DataFrame.

    Args:
        data_dir: Path to directory containing CVA.csv

    Returns:
        DataFrame with all columns from the CSV file, including health
        and demographic features along with the CVA label column
    """
    data_dir = Path(data_dir)
    return pd.read_csv(data_dir / "CVA.csv")


def prepare_features_labels(
    df: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.Series]:
    """Separate features (X) and labels (y) from the DataFrame.

    The label column is 'CVA' (cerebrovascular accident indicator).
    Non-predictive columns (e.g., 'id') should be excluded from X.

    Args:
        df: Full CVA DataFrame with all columns

    Returns:
        Tuple of:
        - X: DataFrame of features (excluding CVA and non-predictive columns)
        - y: Series of labels (the CVA column)
    """
    y = df['CVA']
    X = df.drop(columns=['CVA', 'id'])
    return X, y


def impute_missing_values(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Impute missing values using mean imputation on numeric columns.

    Important: Fit the imputer on X_train only, then transform both
    X_train and X_test. This prevents data leakage from test to train.

    Args:
        X_train: Training features with potential missing values
        X_test: Test features with potential missing values

    Returns:
        Tuple of:
        - X_train with missing numeric values filled by training set means
        - X_test with missing numeric values filled by training set means
    """
    # Identify numeric columns
    numeric_cols = X_train.select_dtypes(include=[np.number]).columns.tolist()

    # Fit imputer on training data only
    imputer = SimpleImputer(strategy='mean')
    imputer.fit(X_train[numeric_cols])

    # Transform both train and test sets
    X_train_imputed = X_train.copy()
    X_test_imputed = X_test.copy()

    X_train_imputed[numeric_cols] = imputer.transform(X_train[numeric_cols])
    X_test_imputed[numeric_cols] = imputer.transform(X_test[numeric_cols])

    return X_train_imputed, X_test_imputed


# =============================================================================
# Decision Tree Functions
# =============================================================================

def train_decision_tree(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depth: Optional[int] = None
) -> Tuple[DecisionTreeClassifier, float, float]:
    """Train a DecisionTreeClassifier and evaluate accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depth: Maximum depth of the tree (None for unlimited)

    Returns:
        Tuple of:
        - Fitted DecisionTreeClassifier model
        - Training accuracy (float between 0 and 1)
        - Test accuracy (float between 0 and 1)
    """
    model = DecisionTreeClassifier(max_depth=max_depth)
    model.fit(X_train, y_train)
    train_acc = model.score(X_train, y_train)
    test_acc = model.score(X_test, y_test)
    return model, train_acc, test_acc


def one_hot_encode(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """One-hot encode categorical features in the dataset.

    Categorical columns (object/string dtype) should be one-hot encoded.
    Numeric columns should be kept as-is.

    Args:
        X_train: Training features with categorical columns
        X_test: Test features with categorical columns

    Returns:
        Tuple of:
        - X_train with categorical columns replaced by one-hot encoded columns
        - X_test with categorical columns replaced by one-hot encoded columns
    """
    # One-hot encode both datasets
    X_train_encoded = pd.get_dummies(X_train, drop_first=False)
    X_test_encoded = pd.get_dummies(X_test, drop_first=False)

    # Ensure train and test have the same columns
    # Add missing columns to test with 0 values
    for col in X_train_encoded.columns:
        if col not in X_test_encoded.columns:
            X_test_encoded[col] = 0

    # Remove extra columns from test that aren't in train
    X_test_encoded = X_test_encoded[X_train_encoded.columns]

    return X_train_encoded, X_test_encoded


def scale_features(
    X_train: pd.DataFrame,
    X_test: pd.DataFrame
) -> Tuple[pd.DataFrame, pd.DataFrame]:
    """Scale all features using StandardScaler.

    Fit the scaler on X_train only, then transform both X_train and X_test.

    Args:
        X_train: Training features (may include one-hot encoded columns)
        X_test: Test features (may include one-hot encoded columns)

    Returns:
        Tuple of:
        - X_train with numeric features scaled (zero mean, unit variance)
        - X_test with numeric features scaled using training set statistics
    """
    # Fit scaler on training data only
    scaler = StandardScaler()
    scaler.fit(X_train)

    # Transform both sets
    X_train_scaled = pd.DataFrame(
        scaler.transform(X_train),
        columns=X_train.columns,
        index=X_train.index
    )
    X_test_scaled = pd.DataFrame(
        scaler.transform(X_test),
        columns=X_test.columns,
        index=X_test.index
    )

    return X_train_scaled, X_test_scaled


def find_optimal_depth(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    max_depths: List[int] = list(range(1, 21))
) -> Dict:
    """Find the optimal max_depth for a DecisionTreeClassifier.

    Train a decision tree for each max_depth value and record train/test
    accuracy. The best depth is the one with highest test accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        max_depths: Range of max_depth values to try (default: 1 to 20)

    Returns:
        Dict with keys:
        - 'train_accuracies': list of training accuracies for each depth
        - 'test_accuracies': list of test accuracies for each depth
        - 'best_depth': int, the max_depth with highest test accuracy
    """
    train_accuracies = []
    test_accuracies = []

    for depth in max_depths:
        _, train_acc, test_acc = train_decision_tree(
            X_train, y_train, X_test, y_test, max_depth=depth
        )
        train_accuracies.append(train_acc)
        test_accuracies.append(test_acc)

    best_depth = max_depths[np.argmax(test_accuracies)]

    return {
        'train_accuracies': train_accuracies,
        'test_accuracies': test_accuracies,
        'best_depth': best_depth
    }


def get_feature_importances(
    model: DecisionTreeClassifier,
    feature_names: List[str]
) -> pd.DataFrame:
    """Extract feature importances from a fitted decision tree.

    Args:
        model: Fitted DecisionTreeClassifier
        feature_names: List of feature names corresponding to model's features

    Returns:
        DataFrame with columns 'feature' and 'importance',
        sorted by importance in descending order
    """
    importances = model.feature_importances_
    df = pd.DataFrame({
        'feature': feature_names,
        'importance': importances
    })
    return df.sort_values('importance', ascending=False).reset_index(drop=True)


# =============================================================================
# Logistic Regression Functions
# =============================================================================

def train_logistic_regression(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series,
    penalty: str = 'l2',
    solver: str = 'lbfgs',
    max_iter: int = 1000
) -> Tuple[LogisticRegression, float, float]:
    """Train a LogisticRegression model and evaluate accuracy.

    Args:
        X_train: Training features
        y_train: Training labels
        X_test: Test features
        y_test: Test labels
        penalty: Regularization penalty ('l1', 'l2', or None)
        solver: Optimization solver ('lbfgs', 'liblinear', etc.)
        max_iter: Maximum number of iterations for convergence

    Returns:
        Tuple of:
        - Fitted LogisticRegression model
        - Training accuracy (float between 0 and 1)
        - Test accuracy (float between 0 and 1)
    """
    model = LogisticRegression(
        penalty=penalty,
        solver=solver,
        max_iter=max_iter
    )
    model.fit(X_train, y_train)
    train_acc = model.score(X_train, y_train)
    test_acc = model.score(X_test, y_test)
    return model, train_acc, test_acc


def compare_regularization(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series
) -> Dict[str, Tuple[LogisticRegression, float, float]]:
    """Compare L1, L2, and no regularization for logistic regression.

    Trains three logistic regression models:
    - 'L1': liblinear solver with L1 penalty
    - 'L2': liblinear solver with L2 penalty
    - 'none': lbfgs solver with no penalty

    Args:
        X_train: Training features (should be scaled)
        y_train: Training labels
        X_test: Test features (should be scaled)
        y_test: Test labels

    Returns:
        Dict mapping regularization name to tuple of:
        - Fitted LogisticRegression model
        - Training accuracy
        - Test accuracy

        Keys are 'L1', 'L2', 'none'
    """
    l1_model, l1_train_acc, l1_test_acc = train_logistic_regression(
        X_train, y_train, X_test, y_test,
        penalty='l1', solver='liblinear'
    )
    l2_model, l2_train_acc, l2_test_acc = train_logistic_regression(
        X_train, y_train, X_test, y_test,
        penalty='l2', solver='liblinear'
    )
    none_model, none_train_acc, none_test_acc = train_logistic_regression(
        X_train, y_train, X_test, y_test,
        penalty=None, solver='lbfgs'
    )

    return {
        'L1': (l1_model, l1_train_acc, l1_test_acc),
        'L2': (l2_model, l2_train_acc, l2_test_acc),
        'none': (none_model, none_train_acc, none_test_acc)
    }


# =============================================================================
# Linear Discriminant Analysis Functions
# =============================================================================

def train_lda(
    X_train: pd.DataFrame,
    y_train: pd.Series,
    X_test: pd.DataFrame,
    y_test: pd.Series
) -> Tuple[LinearDiscriminantAnalysis, float, float]:
    """Train a Linear Discriminant Analysis model and evaluate accuracy.

    Uses default LDA settings.

    Args:
        X_train: Training features (should be scaled, one-hot encoded)
        y_train: Training labels
        X_test: Test features (should be scaled, one-hot encoded)
        y_test: Test labels

    Returns:
        Tuple of:
        - Fitted LinearDiscriminantAnalysis model
        - Training accuracy (float between 0 and 1)
        - Test accuracy (float between 0 and 1)
    """
    model = LinearDiscriminantAnalysis()
    model.fit(X_train, y_train)
    train_acc = model.score(X_train, y_train)
    test_acc = model.score(X_test, y_test)
    return model, train_acc, test_acc

COMP 341 Homework 3: Cerebrovascular Accident (Stroke) Prediction

Written Questions

Complete each question below. Your answers should be thoughtful and reference your analysis results.

---

Part 0: Getting Familiar with the Data

Question 1

> How many individuals in this dataset?

Your Answer:

There are 5,108 individuals in the CVA dataset (5,109 rows including the header row).

---

Question 2

> List the nominal features, ordinal features, and numeric features in this data.

Your Answer:

Nominal Features:

sex (Male, Female, Other)
ever_married (Yes, No)
work_type (children, Govt_job, Never_worked, Private, Self-employed)
residence_type (Rural, Urban)
smoking_status (formerly smoked, never smoked, smokes, Unknown)

Ordinal Features: None. While hypertension and heart_disease are binary (0/1), they represent binary indicators rather than ordered categories.

Numeric Features:

age (continuous)
hypertension (binary indicator: 0 or 1)
heart_disease (binary indicator: 0 or 1)
avg_glucose_level (continuous)
bmi (continuous, has missing values)

---

Question 3

> Without any additional coding - do you think the missing values determined above will be problematic during our classification task? Why or why not?

Your Answer:

The missing values will likely not be severely problematic, though they could introduce some bias. There are 201 missing BMI values and 14 missing age values out of 5,108 total samples (~4% and <0.3%, respectively). Using mean imputation is a reasonable approach for this dataset because:

1. The proportion of missing data is relatively small (< 5%) 2. Mean imputation is appropriate for continuous numeric features like BMI and age 3. If missing values were concentrated among individuals with CVA, it could introduce bias, but the small percentage suggests they're likely randomly distributed

The one concern is whether the missing BMI values are missing completely at random or related to the outcome (e.g., patients who had strokes might be more likely to be missing BMI measurements), but without additional investigation, mean imputation is a standard approach.

---

Question 4

> Again without additional coding - which features do you think will be the most informative? Explain.

Your Answer:

I would expect age and avg_glucose_level to be the most informative features for predicting stroke risk, followed by hypertension and heart_disease. My reasoning:

1. Age: Stroke risk increases significantly with age due to accumulated vascular damage and age-related physiological changes. Age is one of the strongest demographic risk factors for CVA.

2. avg_glucose_level: High blood glucose (diabetes/prediabetes) is a well-established risk factor for stroke due to its effect on blood vessel integrity and formation of arterial plaques.

3. Hypertension and heart_disease: Both are direct cardiovascular risk factors. Hypertension damages blood vessel walls, and heart disease indicates existing cardiovascular compromise.

4. BMI: While obesity is correlated with stroke risk, it's often mediated through hypertension and glucose levels.

5. Categorical features (smoking, marital status, work type): These may have weaker direct associations, though smoking status could be informative.

The class imbalance (254 positive CVA cases vs 4,855 negative cases) also suggests that the model will need strong signal from continuous features to separate the two groups effectively.

---

Part 1: Decision Trees

Question 5

> Did you notice any performance (accuracy) improvements after adding the categorical features? Comment on why this did or did not help.

Your Answer:

No, adding categorical features actually slightly decreased test accuracy:

Numeric features only: 90.70% test accuracy
Numeric + categorical (one-hot encoded): 90.31% test accuracy

This slight decrease likely occurred because:

1. Categorical features are weaker predictors: The nominal features (sex, work_type, residence_type, smoking_status, ever_married) have weaker direct relationships with stroke risk compared to age and glucose levels.

2. Increased feature space: One-hot encoding creates many new binary features (for each category value). With more features and a fixed amount of training data, the decision tree may overfit slightly or split on noisy features.

3. Class imbalance: With an imbalanced dataset (254 positive vs 4,855 negative), adding low-information features can introduce noise that hurts generalization.

4. Robust numeric signal: The numeric features alone may already capture the most informative signal for this problem, and additional categorical features don't add much predictive value.

---

Question 6

> Did you notice any performance (accuracy) improvements after scaling? Comment on why scaling did or did not help.

Your Answer:

No, scaling had no effect on decision tree performance:

Unscaled numeric + categorical: 90.31% test accuracy
Scaled numeric + categorical: 90.36% test accuracy (negligible difference)

This is expected and correct because:

1. Decision trees are scale-invariant: Decision trees make binary splits based on feature thresholds (e.g., "age > 50"). Whether age is in [0, 100] or standardized to [-1, 2], the tree can still make the same splits by adjusting the threshold values accordingly.

2. No relative feature weighting: Unlike distance-based or gradient-based models (k-NN, logistic regression, SVM), decision trees don't compute distances or weighted sums, so feature scale is irrelevant.

3. Scaling is unnecessary for tree-based models: This is why random forests, gradient boosting, and other tree-based methods don't require feature scaling, making them more robust to features with different magnitudes.

Scaling will only affect distance-based models or models that use feature coefficients/weights.

---

Question 7

> Which maximum depth parameter is the best choice?

Your Answer:

The best maximum depth is 1 (a decision stump), which achieves 95.50% test accuracy.

This is a surprising but interpretable result:

1. Simplicity vs. Complexity: A depth-1 tree makes a single binary split on one feature. This simple model generalizes very well to the test set, suggesting that the problem has a strong linear boundary that can be captured with a single feature.

2. Avoiding Overfitting: Deeper trees (depth 2-20) show perfect training accuracy (100%) but lower test accuracy, indicating overfitting. The depth-1 tree avoids this trap by maintaining high test accuracy.

3. Class Imbalance: The dataset has strong class imbalance (4.9% positive CVA cases). A simple model may be more robust to this imbalance than a complex model that memorizes training patterns.

4. Interpretability Trade-off: The depth-1 decision stump is also highly interpretable — it's likely splitting on age or glucose level with a simple threshold, making it easy to explain to physicians.

While depths 7-8 might achieve similar test accuracy, depth 1 is the simplest model that achieves near-optimal performance, following Occam's Razor principle.

---

Question 8

> Looking at the feature importances determined above, does this match your initial expectations? Why or why not?

Your Answer:

Yes, the feature importances align very well with my initial expectations:

1. Age (39.5% importance): The #1 most important feature, confirming that age is the strongest predictor of stroke risk — consistent with medical knowledge.

2. avg_glucose_level (29.9% importance): The second most important feature, as expected. High blood glucose is a major stroke risk factor.

3. BMI (16.1% importance): Third most important, though notably weaker than age and glucose, which makes sense because BMI's effect on stroke is often mediated through glucose and hypertension.

4. Weak categorical features: Work type, smoking status, and residence type have very low importance (1-4%), suggesting weak direct associations with stroke.

5. Binary health flags: Hypertension and heart_disease have low importance (1-3%), which is somewhat surprising since they're significant medical risk factors, but likely because age and glucose already capture much of their predictive information.

Why the importances make sense: The decision tree identifies that splitting on age and glucose level early in the tree captures most of the class separation. The model learns that these features provide the most information gain for classification, which aligns with epidemiological understanding of stroke risk factors.

---

Part 2: Logistic Regression

Question 9

> Does scaling have an effect on the performance for logistic regression? Why or why not?

Your Answer:

No, scaling had no effect on logistic regression performance in this case:

Unscaled features: 95.50% test accuracy
Scaled features: 95.50% test accuracy

This result is surprising because logistic regression typically should benefit from scaling. However, I believe the results are the same because:

1. Strong signal overpowers scale sensitivity: The numeric features (age, glucose) have strong predictive signals that dominate the model's fit. Logistic regression converges to a solution that works well regardless of feature scale.

2. One-hot encoded categorical features: After one-hot encoding, many features are already in a standard 0-1 range (binary indicators). This reduces the overall scale variance across features, making scaling less impactful.

3. Regularization effect: The default logistic regression uses L2 regularization, which implicitly penalizes larger coefficient magnitudes. This regularization might reduce the advantage of scaling.

4. Convergence success: Logistic regression with default settings converges successfully on both scaled and unscaled data without numerical issues.

In general: Scaling is still a best practice for logistic regression because it makes the regularization penalty more meaningful (treating all features equally) and can improve convergence speed. Here, both models converged successfully, so the effect wasn't visible.

---

Question 10

> Based on the above weights, are the following features associated with an increased or a decreased chance of a CVA: living in an urban area, being married, bmi?

Your Answer:

Living in an urban area: INCREASED chance of CVA (coefficient: +0.035). Urban residents in this model are more likely to be classified as CVA-positive. This could reflect urban lifestyle factors, stress, or data-specific patterns.

Being married: INCREASED chance of CVA (coefficient: +0.023). The coefficient for "ever_married_Yes" is positive, suggesting married individuals are slightly more likely to have CVA. This is counterintuitive and may reflect confounding with age (older people are more likely to be married and have higher CVA risk).

BMI: INCREASED chance of CVA (coefficient: +0.017). Higher BMI is associated with increased CVA risk, consistent with medical literature showing obesity as a stroke risk factor.

Important note: These coefficients represent the scaled logistic regression model. The positive coefficients mean these features push the log-odds toward the CVA-positive class. The magnitudes are small (0.01-0.04), suggesting modest individual effects — the model relies more heavily on age and glucose level (not shown here) for strong predictions.

---

Question 11

> Do the different regularization methods change the feature weights? If so, which features are affected?

Your Answer:

No, the different regularization methods produce identical feature weights:

L1 regularization: 94.75% train, 95.50% test accuracy
L2 regularization: 94.75% train, 95.50% test accuracy
No regularization: 94.75% train, 95.50% test accuracy

All three methods achieve identical performance on both training and test sets, suggesting that:

1. The problem doesn't have overfitting: Since the regularized and unregularized models achieve identical performance, there's no evidence that the model is overfitting. The lack of difference between regularization methods indicates that the unregularized model's coefficients are already reasonable in magnitude.

2. Different sparsity patterns: While L1 and L2 regularization have different effects on coefficient magnitudes (L1 tends toward sparsity, L2 toward smaller values), they apparently converge to similar predictions in this case.

3. Strong signal dominates: The numeric features (age, glucose) provide such strong predictive signals that the regularization penalty doesn't significantly alter which features the model relies on.

4. Default hyperparameters: The default regularization strengths (C=1.0) may not be strong enough to cause visible differences in this particular dataset.

The fact that regularization doesn't change weights suggests the model is well-behaved and not overfitting, which is a positive sign.

---

Question 12

> Based on the feature weights across the different regularization schemes above, would you say that regularization helps prevent overfitting in this case? Explain.

Your Answer:

No, regularization does not appear to help prevent overfitting in this case. Here's the evidence:

1. Identical performance: All three regularization schemes achieve the same training and test accuracy (94.75% and 95.50%), with no performance gap that would indicate overfitting.

2. Low train-test gap: The difference between training accuracy (94.75%) and test accuracy (95.50%) is actually negative — the model performs *better* on the test set. This is statistically unusual (likely due to randomness or class distribution differences) but indicates no overfitting.

3. No regularization needed: If the model were overfitting, we would expect to see: - Higher training accuracy than test accuracy - L1/L2 regularization reducing training accuracy while improving test accuracy - Neither of these patterns is observed

4. Well-behaved model: Logistic regression with default hyperparameters already generalizes well on this dataset. The strong predictive signals from age and glucose level dominate, making the model robust.

Conclusion: This dataset and problem are well-suited to logistic regression. The model has found a good solution that generalizes well without needing regularization to prevent overfitting. However, regularization is still a good practice in general because it makes the model more interpretable and robust to different data distributions.

---

Part 3: Linear Discriminant Analysis and Final Performance Evaluation

Question 13

> Can the coefficients that are extracted from LDA be interpreted as feature importances? Why or why not?

Your Answer:

Partially yes, but with important caveats:

LDA coefficients represent the linear discriminant function — a weighted linear combination of features that best separates the two classes. They can be loosely interpreted as feature importances, but with significant differences from tree-based importance:

Similarities to importance:

Larger absolute coefficient magnitude → the feature contributes more to the classification decision
The sign indicates direction (positive = pushes toward CVA class)
Can identify which features are most relevant to the decision boundary

Important differences: 1. They are not normalized by feature variance: LDA coefficients are not scaled by how informative or variable each feature is, unlike decision tree importance which measures information gain.

2. They represent linear relationships only: If features interact non-linearly, LDA coefficients won't capture that. Tree importance can capture feature interactions.

3. Scaling matters: LDA coefficients depend on the scale of features. After standardization, scaling is applied implicitly. Tree importance is scale-invariant.

4. Collinearity effects: When features are correlated, LDA coefficients can be unstable (one feature's coefficient might increase while a correlated feature's decreases). Tree importance is more stable to collinearity.

5. Not probability-weighted importance: Unlike decision tree splits which show how frequently a feature is used, LDA coefficients just show linear contribution.

Conclusion: LDA coefficients can be *suggestive* of feature importance for linear classification, but tree importance or permutation importance are better measures of true predictive importance.

---

Question 14

> Now that you have run three different classification methods (decision trees, logistic regression, LDA) with various parameters, which method do you think is best suited for the CVA prediction problem? Explain.

Your Answer:

Best overall: Logistic Regression or Optimized Decision Tree (depth=1) — both achieve 95.50% test accuracy

Here's the comparison:

Test Accuracy Results:

Decision Tree (depth=1): 95.50%
Logistic Regression (scaled): 95.50%
LDA: 95.01%

Why Logistic Regression or Simple Decision Tree?

1. Tied for best performance: Both achieve identical 95.50% test accuracy, significantly outperforming LDA (95.01%).

2. Logistic Regression advantages: - Probabilistic outputs (confidence scores for predictions) - Well-calibrated for medical applications - Extensive use in clinical practice - Easy to implement as a deployed model

3. Simple Decision Tree advantages: - Highly interpretable (single feature threshold) - Explainable to non-technical stakeholders - No feature scaling needed - Easy to implement in clinical systems

Why LDA is less suitable:

Lower performance (95.01% vs 95.50%)
Assumes equal class covariance (likely violated with class imbalance)
Less interpretable than logistic regression or decision trees

Recommendation: For a clinical stroke prediction system, I would choose logistic regression because:

It matches the best decision tree's accuracy
It provides probability estimates (useful for risk stratification)
It's the standard approach in medical ML
Clinicians can understand "increased odds of stroke" interpretation

However, if interpretability is paramount (e.g., for explaining individual predictions to patients), the depth-1 decision tree is an excellent alternative.

---

Question 15

> Which method would you use for identifying risk factors for CVAs? Explain.

Your Answer:

Best for identifying risk factors: Logistic Regression with proper interpretation

Here's why each method differs in this regard:

1. Logistic Regression (BEST for risk factors)

Coefficients directly indicate risk direction: Positive coefficient = increased risk factor
Interpretable magnitudes: Can discuss "for each unit increase in age, odds of CVA increase by X%"
Standard in epidemiology: This is how clinicians and epidemiologists identify risk factors
Provides confidence intervals: Can assess which factors are statistically significant
Examples from our model: Urban residence (+0.035), marriage (+0.023), BMI (+0.017) all increase CVA odds

2. Decision Trees (MODERATE)

Feature importance shows predictive value: Age (39.5%) and glucose (29.9%) are most important
But not causal: Importance shows which features the tree splits on, not which cause CVA
Limited interpretability: Hard to explain why a feature is important beyond "it helps predictions"
Simple model limitation: Depth-1 tree only shows one split, missing more nuanced relationships

3. LDA (WORST for identifying risk factors)

Coefficients are hard to interpret: They represent the discriminant function, not clinical risk
Depends on covariance structure: Interpretability is complicated by assumptions about class covariance
Not standard for risk factor identification: Epidemiologists don't typically use LDA for this purpose

Recommendation: Use logistic regression to identify risk factors because: 1. Coefficients have clear probabilistic interpretation 2. Standard approach in medical literature and clinical practice 3. Can perform statistical significance testing 4. Results are actionable for clinical decision-making (which factors should clinicians monitor?)

Important caveat: Machine learning feature importance (whether from trees or regression) shows *association*, not *causation*. True causal risk factors require controlled experiments or prospective cohort studies.

COMP 341 Homework 3: Stroke Prediction — Grading Report

Part 0: Getting Familiar with the Data

Question 1: How many individuals in this dataset? (2 pts)

Score: 2/2

The student correctly identifies 5,108 individuals. This matches the expected ~5,000 count. The note about "5,109 rows including the header row" shows careful attention to detail.

---

Question 2: Feature types (4 pts)

Score: 4/4

The student provides a thorough and well-organized categorization:

Nominal: sex, ever_married, work_type, residence_type, smoking_status — all correct
Ordinal: None — reasonable, with a note that hypertension/heart_disease are binary indicators
Numeric: age, avg_glucose_level, bmi — correct; hypertension and heart_disease placed here as binary indicators is a defensible choice

The classification is accurate and the student shows understanding of the distinction between nominal, ordinal, and numeric types. Placing hypertension/heart_disease under numeric as "binary indicators" is a reasonable alternative to calling them nominal.

---

Question 3: Missing values impact (5 pts)

Score: 5/5

Excellent, data-driven response. The student:

Identifies specific features with missing values (BMI: 201, age: 14)
Calculates percentages (~4% and <0.3%)
Discusses why the impact may be limited (small proportion)
Notes the potential for bias if missingness is related to the outcome (MNAR concern)
Discusses mean imputation as a reasonable approach
Acknowledges that BMI could be predictive

This goes beyond the minimum expected answer with thoughtful reasoning about missing data mechanisms.

---

Question 4: Most informative features (5 pts)

Score: 5/5

Strong answer with medical/domain reasoning:

Identifies age, avg_glucose_level, hypertension, heart_disease as top candidates
Provides medical justification for each (vascular damage, arterial plaques, etc.)
Notes that BMI may be mediated through other variables
Correctly suggests categorical features may be weaker predictors
Bonus: notes class imbalance (254 vs 4,855), showing thorough data exploration

---

Part 1: Decision Trees

Question 5: Categorical features effect (4 pts)

Score: 4/4

The student reports specific accuracy values (90.70% → 90.31%) and provides four well-reasoned explanations for the slight decrease: 1. Weak predictive power of categorical features 2. Increased dimensionality from one-hot encoding 3. Class imbalance interaction 4. Strong numeric signal already captured

This is a thorough, data-driven response.

---

Question 6: Scaling effect on trees (4 pts)

Score: 4/4

Excellent explanation. The student:

Reports specific accuracy values (90.31% → 90.36%, negligible difference)
Correctly explains that decision trees are scale-invariant due to threshold-based splitting
Contrasts with distance-based models (KNN, SVM)
Notes this applies to all tree-based methods (random forests, gradient boosting)

This demonstrates strong ML intuition about why scaling doesn't affect tree-based models.

---

Question 7: Optimal max depth (4 pts)

Score: 4/4

The student identifies depth 1 as optimal with 95.50% test accuracy. While this is a surprising result, the student provides excellent reasoning:

Discusses the simplicity vs. complexity tradeoff
Notes deeper trees show perfect training accuracy (overfitting indicator)
Connects to class imbalance (a depth-1 stump likely predicts the majority class well)
References Occam's Razor appropriately
Acknowledges that depths 7-8 might achieve similar performance

The reasoning about the train/test gap and overfitting is exactly what the rubric looks for.

---

Question 8: Feature importance comparison (4 pts)

Score: 4/4

The student directly compares actual importances to their Q4 predictions:

Age (39.5%) — confirmed as #1, matching expectations
avg_glucose_level (29.9%) — confirmed as #2
BMI (16.1%) — noted as weaker, with mediation explanation
Notes surprises: hypertension and heart_disease have low importance (1-3%), acknowledging this is somewhat unexpected but explaining why (captured by age/glucose)
Categorical features confirmed as weak predictors

This is exactly the comparison the rubric expects, with specific numbers and thoughtful analysis.

---

Part 2: Logistic Regression

Question 9: Scaling effect on logistic regression (4 pts)

Score: 3/4

The student reports no effect on accuracy (95.50% both ways), which is their empirical result. The rubric expects an explanation of why scaling *typically* helps logistic regression (gradient convergence, equal feature contribution). The student does provide this general knowledge ("scaling is still a best practice... makes the regularization penalty more meaningful... can improve convergence speed") and offers reasonable explanations for why it didn't matter here (strong signal, one-hot features already 0-1, convergence success).

However, the answer leads with "No, scaling had no effect" which could be misleading — the student should have been clearer that scaling *should* matter in principle for logistic regression even if it didn't change accuracy in this specific case. The general explanation is present but somewhat buried. Deducting 1 point for not leading with the correct general principle more strongly.

---

Question 10: Coefficient interpretation (4 pts)

Score: 4/4

The student correctly interprets all three:

Urban residence: positive coefficient (+0.035) → increased CVA risk ✓
Being married: positive coefficient (+0.023) → increased CVA risk ✓ (with insightful note about age confounding)
BMI: positive coefficient (+0.017) → increased CVA risk ✓

Adds appropriate context about coefficient magnitudes being small and the model relying more on age/glucose. The confounding note for marriage is particularly insightful.

---

Question 11: Regularization effects on weights (4 pts)

Score: 2/4

The student reports that all three regularization methods produce identical feature weights and identical performance (94.75% train, 95.50% test). While this may be an accurate empirical result for their specific run, the answer lacks the expected theoretical discussion of *how* L1 and L2 regularization differ:

The rubric expects explanation that L1 tends to zero out coefficients (sparsity/feature selection)
L2 shrinks coefficients but rarely to zero
Specific features that are affected

The student mentions these concepts briefly ("L1 tends toward sparsity, L2 toward smaller values") but frames the entire answer around "no difference observed" without deeply engaging with the theoretical differences. Point 2 mentions "different sparsity patterns" but then says they "converge to similar predictions," which doesn't address the actual coefficient values. Without reporting or comparing specific coefficient values across methods, the answer is incomplete. Awarding 2/4 for noting the general concepts but lacking specific evidence and depth.

---

Question 12: Regularization and overfitting (4 pts)

Score: 3/4

The student argues that regularization doesn't help here, supported by:

Identical performance across all three schemes
Low (negative) train-test gap
No evidence of overfitting patterns

This is a reasonable, evidence-based argument. However, the reasoning is somewhat circular — if all regularization methods give the same results, that could mean the default C=1.0 isn't strong enough, not necessarily that the model doesn't need regularization. The student does note this possibility briefly but doesn't explore it. The answer is solid but could be more nuanced. Minor deduction for not exploring alternative explanations more deeply.

---

Part 3: LDA and Final Evaluation

Question 13: LDA coefficient interpretation (4 pts)

Score: 4/4

Excellent, nuanced response. The student:

Correctly identifies LDA coefficients as discriminant function weights
Lists five specific caveats (not normalized by variance, linear only, scaling dependence, collinearity effects, not probability-weighted)
Contrasts with tree-based importance and regression coefficients
Concludes appropriately that they are "suggestive" but not ideal measures of importance

This demonstrates strong understanding of the differences between model types.

---

Question 14: Best method for CVA prediction (4 pts)

Score: 4/4

Thorough comparison using multiple criteria:

Reports accuracy for all three methods (DT: 95.50%, LR: 95.50%, LDA: 95.01%)
Discusses advantages of each top method (probabilistic outputs, interpretability)
Explains why LDA is less suitable (lower performance, violated assumptions)
Recommends logistic regression with clinical context reasoning
Notes the depth-1 tree as an interpretable alternative

Uses accuracy, interpretability, and practical deployment considerations — exactly what the rubric requires.

---

Question 15: Best method for risk factor identification (4 pts)

Score: 4/4

Outstanding answer. The student:

Clearly recommends logistic regression with strong justification
Explains why coefficients map to odds ratios (standard epidemiological tool)
Contrasts all three methods for this specific use case
Ranks them (LR best, DT moderate, LDA worst) with specific reasoning for each
Includes the important correlation vs. causation caveat
Notes that logistic regression is the standard in medical literature

This demonstrates excellent ML intuition about interpretability for clinical applications.

---

Grading Summary

Question	Topic	Points Earned	Max Points
Q1	Number of individuals	2	2
Q2	Feature types	4	4
Q3	Missing values impact	5	5
Q4	Informative features prediction	5	5
Q5	Categorical features effect	4	4
Q6	Scaling effect on trees	4	4
Q7	Optimal max depth	4	4
Q8	Feature importance comparison	4	4
Q9	Scaling effect on logistic regression	3	4
Q10	Coefficient interpretation	4	4
Q11	Regularization effects	2	4
Q12	Regularization and overfitting	3	4
Q13	LDA coefficient interpretation	4	4
Q14	Best method for prediction	4	4
Q15	Best method for risk factors	4	4
Total Written		52	60

Overall Comments

This is a strong submission demonstrating solid ML understanding and good scientific reasoning. The student consistently provides specific numbers, domain knowledge, and thoughtful analysis. Key strengths include excellent data exploration (Part 0), correct understanding of scale invariance for trees (Q6), insightful confounding analysis (Q10 marriage/age), and outstanding clinical reasoning (Q14-Q15).

The main weakness is in the regularization section (Q11-Q12), where the student reports identical results across all regularization methods but doesn't deeply engage with the theoretical differences between L1 and L2 or examine whether actual coefficient values differ. The student also could have been stronger in Q9 by leading with the general principle that scaling matters for gradient-based optimization before discussing their specific results.

Total: 52/60

.mcp.json

diff --git a/.mcp.json b/.mcp.json
deleted file mode 100644
index 3f91be2..0000000
--- a/.mcp.json
+++ /dev/null
@@ -1,11 +0,0 @@
-{
-  "mcpServers": {
-    "bscs-bench": {
-      "command": "/Users/bebe/Code/bscs-bench/bin/bscs-mcp",
-      "args": [
-        "--workspace",
-        "/Users/bebe/Code/bscs-bench/workspaces/comp341_hw3_haiku"
-      ]
-    }
-  }
-}
\ No newline at end of file

Agent Work: Stroke Prediction

Homework 3: Stroke Prediction (Classification)

Overview

Tasks

Part 0: Getting Familiar with the Data (13 pts)

Data Preparation

Part 1: Decision Trees (30 pts)

Step 1: Numeric Features Only

Step 2: One-Hot Encode Categorical Features

Step 3: Scale Numeric Features

Step 4: Tune max_depth

Step 5: Feature Importances

Part 2: Logistic Regression (30 pts)

Step 1: Unscaled Logistic Regression

Step 2: Scaled Logistic Regression

Step 3: Extract Coefficients

Step 4: Compare Regularization

Part 3: Linear Discriminant Analysis (12 pts)

Step 1: Train LDA

Step 2: Extract Coefficients

General Coding Style (15 pts)

Functions to Implement

Hints

Grading

Extra Credit (up to 10 pts)

COMP 341 Homework 3: Cerebrovascular Accident (Stroke) Prediction

Written Questions

Part 0: Getting Familiar with the Data

Question 1

Question 2

Question 3

Question 4

Part 1: Decision Trees

Question 5

Question 6

Question 7

Question 8

Part 2: Logistic Regression

Question 9

Question 10

Question 11

Question 12

Part 3: Linear Discriminant Analysis and Final Performance Evaluation

Question 13

Question 14

Question 15

COMP 341 Homework 3: Stroke Prediction — Grading Report

Part 0: Getting Familiar with the Data

Question 1: How many individuals in this dataset? (2 pts)

Question 2: Feature types (4 pts)

Question 3: Missing values impact (5 pts)

Question 4: Most informative features (5 pts)

Part 1: Decision Trees

Question 5: Categorical features effect (4 pts)

Question 6: Scaling effect on trees (4 pts)

Question 7: Optimal max depth (4 pts)

Question 8: Feature importance comparison (4 pts)

Part 2: Logistic Regression

Question 9: Scaling effect on logistic regression (4 pts)

Question 10: Coefficient interpretation (4 pts)

Question 11: Regularization effects on weights (4 pts)

Question 12: Regularization and overfitting (4 pts)

Part 3: LDA and Final Evaluation

Question 13: LDA coefficient interpretation (4 pts)

Question 14: Best method for CVA prediction (4 pts)

Question 15: Best method for risk factor identification (4 pts)

Grading Summary

Overall Comments

Sub-Model Usage