{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "aed4bd9d",
   "metadata": {},
   "source": [
    "# Load Libraries (to make life easier)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fd4e65aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc646e7",
   "metadata": {},
   "source": [
    "# Load Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "18cef1d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Hours Studied</th>\n",
       "      <th>Previous Scores</th>\n",
       "      <th>Extracurricular Activities</th>\n",
       "      <th>Sleep Hours</th>\n",
       "      <th>Sample Question Papers Practiced</th>\n",
       "      <th>Performance Index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7</td>\n",
       "      <td>99</td>\n",
       "      <td>Yes</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>91.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>82</td>\n",
       "      <td>No</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>65.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>51</td>\n",
       "      <td>Yes</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>45.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>52</td>\n",
       "      <td>Yes</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>36.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>75</td>\n",
       "      <td>No</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>66.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Hours Studied  Previous Scores Extracurricular Activities  Sleep Hours  \\\n",
       "0              7               99                        Yes            9   \n",
       "1              4               82                         No            4   \n",
       "2              8               51                        Yes            7   \n",
       "3              5               52                        Yes            5   \n",
       "4              7               75                         No            8   \n",
       "\n",
       "   Sample Question Papers Practiced  Performance Index  \n",
       "0                                 1               91.0  \n",
       "1                                 2               65.0  \n",
       "2                                 2               45.0  \n",
       "3                                 2               36.0  \n",
       "4                                 5               66.0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load Data (I put it in github)\n",
    "df = pd.read_csv('https://raw.githubusercontent.com/youronlydimwit/Data_ScienceUse_Cases/main/Predictions/Data/Student_Performance.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c192ce2",
   "metadata": {},
   "source": [
    "# See Basic Information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "93050c50",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 10000 entries, 0 to 9999\n",
      "Data columns (total 6 columns):\n",
      " #   Column                            Non-Null Count  Dtype  \n",
      "---  ------                            --------------  -----  \n",
      " 0   Hours Studied                     10000 non-null  int64  \n",
      " 1   Previous Scores                   10000 non-null  int64  \n",
      " 2   Extracurricular Activities        10000 non-null  object \n",
      " 3   Sleep Hours                       10000 non-null  int64  \n",
      " 4   Sample Question Papers Practiced  10000 non-null  int64  \n",
      " 5   Performance Index                 10000 non-null  float64\n",
      "dtypes: float64(1), int64(4), object(1)\n",
      "memory usage: 468.9+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "abd1878e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Hours Studied</th>\n",
       "      <th>Previous Scores</th>\n",
       "      <th>Sleep Hours</th>\n",
       "      <th>Sample Question Papers Practiced</th>\n",
       "      <th>Performance Index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>10000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>4.992900</td>\n",
       "      <td>69.445700</td>\n",
       "      <td>6.530600</td>\n",
       "      <td>4.583300</td>\n",
       "      <td>55.224800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.589309</td>\n",
       "      <td>17.343152</td>\n",
       "      <td>1.695863</td>\n",
       "      <td>2.867348</td>\n",
       "      <td>19.212558</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>10.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>40.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>5.000000</td>\n",
       "      <td>69.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>55.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.000000</td>\n",
       "      <td>85.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>71.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.000000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>100.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Hours Studied  Previous Scores   Sleep Hours  \\\n",
       "count   10000.000000     10000.000000  10000.000000   \n",
       "mean        4.992900        69.445700      6.530600   \n",
       "std         2.589309        17.343152      1.695863   \n",
       "min         1.000000        40.000000      4.000000   \n",
       "25%         3.000000        54.000000      5.000000   \n",
       "50%         5.000000        69.000000      7.000000   \n",
       "75%         7.000000        85.000000      8.000000   \n",
       "max         9.000000        99.000000      9.000000   \n",
       "\n",
       "       Sample Question Papers Practiced  Performance Index  \n",
       "count                      10000.000000       10000.000000  \n",
       "mean                           4.583300          55.224800  \n",
       "std                            2.867348          19.212558  \n",
       "min                            0.000000          10.000000  \n",
       "25%                            2.000000          40.000000  \n",
       "50%                            5.000000          55.000000  \n",
       "75%                            7.000000          71.000000  \n",
       "max                            9.000000         100.000000  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79667c41",
   "metadata": {},
   "source": [
    "The Column `Extracurricular Activities` is an Object Data type. We need to change it into a numerical through encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ff70a807",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Yes', 'No'], dtype=object)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# See the values\n",
    "df['Extracurricular Activities'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0b6fa2d5",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 10000 entries, 0 to 9999\n",
      "Data columns (total 6 columns):\n",
      " #   Column                            Non-Null Count  Dtype  \n",
      "---  ------                            --------------  -----  \n",
      " 0   Hours Studied                     10000 non-null  int64  \n",
      " 1   Previous Scores                   10000 non-null  int64  \n",
      " 2   Extracurricular Activities        10000 non-null  int64  \n",
      " 3   Sleep Hours                       10000 non-null  int64  \n",
      " 4   Sample Question Papers Practiced  10000 non-null  int64  \n",
      " 5   Performance Index                 10000 non-null  float64\n",
      "dtypes: float64(1), int64(5)\n",
      "memory usage: 468.9 KB\n"
     ]
    }
   ],
   "source": [
    "# Encode the Extracurricular Activities Column\n",
    "df['Extracurricular Activities'] = df['Extracurricular Activities'].apply(lambda x: 1 if x == 'Yes' else 0)\n",
    "df['Extracurricular Activities'].astype('int64')\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b40617da",
   "metadata": {},
   "source": [
    "And now they're all numerical data types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f5bda1df",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 12536 characters]\n",
      "text/plain": [
       "<Figure size 1080x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Attempt to see outliers through boxplots\n",
    "plt.figure(figsize=(15,5))\n",
    "sns.boxplot(data=df)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08e320dc",
   "metadata": {},
   "source": [
    "Data looks clean with no apparent outliers. Next, let's try to do a visual inspection of Linearity.\n",
    "To do that, We can plot every column against the target column, `Performance Index`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "68f1e2dd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 150772 characters]\n",
      "text/plain": [
       "<Figure size 576x936 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Features and target\n",
    "features = ['Hours Studied', 'Previous Scores', 'Extracurricular Activities', 'Sleep Hours', 'Sample Question Papers Practiced']\n",
    "\n",
    "# Create subplots\n",
    "fig, axs = plt.subplots(5, 1, figsize=(8, 13))\n",
    "\n",
    "# Iterate over each feature\n",
    "for i, feature in enumerate(features):\n",
    "    # Scatter plot\n",
    "    sns.scatterplot(x=df[feature], y=df['Performance Index'], ax=axs[i])\n",
    "    \n",
    "    # Linear regression line\n",
    "    sns.regplot(x=df[feature], y=df['Performance Index'], scatter=False, color='red', ax=axs[i])\n",
    "    \n",
    "    # Add title\n",
    "    axs[i].set_title(f'{feature} vs Performance Index')\n",
    "\n",
    "# Adjust layout\n",
    "plt.tight_layout()\n",
    "\n",
    "# Show plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22cf1913",
   "metadata": {},
   "source": [
    "From visual inspection only, we can see from the lines that `Hours Studied` and `Previous Scores` are the two features that visually has the most obvious Linearity Relationship against `Performance Index`. We can expect the two features to be important in determining the predicted value."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5af45b75",
   "metadata": {},
   "source": [
    "# Running (Multiple) Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d593d38c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f50c1f5b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define features and target\n",
    "X = df[['Hours Studied', 'Previous Scores', 'Extracurricular Activities', 'Sleep Hours', 'Sample Question Papers Practiced']]\n",
    "y = df['Performance Index']\n",
    "\n",
    "# Split the data\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "787130e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create and fit the model\n",
    "model = LinearRegression()\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "# Run model\n",
    "predictions = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "df940c4c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature</th>\n",
       "      <th>Coefficient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Intercept</td>\n",
       "      <td>-33.921946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hours Studied</td>\n",
       "      <td>2.852484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Previous Scores</td>\n",
       "      <td>1.016988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Extracurricular Activities</td>\n",
       "      <td>0.608617</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Sleep Hours</td>\n",
       "      <td>0.476941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Sample Question Papers Practiced</td>\n",
       "      <td>0.191831</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            Feature  Coefficient\n",
       "0                         Intercept   -33.921946\n",
       "1                     Hours Studied     2.852484\n",
       "2                   Previous Scores     1.016988\n",
       "3        Extracurricular Activities     0.608617\n",
       "4                       Sleep Hours     0.476941\n",
       "5  Sample Question Papers Practiced     0.191831"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a DataFrame for the coefficients\n",
    "coef_df = pd.DataFrame({\n",
    "    'Feature': X.columns,\n",
    "    'Coefficient': model.coef_\n",
    "})\n",
    "\n",
    "# Add the intercept to the DataFrame\n",
    "intercept_df = pd.DataFrame({\n",
    "    'Feature': ['Intercept'],\n",
    "    'Coefficient': [model.intercept_]\n",
    "})\n",
    "\n",
    "# Combine both DataFrames\n",
    "coef_df = pd.concat([intercept_df, coef_df], ignore_index=True)\n",
    "coef_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acf88422",
   "metadata": {},
   "source": [
    "Based on the coefficients (which reflect the influence of each feature on the target variable) shown in the table above, our observations align with expectations from previous visualizations. Notably, `Hours Studied` and `Previous Scores` stand out with the highest coefficients among all features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "4280581e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error: 4.082628398521853\n",
      "R^2 Score: 0.9889832909573145\n"
     ]
    }
   ],
   "source": [
    "# Linear Regression Metrics\n",
    "from sklearn.metrics import mean_squared_error, r2_score\n",
    "\n",
    "mse = mean_squared_error(y_test, predictions)\n",
    "r2 = r2_score(y_test, predictions)\n",
    "\n",
    "print(\"Mean Squared Error:\", mse)\n",
    "print(\"R^2 Score:\", r2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1130bbd2",
   "metadata": {},
   "source": [
    "## From Results Above, The Formula Can be Put Together As Follows:\n",
    "**Performance Index** = -33.92 + (2.85 x Hours Studied) + (1.01 x Previous Scores) + (0.60 x Extracurricular Activities) + (0.47 x Sleep Hours) + (0.19 x Sample Question Papers Practiced)\n",
    "\n",
    "## Metrics Interpretation\n",
    "- On average, the predicted Performance Index values from the model are about 4.08 units squared away from the actual Performance Index values.\n",
    "- About 98.9% of the variance in the Performance Index can be explained by the independent variables included in your model. This indicates a very strong fit of the model to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c3b27bb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
