{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The objective of this small project is to train a model to predict whether a breast lesion is malignant or benign, from data obtained by biopsies.\n", "\n", "The data comes from the University of Wisconsin. It consists of features from digitized images of fine needle aspirates of a breast mass.\n", "\n", "\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The features describe characteristics of the cell nuclei present in the image. Namely, ten real-valued features are computed for each cell nucleus in the digital image:\n", "\n", "1) radius (mean of distances from center to points on the perimeter)\n", "\n", "2) texture (standard deviation of gray-scale values)\n", "\n", "3) perimeter\n", "\n", "4) area\n", "\n", "5) smoothness (local variation in radius lengths)\n", "\n", "6) compactness (perimeter^2 / area - 1.0)\n", "\n", "7) concavity (severity of concave portions of the contour)\n", "\n", "8) concave points (number of concave portions of the contour)\n", "\n", "9) symmetry\n", "\n", "10) fractal dimension\n", "\n", "\"\"\n", "\n", "(picture from Chain, K., Legesse, T., Heath, J.E. and Staats, P.N. (2019), Digital image‐assisted quantitative nuclear analysis improves diagnostic accuracy of thyroid fine‐needle aspiration cytology. Cancer Cytopathology, 127: 501-513. doi:10.1002/cncy.22120)\n", "\n", "The mean value, standard error and and \"worst\" or largest (mean of the three largest values) of these features were computed for each image, resulting in 30 features per instance (patient). The first and two columns correspond to the patient IDs and the diagnosis outcome (M=malignant, B=benign)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can find more information and other data sets at [this link](https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29).\n", "\n", "Credit: a lot of this tutorial has been adapted from [here](https://towardsdatascience.com/breast-cancer-cell-type-classifier-ace4e82f9a79)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Load data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the function `pd.read_csv` load the data set with no header" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "url_data = \"https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the method `head()` of a dataframe, display the first 5 lines of the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How many patients are present in the data set?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Change the name of the columns of the `df` so that they match the features. Use the `columns` attribute of a `df`" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "features_names = [\n", " 'radius',\n", " 'texture',\n", " 'perimeter',\n", " 'area',\n", " 'smoothness',\n", " 'compactness',\n", " 'concavity',\n", " 'concave_points',\n", " 'symmetry',\n", " 'fractal_dimension'\n", "]\n", "features_names_SE = [feature+'_SE' for feature in features_names]\n", "features_names_W = [feature+'_W' for feature in features_names]" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "column_names = ['ID', 'diagnosis']+features_names+features_names_SE+features_names_W" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "df.columns = column_names" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Drop the 'ID' column (function `drop()`), as well as the columns with standard errors and worst values of the parameters." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Check if there are any missing values. To do so, first define a function `num_missing()` that takes a Series (=a column) and returns the number of missing values in it (use `pd.isna()`). Then use the `apply()` method to apply this function to each column of `df`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Check the types of every column (`dtypes` attribute)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What is the type of the entries of the 'diagnosis' column?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Data exploration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How many patients have a benign lesion and how many have a malignant tumor? What are the corresponding percentages?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the method `value_counts()` and the Series method `plot(kind='bar')`, visualize these numbers as bars" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the `swarmplot()` function of the seaborn package, plot swarmplots comparing radius, area and concavity between bening and malignant" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use a for loop over the `features_names` and plot the histograms of each feature, with one histogram for the benign lesions and one histogram for the malignant tumors (both histograms on same figure, colors red and green). Use transparency (parameter `alpha=0.5`) when plotting histogram (method `.hist()`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Same with kernel density estimate (kde) plots (method `.plot.kde()` or function `sns.kde()`)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the `sns.pairplot()` function from seaborn, with `palette=('r','g')` and `hue='diagnosis'` to plot benign and malignant in different colors, plot all 2x2 plots of features against each other" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Do you observe correlations? Do they make sense?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use the `.corr()` method to compute the correlation matrix of the data. Use the `plt.matshow()` function (a pretty cmap is `plt.cm.Blues`) to display it. Use the `plt.xticks(, rotation='vertical')` and `plt.yticks()` functions if you want to display names of features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Preparation of the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Define `X` as containing only the features and `y` containing only the outcome (diagnosis)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Split the data set into a training (`X_train` and `y_train`) and a test set (`X_test` and `y_test`). Put 30% of the data in the test set. Use the `train_test_split()` function from `sklearn`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Look at the variances of the features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There is a wide heteregeneity of values. This suggests to rescale the data, i.e. subtracting mean and dividing by standard deviation. Do this using the `StandardScaler()` function from `sklearn.preprocessing`. Apply the same scaler to `X_train` and `X_test`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Fit model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fit a random forest classifier to the data with `max_depth = 5`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute predictions on the test set (`.predict()` method)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute the accuracy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute the confusion matrix of the prediction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Print the classification report from `sklearn.metrics` to print other metrics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note: sensitivity corresponds to the recall in what is defined as the positive class (say, malignant here) and specificity to the recall in what is defined as the negative class (say, benign). Likewise, precisions correspond to positive and predictive values" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Map the benign entries to 0 and malignant to 1" ] }, { "cell_type": "code", "execution_count": 130, "metadata": {}, "outputs": [], "source": [ "y_test_01 = y_test.map({'B':0, 'M':1})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the probability predictions (of the malignant class) of the random forest classifier as scores, plot the ROC curve and compute AUC. NB: you can obtain the order used for the classes using the attribute `classes_` of the classifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the attribute `feature_importances_` return the list of features ranked by order of importance in the model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plot their relative importances as a barplot with all features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Cross-validation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To evaluate the performances of the model not only on one test-set but on multiple ones, we can use cross-validation. Let's use 10 folds. This means that the data set will be cut ten times into a learning and a test set and each time we will train a random forest classifier." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use the function `cross_validate` from `sklearn.model_selection` to perform the 10-folds cross-validation. Map the `y_train` data to `0` and `1` before. Select the `roc_auc` and `accuracy` as metrics to be computed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute the mean AUC and mean accuracy" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "165px" }, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 2 }