{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Postprocessing parameter in compile method\n",
"\n",
"Compile method is a method that creates the explainer you need for your model.
This compile method has many parameters, and among those is `postprocessing` parameter, that will be explained in this tutorial.
\n",
"This parameter allows to **modify** the dataset with several techniques, for a better visualization.\n",
"This tutorial presents the different way you can modify data, and the right syntax to do it.\n",
"\n",
"Contents:\n",
"- Loading dataset and fitting a model.\n",
"\n",
"- Creating our SmartExplainer and compiling it without postprocessing.\n",
"\n",
"- New SmartExplainer with postprocessing parameter.\n",
"\n",
"\n",
"Data from Kaggle: [Titanic](https://www.kaggle.com/c/titanic/data)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.ensemble import RandomForestClassifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Building Supervized Model "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### First step : Importing our dataset"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from shapash.data.data_loader import data_loading\n",
"titanic_df, titanic_dict = data_loading('titanic')\n",
"y_df=titanic_df['Survived']\n",
"X_df=titanic_df[titanic_df.columns.difference(['Survived'])]"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n", " | Survived | \n", "Pclass | \n", "Name | \n", "Sex | \n", "Age | \n", "SibSp | \n", "Parch | \n", "Fare | \n", "Embarked | \n", "Title | \n", "
---|---|---|---|---|---|---|---|---|---|---|
PassengerId | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
1 | \n", "0 | \n", "Third class | \n", "Braund Owen Harris | \n", "male | \n", "22.0 | \n", "1 | \n", "0 | \n", "7.25 | \n", "Southampton | \n", "Mr | \n", "
2 | \n", "1 | \n", "First class | \n", "Cumings John Bradley (Florence Briggs Thayer) | \n", "female | \n", "38.0 | \n", "1 | \n", "0 | \n", "71.28 | \n", "Cherbourg | \n", "Mrs | \n", "
3 | \n", "1 | \n", "Third class | \n", "Heikkinen Laina | \n", "female | \n", "26.0 | \n", "0 | \n", "0 | \n", "7.92 | \n", "Southampton | \n", "Miss | \n", "
4 | \n", "1 | \n", "First class | \n", "Futrelle Jacques Heath (Lily May Peel) | \n", "female | \n", "35.0 | \n", "1 | \n", "0 | \n", "53.10 | \n", "Southampton | \n", "Mrs | \n", "
5 | \n", "0 | \n", "Third class | \n", "Allen William Henry | \n", "male | \n", "35.0 | \n", "0 | \n", "0 | \n", "8.05 | \n", "Southampton | \n", "Mr | \n", "
\n", " | Age | \n", "Embarked | \n", "Fare | \n", "Name | \n", "Parch | \n", "Pclass | \n", "Sex | \n", "SibSp | \n", "Title | \n", "
---|---|---|---|---|---|---|---|---|---|
PassengerId | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
863 | \n", "48.0 | \n", "Southampton | \n", "25.93 | \n", "Swift Frederick Joel (Margaret Welles Barron) | \n", "0 | \n", "First class | \n", "female | \n", "0 | \n", "Mrs | \n", "
224 | \n", "29.5 | \n", "Southampton | \n", "7.90 | \n", "Nenkoff Christo | \n", "0 | \n", "Third class | \n", "male | \n", "0 | \n", "Mr | \n", "
85 | \n", "17.0 | \n", "Southampton | \n", "10.50 | \n", "Ilett Bertha | \n", "0 | \n", "Second class | \n", "female | \n", "0 | \n", "Miss | \n", "
681 | \n", "29.5 | \n", "Queenstown | \n", "8.14 | \n", "Peters Katie | \n", "0 | \n", "Third class | \n", "female | \n", "0 | \n", "Miss | \n", "
536 | \n", "7.0 | \n", "Southampton | \n", "26.25 | \n", "Hart Eva Miriam | \n", "2 | \n", "Second class | \n", "female | \n", "0 | \n", "Miss | \n", "
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
507 | \n", "33.0 | \n", "Southampton | \n", "26.00 | \n", "Quick Frederick Charles (Jane Richards) | \n", "2 | \n", "Second class | \n", "female | \n", "0 | \n", "Mrs | \n", "
468 | \n", "56.0 | \n", "Southampton | \n", "26.55 | \n", "Smart John Montgomery | \n", "0 | \n", "First class | \n", "male | \n", "0 | \n", "Mr | \n", "
741 | \n", "29.5 | \n", "Southampton | \n", "30.00 | \n", "Hawksford Walter James | \n", "0 | \n", "First class | \n", "male | \n", "0 | \n", "Mr | \n", "
355 | \n", "29.5 | \n", "Cherbourg | \n", "7.22 | \n", "Yousif Wazli | \n", "0 | \n", "Third class | \n", "male | \n", "0 | \n", "Mr | \n", "
450 | \n", "52.0 | \n", "Southampton | \n", "30.50 | \n", "Peuchen Arthur Godfrey | \n", "0 | \n", "First class | \n", "male | \n", "0 | \n", "Major | \n", "
223 rows × 9 columns
\n", "\n", " | Age | \n", "Embarked | \n", "Fare | \n", "Name | \n", "Parch | \n", "Pclass | \n", "Sex | \n", "SibSp | \n", "Title | \n", "
---|---|---|---|---|---|---|---|---|---|
PassengerId | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
863 | \n", "48.0 years old | \n", "SOUTHAMPTON | \n", "$25.93 | \n", "Swift Frederick Joel (Margaret Welles Barron) | \n", "0 | \n", "First | \n", "Woman | \n", "0 | \n", "Mrs | \n", "
224 | \n", "29.5 years old | \n", "SOUTHAMPTON | \n", "$7.9 | \n", "Nenkoff Christo | \n", "0 | \n", "Third | \n", "Man | \n", "0 | \n", "Mr | \n", "
85 | \n", "17.0 years old | \n", "SOUTHAMPTON | \n", "$10.5 | \n", "Ilett Bertha | \n", "0 | \n", "Second | \n", "Woman | \n", "0 | \n", "Miss | \n", "
681 | \n", "29.5 years old | \n", "QUEENSTOWN | \n", "$8.14 | \n", "Peters Katie | \n", "0 | \n", "Third | \n", "Woman | \n", "0 | \n", "Miss | \n", "
536 | \n", "7.0 years old | \n", "SOUTHAMPTON | \n", "$26.25 | \n", "Hart Eva Miriam | \n", "2 | \n", "Second | \n", "Woman | \n", "0 | \n", "Miss | \n", "
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
507 | \n", "33.0 years old | \n", "SOUTHAMPTON | \n", "$26.0 | \n", "Quick Frederick Charles (Jane Richards) | \n", "2 | \n", "Second | \n", "Woman | \n", "0 | \n", "Mrs | \n", "
468 | \n", "56.0 years old | \n", "SOUTHAMPTON | \n", "$26.55 | \n", "Smart John Montgomery | \n", "0 | \n", "First | \n", "Man | \n", "0 | \n", "Mr | \n", "
741 | \n", "29.5 years old | \n", "SOUTHAMPTON | \n", "$30.0 | \n", "Hawksford Walter James | \n", "0 | \n", "First | \n", "Man | \n", "0 | \n", "Mr | \n", "
355 | \n", "29.5 years old | \n", "CHERBOURG | \n", "$7.22 | \n", "Yousif Wazli | \n", "0 | \n", "Third | \n", "Man | \n", "0 | \n", "Mr | \n", "
450 | \n", "52.0 years old | \n", "SOUTHAMPTON | \n", "$30.5 | \n", "Peuchen Arthur Godfrey | \n", "0 | \n", "First | \n", "Man | \n", "0 | \n", "Major | \n", "
223 rows × 9 columns
\n", "\n", " | pred | \n", "feature_1 | \n", "value_1 | \n", "contribution_1 | \n", "feature_2 | \n", "value_2 | \n", "contribution_2 | \n", "feature_3 | \n", "value_3 | \n", "contribution_3 | \n", "... | \n", "contribution_6 | \n", "feature_7 | \n", "value_7 | \n", "contribution_7 | \n", "feature_8 | \n", "value_8 | \n", "contribution_8 | \n", "feature_9 | \n", "value_9 | \n", "contribution_9 | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
863 | \n", "1 | \n", "Title of passenger | \n", "Mrs | \n", "0.163479 | \n", "Sex | \n", "Woman | \n", "0.154309 | \n", "Ticket class | \n", "First | \n", "0.130221 | \n", "... | \n", "0.0406219 | \n", "Name, First name | \n", "Swift Frederick Joel (Margaret Welles Barron) | \n", "-0.0381955 | \n", "Port of embarkation | \n", "SOUTHAMPTON | \n", "-0.0147327 | \n", "Relatives like children or parents | \n", "0 | \n", "-0.00538103 | \n", "
224 | \n", "0 | \n", "Title of passenger | \n", "Mr | \n", "0.094038 | \n", "Sex | \n", "Man | \n", "0.0696282 | \n", "Age | \n", "29.5 years old | \n", "0.0658556 | \n", "... | \n", "0.0151605 | \n", "Relatives such as brother or wife | \n", "0 | \n", "-0.00855039 | \n", "Relatives like children or parents | \n", "0 | \n", "0.00124433 | \n", "Name, First name | \n", "Nenkoff Christo | \n", "-0.000577095 | \n", "
85 | \n", "1 | \n", "Title of passenger | \n", "Miss | \n", "0.190529 | \n", "Sex | \n", "Woman | \n", "0.135507 | \n", "Ticket class | \n", "Second | \n", "0.0809714 | \n", "... | \n", "-0.025286 | \n", "Relatives like children or parents | \n", "0 | \n", "-0.0238222 | \n", "Relatives such as brother or wife | \n", "0 | \n", "0.0209045 | \n", "Age | \n", "17.0 years old | \n", "-0.00702283 | \n", "
681 | \n", "1 | \n", "Title of passenger | \n", "Miss | \n", "0.237477 | \n", "Port of embarkation | \n", "QUEENSTOWN | \n", "0.143451 | \n", "Sex | \n", "Woman | \n", "0.127931 | \n", "... | \n", "0.0243567 | \n", "Relatives like children or parents | \n", "0 | \n", "0.0165205 | \n", "Passenger fare | \n", "$8.14 | \n", "-0.0109633 | \n", "Age | \n", "29.5 years old | \n", "0.00327866 | \n", "
536 | \n", "1 | \n", "Title of passenger | \n", "Miss | \n", "0.210166 | \n", "Ticket class | \n", "Second | \n", "0.168247 | \n", "Sex | \n", "Woman | \n", "0.0876445 | \n", "... | \n", "0.0147503 | \n", "Relatives like children or parents | \n", "2 | \n", "0.0125069 | \n", "Port of embarkation | \n", "SOUTHAMPTON | \n", "-0.0119119 | \n", "Name, First name | \n", "Hart Eva Miriam | \n", "0.00654165 | \n", "
... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
507 | \n", "1 | \n", "Title of passenger | \n", "Mrs | \n", "0.215332 | \n", "Sex | \n", "Woman | \n", "0.194419 | \n", "Ticket class | \n", "Second | \n", "0.166437 | \n", "... | \n", "-0.0079185 | \n", "Relatives like children or parents | \n", "2 | \n", "0.00407485 | \n", "Age | \n", "33.0 years old | \n", "-0.00263589 | \n", "Name, First name | \n", "Quick Frederick Charles (Jane Richards) | \n", "0.00162901 | \n", "
468 | \n", "0 | \n", "Sex | \n", "Man | \n", "0.100602 | \n", "Passenger fare | \n", "$26.55 | \n", "-0.099794 | \n", "Title of passenger | \n", "Mr | \n", "0.0967768 | \n", "... | \n", "0.0243706 | \n", "Port of embarkation | \n", "SOUTHAMPTON | \n", "0.0124424 | \n", "Relatives such as brother or wife | \n", "0 | \n", "-0.0108301 | \n", "Relatives like children or parents | \n", "0 | \n", "-0.00332632 | \n", "
741 | \n", "0 | \n", "Title of passenger | \n", "Mr | \n", "0.131861 | \n", "Sex | \n", "Man | \n", "0.110845 | \n", "Age | \n", "29.5 years old | \n", "0.104878 | \n", "... | \n", "0.0339308 | \n", "Relatives such as brother or wife | \n", "0 | \n", "-0.00715564 | \n", "Name, First name | \n", "Hawksford Walter James | \n", "0.00165882 | \n", "Relatives like children or parents | \n", "0 | \n", "-0.00137946 | \n", "
355 | \n", "0 | \n", "Title of passenger | \n", "Mr | \n", "0.12679 | \n", "Sex | \n", "Man | \n", "0.0933251 | \n", "Age | \n", "29.5 years old | \n", "0.0717939 | \n", "... | \n", "-0.0271103 | \n", "Name, First name | \n", "Yousif Wazli | \n", "0.0163174 | \n", "Relatives such as brother or wife | \n", "0 | \n", "-0.0108501 | \n", "Relatives like children or parents | \n", "0 | \n", "-0.000543508 | \n", "
450 | \n", "0 | \n", "Sex | \n", "Man | \n", "0.13572 | \n", "Title of passenger | \n", "Major | \n", "-0.0723023 | \n", "Age | \n", "52.0 years old | \n", "0.0690373 | \n", "... | \n", "0.027384 | \n", "Relatives such as brother or wife | \n", "0 | \n", "-0.0134144 | \n", "Relatives like children or parents | \n", "0 | \n", "0.00256623 | \n", "Name, First name | \n", "Peuchen Arthur Godfrey | \n", "0.00229483 | \n", "
223 rows × 28 columns
\n", "