Skip to content

Instantly share code, notes, and snippets.

@alonsosilvaallende

alonsosilvaallende/concordance_index_2026-01.ipynb

Created January 16, 2026 12:44
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save alonsosilvaallende/817bb7c49cd2b9f5407c56fb15f536ab to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save alonsosilvaallende/817bb7c49cd2b9f5407c56fb15f536ab to your computer and use it in GitHub Desktop.
Download ZIP
concordance_index_2026-01.ipynb
Display the source blob
Display the rendered blob
Raw
concordance_index_2026-01.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/alonsosilvaallende/817bb7c49cd2b9f5407c56fb15f536ab/concordance_index_2026-01.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yujaTHwquzdV"
},
"source": [
"# Concordance index"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Tg64n0Jjw-3Z",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "28c7484f-37a2-4ae0-c436-4083c89273e9"
},
"source": [
"%pip install --quiet lifelines"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m349.3/349.3 kB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m117.3/117.3 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Building wheel for autograd-gamma (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "EIejd1FnxF0T"
},
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"from matplotlib import pyplot as plt\n",
"\n",
"# import concordance index from lifelines\n",
"from lifelines.utils import concordance_index"
],
"execution_count": 2,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "bD2oGv0aW1du"
},
"source": [
"The _concordance index_ or _c-index_ is a metric to evaluate the predictions made by an algorithm. It is defined as the proportion of concordant pairs divided by the total number of possible evaluation pairs. Let's see through some examples what does this definition mean in practice.\n",
"\n",
"Suppose we are a telecom operator with 5 customers (Alice, Bob, Carol, Dave, and Eve) and we're trying to predict who are going to unsubscribe from our services (also known as churn) first.\n",
"\n",
"Suppose that Alice left after 1 year, Bob after 2 years, Carol after 3 years, Dave after 4 years and Eve after 5 years.\n",
"\n",
"Suppose that our algorithm made the following prediction: Alice will leave after 1 year, Bob after 2 years, Carol after 3 years, Dave after 4 years, and Eve after 5 years. In that case, the concordance index is equal to its maximum value 1."
]
},
{
"cell_type": "code",
"metadata": {
"id": "70XQmwaKXTjT",
"outputId": "b3e5dc33-4771-48f0-ff69-9f806b0b04b6",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [1, 2, 3, 4, 5]\n",
"df = pd.DataFrame(data={'Churn times': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn times Predictions\n",
"Alice 1 1\n",
"Bob 2 2\n",
"Carol 3 3\n",
"Dave 4 4\n",
"Eve 5 5\n",
"Concordance index: 1.0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fELOvbV0am9N"
},
"source": [
"However, the concordance index is interested on the **order of the predictions**, not the predictions themselves. Assume now that the algorithm predicted that: Alice will leave after 2 years, Bob after 3 years, Carol after 5 years, Dave after 8 years and Eve after 14 years. In that case, the concordance index is still equal to its maximum value 1 since the order of the predictions is right."
]
},
{
"cell_type": "code",
"metadata": {
"id": "oH2oytkpXLCj",
"outputId": "45c90772-3c5d-4716-970a-798b14b95449",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [2, 3, 5, 8, 14]\n",
"df = pd.DataFrame(data={'Churn times': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn times Predictions\n",
"Alice 1 2\n",
"Bob 2 3\n",
"Carol 3 5\n",
"Dave 4 8\n",
"Eve 5 14\n",
"Concordance index: 1.0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9IpSAjRWb5Ab"
},
"source": [
"This is very different from other evaluation measures such as [mean squared error](https://en.wikipedia.org/wiki/Mean_squared_error) or [mean absolute error](https://en.wikipedia.org/wiki/Mean_absolute_error). Actually for any strictly increasing function the concordance index will still be equal to 1. Here is an example with the logarithm function:"
]
},
{
"cell_type": "code",
"metadata": {
"outputId": "58816614-6aff-40fa-b2a5-42d1ad2f819b",
"id": "1woJm4ZgcKeU",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = np.log(events)\n",
"df = pd.DataFrame(data={'Churn times': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn times Predictions\n",
"Alice 1 0.000000\n",
"Bob 2 0.693147\n",
"Carol 3 1.098612\n",
"Dave 4 1.386294\n",
"Eve 5 1.609438\n",
"Concordance index: 1.0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TB1Xs8SlbVwr"
},
"source": [
"Let's consider the case where the algorithm got the order of the predictions completely wrong. Let's say the algorithm predicted that Eve will churn first after 1 year, Dave after 2 years, Carol after 3 years, Dave after 4 years, and Alice after 5 years, so the order is completely reversed (although notice that Carol did leave after 3 years!). In that case, the concordance index will be equal to its minimum value 0 since the order of the predictions is wrong."
]
},
{
"cell_type": "code",
"metadata": {
"id": "13hvZDL6ddsn",
"outputId": "52c85c1b-a853-47fa-b676-4da9a85b8690",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [5, 4, 3, 2, 1]\n",
"df = pd.DataFrame(data={'Churn times': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn times Predictions\n",
"Alice 1 5\n",
"Bob 2 4\n",
"Carol 3 3\n",
"Dave 4 2\n",
"Eve 5 1\n",
"Concordance index: 0.0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fEq7I440kuM1"
},
"source": [
"Actually the concordance index will be zero for every strictly decreasing function of the churn times. Here is an example for $f(x)=1/x$:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "mTXu1ZT_k_Aa",
"outputId": "72d59609-62e7-47af-9860-3d4b500bc419",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [1/event for event in events]\n",
"df = pd.DataFrame(data={'Churn dates': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn dates Predictions\n",
"Alice 1 1.000000\n",
"Bob 2 0.500000\n",
"Carol 3 0.333333\n",
"Dave 4 0.250000\n",
"Eve 5 0.200000\n",
"Concordance index: 0.0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wD6u_HRjdkaS"
},
"source": [
"Now comes the interesting part. What does the concordance index do when the order of the predictions is not completely right, nor completely wrong? Which score do you give to the following prediction:\n",
"+ *Prediction 1*: Alice will leave after 3 years, Bob after 2 years, Carol after 1 year, Dave after 5 years, and Eve after 4 years.\n",
"\n",
"Well, the concordance index lists all the possible pairs, in our case there are 10 potential pairs:\n",
"1. Alice and Bob\n",
"2. Alice and Carol\n",
"3. Alice and Dave\n",
"4. Alice and Eve\n",
"5. Bob and Carol\n",
"6. Bob and Dave\n",
"7. Bob and Eve\n",
"8. Carol and Dave\n",
"9. Carol and Eve\n",
"10. Dave and Eve\n",
"\n",
"and then it counts how many of the order of these predictions of pairs were right. For example, for *Prediction 1*:\n",
"\n",
"1. <font color='red'>Alice is predicted to churn after Bob, while in reality Bob churns after Alice.</font>\n",
"2. <font color='red'>Alice is predicted to churn after Carol, while Carol churned after Alice.</font>\n",
"3. <font color='blue'>Alice is predicted to churn before Dave, and Alice did churn before Dave.</font>\n",
"4. <font color='blue'>Alice is predicted to churn before Eve, and Alice did churn before Eve.</font>\n",
"5. <font color='red'>Bob is predicted to churn after Carol, while Carol churned after Bob.</font>\n",
"6. <font color='blue'>Bob is predicted to churn before Dave, and Bob did churn before Dave.</font>\n",
"7. <font color='blue'>Bob is predicted to churn before Eve, and Bob did churn before Eve.</font>\n",
"8. <font color='blue'>Carol is predicted to churn before Dave, and Carol did churn before Dave.</font>\n",
"9. <font color='blue'>Carol is predicted to churn before Eve, and Carol did churn before Eve.</font>\n",
"10. <font color='red'>Dave is predicted to churn after Eve, while Eve churned after Dave.</font>\n",
"\n",
"There are 6 concordant pairs out of 10 possible pairs so the concordance index is equal to 6/10 or 0.6.\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "c6Cn95BBb9HD",
"outputId": "a2b6f47b-fc63-42a1-f975-fd184be194fc",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [3, 2, 1, 5, 4]\n",
"df = pd.DataFrame(data={'Churn times': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn times Predictions\n",
"Alice 1 3\n",
"Bob 2 2\n",
"Carol 3 1\n",
"Dave 4 5\n",
"Eve 5 4\n",
"Concordance index: 0.6\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xfLLk7J6eOwY"
},
"source": [
"What happens in case of ties? Ties are counted as half concordant pair:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2rgbeMkjeBcZ",
"outputId": "c50d4dc5-5244-4560-a79c-598129f3ef4d",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [1, 2, 3, 4, 4]\n",
"df = pd.DataFrame(data={'Churn times': events, 'Predictions': preds}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds)}')"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Churn times Predictions\n",
"Alice 1 1\n",
"Bob 2 2\n",
"Carol 3 3\n",
"Dave 4 4\n",
"Eve 5 4\n",
"Concordance index: 0.95\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Another interesting characteristic of the concordance index is that it supports right censoring, i.e., the case when by the end of the study, the event of interest (for example, in medicine 'death of a patient' or in our example 'churn of a customer') has only occurred for a subset of the observations.\n",
"\n",
"Up to now, we never had to specify if our observations were right censored or not because the concordance index default option is that all events are observed. Let's change that."
],
"metadata": {
"id": "KiMQ2Pmtx-Zu"
}
},
{
"cell_type": "markdown",
"source": [
"Let's restrict our analysis to only Alice and Bob. Suppose our predictions for Alice and Bob are:\n",
"\n",
" - Alice will leave after 1 year\n",
" - Bob will leave after 2 years"
],
"metadata": {
"id": "nbv-aD89yOkp"
}
},
{
"cell_type": "markdown",
"source": [
"There are 4 possible cases:"
],
"metadata": {
"id": "gMmMq31WyjIC"
}
},
{
"cell_type": "markdown",
"source": [
"Case #1: Both events are observed"
],
"metadata": {
"id": "Yv2ADey1y47C"
}
},
{
"cell_type": "code",
"source": [
"concordance_index([1, 2], [1, 2], [True, True]).item()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "sWb7xlS4wbmL",
"outputId": "603f52cd-d9f0-4813-e3f4-f54967441468"
},
"execution_count": 16,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"1.0"
]
},
"metadata": {},
"execution_count": 16
}
]
},
{
"cell_type": "markdown",
"source": [
"Case #2: Both events are right-censored"
],
"metadata": {
"id": "IbEIvxlvzQXm"
}
},
{
"cell_type": "code",
"source": [
"concordance_index([1, 2], [1, 2], [False, False]).item()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 287
},
"id": "ZiNULbgvw1tt",
"outputId": "79d98d51-6812-4097-9c2c-8cc80ef4947e"
},
"execution_count": 18,
"outputs": [
{
"output_type": "error",
"ename": "ZeroDivisionError",
"evalue": "No admissable pairs in the dataset.",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mZeroDivisionError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m/tmp/ipython-input-2783031866.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mconcordance_index\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/usr/local/lib/python3.12/dist-packages/lifelines/utils/concordance.py\u001b[0m in \u001b[0;36mconcordance_index\u001b[0;34m(event_times, predicted_scores, event_observed)\u001b[0m\n\u001b[1;32m 92\u001b[0m \u001b[0mnum_correct\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_tied\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_pairs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_concordance_summary_statistics\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mevent_times\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpredicted_scores\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mevent_observed\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 93\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 94\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_concordance_ratio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_correct\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_tied\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_pairs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 95\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 96\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.12/dist-packages/lifelines/utils/concordance.py\u001b[0m in \u001b[0;36m_concordance_ratio\u001b[0;34m(num_correct, num_tied, num_pairs)\u001b[0m\n\u001b[1;32m 97\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_concordance_ratio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_correct\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_tied\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_pairs\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 98\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnum_pairs\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 99\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mZeroDivisionError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"No admissable pairs in the dataset.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 100\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mnum_correct\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mnum_tied\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mnum_pairs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 101\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mZeroDivisionError\u001b[0m: No admissable pairs in the dataset."
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Case #3: There is one event observed and one censored and the observed event happens **before** the censored event"
],
"metadata": {
"id": "phE3PbhjzbOx"
}
},
{
"cell_type": "code",
"source": [
"concordance_index([1, 2], [1, 2], [True, False]).item()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "YzeCtTmdwtio",
"outputId": "00fc5982-44b7-423c-ba01-f9ce13853e08"
},
"execution_count": 17,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"1.0"
]
},
"metadata": {},
"execution_count": 17
}
]
},
{
"cell_type": "markdown",
"source": [
"Case #4: There is one event observed and one censored and the observed event happens **after** the censored event"
],
"metadata": {
"id": "dbpuPOwnzwnr"
}
},
{
"cell_type": "code",
"source": [
"concordance_index([1, 2], [1, 2], [False, True]).item()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 287
},
"id": "2_ZR_hVExBLt",
"outputId": "a0659633-0f9f-4acc-f1c8-bdf3706257b5"
},
"execution_count": 19,
"outputs": [
{
"output_type": "error",
"ename": "ZeroDivisionError",
"evalue": "No admissable pairs in the dataset.",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mZeroDivisionError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m/tmp/ipython-input-1563998091.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mconcordance_index\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/usr/local/lib/python3.12/dist-packages/lifelines/utils/concordance.py\u001b[0m in \u001b[0;36mconcordance_index\u001b[0;34m(event_times, predicted_scores, event_observed)\u001b[0m\n\u001b[1;32m 92\u001b[0m \u001b[0mnum_correct\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_tied\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_pairs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_concordance_summary_statistics\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mevent_times\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpredicted_scores\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mevent_observed\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 93\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 94\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_concordance_ratio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_correct\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_tied\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_pairs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 95\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 96\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.12/dist-packages/lifelines/utils/concordance.py\u001b[0m in \u001b[0;36m_concordance_ratio\u001b[0;34m(num_correct, num_tied, num_pairs)\u001b[0m\n\u001b[1;32m 97\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_concordance_ratio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_correct\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_tied\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_pairs\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 98\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mnum_pairs\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 99\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mZeroDivisionError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"No admissable pairs in the dataset.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 100\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mnum_correct\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mnum_tied\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mnum_pairs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 101\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mZeroDivisionError\u001b[0m: No admissable pairs in the dataset."
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "T6nJm6Rkcev0"
},
"source": [
"\n",
"\n",
"Suppose that Alice churned after 1 year, Bob after 2 years, <font color='green'>Carol haven't churned after 3 years</font>, Dave churned after 4 years and Eve after 5 years.\n",
"\n",
"and our prediction is the following:\n",
"\n",
"+ *Prediction 2*: Alice will churn after 1 year, Bob after 2 years, Carol after 3 years, Dave after 5 years, and Eve after 4 years.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "USiqbR6gjZ1q"
},
"source": [
"Now we cannot give a score to the pairs (Carol, Dave) and (Carol, Eve) since we simply don't know who churned first. Therefore we only have 8 potential pairs:\n",
"\n",
"1. <font color='blue'>Alice is predicted to churn before Bob, and Alice did churn before Bob.</font>\n",
"2. <font color='blue'>Alice is predicted to churn before Carol, and Alice did churn before Carol.</font>\n",
"3. <font color='blue'>Alice is predicted to churn before Dave, and Alice did churn before Dave.</font>\n",
"4. <font color='blue'>Alice is predicted to churn before Eve, and Alice did churn before Eve.</font>\n",
"5. <font color='blue'>Bob is predicted to churn before Carol, and Bob did churn before Carol.</font>\n",
"6. <font color='blue'>Bob is predicted to churn before Dave, and Bob did churn before Dave.</font>\n",
"7. <font color='blue'>Bob is predicted to churn before Eve, and Bob did churn before Eve.</font>\n",
"8. <font color='red'>Dave is predicted to churn after Eve, while Eve churned after Dave.</font>\n",
"\n",
"Therefore the concordance index is equal to 7/8 or 0.875."
]
},
{
"cell_type": "code",
"metadata": {
"id": "FGGblZbKeuEH",
"outputId": "2e07db65-203b-45bc-af4b-5b5247e76771",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"times = [1, 2, 3, 4, 5]\n",
"preds = [1, 2, 3, 5, 4]\n",
"event_obs = [True, True, False, True, True]\n",
"df = pd.DataFrame(data={'Times': times, 'Predictions': preds, 'Event observed?': event_obs}, index=names)\n",
"print(df)\n",
"print(f'Concordance index: {concordance_index(events, preds, event_obs)}')"
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Times Predictions Event observed?\n",
"Alice 1 1 True\n",
"Bob 2 2 True\n",
"Carol 3 3 False\n",
"Dave 4 5 True\n",
"Eve 5 4 True\n",
"Concordance index: 0.875\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rJYxcCPYdKd0"
},
"source": [
"Let's do random predictions and see the concordance index we obtain."
]
},
{
"cell_type": "code",
"metadata": {
"id": "sQx4zxpt0sCA",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "fc844be0-4e9b-479b-f73b-97f09f517756"
},
"source": [
"# choose 100 random seeds\n",
"np.random.seed(0)\n",
"seeds = np.random.permutation(1000)[:100]\n",
"\n",
"actuals = np.arange(0,10,1)\n",
"ci_random = []\n",
"for _seed in seeds:\n",
" np.random.seed(_seed); preds_random = np.random.permutation(np.arange(0,10,1))\n",
" ci_random.append(concordance_index(actuals,preds_random))\n",
"print(f'Concordance index of random predictions: mean: {np.mean(ci_random):.3f}, std.dev.: {np.std(ci_random):.3f}')"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Concordance index of random predictions: mean: 0.517, std.dev.: 0.127\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "02zdCT6xobOS"
},
"source": [
"Let's see the concordance indexes of our random predictions in a box plot:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Wc4yEkVG4vlk",
"outputId": "cd547037-bfaf-4838-a592-c469a7288789",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 513
}
},
"source": [
"data = ci_random\n",
"green_diamond = dict(markerfacecolor='g', marker='D')\n",
"\n",
"fig1, ax1 = plt.subplots(figsize=(7,7))\n",
"ax1.set_title('Concordance indexes of random predictions')\n",
"ax1.boxplot(data, notch=False, flierprops=green_diamond)\n",
"ax1.set_xlabel('Random predictions')\n",
"ax1.set_ylabel('Concordance index')\n",
"plt.tight_layout()\n",
"# plt.show()\n",
"plt.savefig('Concordance.png')"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAHwCAYAAABZrD3mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcJXV97//X20GEEVSU0SjbEAQV\nE4PagttVxhCDShCNV8ElQryiuWJMNEZy9SKiJi4/jbkJMeICxg2JAe7IRZHoqHELMyguQFDCEgaN\nDAIujArI5/dHVeuZppcDTM3pb/N6Ph7n0V1V36r6nKrT591V9T11UlVIkqS23GnSBUiSpFvPAJck\nqUEGuCRJDTLAJUlqkAEuSVKDDHBJkhpkgGvJSHJSkjdMuo5pSZ6T5FO3cd6VSSrJVpu7riEkeUOS\nq5P814TWf1mSAyax7s0pyeFJvjAy/JMkv34blnObX3tqhwGuWSV5dpJ1/RvI95J8IsljJ11XS6rq\nQ1X1xEnXMbQkuwKvAPauql+bdD1LSVVtV1WXzNdmtn/27iivvTs6A1y3kOTlwDuAvwTuA+wK/D3w\n1EnWNa2Vo9I7kF2BH1TVVeM0vqPsvzvK89TkGODaRJK7A8cBL6mqU6vq+qq6sao+XlWv7NvcJck7\nkny3f7wjyV36afsnWZ/kFUmu6o/ejxhZ/rZJ3pbk8iQ/TPKFJNv20w5Ocn6S65J8NsmDRua7LMmr\nknwDuD7JVkkemuSrSX6c5KPANiPtd0hyRpINSa7tf995ZPpnk7w+yRf7+T+VZMeR6Y9N8qW+liuS\nHD7y3P+/JP+Z5PtJ/mG6/lm25czToZXkxUm+0y/3+CTppy3rl3t1kkuAp8zcL0ne22/PK/tT1sv6\nae9M8s8jbd+c5NMjyz4oyXn9Or+U5CEjbV/VL+/HSS5K8ttzvS6S/GO/PS9P8pokd+pPW58N3K8/\nW3PSLPNOvyZe1Z9iP3Ez7J/n9XX8IMmrZ6xvnNfnn4+8Pg9J8uQk305yTZL/Nds26Oc/qd/nZ/d1\nfS7JbjP28UuSfAf4Tj/ugX37a/pt/MyR9vdKsjrJj5KcA+wxY32V5P7973P97Xy+b35dvw8eNctr\n79FJ1vbzrU3y6HG2dZJtknyw387X9fPeZ67toy2sqnz4+OUDOBC4CdhqnjbHAV8B7g2sAL4EvL6f\ntn8//3HAnYEnAxuBHfrpxwOfBXYClgGPBu4C7AVcD/xOP9+fAxcDW/fzXQacB+wCbAtsDVwO/Gnf\n/hnAjcAb+vb3An4fWA5sD/wTcPrIc/gs8B/9erfth9/UT9sN+DFwWL/sewH79NP+GlgN3LNf7seB\nv5pjOx0OfGFkuIAzgHvQHbVuAA7sp70Y+Pf++d0TWNO336qffhrwLuCu/XY/B3hRP2058O1+ff8N\nuBrYuZ/2UOAqYL9+ez+/35Z3AR4AXAHcr2+7Ethjjufyj8D/7Z/zyn59LxjZ5+vneb3sT/eaeHO/\n3m1v5/7ZG/gJ8Lh+eW/vl3/ArXh9HtPv2xf2++HDfR0PBn4K7D7Hczmpf21Mr/tvZtnHZ/f7cNt+\nf10BHAFs1e+Pq+kuNwCcDJzSt/sN4MpZlnf/Bf52VjLyWpn52utruRZ4Xl/DYf3wvcbY1i+ie40v\n79f5cOBuk36f8tHv50kX4GNxPYDnAP+1QJv/AJ48Mvy7wGX97/v3b4CjbyZXAY+kO+PzU+C3Zlnm\n/wZOGRm+U/9mtn8/fBnwhyPTHwd8F8jIuC/RB/gsy98HuHZk+LPAa0aG/yfwyf73vwBOm2UZofsn\nY4+RcY8CLp1jnb98E+2HC3jsyPApwNH9758BXjwy7YnTb8p0lzF+Dmw7Mv0wYM3I8H7ANXT/1Bw2\nMv6d9OE1Mu4i4PHA/ft9cwBw53n29zLgBvrQ6ce9CPjsyD5fKMBvALaZp82t2T/HACePTLtrv/zp\nAB/n9bmsH96+3877jbQ/FzhkjjpPmrHu7YBfALuM7OMnjEx/FvCvM5bxLuC1/Xa9EXjgyLS/nOU1\nc3/m/9tZyfwB/jzgnBnzfBk4fIxt/Yd0f1cPme89wcdkHl6j0Uw/AHZMslVV3TRHm/vRBcW0y/tx\nv1zGjHk30r3R7Uh3mvs/FlpmVd2c5Aq6o41pV8xof2X17zIjdQCQZDnd0fKBwA796O2TLKuqX/TD\noz2mp2uE7ih4thpX0B2JnNufnYYu1JfN0nYuc63zfmz6/Ea37250R4vfG1nvnUbbV9W/9afe7033\nj8HovM9P8tKRcVvTHXV/LsmfAMcCD05yFvDyqvrujJp37Nc/c5/vxPg2VNXPpgdu5/7ZZFtV1fVJ\nfjDSdpzX5/Q6ftr//P7I9J+OrGs2o+v+SZJrZtQ0uh93A/ZLct3IuK2AD9C9nrZi7v0+ar6/nYXM\n3B7T6xndf3Nt6w/Q/T2cnOQewAeBV1fVjbehDm1mXgPXTF+mO9o7ZJ4236V7Y5q2az9uIVcDP2PG\ndb7Zltlfv92F7ih82mhYfw/YKSOJ1tcx7RV0p4j3q6q70R2xQxe4C7lijhqvpntzf3BV3aN/3L2q\n5nuzH9f36J7vtNHncgXdPtlxZL13q6oHTzdI8hK606nfpbv8MDrvG0fmu0dVLa+qjwBU1Yer6rF0\n277oTnPPdDXdkeLMfX7lLG3nMvNrD2/P/tlkW/X/DNxrZPptfX2Oa3Td29Gdoh5d/uhzvQL43Izt\nv11V/RHdqfubmHu/j5rvb2ehr5ScuT2m17Pg/quu/8vrqmpvulP2BwF/sNB82jIMcG2iqn5Id4ry\n+L5zz/Ikd07ypCRv6Zt9BHhNkhV9Z5dj6P4zX2jZNwPvA96e5H7pOm49qu9gdArwlCS/neTOdG/w\nP6c7fTebL9O9+f1xX9/TgX1Hpm9PF7bXJbkn3SnLcX0IOCDJM9N1lrtXkn36+t8N/HWSewMk2SnJ\n796KZc/llP657JxkB+Do6QlV9T3gU8DbktwtXeexPZI8vq9hL+ANwHPpTpf+eZJ9+tnfDbw4yX7p\n3DXJU5Jsn+QBSZ7Qb/+f0W2vm2cW1h+tngK8sZ9vN+DljLHP53F79s/HgIPSdTTcmu6a9+h72W16\nfd4KTx5Z9+uBr1TVFXO0PQPYK12nuzv3j0ckeVC/XU8Fju3/zvam66NwCwv87Wyg229zfV78zL6G\nZ/ev52fR9SM4Y6EnmmRVkt9M12HyR3T/yN3iNaLJMMB1C1X1Nro36NfQvTlcARwFnN43eQOwDvgG\n8E3gq/24cfxZP89aumu2bwbuVFUX0QXQ39Idbfwe8HtVdcMcNd4APJ3uWt81dNcaTx1p8g66DjlX\n03Vo+uSY9VFV/0nX+e4V/bLPA36rn/wqus51X0nyI+Bf6I4kb693A2cBX6fbnqfOmP4HdKe+L6Dr\ngPQx4L7pPqr0QeDNVfX1qvoO8L+ADyS5S1Wto+uo9Xf9fBfTbTPojtjfRLeN/ovu9PtfzFHfS+mu\n/18CfIGu09f7bsfzvT3753zgJX0N36N7XutHmtye1+c4Pkz3D8c1dJ26njtPrT+m689wKN2R8H/x\nq8580P1dbdePPwk4cZ71zvW3sxF4I/DFvqf4I2fU8AO6I+dX0F0i+3PgoKq6eozn+mt0r7UfARcC\nn6M7ra5FIJteQpQkzSXdx+TWV9VrJl2L5BG4JEkNMsAlSWrQoKfQkxxId6ODZcB7qupNM6bvRncd\nbQXdNZ3nVtX6WyxIkiRtYrAA73stfpvuzlrr6TpeHFZVF4y0+SfgjKp6f5InAEdU1fMGKUiSpCVk\nyBu57AtcXP036SQ5me7LMC4YabM3XW9n6G4deToL2HHHHWvlypWbt1JJkhaJc8899+qqWrFQuyED\nfCc2vcPQerrbPY76Ot1Hgf4GeBrdnZju1X/sYVYrV65k3bp1m7tWSZIWhSRz3ZFvE5PuxPZnwOOT\nfI3u3sxX0t1XeBNJjkz33dTrNmzYsKVrlCRp0RkywK9k01sE7syMW/dV1Xer6ulV9VDg1f240XsG\nT7c7oaqmqmpqxYoFzypIkrTkDRnga4E9k+ze33LwULqvYfylJDsmma7hL7h9d3aSJOkOY7AA77+N\n6ii620NeSPdVkecnOS7JwX2z/YGLknyb7isT3zhUPZIkLSXN3Up1amqq7MQmSVqqkpxbVVMLtZt0\nJzZJknQbGOCSJDXIAJckqUEGuCRJDTLAJUlqkAEuSVKDDHBJkhpkgEuS1CADXJKkBhngkua0Zs0a\nVu65kjVr1ky6FEkzGOCSZrVmzRoOetpBXL775Rz0tIMMcWmRMcAl3cJ0eG88ZCM8BjYestEQlxYZ\nA1zSJjYJ7937kbsb4tJiY4BL2sQRRx7Bxn1Hwnva7rBx340cceQRE6lL0qYMcEmbOPGEE1l+znK4\ndMaES2H5Ocs58YQTJ1KXpE0Z4JI2sWrVKs447QyWnz4S4pfC8tOXc8ZpZ7Bq1aqJ1iepY4BLuoVN\nQvyLhre0GBngkmY1HeK7Xbqb4S0tQltNugBJi9eqVau47DuXTboMSbPwCFySpAYZ4JIkNcgAlySp\nQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQGGeCSJDXIAJckqUEGuCRJDTLAJUlqkAEu\nSVKDDHBJkhpkgEuS1CADXJKkBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktQg\nA1ySpAYZ4JIkNcgAlySpQQa4JEkNGjTAkxyY5KIkFyc5epbpuyZZk+RrSb6R5MlD1iNJ0lIxWIAn\nWQYcDzwJ2Bs4LMneM5q9Bjilqh4KHAr8/VD1SJK0lAx5BL4vcHFVXVJVNwAnA0+d0aaAu/W/3x34\n7oD1SJK0ZGw14LJ3Aq4YGV4P7DejzbHAp5K8FLgrcMBsC0pyJHAkwK677rrZC5WWsiSTLgGAqpp0\nCdKSMulObIcBJ1XVzsCTgQ8kuUVNVXVCVU1V1dSKFSu2eJFSy6rqdj02xzIMb2nzGzLArwR2GRne\nuR836gXAKQBV9WVgG2DHAWuSJGlJGDLA1wJ7Jtk9ydZ0ndRWz2jzn8BvAyR5EF2AbxiwJkmSloTB\nAryqbgKOAs4CLqTrbX5+kuOSHNw3ewXwwiRfBz4CHF6ea5MkaUFDdmKjqs4Ezpwx7piR3y8AHjNk\nDZIkLUWT7sQmSZJuAwNckqQGGeCSJDXIAJckqUEGuCRJDTLAJUlqkAEuSVKDDHBJkhpkgEuS1CAD\nXJKkBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktQgA1ySpAYZ4JIkNcgAlySp\nQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQGGeCSJDXIAJckqUEGuCRJDTLAJUlqkAEu\nSVKDDHBJkhpkgEuS1CADXJKkBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktQg\nA1ySpAYZ4JIkNcgAlySpQQa4JEkNMsAlSWqQAS5JUoMGDfAkBya5KMnFSY6eZfpfJzmvf3w7yXVD\n1iNJ0lKx1VALTrIMOB74HWA9sDbJ6qq6YLpNVf3pSPuXAg8dqh5JkpaSIY/A9wUurqpLquoG4GTg\nqfO0Pwz4yID1SJK0ZAwZ4DsBV4wMr+/H3UKS3YDdgc/MMf3IJOuSrNuwYcNmL1SSpNYslk5shwIf\nq6pfzDaxqk6oqqmqmlqxYsUWLk2SpMVnyAC/EthlZHjnftxsDsXT55IkjW3IAF8L7Jlk9yRb04X0\n6pmNkjwQ2AH48oC1SJK0pAwW4FV1E3AUcBZwIXBKVZ2f5LgkB480PRQ4uapqqFokSVpqBvsYGUBV\nnQmcOWPcMTOGjx2yBkmSlqLF0olNkiTdCga4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQG\nGeCSJDXIAJckqUEGuCRJDTLAJUlqkAEuSVKDDHBJkhpkgEuS1CADXJKkBhngkiQ1yACXJKlBBrgk\nSQ0ywCVJapABLklSgwxwSZIaZIBLktQgA1ySpAYZ4JIkNcgAlySpQQa4JEkNMsAlSWqQAS5JUoMM\ncEmSGmSAS5LUIANckqQGGeCSJDXIAJckqUEGuCRJDTLAJUlqkAEuSVKDDHBJkhpkgEuS1CADXJKk\nBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktSgQQM8yYFJLkpycZKj52jzzCQX\nJDk/yYeHrEeSpKViq6EWnGQZcDzwO8B6YG2S1VV1wUibPYG/AB5TVdcmufdQ9UiStJQMeQS+L3Bx\nVV1SVTcAJwNPndHmhcDxVXUtQFVdNWA9kiQtGUMG+E7AFSPD6/txo/YC9kryxSRfSXLggPVIkrRk\nDHYK/Vasf09gf2Bn4PNJfrOqrhttlORI4EiAXXfddUvXKE3MPe95T6699tpJl0GSia5/hx124Jpr\nrploDdJiM2SAXwnsMjK8cz9u1Hrg36rqRuDSJN+mC/S1o42q6gTgBICpqakarGJpkbn22mup8iU/\n6X8gpMVoyFPoa4E9k+yeZGvgUGD1jDan0x19k2RHulPqlwxYkyRJS8KCAZ7kBTOGlyV57ULzVdVN\nwFHAWcCFwClVdX6S45Ic3Dc7C/hBkguANcArq+oHt/ZJSJJ0R5OFTs/1n82+B/AC4J7AScDnqurP\nBq9uFlNTU7Vu3bpJrFra4pJ4Ch23g+5YkpxbVVMLtVvwGnhVPTvJs4BvAtcDz66qL26GGiVJ0m00\nzin0PYGXAf8MXA48L8nyoQuTJElzG6cT28eBY6rqRcDjge8wo5e4JEnassb5GNm+VfUjgOouQr0t\nyceHLUuSJM1nnCPwm5L87yTvhl+eUt9r2LIkSdJ8xgnwE4GfA4/qh68E3jBYRZIkaUHjBPgeVfUW\n4EaAqtoIeFskSZImaJwAvyHJtkABJNmD7ohckiRNyDid2F4LfBLYJcmHgMcAhw9ZlCRJmt84N3I5\nO8lXgUfSnTp/WVVdPXhlkiRpTnMGeJKHzRj1vf7nrkl2raqvDleWJEmaz3xH4G/rf24DTAFfpzsC\nfwiwjl/1SpckSVvYnJ3YqmpVVa2iO/J+WFVNVdXDgYdyy+/1liRJW9A4vdAfUFXfnB6oqm8BDxqu\nJEmStJBxeqF/I8l7gA/2w88BvjFcSZIkaSHjBPgRwB/RfSMZwOeBdw5WkSRJWtA4HyP7GfDX/UOS\nJC0CCwZ4kscAxwK7jbavql8frixJkjSfcU6hvxf4U+Bc4BfDliNJksYxToD/sKo+MXglkiRpbOME\n+JokbwVOZeRLTLwTmyRJkzNOgO/X/5waGVfAEzZ/OZIkaRzj9EJftSUKkSRJ45vvy0yeW1UfTPLy\n2aZX1duHK0uSJM1nviPwu/Y/t98ShUiSpPHNGeBV9a7+5+u2XDmSJGkc43yZiSRJWmQMcEmSGmSA\nS5LUoAUDPMl9krw3ySf64b2TvGD40iRJ0lzGOQI/CTgLuF8//G3gT4YqSJIkLWycAN+xqk4Bbgao\nqpvwS00kSZqocW6len2Se9HdPpUkjwR+OGhVkgCo194Njr37pMuYuHrt3SZdgrTojBPgLwdWA3sk\n+SKwAnjGoFVJAiCv+xFVNekyJi4Jdeykq5AWl3Huhf7VJI8HHgAEuKiqbhy8MkmSNKdxeqG/BNiu\nqs6vqm8B2yX5n8OXJkmS5jJOJ7YXVtV10wNVdS3wwuFKkiRJCxknwJclyfRAkmXA1sOVJEmSFjJO\nJ7ZPAh9N8q5++EX9OEmSNCHjBPir6EL7j/rhs4H3DFaRJEla0Di90G8G3tk/JEnSIrBggCd5DHAs\nsFvfPkBV1a8PW5okSZrLOKfQ3wv8KXAu3kJVkqRFYZwA/2FVfWLwSiRJ0tjGCfA1Sd4KnAr8fHpk\nVX11sKokSdK8xgnw/fqfUyPjCnjC5i9HkiSNY5xe6Ku2RCGSJGl84xyBk+QpwIOBbabHVdVxY8x3\nIPA3wDLgPVX1phnTDwfeClzZj/q7qvIz5pIkLWCcj5H9A7AcWEV3A5dnAOeMMd8y4Hjgd4D1wNok\nq6vqghlNP1pVR93awiVJuiMb517oj66qPwCurarXAY8C9hpjvn2Bi6vqkqq6ATgZeOptL1WSJE0b\nJ8B/2v/cmOR+wI3AfceYbyfgipHh9f24mX4/yTeSfCzJLrMtKMmRSdYlWbdhw4YxVi1J0tI2ToCf\nkeQedNeqvwpcBnxkM63/48DKqnoI3T3W3z9bo6o6oaqmqmpqxYoVm2nVkiS1a5xe6K/vf/3nJGcA\n21TVD8dY9pXA6BH1zvyqs9r0sn8wMvge4C1jLFeSpDu8OQM8ydPnmUZVnbrAstcCeybZnS64DwWe\nPWM5962q7/WDBwMXjlW1JEl3cPMdgf9e//PewKOBz/TDq4Av0d2ZbU5VdVOSo4Cz6D5G9r6qOj/J\nccC6qloN/HGSg4GbgGuAw2/rE5Ek6Y4kVTV/g+RTwPOnj5ST3Bc4qap+dwvUdwtTU1O1bt26Saxa\n2uL6s12TLmPi3A66I0lyblVNLdRunE5su4yc5gb4PrDrba5MkiTdbuPcie3TSc7iVz3PnwX8y3Al\nSZKkhYzTC/2oJE8DHtePOqGqThu2LEmSNJ95A7y/Heq/9F9oYmhLkrRIzHsNvKp+Adyc5O5bqB5J\nkjSGca6B/wT4ZpKzgeunR1bVHw9WlSRJmtc4AX4qC3zmW5IkbVnjdGJ7f5Kt+dU3kF1UVTcOW5Yk\nSZrPON8Hvj/dl4xcBgTYJcnzq+rzw5YmSZLmMs4p9LcBT6yqiwCS7EX3mfCHD1mYJEma2zh3Yrvz\ndHgDVNW3gTsPV5IkSVrIOEfg65K8B/hgP/wcwJuRS5I0QeME+B8BLwGmPzb2r8DfD1aRJEla0DgB\nvhXwN1X1dvjl3dnuMmhVkiRpXuNcA/80sO3I8Lb4ZSaSJE3UOAG+TVX9ZHqg/335cCVJkqSFjBPg\n1yd52PRAkocDPx2uJEmStJBxroH/CfBPSb5LdyOXX6P7TnBJkjQh49xKdW2SBwIP6Ed5K1VJkiZs\nnCNwgEcAK/v2D0tCVf3jYFVJkqR5jXMv9A8AewDnAb/oRxdggEuSNCHjHIFPAXtXVQ1djCRJGs84\nvdC/RddxTZIkLRLjHIHvCFyQ5Bzg59Mjq+rgwaqSJEnzGifAjx26CEmSdOuM8zGyzyW5D11PdIBz\nquqqYcuSJEnzWfAaeJJnAucA/x14JvBvSZ4xdGGSJGlu45xCfzXwiOmj7iQr6L7M5GNDFiZJkuY2\nTi/0O804Zf6DMeeTJEkDGecI/JNJzgI+0g8/C/jEcCVJkqSFjNOJ7ZVJng48th91QlWdNmxZkiRp\nPnMGeJL7A/epqi9W1anAqf34xybZo6r+Y0sVKUmSNjXftex3AD+aZfwP+2mSJGlC5gvw+1TVN2eO\n7MetHKwiSZK0oPkC/B7zTNt2cxciSZLGN1+Ar0vywpkjk/wP4NzhSpIkSQuZrxf6nwCnJXkOvwrs\nKWBr4GlDFyZJkuY2Z4BX1feBRydZBfxGP/r/VdVntkhlkiRpTuN8DnwNsGYL1CJpFkkmXcLE7bDD\nDpMuQVp0xrkTm6QJqapJl0CSRVGHpE15T3NJkhpkgEuS1CADXJKkBhngkiQ1yACXJKlBBrgkSQ0y\nwCVJatCgAZ7kwCQXJbk4ydHztPv9JJVkash6JElaKgYL8CTLgOOBJwF7A4cl2XuWdtsDLwP+baha\nJElaaoY8At8XuLiqLqmqG4CTgafO0u71wJuBnw1YiyRJS8qQAb4TcMXI8Pp+3C8leRiwS1X9v/kW\nlOTIJOuSrNuwYcPmr1SSpMZMrBNbkjsBbwdesVDbqjqhqqaqamrFihXDFydJ0iI3ZIBfCewyMrxz\nP27a9nRfU/rZJJcBjwRW25FNkqSFDRnga4E9k+yeZGvgUGD19MSq+mFV7VhVK6tqJfAV4OCqWjdg\nTZIkLQmDBXhV3QQcBZwFXAicUlXnJzkuycFDrVeSpDuCQb8PvKrOBM6cMe6YOdruP2QtkiQtJd6J\nTZKkBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktQgA1ySpAYZ4JIkNcgAlySp\nQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQGGeCSJDXIAJckqUEGuCRJDTLAJUlqkAEu\nSVKDDHBJkhpkgEuS1CADXJKkBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktQg\nA1ySpAYZ4JIkNcgAlySpQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQGGeCSJDXIAJck\nqUEGuCRJDTLAJUlqkAEuSVKDBg3wJAcmuSjJxUmOnmX6i5N8M8l5Sb6QZO8h65EkaakYLMCTLAOO\nB54E7A0cNktAf7iqfrOq9gHeArx9qHokSVpKhjwC3xe4uKouqaobgJOBp442qKofjQzeFagB65Ek\nacnYasBl7wRcMTK8HthvZqMkLwFeDmwNPGHAeiRJWjIm3omtqo6vqj2AVwGvma1NkiOTrEuybsOG\nDVu2QEmSFqEhA/xKYJeR4Z37cXM5GThktglVdUJVTVXV1IoVKzZjiZIktWnIAF8L7Jlk9yRbA4cC\nq0cbJNlzZPApwHcGrEeSpCVjsGvgVXVTkqOAs4BlwPuq6vwkxwHrqmo1cFSSA4AbgWuB5w9VjyRJ\nS8mQndioqjOBM2eMO2bk95cNuX5JkpaqiXdikyRJt54BLklSgwxwSZIaZIBLktQgA1ySpAYZ4JIk\nNcgAlySpQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQGGeCSJDXIAJckqUEGuCRJDTLA\nJUlqkAEuSVKDDHBJkhpkgEuS1CADXJKkBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIa\nZIBLktQgA1ySpAYZ4JIkNcgAlySpQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANckqQGGeCS\nJDVoq0kXIGlYSRbFMqrqdi9D0q8Y4NISZ3BKS5On0CVJapABLklSgwxwSZIaZIBLktQgA1ySpAYZ\n4JIkNcgAlySpQYMGeJIDk1w8xt3tAAAHiklEQVSU5OIkR88y/eVJLkjyjSSfTrLbkPVIkrRUDBbg\nSZYBxwNPAvYGDkuy94xmXwOmquohwMeAtwxVjyRJS8mQR+D7AhdX1SVVdQNwMvDU0QZVtaaqNvaD\nXwF2HrAeSZKWjCEDfCfgipHh9f24ubwA+MRsE5IcmWRdknUbNmzYjCVKktSmRdGJLclzgSngrbNN\nr6oTqmqqqqZWrFixZYuTJGkRGvLLTK4EdhkZ3rkft4kkBwCvBh5fVT8fsB5JkpaMIY/A1wJ7Jtk9\nydbAocDq0QZJHgq8Czi4qq4asBZJkpaUwQK8qm4CjgLOAi4ETqmq85Mcl+Tgvtlbge2Af0pyXpLV\ncyxOkiSNGPT7wKvqTODMGeOOGfn9gCHXL0nSUrUoOrFJkqRbxwCXJKlBBrgkSQ0ywCVJapABLklS\ngwxwSZIaZIBLktQgA1ySpAYZ4JIkNcgAlySpQQa4JEkNMsAlSWqQAS5JUoMMcEmSGmSAS5LUIANc\n0pzWrFnDyj1XsmbNmkmXImkGA1zSrNasWcNBTzuIy3e/nIOedpAhLi0yBrikW5gO742HbITHwMZD\nNhri0iJjgEvaxCbhvXs/cndDXFpsDHBJmzjiyCPYuO9IeE/bHTbuu5EjjjxiInVJ2pQBLmkTJ55w\nIsvPWQ6XzphwKSw/ZzknnnDiROqStCkDXNImVq1axRmnncHy00dC/FJYfvpyzjjtDFatWjXR+iR1\nDHBJt7BJiH/R8JYWIwNc0qymQ3y3S3czvKVFaKtJFyBp8Vq1ahWXfeeySZchaRYegUuS1CADXJKk\nBhngkiQ1yACXJKlBBrgkSQ0ywCVJapABLklSgwxwSZIaZIBLktQgA1ySpAYZ4JIkNcgAlySpQQa4\nJEkNMsAlSWpQqmrSNdwqSTYAl0+6DukOZEfg6kkXId2B7FZVKxZq1FyAS9qykqyrqqlJ1yFpU55C\nlySpQQa4JEkNMsAlLeSESRcg6Za8Bi5JUoM8ApckqUEGuCRJDTLAJc0qyfuSXJXkW5OuRdItGeCS\n5nIScOCki5A0OwNc0qyq6vPANZOuQ9LsDHBJkhpkgEuS1CADXJKkBhngkiQ1yACXNKskHwG+DDwg\nyfokL5h0TZJ+xVupSpLUII/AJUlqkAEuSVKDDHBJkhpkgEuS1CADXJKkBhng0oCS/CLJeUm+leTj\nSe6xmZa7soVvCUtybJI/638/LskB87TdJ8mTR4YPTnL0lqhTapEBLg3rp1W1T1X9Bt0Xg7xk0gXd\nXkm2ui3zVdUxVfUv8zTZB3jySPvVVfWm27Iu6Y7AAJe2nC8DOwEk2S7Jp5N8Nck3kzy1H78yyYVJ\n3p3k/CSfSrJtP+3hSb6e5OuM/COQZJskJ/bL+VqSVf34w5OcnuTsJJclOSrJy/s2X0lyz5kFJjkp\nyT8kWZfk20kOGlnW6iSfAT7dj3tlkrVJvpHkdSPLeHU/7xeAB8xY9jP63x+R5Ev98zknyd2B44Bn\n9WcsntWv8+9Gtstn+nV9OsmuI8v8P/2yLhlZ/n2TfH7k7Md/21w7UVosDHBpC0iyDPhtYHU/6mfA\n06rqYcAq4G1J0k/bEzi+qh4MXAf8fj/+ROClVfVbMxb/EqCq6jeBw4D3J9mmn/YbwNOBRwBvBDZW\n1UPp/pn4gznKXQnsCzwF+IeRZT0MeEZVPT7JE/s696U7cn54kscleThwKL86mn7ELNtia+CjwMv6\n53IAcD1wDPDR/ozFR2fM9rfA+6vqIcCHgP8zMu2+wGOBg4DpI/ZnA2dV1T7AbwHnzfFcpWbdplNh\nksa2bZLz6I68LwTO7scH+MskjwNu7qffp592aVVNB865wMr+2vk9+u/oBvgA8KT+98fSBRxV9e9J\nLgf26qetqaofAz9O8kPg4/34bwIPmaPmU6rqZuA7SS4BHtiPP7uqpr8f/In942v98HZ0gb49cFpV\nbQRIsppbegDwvapa29f8o77tHOUA8Ci6f0Smn/tbRqad3td7QZLpbbgWeF+SO/fTDXAtOR6BS8P6\naX8UuBtdaE+f+n4OsAJ4eD/9+8D0ke7PR+b/BbfvH+3RZd08MnzzPMudeX/l6eHrR8YF+Kv+aHmf\nqrp/Vb33dtR5e4w+xwD0/+g8DrgSOCnJXGcbpGYZ4NIW0B+R/jHwir4T2N2Bq6rqxv6a9W4LzH8d\ncF2Sx/ajnjMy+V+nh5PsBewKXHQ7yv3vSe6UZA/g1+dY1lnAHybZrl/vTknuDXweOCTJtkm2B35v\nlnkvAu6b5BH9vNv32+THdEfws/kS3al56J7rv873BJLsBny/qt4NvIfu9L+0pHgKXdpCquprSb5B\nd536Q8DHk3wTWAf8+xiLOILutHABnxoZ//fAO/tl3QQcXlU/X+CU9Hz+EzgHuBvw4qr62cxlVdWn\nkjwI+HI/7SfAc6vqq0k+CnwduIruVPbMeW9I8izgb/sOej+luw6+Bji6v+TwVzNmeylwYpJXAhv6\nbTGf/YFXJrmxr80jcC05fhuZpF9KchJwRlV9bNK1SJqfp9AlSWqQR+CSJDXII3BJkhpkgEuS1CAD\nXJKkBhngkiQ1yACXJKlB/z/R8Rd7R92BhgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 504x504 with 1 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "w9gJdtN7q6jp"
},
"source": [
"## Difference between libraries"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yj0u5J8uyOLZ"
},
"source": [
"Lifelines concordance index inputs are:\n",
"\n",
"+ event/censoring times\n",
"+ predicted scores\n",
"+ event observed (*optional*). The default assumes all events observed.\n",
"\n",
"Lifelines concordance index output is:\n",
"\n",
"+ concordance index\n",
"\n",
"Let's compare the concordance index of [lifelines](https://lifelines.readthedocs.io/en/latest/) with the one provided by [scikit-survival](https://scikit-survival.readthedocs.io/en/latest/)."
]
},
{
"cell_type": "code",
"metadata": {
"id": "-nHs7CGyfIcK",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3c1fa956-3ad7-4755-c82a-16af1b77a490"
},
"source": [
"%pip install -q scikit-survival"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/4.0 MB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.8/4.0 MB\u001b[0m \u001b[31m24.2 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m \u001b[32m4.0/4.0 MB\u001b[0m \u001b[31m68.2 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.0/4.0 MB\u001b[0m \u001b[31m42.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/222.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m222.1/222.1 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "ohcR8VX0dEgZ"
},
"source": [
"from lifelines.utils import concordance_index as ci_lifelines\n",
"from sksurv.metrics import concordance_index_censored as ci_scikit"
],
"execution_count": 12,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "V7xuP687dV2J"
},
"source": [
"Scikit-survival's concordance index inputs are:\n",
"+ event observed\n",
"+ event/censoring times\n",
"+ predicted risks\n",
"+ tied tolerance (float, optional, default: 1e-8) – The tolerance value for considering ties. If the absolute difference between risk scores is smaller or equal than tied_tol, risk scores are considered tied.\n",
"\n",
"Scikit-survival's concordance index outputs are:\n",
"\n",
"+ Concordance index\n",
"+ Number of concordant pairs\n",
"+ Number of discordant pairs\n",
"+ Number of pairs having tied estimated risks\n",
"+ Number of comparable pairs sharing the same time\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NMY28CGB15IL"
},
"source": [
"Notice that lifelines gives the concordance between the actuals and the predicted *scores*, while scikit-survival gives the concordance between the actuals and the predicted *risks*, so over the same lists, they are the complete opposite.\n",
"\n",
"$$\n",
"\\textrm{Concordance-index (lifelines)}= 1-\\textrm{Concordance-index (scikit-survival)}\n",
"$$\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "DEFgM_FKxIzk",
"outputId": "157f27ee-6342-4544-ece8-93d70b004867",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [1, 2, 3, 4, 5]\n",
"event_obs = [True, True, True, True, True]\n",
"df = pd.DataFrame(data={'Times': times, 'Predictions': preds, 'Event observed?': event_obs}, index=names)\n",
"print(df)\n",
"print(f'Concordance index (lifelines): {ci_lifelines(events, preds, event_obs)}')\n",
"print(f'Concordance-index (scikit-survival): {ci_scikit(event_obs, events, preds)[0]}')"
],
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Times Predictions Event observed?\n",
"Alice 1 1 True\n",
"Bob 2 2 True\n",
"Carol 3 3 True\n",
"Dave 4 4 True\n",
"Eve 5 5 True\n",
"Concordance index (lifelines): 1.0\n",
"Concordance-index (scikit-survival): 0.0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EyT74Q4i3Kpy"
},
"source": [
"As mentioned above, scikit survival also gives the number of concordant pairs, number of discordant pairs, number of pairs having tied estimated risks, and the number of comparable pairs sharing the same time."
]
},
{
"cell_type": "code",
"metadata": {
"id": "0RQDwjdqfrbF",
"outputId": "41fd7b65-0b63-46ea-db4c-c3a7e8b32270",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"events = [1, 2, 3, 4, 5]\n",
"preds = [5, 4, 3, 2, 1]\n",
"event_obs = [True, True, True, True, True]\n",
"df = pd.DataFrame(data={'Times': times, 'Predictions': preds, 'Event observed?': event_obs}, index=names)\n",
"print(df)\n",
"print(f'Concordance-index (scikit-survival): {ci_scikit(event_obs, events, preds)[0]}')\n",
"print(f'Number of concordant pairs (scikit-survival): {ci_scikit(event_obs, events, preds)[1]}')\n",
"print(f'Number of discordant pairs (scikit-survival): {ci_scikit(event_obs, events, preds)[2]}')\n",
"print(f'Number of pairs having tied estimated risks (scikit-survival): {ci_scikit(event_obs, events, preds)[3]}')\n",
"print(f'Number of comparable pairs sharing the same time (scikit-survival): {ci_scikit(event_obs, events, preds)[4]}')"
],
"execution_count": 14,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" Times Predictions Event observed?\n",
"Alice 1 5 True\n",
"Bob 2 4 True\n",
"Carol 3 3 True\n",
"Dave 4 2 True\n",
"Eve 5 1 True\n",
"Concordance-index (scikit-survival): 1.0\n",
"Number of concordant pairs (scikit-survival): 10\n",
"Number of discordant pairs (scikit-survival): 0\n",
"Number of pairs having tied estimated risks (scikit-survival): 0\n",
"Number of comparable pairs sharing the same time (scikit-survival): 0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eokykj3s2V3T"
},
"source": [
"Let's see a more interesting example:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "zxxz9nUd2_ok",
"outputId": "9a860fde-3230-423e-e9a8-e57b81abd6a7",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 208
}
},
"source": [
"names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']\n",
"times = [1, 2, 3, 4, 5]\n",
"preds = [4, 5, 3, 2, 1]\n",
"event_obs = [True, True, False, True, True]\n",
"df = pd.DataFrame(data={'Times': times, 'Predictions': preds, 'Event observed?': event_obs}, index=names)\n",
"print(df)\n",
"print(f'Concordance-index (scikit-survival): {ci_scikit(event_obs, events, preds)[0]}')\n",
"print(f'Number of concordant pairs (scikit-survival): {ci_scikit(event_obs, events, preds)[1]}')\n",
"print(f'Number of discordant pairs (scikit-survival): {ci_scikit(event_obs, events, preds)[2]}')\n",
"print(f'Number of pairs having tied estimated risks (scikit-survival): {ci_scikit(event_obs, events, preds)[3]}')\n",
"print(f'Number of comparable pairs sharing the same time (scikit-survival): {ci_scikit(event_obs, events, preds)[4]}')"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
" Times Predictions Event observed?\n",
"Alice 1 4 True\n",
"Bob 2 5 True\n",
"Carol 3 3 False\n",
"Dave 4 2 True\n",
"Eve 5 1 True\n",
"Concordance-index (scikit-survival): 0.875\n",
"Number of concordant pairs (scikit-survival): 7\n",
"Number of discordant pairs (scikit-survival): 1\n",
"Number of pairs having tied estimated risks (scikit-survival): 0\n",
"Number of comparable pairs sharing the same time (scikit-survival): 0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5eulYERhomKP"
},
"source": [
"## Conclusions\n",
"\n",
"Concordance index is a useful metric to evaluate the predictions made by an algorithm. It can consider the case of right-censoring, i.e., when by the end of the study, the event of interest (for example, in medicine 'death of a patient' or in our example 'churn of a customer') has only occurred for a subset of the observations. We have seen two libraries (lifelines and scikit-survival) who implement the concordance index but with some subtle differences."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nRE2p7s1b_Rn"
},
"source": [
"**Sources**\n",
"\n",
"+ Harrell F.E Jr., Lee K.L., Mark D.B., [\"Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors\"](https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4%3C361::AID-SIM168%3E3.0.CO;2-4), Statistics in Medicine, 15(4), 361-87, 1996.\n",
"+ [Lifelines' concordance index](https://lifelines.readthedocs.io/en/latest/lifelines.utils.html#lifelines.utils.concordance_index)\n",
"+ [Scikit-survival's concordance index](https://scikit-survival.readthedocs.io/en/latest/generated/sksurv.metrics.concordance_index_censored.html#sksurv.metrics.concordance_index_censored)"
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment