Skip to content

Instantly share code, notes, and snippets.

@wlattner

wlattner/dplyr_to_pandas.ipynb

Created July 1, 2016 00:31
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save wlattner/4c34b9f0c6fe6dcd79983bc9fe6e70a2 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save wlattner/4c34b9f0c6fe6dcd79983bc9fe6e70a2 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
dplyr_to_pandas.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"cases = pd.read_csv(\"https://raw.githubusercontent.com/rstudio/EDAWR/master/data-raw/cases.csv\")\n",
"storms = pd.read_csv(\"https://raw.githubusercontent.com/rstudio/EDAWR/master/data-raw/storms.csv\")\n",
"pollution = pd.read_csv(\"https://raw.githubusercontent.com/rstudio/EDAWR/master/data-raw/pollution.csv\")\n",
"tb = pd.read_csv(\"https://raw.githubusercontent.com/rstudio/EDAWR/master/data-raw/tb.csv\")\n",
"iris = pd.read_csv(\"./iris.csv\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Gather columns into rows."
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"```r\n",
"> tidyr::gather(cases, \"year\", \"n\", 2:4)\n",
" country year n\n",
"1 FR 2011 7000\n",
"2 DE 2011 5800\n",
"3 US 2011 15000\n",
"4 FR 2012 6900\n",
"5 DE 2012 6000\n",
"6 US 2012 14000\n",
"7 FR 2013 7000\n",
"8 DE 2013 6200\n",
"9 US 2013 13000\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>country</th>\n",
" <th>year</th>\n",
" <th>n</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>FR</td>\n",
" <td>2011</td>\n",
" <td>7000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>DE</td>\n",
" <td>2011</td>\n",
" <td>5800</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>US</td>\n",
" <td>2011</td>\n",
" <td>15000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>FR</td>\n",
" <td>2012</td>\n",
" <td>6900</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>DE</td>\n",
" <td>2012</td>\n",
" <td>6000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>US</td>\n",
" <td>2012</td>\n",
" <td>14000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>FR</td>\n",
" <td>2013</td>\n",
" <td>7000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>DE</td>\n",
" <td>2013</td>\n",
" <td>6200</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>US</td>\n",
" <td>2013</td>\n",
" <td>13000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" country year n\n",
"0 FR 2011 7000\n",
"1 DE 2011 5800\n",
"2 US 2011 15000\n",
"3 FR 2012 6900\n",
"4 DE 2012 6000\n",
"5 US 2012 14000\n",
"6 FR 2013 7000\n",
"7 DE 2013 6200\n",
"8 US 2013 13000"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pd.melt(cases, id_vars=\"country\", var_name=\"year\", value_name=\"n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Spread rows into columns."
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"```r\n",
"> tidyr::spread(pollution, size, amount)\n",
" city large small\n",
"1 Beijing 121 56\n",
"2 London 22 16\n",
"3 New York 23 14\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>size</th>\n",
" <th>large</th>\n",
" <th>small</th>\n",
" </tr>\n",
" <tr>\n",
" <th>city</th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>Beijing</th>\n",
" <td>121</td>\n",
" <td>56</td>\n",
" </tr>\n",
" <tr>\n",
" <th>London</th>\n",
" <td>22</td>\n",
" <td>16</td>\n",
" </tr>\n",
" <tr>\n",
" <th>New York</th>\n",
" <td>23</td>\n",
" <td>14</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
"size large small\n",
"city \n",
"Beijing 121 56\n",
"London 22 16\n",
"New York 23 14"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pollution.pivot(index=\"city\", columns=\"size\", values=\"amount\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Separate one column into several."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> tidyr::separate(storms, date, c(\"y\", \"m\", \"d\"))\n",
"# A tibble: 6 x 6\n",
" storm wind pressure y m d\n",
"* <chr> <int> <int> <chr> <chr> <chr>\n",
"1 Alberto 110 1007 2000 08 03\n",
"2 Alex 45 1009 1998 07 27\n",
"3 Allison 65 1005 1995 06 03\n",
"4 Ana 40 1013 1997 06 30\n",
"5 Arlene 50 1010 1999 06 11\n",
"6 Arthur 45 1010 1996 06 17\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>storm</th>\n",
" <th>wind</th>\n",
" <th>pressure</th>\n",
" <th>d</th>\n",
" <th>m</th>\n",
" <th>y</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Alberto</td>\n",
" <td>110</td>\n",
" <td>1007</td>\n",
" <td>03</td>\n",
" <td>08</td>\n",
" <td>2000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Alex</td>\n",
" <td>45</td>\n",
" <td>1009</td>\n",
" <td>27</td>\n",
" <td>07</td>\n",
" <td>1998</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Allison</td>\n",
" <td>65</td>\n",
" <td>1005</td>\n",
" <td>03</td>\n",
" <td>06</td>\n",
" <td>1995</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Ana</td>\n",
" <td>40</td>\n",
" <td>1013</td>\n",
" <td>30</td>\n",
" <td>06</td>\n",
" <td>1997</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Arlene</td>\n",
" <td>50</td>\n",
" <td>1010</td>\n",
" <td>11</td>\n",
" <td>06</td>\n",
" <td>1999</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>Arthur</td>\n",
" <td>45</td>\n",
" <td>1010</td>\n",
" <td>17</td>\n",
" <td>06</td>\n",
" <td>1996</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" storm wind pressure d m y\n",
"0 Alberto 110 1007 03 08 2000\n",
"1 Alex 45 1009 27 07 1998\n",
"2 Allison 65 1005 03 06 1995\n",
"3 Ana 40 1013 30 06 1997\n",
"4 Arlene 50 1010 11 06 1999\n",
"5 Arthur 45 1010 17 06 1996"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"storms.assign(\n",
" y = storms.date.map(lambda x: x.split(\"-\")[0]),\n",
" m = storms.date.map(lambda x: x.split(\"-\")[1]),\n",
" d = storms.date.map(lambda x: x.split(\"-\")[2])\n",
").drop(['date'], axis=1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extract rows that meet logical criteria"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::filter(iris, Sepal.Length > 7)\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"1 7.1 3.0 5.9 2.1 virginica\n",
"2 7.6 3.0 6.6 2.1 virginica\n",
"3 7.3 2.9 6.3 1.8 virginica\n",
"4 7.2 3.6 6.1 2.5 virginica\n",
"5 7.7 3.8 6.7 2.2 virginica\n",
"6 7.7 2.6 6.9 2.3 virginica\n",
"7 7.7 2.8 6.7 2.0 virginica\n",
"8 7.2 3.2 6.0 1.8 virginica\n",
"9 7.2 3.0 5.8 1.6 virginica\n",
"10 7.4 2.8 6.1 1.9 virginica\n",
"11 7.9 3.8 6.4 2.0 virginica\n",
"12 7.7 3.0 6.1 2.3 virginica\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>102</th>\n",
" <td>7.1</td>\n",
" <td>3.0</td>\n",
" <td>5.9</td>\n",
" <td>2.1</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>105</th>\n",
" <td>7.6</td>\n",
" <td>3.0</td>\n",
" <td>6.6</td>\n",
" <td>2.1</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>107</th>\n",
" <td>7.3</td>\n",
" <td>2.9</td>\n",
" <td>6.3</td>\n",
" <td>1.8</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>109</th>\n",
" <td>7.2</td>\n",
" <td>3.6</td>\n",
" <td>6.1</td>\n",
" <td>2.5</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>117</th>\n",
" <td>7.7</td>\n",
" <td>3.8</td>\n",
" <td>6.7</td>\n",
" <td>2.2</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>118</th>\n",
" <td>7.7</td>\n",
" <td>2.6</td>\n",
" <td>6.9</td>\n",
" <td>2.3</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>122</th>\n",
" <td>7.7</td>\n",
" <td>2.8</td>\n",
" <td>6.7</td>\n",
" <td>2.0</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>125</th>\n",
" <td>7.2</td>\n",
" <td>3.2</td>\n",
" <td>6.0</td>\n",
" <td>1.8</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>129</th>\n",
" <td>7.2</td>\n",
" <td>3.0</td>\n",
" <td>5.8</td>\n",
" <td>1.6</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>130</th>\n",
" <td>7.4</td>\n",
" <td>2.8</td>\n",
" <td>6.1</td>\n",
" <td>1.9</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>131</th>\n",
" <td>7.9</td>\n",
" <td>3.8</td>\n",
" <td>6.4</td>\n",
" <td>2.0</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>135</th>\n",
" <td>7.7</td>\n",
" <td>3.0</td>\n",
" <td>6.1</td>\n",
" <td>2.3</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"102 7.1 3.0 5.9 2.1 virginica\n",
"105 7.6 3.0 6.6 2.1 virginica\n",
"107 7.3 2.9 6.3 1.8 virginica\n",
"109 7.2 3.6 6.1 2.5 virginica\n",
"117 7.7 3.8 6.7 2.2 virginica\n",
"118 7.7 2.6 6.9 2.3 virginica\n",
"122 7.7 2.8 6.7 2.0 virginica\n",
"125 7.2 3.2 6.0 1.8 virginica\n",
"129 7.2 3.0 5.8 1.6 virginica\n",
"130 7.4 2.8 6.1 1.9 virginica\n",
"131 7.9 3.8 6.4 2.0 virginica\n",
"135 7.7 3.0 6.1 2.3 virginica"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris[iris['Sepal.Length'] > 7]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Remove duplicate rows."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::distinct(iris)\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"1 5.1 3.5 1.4 0.2 setosa\n",
"2 4.9 3.0 1.4 0.2 setosa\n",
"3 4.7 3.2 1.3 0.2 setosa\n",
"4 4.6 3.1 1.5 0.2 setosa\n",
"5 5.0 3.6 1.4 0.2 setosa\n",
"6 5.4 3.9 1.7 0.4 setosa\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>5.1</td>\n",
" <td>3.5</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>4.9</td>\n",
" <td>3.0</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>4.7</td>\n",
" <td>3.2</td>\n",
" <td>1.3</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4.6</td>\n",
" <td>3.1</td>\n",
" <td>1.5</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5.0</td>\n",
" <td>3.6</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"0 5.1 3.5 1.4 0.2 setosa\n",
"1 4.9 3.0 1.4 0.2 setosa\n",
"2 4.7 3.2 1.3 0.2 setosa\n",
"3 4.6 3.1 1.5 0.2 setosa\n",
"4 5.0 3.6 1.4 0.2 setosa"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.drop_duplicates().head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Randomly select fraction of rows."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::sample_frac(iris, 0.5, replace = TRUE)\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"94 5.0 2.3 3.3 1.0 versicolor\n",
"24 5.1 3.3 1.7 0.5 setosa\n",
"4 4.6 3.1 1.5 0.2 setosa\n",
"139 6.0 3.0 4.8 1.8 virginica\n",
"82 5.5 2.4 3.7 1.0 versicolor\n",
"128 6.1 3.0 4.9 1.8 virginica\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>36</th>\n",
" <td>5.5</td>\n",
" <td>3.5</td>\n",
" <td>1.3</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>37</th>\n",
" <td>4.9</td>\n",
" <td>3.6</td>\n",
" <td>1.4</td>\n",
" <td>0.1</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>132</th>\n",
" <td>6.4</td>\n",
" <td>2.8</td>\n",
" <td>5.6</td>\n",
" <td>2.2</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>106</th>\n",
" <td>4.9</td>\n",
" <td>2.5</td>\n",
" <td>4.5</td>\n",
" <td>1.7</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>97</th>\n",
" <td>6.2</td>\n",
" <td>2.9</td>\n",
" <td>4.3</td>\n",
" <td>1.3</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"36 5.5 3.5 1.3 0.2 setosa\n",
"37 4.9 3.6 1.4 0.1 setosa\n",
"132 6.4 2.8 5.6 2.2 virginica\n",
"106 4.9 2.5 4.5 1.7 virginica\n",
"97 6.2 2.9 4.3 1.3 versicolor"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.sample(frac=0.5, replace=True).head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Randomly select n rows."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::sample_n(iris, 10, replace = TRUE)\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"13 4.8 3.0 1.4 0.1 setosa\n",
"135 6.1 2.6 5.6 1.4 virginica\n",
"69 6.2 2.2 4.5 1.5 versicolor\n",
"31 4.8 3.1 1.6 0.2 setosa\n",
"27 5.0 3.4 1.6 0.4 setosa\n",
"100 5.7 2.8 4.1 1.3 versicolor\n",
"86 6.0 3.4 4.5 1.6 versicolor\n",
"150 5.9 3.0 5.1 1.8 virginica\n",
"125 6.7 3.3 5.7 2.1 virginica\n",
"122 5.6 2.8 4.9 2.0 virginica\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>81</th>\n",
" <td>5.5</td>\n",
" <td>2.4</td>\n",
" <td>3.7</td>\n",
" <td>1.0</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" <tr>\n",
" <th>107</th>\n",
" <td>7.3</td>\n",
" <td>2.9</td>\n",
" <td>6.3</td>\n",
" <td>1.8</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>143</th>\n",
" <td>6.8</td>\n",
" <td>3.2</td>\n",
" <td>5.9</td>\n",
" <td>2.3</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>127</th>\n",
" <td>6.1</td>\n",
" <td>3.0</td>\n",
" <td>4.9</td>\n",
" <td>1.8</td>\n",
" <td>virginica</td>\n",
" </tr>\n",
" <tr>\n",
" <th>77</th>\n",
" <td>6.7</td>\n",
" <td>3.0</td>\n",
" <td>5.0</td>\n",
" <td>1.7</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>5.4</td>\n",
" <td>3.9</td>\n",
" <td>1.7</td>\n",
" <td>0.4</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>64</th>\n",
" <td>5.6</td>\n",
" <td>2.9</td>\n",
" <td>3.6</td>\n",
" <td>1.3</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>4.8</td>\n",
" <td>3.0</td>\n",
" <td>1.4</td>\n",
" <td>0.1</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>55</th>\n",
" <td>5.7</td>\n",
" <td>2.8</td>\n",
" <td>4.5</td>\n",
" <td>1.3</td>\n",
" <td>versicolor</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>5.4</td>\n",
" <td>3.4</td>\n",
" <td>1.5</td>\n",
" <td>0.4</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"81 5.5 2.4 3.7 1.0 versicolor\n",
"107 7.3 2.9 6.3 1.8 virginica\n",
"143 6.8 3.2 5.9 2.3 virginica\n",
"127 6.1 3.0 4.9 1.8 virginica\n",
"77 6.7 3.0 5.0 1.7 versicolor\n",
"5 5.4 3.9 1.7 0.4 setosa\n",
"64 5.6 2.9 3.6 1.3 versicolor\n",
"12 4.8 3.0 1.4 0.1 setosa\n",
"55 5.7 2.8 4.5 1.3 versicolor\n",
"31 5.4 3.4 1.5 0.4 setosa"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.sample(n=10, replace=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Select rows by position"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::slice(iris, 10:15)\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"1 4.9 3.1 1.5 0.1 setosa\n",
"2 5.4 3.7 1.5 0.2 setosa\n",
"3 4.8 3.4 1.6 0.2 setosa\n",
"4 4.8 3.0 1.4 0.1 setosa\n",
"5 4.3 3.0 1.1 0.1 setosa\n",
"6 5.8 4.0 1.2 0.2 setosa\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>4.9</td>\n",
" <td>3.1</td>\n",
" <td>1.5</td>\n",
" <td>0.1</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>5.4</td>\n",
" <td>3.7</td>\n",
" <td>1.5</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td>4.8</td>\n",
" <td>3.4</td>\n",
" <td>1.6</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>4.8</td>\n",
" <td>3.0</td>\n",
" <td>1.4</td>\n",
" <td>0.1</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td>4.3</td>\n",
" <td>3.0</td>\n",
" <td>1.1</td>\n",
" <td>0.1</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"9 4.9 3.1 1.5 0.1 setosa\n",
"10 5.4 3.7 1.5 0.2 setosa\n",
"11 4.8 3.4 1.6 0.2 setosa\n",
"12 4.8 3.0 1.4 0.1 setosa\n",
"13 4.3 3.0 1.1 0.1 setosa"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.iloc[9:14] # 0-based vs 1-based indexing in R"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Select and order by top n entires."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::top_n(storms, 2, date)\n",
"# A tibble: 2 x 4\n",
" storm wind pressure date\n",
" <chr> <int> <int> <date>\n",
"1 Alberto 110 1007 2000-08-03\n",
"2 Arlene 50 1010 1999-06-11\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>storm</th>\n",
" <th>wind</th>\n",
" <th>pressure</th>\n",
" <th>date</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Alberto</td>\n",
" <td>110</td>\n",
" <td>1007</td>\n",
" <td>2000-08-03</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Arlene</td>\n",
" <td>50</td>\n",
" <td>1010</td>\n",
" <td>1999-06-11</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" storm wind pressure date\n",
"0 Alberto 110 1007 2000-08-03\n",
"4 Arlene 50 1010 1999-06-11"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"storms.sort_values('date', ascending=False).head(n=2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Select columns by name."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::select(iris, Sepal.Width, Petal.Length, Species)\n",
" Sepal.Width Petal.Length Species\n",
"1 3.5 1.4 setosa\n",
"2 3.0 1.4 setosa\n",
"3 3.2 1.3 setosa\n",
"4 3.1 1.5 setosa\n",
"5 3.6 1.4 setosa\n",
"6 3.9 1.7 setosa\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>3.5</td>\n",
" <td>1.4</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>3.0</td>\n",
" <td>1.4</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3.2</td>\n",
" <td>1.3</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>3.1</td>\n",
" <td>1.5</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>3.6</td>\n",
" <td>1.4</td>\n",
" <td>setosa</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Width Petal.Length Species\n",
"0 3.5 1.4 setosa\n",
"1 3.0 1.4 setosa\n",
"2 3.2 1.3 setosa\n",
"3 3.1 1.5 setosa\n",
"4 3.6 1.4 setosa"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris[['Sepal.Width', 'Petal.Length', 'Species']].head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summarize data into a single row of values."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::summarize(iris, avg = mean(Sepal.Length))\n",
" avg\n",
"1 5.843333\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"5.8433333333333337"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris['Sepal.Length'].mean()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Apply a summary function to each column"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::summarize_each(iris, dplyr::funs(mean))\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"1 5.843333 3.057333 3.758 1.199333 NA\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"Sepal.Length 5.843333\n",
"Sepal.Width 3.057333\n",
"Petal.Length 3.758000\n",
"Petal.Width 1.199333\n",
"dtype: float64"
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.drop(['Species'], axis=1).apply(np.mean)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Count number of rows with each unique value of a variable."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::count(iris, Species)\n",
"# A tibble: 3 x 2\n",
" Species n\n",
" <fctr> <int>\n",
"1 setosa 50\n",
"2 versicolor 50\n",
"3 virginica 50\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"Species\n",
"setosa 50\n",
"versicolor 50\n",
"virginica 50\n",
"dtype: int64"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.groupby('Species').size()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::mutate(iris, sepal = Sepal.Length + Sepal.Width)\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species sepal\n",
"1 5.1 3.5 1.4 0.2 setosa 8.6\n",
"2 4.9 3.0 1.4 0.2 setosa 7.9\n",
"3 4.7 3.2 1.3 0.2 setosa 7.9\n",
"4 4.6 3.1 1.5 0.2 setosa 7.7\n",
"5 5.0 3.6 1.4 0.2 setosa 8.6\n",
"6 5.4 3.9 1.7 0.4 setosa 9.3\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" <th>sepal</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>5.1</td>\n",
" <td>3.5</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" <td>8.6</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>4.9</td>\n",
" <td>3.0</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" <td>7.9</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>4.7</td>\n",
" <td>3.2</td>\n",
" <td>1.3</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" <td>7.9</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4.6</td>\n",
" <td>3.1</td>\n",
" <td>1.5</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" <td>7.7</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5.0</td>\n",
" <td>3.6</td>\n",
" <td>1.4</td>\n",
" <td>0.2</td>\n",
" <td>setosa</td>\n",
" <td>8.6</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species sepal\n",
"0 5.1 3.5 1.4 0.2 setosa 8.6\n",
"1 4.9 3.0 1.4 0.2 setosa 7.9\n",
"2 4.7 3.2 1.3 0.2 setosa 7.9\n",
"3 4.6 3.1 1.5 0.2 setosa 7.7\n",
"4 5.0 3.6 1.4 0.2 setosa 8.6"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.assign(\n",
" sepal = iris['Sepal.Length'] + iris['Sepal.Width']\n",
").head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Apply window function to each column."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::mutate_each(iris, funs(min_rank))\n",
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"1 33 126 12 6 1\n",
"2 17 58 12 6 1\n",
"3 10 95 5 6 1\n",
"4 6 84 25 6 1\n",
"5 23 132 12 6 1\n",
"6 47 145 45 42 1\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sepal.Length</th>\n",
" <th>Sepal.Width</th>\n",
" <th>Petal.Length</th>\n",
" <th>Petal.Width</th>\n",
" <th>Species</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>33.0</td>\n",
" <td>126.0</td>\n",
" <td>12.0</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>17.0</td>\n",
" <td>58.0</td>\n",
" <td>12.0</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>10.0</td>\n",
" <td>95.0</td>\n",
" <td>5.0</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>6.0</td>\n",
" <td>84.0</td>\n",
" <td>25.0</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>23.0</td>\n",
" <td>132.0</td>\n",
" <td>12.0</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sepal.Length Sepal.Width Petal.Length Petal.Width Species\n",
"0 33.0 126.0 12.0 6.0 1.0\n",
"1 17.0 58.0 12.0 6.0 1.0\n",
"2 10.0 95.0 5.0 6.0 1.0\n",
"3 6.0 84.0 25.0 6.0 1.0\n",
"4 23.0 132.0 12.0 6.0 1.0"
]
},
"execution_count": 49,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.apply(lambda x: x.rank(method=\"min\")).head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Compute one or more new columns. Drop original columns."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```r\n",
"> dplyr::transmute(iris, sepal = Sepal.Length + Sepal.Width)\n",
" sepal\n",
"1 8.6\n",
"2 7.9\n",
"3 7.9\n",
"4 7.7\n",
"5 8.6\n",
"6 9.3\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>sepal</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>8.6</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>7.9</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>7.9</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>7.7</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>8.6</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" sepal\n",
"0 8.6\n",
"1 7.9\n",
"2 7.9\n",
"3 7.7\n",
"4 8.6"
]
},
"execution_count": 56,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"iris.assign(\n",
" sepal = iris['Sepal.Length'] + iris['Sepal.Width']\n",
")[['sepal',]].head()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment