Created
November 22, 2021 09:15
-
-
Save olayinka04/975d41b85e3bed16aee9f5eeec3f3b5d to your computer and use it in GitHub Desktop.
DE IBM 3 - Python Project for Data Engineering
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"cells":[{"cell_type":"markdown","source":["<center>\n"," <img src=\"https://gitlab.com/ibm/skills-network/courses/placeholder101/-/raw/master/labs/module%201/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\" />\n","</center>\n"],"metadata":{}},{"cell_type":"markdown","source":["# Peer Review Assignment - Data Engineer - Extract API Data\n"],"metadata":{}},{"cell_type":"markdown","source":["Estimated time needed: **20** minutes\n"],"metadata":{}},{"cell_type":"markdown","source":["## Objectives\n","\n","In this part you will:\n","\n","* Collect exchange rate data using an API\n","* Store the data as a CSV\n"],"metadata":{}},{"cell_type":"markdown","source":["For this lab, we are going to be using Python and several Python libraries. Some of these libraries might be installed in your lab environment or in SN Labs. Others may need to be installed by you. The cells below will install these libraries when executed.\n"],"metadata":{}},{"cell_type":"code","execution_count":null,"source":["#!pip install pandas\n","#!pip install requests"],"outputs":[],"metadata":{}},{"cell_type":"markdown","source":["## Imports\n","\n","Import any additional libraries you may need here.\n"],"metadata":{}},{"cell_type":"code","execution_count":1,"source":["import requests\n","import pandas as pd"],"outputs":[],"metadata":{}},{"cell_type":"markdown","source":["## Extract Data Using an API\n"],"metadata":{}},{"cell_type":"markdown","source":["Using ExchangeRate-API we will extract currency exchange rate data. Use the below steps to get the access key and to get the data.\n","\n","1. Open the url : [https://exchangeratesapi.io/](https://exchangeratesapi.io/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0221ENSkillsNetwork23455645-2021-01-01) and create a free account.\n","2. Once the account is created. You will get the Get the Free API key option on the top as shown below:\n","\n","<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0221EN-SkillsNetwork/labs/module%206/Final%20Assignment/Images/getapi.png\"/>\n","\n","3. Copy the API key and use in the url in Question 1.\n"],"metadata":{}},{"cell_type":"markdown","source":["### Call the API\n","\n","<b> Question 1</b> Using the `requests` library call the endpoint given above and save the text, remember the first few characters of the output:\n"],"metadata":{}},{"cell_type":"code","execution_count":5,"source":["# Write your code here\n","url = \"http://api.exchangeratesapi.io/v1/latest?base=EUR&access_key=3ae8a41ae8b314fe2c76abee00969ef2\" #Make sure to change ******* to your API key.\n","response = requests.get(url)\n","response.json()"],"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'success': True,\n"," 'timestamp': 1629978784,\n"," 'base': 'EUR',\n"," 'date': '2021-08-26',\n"," 'rates': {'AED': 4.325492,\n"," 'AFN': 101.43806,\n"," 'ALL': 121.819679,\n"," 'AMD': 580.026085,\n"," 'ANG': 2.113768,\n"," 'AOA': 749.061146,\n"," 'ARS': 114.679114,\n"," 'AUD': 1.620745,\n"," 'AWG': 2.120293,\n"," 'AZN': 2.006053,\n"," 'BAM': 1.955985,\n"," 'BBD': 2.377689,\n"," 'BDT': 100.507186,\n"," 'BGN': 1.958644,\n"," 'BHD': 0.443971,\n"," 'BIF': 2336.345792,\n"," 'BMD': 1.177614,\n"," 'BND': 1.592727,\n"," 'BOB': 8.119445,\n"," 'BRL': 6.136426,\n"," 'BSD': 1.177594,\n"," 'BTC': 2.5066732e-05,\n"," 'BTN': 87.353941,\n"," 'BWP': 13.186948,\n"," 'BYN': 2.960735,\n"," 'BYR': 23081.225896,\n"," 'BZD': 2.373689,\n"," 'CAD': 1.484164,\n"," 'CDF': 2351.69458,\n"," 'CHF': 1.078894,\n"," 'CLF': 0.033425,\n"," 'CLP': 922.301766,\n"," 'CNY': 7.633177,\n"," 'COP': 4552.06524,\n"," 'CRC': 732.038163,\n"," 'CUC': 1.177614,\n"," 'CUP': 31.20676,\n"," 'CVE': 110.273762,\n"," 'CZK': 25.557514,\n"," 'DJF': 209.636656,\n"," 'DKK': 7.436571,\n"," 'DOP': 67.016325,\n"," 'DZD': 159.201324,\n"," 'EGP': 18.520796,\n"," 'ERN': 17.669871,\n"," 'ETB': 53.696366,\n"," 'EUR': 1,\n"," 'FJD': 2.476109,\n"," 'FKP': 0.850947,\n"," 'GBP': 0.856802,\n"," 'GEL': 3.674223,\n"," 'GGP': 0.850947,\n"," 'GHS': 7.112665,\n"," 'GIP': 0.850947,\n"," 'GMD': 60.246932,\n"," 'GNF': 11543.317163,\n"," 'GTQ': 9.114524,\n"," 'GYD': 246.17956,\n"," 'HKD': 9.169664,\n"," 'HNL': 28.031227,\n"," 'HRK': 7.489527,\n"," 'HTG': 116.287239,\n"," 'HUF': 348.991653,\n"," 'IDR': 17010.569082,\n"," 'ILS': 3.792383,\n"," 'IMP': 0.850947,\n"," 'INR': 87.321753,\n"," 'IQD': 1718.10651,\n"," 'IRR': 49683.516564,\n"," 'ISK': 148.579348,\n"," 'JEP': 0.850947,\n"," 'JMD': 179.482261,\n"," 'JOD': 0.834904,\n"," 'JPY': 129.636424,\n"," 'KES': 129.266391,\n"," 'KGS': 99.743584,\n"," 'KHR': 4802.181553,\n"," 'KMF': 493.243476,\n"," 'KPW': 1059.852408,\n"," 'KRW': 1374.605154,\n"," 'KWD': 0.35445,\n"," 'KYD': 0.981378,\n"," 'KZT': 502.829952,\n"," 'LAK': 11287.196813,\n"," 'LBP': 1780.741487,\n"," 'LKR': 234.928966,\n"," 'LRD': 202.372542,\n"," 'LSL': 17.629132,\n"," 'LTL': 3.477187,\n"," 'LVL': 0.712327,\n"," 'LYD': 5.332424,\n"," 'MAD': 10.558739,\n"," 'MDL': 20.837456,\n"," 'MGA': 4619.097006,\n"," 'MKD': 61.619896,\n"," 'MMK': 1938.310907,\n"," 'MNT': 3357.259862,\n"," 'MOP': 9.44475,\n"," 'MRO': 420.407841,\n"," 'MUR': 50.696058,\n"," 'MVR': 18.217762,\n"," 'MWK': 956.803022,\n"," 'MXN': 23.904396,\n"," 'MYR': 4.940133,\n"," 'MZN': 75.043445,\n"," 'NAD': 17.629132,\n"," 'NGN': 484.576176,\n"," 'NIO': 41.351286,\n"," 'NOK': 10.357082,\n"," 'NPR': 139.766105,\n"," 'NZD': 1.688704,\n"," 'OMR': 0.453393,\n"," 'PAB': 1.177594,\n"," 'PEN': 4.809482,\n"," 'PGK': 4.187733,\n"," 'PHP': 58.838277,\n"," 'PKR': 196.362902,\n"," 'PLN': 4.576193,\n"," 'PYG': 8158.348213,\n"," 'QAR': 4.287696,\n"," 'RON': 4.931732,\n"," 'RSD': 117.589343,\n"," 'RUB': 87.109844,\n"," 'RWF': 1187.914385,\n"," 'SAR': 4.416976,\n"," 'SBD': 9.481646,\n"," 'SCR': 15.156031,\n"," 'SDG': 521.097484,\n"," 'SEK': 10.237642,\n"," 'SGD': 1.592516,\n"," 'SHP': 1.622038,\n"," 'SLL': 12158.860258,\n"," 'SOS': 688.903731,\n"," 'SRD': 25.169109,\n"," 'STD': 24374.223226,\n"," 'SVC': 10.303819,\n"," 'SYP': 1480.93617,\n"," 'SZL': 17.565296,\n"," 'THB': 38.543561,\n"," 'TJS': 13.436268,\n"," 'TMT': 4.121647,\n"," 'TND': 3.282601,\n"," 'TOP': 2.665298,\n"," 'TRY': 9.850793,\n"," 'TTD': 8.006936,\n"," 'TWD': 32.907823,\n"," 'TZS': 2730.826975,\n"," 'UAH': 31.577409,\n"," 'UGX': 4162.730746,\n"," 'USD': 1.177614,\n"," 'UYU': 50.430007,\n"," 'UZS': 12546.036833,\n"," 'VEF': 251809370669.2647,\n"," 'VND': 26834.280331,\n"," 'VUV': 130.581948,\n"," 'WST': 3.01559,\n"," 'XAF': 656.009123,\n"," 'XAG': 0.049634,\n"," 'XAU': 0.000658,\n"," 'XCD': 3.18256,\n"," 'XDR': 0.829166,\n"," 'XOF': 656.003551,\n"," 'XPF': 119.757324,\n"," 'YER': 295.346157,\n"," 'ZAR': 17.549704,\n"," 'ZMK': 10599.931625,\n"," 'ZMW': 19.341937,\n"," 'ZWL': 379.191088}}"]},"metadata":{},"execution_count":5}],"metadata":{}},{"cell_type":"markdown","source":["### Save as DataFrame\n","\n","<b> Question 2</b> Using the data gathered turn it into a `pandas` dataframe. The dataframe should have the Currency as the index and `Rate` as their columns. Make sure to drop unnecessary columns.\n"],"metadata":{}},{"cell_type":"code","execution_count":8,"source":["# Turn the data into a dataframe\n","dataframe = pd.DataFrame(response.json())\n","dataframe"],"outputs":[{"output_type":"execute_result","data":{"text/plain":[" success timestamp base date rates\n","AED True 1629978784 EUR 2021-08-26 4.325492\n","AFN True 1629978784 EUR 2021-08-26 101.438060\n","ALL True 1629978784 EUR 2021-08-26 121.819679\n","AMD True 1629978784 EUR 2021-08-26 580.026085\n","ANG True 1629978784 EUR 2021-08-26 2.113768\n",".. ... ... ... ... ...\n","YER True 1629978784 EUR 2021-08-26 295.346157\n","ZAR True 1629978784 EUR 2021-08-26 17.549704\n","ZMK True 1629978784 EUR 2021-08-26 10599.931625\n","ZMW True 1629978784 EUR 2021-08-26 19.341937\n","ZWL True 1629978784 EUR 2021-08-26 379.191088\n","\n","[168 rows x 5 columns]"],"text/html":["<div>\n","<style scoped>\n"," .dataframe tbody tr th:only-of-type {\n"," vertical-align: middle;\n"," }\n","\n"," .dataframe tbody tr th {\n"," vertical-align: top;\n"," }\n","\n"," .dataframe thead th {\n"," text-align: right;\n"," }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n"," <thead>\n"," <tr style=\"text-align: right;\">\n"," <th></th>\n"," <th>success</th>\n"," <th>timestamp</th>\n"," <th>base</th>\n"," <th>date</th>\n"," <th>rates</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n"," <th>AED</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>4.325492</td>\n"," </tr>\n"," <tr>\n"," <th>AFN</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>101.438060</td>\n"," </tr>\n"," <tr>\n"," <th>ALL</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>121.819679</td>\n"," </tr>\n"," <tr>\n"," <th>AMD</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>580.026085</td>\n"," </tr>\n"," <tr>\n"," <th>ANG</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>2.113768</td>\n"," </tr>\n"," <tr>\n"," <th>...</th>\n"," <td>...</td>\n"," <td>...</td>\n"," <td>...</td>\n"," <td>...</td>\n"," <td>...</td>\n"," </tr>\n"," <tr>\n"," <th>YER</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>295.346157</td>\n"," </tr>\n"," <tr>\n"," <th>ZAR</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>17.549704</td>\n"," </tr>\n"," <tr>\n"," <th>ZMK</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>10599.931625</td>\n"," </tr>\n"," <tr>\n"," <th>ZMW</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>19.341937</td>\n"," </tr>\n"," <tr>\n"," <th>ZWL</th>\n"," <td>True</td>\n"," <td>1629978784</td>\n"," <td>EUR</td>\n"," <td>2021-08-26</td>\n"," <td>379.191088</td>\n"," </tr>\n"," </tbody>\n","</table>\n","<p>168 rows × 5 columns</p>\n","</div>"]},"metadata":{},"execution_count":8}],"metadata":{}},{"cell_type":"code","execution_count":9,"source":["# Drop unnescessary columns\n","dataframe.drop(columns=[\"success\", \"timestamp\", \"base\", \"date\"], inplace=True)\n","dataframe"],"outputs":[{"output_type":"execute_result","data":{"text/plain":[" rates\n","AED 4.325492\n","AFN 101.438060\n","ALL 121.819679\n","AMD 580.026085\n","ANG 2.113768\n",".. ...\n","YER 295.346157\n","ZAR 17.549704\n","ZMK 10599.931625\n","ZMW 19.341937\n","ZWL 379.191088\n","\n","[168 rows x 1 columns]"],"text/html":["<div>\n","<style scoped>\n"," .dataframe tbody tr th:only-of-type {\n"," vertical-align: middle;\n"," }\n","\n"," .dataframe tbody tr th {\n"," vertical-align: top;\n"," }\n","\n"," .dataframe thead th {\n"," text-align: right;\n"," }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n"," <thead>\n"," <tr style=\"text-align: right;\">\n"," <th></th>\n"," <th>rates</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n"," <th>AED</th>\n"," <td>4.325492</td>\n"," </tr>\n"," <tr>\n"," <th>AFN</th>\n"," <td>101.438060</td>\n"," </tr>\n"," <tr>\n"," <th>ALL</th>\n"," <td>121.819679</td>\n"," </tr>\n"," <tr>\n"," <th>AMD</th>\n"," <td>580.026085</td>\n"," </tr>\n"," <tr>\n"," <th>ANG</th>\n"," <td>2.113768</td>\n"," </tr>\n"," <tr>\n"," <th>...</th>\n"," <td>...</td>\n"," </tr>\n"," <tr>\n"," <th>YER</th>\n"," <td>295.346157</td>\n"," </tr>\n"," <tr>\n"," <th>ZAR</th>\n"," <td>17.549704</td>\n"," </tr>\n"," <tr>\n"," <th>ZMK</th>\n"," <td>10599.931625</td>\n"," </tr>\n"," <tr>\n"," <th>ZMW</th>\n"," <td>19.341937</td>\n"," </tr>\n"," <tr>\n"," <th>ZWL</th>\n"," <td>379.191088</td>\n"," </tr>\n"," </tbody>\n","</table>\n","<p>168 rows × 1 columns</p>\n","</div>"]},"metadata":{},"execution_count":9}],"metadata":{}},{"cell_type":"markdown","source":["### Load the Data\n","\n","Using the dataframe save it as a CSV names `exchange_rates_1.csv`.\n"],"metadata":{}},{"cell_type":"code","execution_count":11,"source":["# Save the Dataframe\n","dataframe.to_csv(\"/home/gustavo/Documents/Working_dir/DataEngineerIBM/Course3/exchange_rates_1.csv\")"],"outputs":[],"metadata":{}},{"cell_type":"markdown","source":["Your CSV should be in this format with more currencies\n","\n","| | Rates |\n","| --- | ---------- |\n","| AED | 4.398618 |\n","| AFN | 92.917693 |\n","| ALL | 123.099093 |\n","| AMD | 621.935674 |\n","| ANG | 2.149648 |\n"],"metadata":{}},{"cell_type":"markdown","source":["## Authors\n"],"metadata":{}},{"cell_type":"markdown","source":["Ramesh Sannareddy, Joseph Santarcangelo and Azim Hirjani\n"],"metadata":{}},{"cell_type":"markdown","source":["### Other Contributors\n"],"metadata":{}},{"cell_type":"markdown","source":["Rav Ahuja\n"],"metadata":{}},{"cell_type":"markdown","source":["## Change Log\n"],"metadata":{}},{"cell_type":"markdown","source":["| Date (YYYY-MM-DD) | Version | Changed By | Change Description |\n","| ----------------- | ------- | ----------------- | ---------------------------------- |\n","| 2021-04-15 | 0.2 | Malika | Updated the lab from USD to EUR |\n","| 2020-11-25 | 0.1 | Ramesh Sannareddy | Created initial version of the lab |\n"],"metadata":{}},{"cell_type":"markdown","source":["Copyright © 2020 IBM Corporation. This notebook and its source code are released under the terms of the [MIT License](https://cognitiveclass.ai/mit-license?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0221ENSkillsNetwork23455645-2021-01-01&cm_mmc=Email_Newsletter-\\_-Developer_Ed%2BTech-\\_-WW_WW-\\_-SkillsNetwork-Courses-IBM-DA0321EN-SkillsNetwork-21426264&cm_mmca1=000026UJ&cm_mmca2=10006555&cm_mmca3=M12345678&cvosrc=email.Newsletter.M12345678&cvo_campaign=000026UJ).\n"],"metadata":{}}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3.8.10 64-bit"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.8.10"},"interpreter":{"hash":"31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"}},"nbformat":4,"nbformat_minor":4} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is very helpful, thanks a loads!! :)