Skip to main content

Rice Paddy Methane Emissions Estimation: Part 1

banner-image

Justin Napolitano

2022-05-21 08:30:32 +0000 UTC


Table of Contents


Series

This is a post in the Rice Paddy Methane Emissions series.
Other posts in this series:

  • Rice Paddy Methane Emissions Estimation: Part 1

  • Methane Emissions Estimation Data Part 1: A Comparison between FAOSTAT and University of Malaysia Estimates

    This post documents the data exploration phase of a project that determines whether global methane emissions produced by rice paddies are undercounted.

    It is fairly code python and pandas heavy.

    The code and data exploration follows the summary below.

    Inspiration for this work

    The University of Malaysia in partnership with Climate TRACE release a paper stating that the UN undercounts rice paddy methane emissions by about 16%. Upon review of their claims, I decided to test the data myself across a 5 year distribution to verify that the claims hold up across multiple distributions.

    University of Malaysia Methodological Deficiencies

    • Comparing a 2019 distribution FAOSTAT to a 2020 distribution.
    • Rounding to 4 signficant figures.
    • Testing only one year of emissions data.
    • Did not publish data to hypothesis testing to determine if emission distributions significantly vary annually and between states.
    • Relying only on satellite data may undercount hectares in cultivation at higher altitudes.

    FaoStat Methodological Deficiencies

    • Relies upon official government statistics which can be manipulated at any point along the bureaucratic paper chain.
    • There is an incentive for certain nations to reduce their counts in order to receive international aid and to meet emissions standards.

    University of Malaysia acknowledging deficiencies (like all good papers should)

    “The difference between harvested rice cultivation area from statistical data and remote-sensing estimates can be due to two factors: (i) MODIS data which have moderate spatial resolution lead to mixed pixels, where rice fields and non-rice fields are combined. This can overestimate area, especially in lowland regions and have a low ability to detect small rice field patches in upland regions (Frolking et al 1999, Seto et al 2000); and (ii) political and policy factors (Yan et al., 2019) such as determination of the amount of subsidies for fertilizers and evaluation of achievement of government programs in the agricultural sector. Other factors that contribute to discrepancy in CH4 emission are from different emission and scale factors that are related to water regime and organic amendment. These values give high uncertainty since the availability of these data are limited and quite variable.”

    My Initial Impressions and findings

    Initial

    The percent difference and the tonnage difference do not support each other. I need to recalculate the totals section to ensure that we are doing things correctly.

    I need to confirm the values, but I’m initially impressed by the fact that the FAOSTAT data reports higher values than the Malaysia data on average. According to the included paper this should not be the case.

    Verified

    I calculated differences totals and means per DataFrame to ensure accuracy prior to the join. I also dropped pre-calcuated values when joining to ensure that the aggregation algorithms to not modify the results.

    The data is now consistent and supports the findings of the University of Malaysia Paper. With that said it is important to note that the differences recorded are far smaller than suggested.

    Import Dependencies

    import pandas as pd
    import matplotlib.pyplot as plt
    import geopandas as gpd
    import folium
    import contextily as cx
    from shapely.geometry import Point, LineString, Polygon
    import numpy as np
    from scipy.spatial import cKDTree
    from geopy.distance import distance
    import scipy.stats as stats
    

    Dependencies

    • Geopandas
    • pandas
    • openpyxl
    • Shapely
    • geopy Distance
    • numpy

    Exploration Plan

    Data Imports

    • /Users/jnapolitano/Projects/wattime-takehome/data/ch4_2015-2021.xlsx
    • /Users/jnapolitano/Projects/wattime-takehome/data/emissions_csv_fao_emiss_csv_ch4_fao_2015_2019_tonnes.xlsx

    Import Data Frames

    Since jupyter caches the data to the notebook json I can import the dataframes that I will be using together.

    If I were to build automated scripts to perform the analysis I would only load the data necessary to perform a process.

    Experiment with Plots for each Set

    I don’t know exactly which plots I want to include in the final report.

    I ’ll plot a few for each data set

    Calculate differences between the datasets

    • create a differences data frame
    • write to file for use
    • plot

    University of Malaysia Emission Estimates

    filepath = "/Users/jnapolitano/Projects/wattime-takehome/wattime-takehome/data/ch4_2015-2021.xlsx"
    
    malaysia_emissions_df = pd.read_excel(filepath)
    
    malaysia_emissions_df
    

    iso3_country country_name tCH4_2015 tCH4_2016 tCH4_2017 tCH4_2018 tCH4_2019 tCH4_2020 tCH4_2021
    0 BGD Bangladesh 2.344420e+06 2.278158e+06 2.098958e+06 2.141231e+06 2.070985e+06 2.106781e+06 1.983974e+06
    1 BRA Brazil 3.410233e+05 3.104189e+05 3.725173e+05 3.717030e+05 3.294713e+05 4.902874e+05 4.544874e+05
    2 CHN China 6.133647e+06 5.859531e+06 6.355071e+06 5.413962e+06 5.603352e+06 6.402353e+06 6.068210e+06
    3 ESP Spain 1.141464e+04 1.334803e+04 1.217299e+04 1.405410e+04 1.148324e+04 1.305461e+04 8.531579e+03
    4 IDN Indonesia 1.283649e+06 1.023129e+06 9.615327e+05 1.176982e+06 1.266668e+06 1.188195e+06 1.009936e+06
    5 IND India 6.219887e+06 5.309413e+06 6.228451e+06 6.589798e+06 7.501556e+06 7.599764e+06 6.567960e+06
    6 IRN Iran (Islamic Republic of) 8.774407e+04 9.180121e+04 9.620217e+04 8.875744e+04 9.500199e+04 9.600254e+04 9.053525e+04
    7 ITA Italy 4.995968e+04 4.937785e+04 5.443679e+04 4.469902e+04 4.566914e+04 5.101547e+04 5.089759e+04
    8 JPN Japan 2.305465e+05 2.284133e+05 2.708935e+05 1.548252e+05 2.332056e+05 2.835167e+05 1.574007e+05
    9 KHM Cambodia 4.954698e+05 5.731698e+05 4.517045e+05 5.592610e+05 5.947277e+05 6.412802e+05 5.644891e+05
    10 KOR Korea (the Republic of) 1.451878e+05 1.274597e+05 1.463222e+05 1.293543e+05 1.327782e+05 1.165467e+05 1.013006e+05
    11 LAO Lao People's Democratic Republic (the) 1.661169e+04 1.696441e+04 1.168063e+04 1.009675e+04 1.461058e+04 2.136270e+04 1.475014e+04
    12 LKA Sri Lanka 8.305626e+04 1.011743e+05 5.911841e+04 9.018914e+04 8.476088e+04 9.248238e+04 8.466966e+04
    13 MMR Myanmar 1.132082e+06 1.290806e+06 1.205169e+06 1.372447e+06 1.256888e+06 1.221904e+06 1.289837e+06
    14 MYS Malaysia 1.057399e+05 1.110049e+05 1.111291e+05 1.066525e+05 1.056287e+05 1.127141e+05 1.069696e+05
    15 NPL Nepal 1.007479e+05 6.667161e+04 8.081300e+04 9.200752e+04 1.164235e+05 7.168401e+04 4.811408e+04
    16 PAK Pakistan 4.852431e+05 5.945922e+05 5.372641e+05 4.532297e+05 6.528548e+05 6.401201e+05 4.849205e+05
    17 PHL Philippines (the) 3.432021e+05 4.073554e+05 3.836830e+05 4.175210e+05 3.584550e+05 4.462836e+05 4.383270e+05
    18 PRK Korea (the Democratic People's Republic of) 1.143217e+05 9.177653e+04 1.085457e+05 8.662578e+04 9.655062e+04 8.581038e+04 7.735988e+04
    19 THA Thailand 1.393798e+06 1.780993e+06 1.164699e+06 9.166575e+05 1.305046e+06 1.520788e+06 8.528673e+05
    20 TWN Taiwan (Province of China) 7.866956e+04 8.089149e+04 8.705634e+04 8.138151e+04 8.990870e+04 8.333327e+04 6.619861e+04
    21 USA United States of America (the) 1.611324e+05 1.618576e+05 1.684799e+05 1.657254e+05 1.691351e+05 1.941455e+05 1.634842e+05
    22 VNM Viet Nam 1.346013e+06 1.483777e+06 1.406437e+06 1.317455e+06 1.269751e+06 1.374450e+06 1.502787e+06
    23 NaN Total 2.270357e+07 2.205208e+07 2.237234e+07 2.179462e+07 2.340491e+07 2.485387e+07 2.218801e+07

    Calculate Co2 Equivalency

    malaysia_emissions_df['tCO2_2015'] = (malaysia_emissions_df['tCH4_2015'] * 25)
    malaysia_emissions_df['tCO2_2016'] = (malaysia_emissions_df['tCH4_2016'] * 25)
    malaysia_emissions_df['tCO2_2017'] = (malaysia_emissions_df['tCH4_2017'] * 25)
    malaysia_emissions_df['tCO2_2018'] = (malaysia_emissions_df['tCH4_2018'] * 25)
    malaysia_emissions_df['tCO2_2019'] = (malaysia_emissions_df['tCH4_2019'] * 25)
    

    Calculate Means

    malaysia_emissions_df.loc['mean'] = malaysia_emissions_df.loc[(malaysia_emissions_df['country_name'] != "Total")].select_dtypes(np.number).mean()
    malaysia_emissions_df.at['mean','country_name'] = 'mean'
    malaysia_emissions_df
    

    iso3_country country_name tCH4_2015 tCH4_2016 tCH4_2017 tCH4_2018 tCH4_2019 tCH4_2020 tCH4_2021 tCO2_2015 tCO2_2016 tCO2_2017 tCO2_2018 tCO2_2019
    0 BGD Bangladesh 2.344420e+06 2.278158e+06 2.098958e+06 2.141231e+06 2.070985e+06 2.106781e+06 1.983974e+06 5.861049e+07 5.695395e+07 5.247394e+07 5.353076e+07 5.177463e+07
    1 BRA Brazil 3.410233e+05 3.104189e+05 3.725173e+05 3.717030e+05 3.294713e+05 4.902874e+05 4.544874e+05 8.525583e+06 7.760473e+06 9.312934e+06 9.292575e+06 8.236783e+06
    2 CHN China 6.133647e+06 5.859531e+06 6.355071e+06 5.413962e+06 5.603352e+06 6.402353e+06 6.068210e+06 1.533412e+08 1.464883e+08 1.588768e+08 1.353491e+08 1.400838e+08
    3 ESP Spain 1.141464e+04 1.334803e+04 1.217299e+04 1.405410e+04 1.148324e+04 1.305461e+04 8.531579e+03 2.853661e+05 3.337007e+05 3.043248e+05 3.513524e+05 2.870810e+05
    4 IDN Indonesia 1.283649e+06 1.023129e+06 9.615327e+05 1.176982e+06 1.266668e+06 1.188195e+06 1.009936e+06 3.209122e+07 2.557824e+07 2.403832e+07 2.942454e+07 3.166670e+07
    5 IND India 6.219887e+06 5.309413e+06 6.228451e+06 6.589798e+06 7.501556e+06 7.599764e+06 6.567960e+06 1.554972e+08 1.327353e+08 1.557113e+08 1.647450e+08 1.875389e+08
    6 IRN Iran (Islamic Republic of) 8.774407e+04 9.180121e+04 9.620217e+04 8.875744e+04 9.500199e+04 9.600254e+04 9.053525e+04 2.193602e+06 2.295030e+06 2.405054e+06 2.218936e+06 2.375050e+06
    7 ITA Italy 4.995968e+04 4.937785e+04 5.443679e+04 4.469902e+04 4.566914e+04 5.101547e+04 5.089759e+04 1.248992e+06 1.234446e+06 1.360920e+06 1.117475e+06 1.141729e+06
    8 JPN Japan 2.305465e+05 2.284133e+05 2.708935e+05 1.548252e+05 2.332056e+05 2.835167e+05 1.574007e+05 5.763662e+06 5.710333e+06 6.772337e+06 3.870631e+06 5.830141e+06
    9 KHM Cambodia 4.954698e+05 5.731698e+05 4.517045e+05 5.592610e+05 5.947277e+05 6.412802e+05 5.644891e+05 1.238675e+07 1.432925e+07 1.129261e+07 1.398153e+07 1.486819e+07
    10 KOR Korea (the Republic of) 1.451878e+05 1.274597e+05 1.463222e+05 1.293543e+05 1.327782e+05 1.165467e+05 1.013006e+05 3.629695e+06 3.186493e+06 3.658056e+06 3.233858e+06 3.319455e+06
    11 LAO Lao People's Democratic Republic (the) 1.661169e+04 1.696441e+04 1.168063e+04 1.009675e+04 1.461058e+04 2.136270e+04 1.475014e+04 4.152924e+05 4.241102e+05 2.920158e+05 2.524186e+05 3.652645e+05
    12 LKA Sri Lanka 8.305626e+04 1.011743e+05 5.911841e+04 9.018914e+04 8.476088e+04 9.248238e+04 8.466966e+04 2.076407e+06 2.529358e+06 1.477960e+06 2.254728e+06 2.119022e+06
    13 MMR Myanmar 1.132082e+06 1.290806e+06 1.205169e+06 1.372447e+06 1.256888e+06 1.221904e+06 1.289837e+06 2.830206e+07 3.227014e+07 3.012923e+07 3.431117e+07 3.142221e+07
    14 MYS Malaysia 1.057399e+05 1.110049e+05 1.111291e+05 1.066525e+05 1.056287e+05 1.127141e+05 1.069696e+05 2.643498e+06 2.775123e+06 2.778227e+06 2.666313e+06 2.640717e+06
    15 NPL Nepal 1.007479e+05 6.667161e+04 8.081300e+04 9.200752e+04 1.164235e+05 7.168401e+04 4.811408e+04 2.518697e+06 1.666790e+06 2.020325e+06 2.300188e+06 2.910588e+06
    16 PAK Pakistan 4.852431e+05 5.945922e+05 5.372641e+05 4.532297e+05 6.528548e+05 6.401201e+05 4.849205e+05 1.213108e+07 1.486480e+07 1.343160e+07 1.133074e+07 1.632137e+07
    17 PHL Philippines (the) 3.432021e+05 4.073554e+05 3.836830e+05 4.175210e+05 3.584550e+05 4.462836e+05 4.383270e+05 8.580052e+06 1.018389e+07 9.592074e+06 1.043803e+07 8.961374e+06
    18 PRK Korea (the Democratic People's Republic of) 1.143217e+05 9.177653e+04 1.085457e+05 8.662578e+04 9.655062e+04 8.581038e+04 7.735988e+04 2.858041e+06 2.294413e+06 2.713641e+06 2.165645e+06 2.413765e+06
    19 THA Thailand 1.393798e+06 1.780993e+06 1.164699e+06 9.166575e+05 1.305046e+06 1.520788e+06 8.528673e+05 3.484495e+07 4.452483e+07 2.911748e+07 2.291644e+07 3.262615e+07
    20 TWN Taiwan (Province of China) 7.866956e+04 8.089149e+04 8.705634e+04 8.138151e+04 8.990870e+04 8.333327e+04 6.619861e+04 1.966739e+06 2.022287e+06 2.176408e+06 2.034538e+06 2.247717e+06
    21 USA United States of America (the) 1.611324e+05 1.618576e+05 1.684799e+05 1.657254e+05 1.691351e+05 1.941455e+05 1.634842e+05 4.028310e+06 4.046440e+06 4.211999e+06 4.143136e+06 4.228377e+06
    22 VNM Viet Nam 1.346013e+06 1.483777e+06 1.406437e+06 1.317455e+06 1.269751e+06 1.374450e+06 1.502787e+06 3.365033e+07 3.709441e+07 3.516092e+07 3.293637e+07 3.174377e+07
    23 NaN Total 2.270357e+07 2.205208e+07 2.237234e+07 2.179462e+07 2.340491e+07 2.485387e+07 2.218801e+07 5.675891e+08 5.513021e+08 5.593084e+08 5.448654e+08 5.851228e+08
    mean NaN mean 9.871116e+05 9.587863e+05 9.727103e+05 9.475920e+05 1.017605e+06 1.080603e+06 9.646959e+05 2.467779e+07 2.396966e+07 2.431776e+07 2.368980e+07 2.544012e+07

    Calculate Means and Totals Across Rows

    mean_series = malaysia_emissions_df[['tCH4_2015','tCH4_2016','tCH4_2017','tCH4_2018','tCH4_2019']].select_dtypes(np.number).mean(axis=1)
    total_series = malaysia_emissions_df[['tCH4_2015','tCH4_2016','tCH4_2017','tCH4_2018','tCH4_2019']].select_dtypes(np.number).sum(axis=1)
    malaysia_emissions_df["Mean_CH4"] = mean_series
    malaysia_emissions_df['Total_CH4'] = total_series 
    
    ## the select np.number is uncecessary, but i'm including anyways as it doesnt really hurt but for a small calculation penalty
    mean_series = malaysia_emissions_df[['tCO2_2015','tCO2_2016','tCO2_2017','tCO2_2018','tCO2_2019']].select_dtypes(np.number).mean(axis=1)
    total_series = malaysia_emissions_df[['tCO2_2015','tCO2_2016','tCO2_2017','tCO2_2018','tCO2_2019']].select_dtypes(np.number).sum(axis=1)
    malaysia_emissions_df["Mean_CO2"] = mean_series
    malaysia_emissions_df['Total_CO2'] = total_series 
    
    malaysia_emissions_df.reset_index(inplace=True, drop = True)
    
    malaysia_emissions_df
    

    iso3_country country_name tCH4_2015 tCH4_2016 tCH4_2017 tCH4_2018 tCH4_2019 tCH4_2020 tCH4_2021 tCO2_2015 tCO2_2016 tCO2_2017 tCO2_2018 tCO2_2019 Mean_CH4 Total_CH4 Mean_CO2 Total_CO2
    0 BGD Bangladesh 2.344420e+06 2.278158e+06 2.098958e+06 2.141231e+06 2.070985e+06 2.106781e+06 1.983974e+06 5.861049e+07 5.695395e+07 5.247394e+07 5.353076e+07 5.177463e+07 2.186750e+06 1.093375e+07 5.466875e+07 2.733438e+08
    1 BRA Brazil 3.410233e+05 3.104189e+05 3.725173e+05 3.717030e+05 3.294713e+05 4.902874e+05 4.544874e+05 8.525583e+06 7.760473e+06 9.312934e+06 9.292575e+06 8.236783e+06 3.450268e+05 1.725134e+06 8.625670e+06 4.312835e+07
    2 CHN China 6.133647e+06 5.859531e+06 6.355071e+06 5.413962e+06 5.603352e+06 6.402353e+06 6.068210e+06 1.533412e+08 1.464883e+08 1.588768e+08 1.353491e+08 1.400838e+08 5.873113e+06 2.936556e+07 1.468278e+08 7.341391e+08
    3 ESP Spain 1.141464e+04 1.334803e+04 1.217299e+04 1.405410e+04 1.148324e+04 1.305461e+04 8.531579e+03 2.853661e+05 3.337007e+05 3.043248e+05 3.513524e+05 2.870810e+05 1.249460e+04 6.247300e+04 3.123650e+05 1.561825e+06
    4 IDN Indonesia 1.283649e+06 1.023129e+06 9.615327e+05 1.176982e+06 1.266668e+06 1.188195e+06 1.009936e+06 3.209122e+07 2.557824e+07 2.403832e+07 2.942454e+07 3.166670e+07 1.142392e+06 5.711960e+06 2.855980e+07 1.427990e+08
    5 IND India 6.219887e+06 5.309413e+06 6.228451e+06 6.589798e+06 7.501556e+06 7.599764e+06 6.567960e+06 1.554972e+08 1.327353e+08 1.557113e+08 1.647450e+08 1.875389e+08 6.369821e+06 3.184910e+07 1.592455e+08 7.962276e+08
    6 IRN Iran (Islamic Republic of) 8.774407e+04 9.180121e+04 9.620217e+04 8.875744e+04 9.500199e+04 9.600254e+04 9.053525e+04 2.193602e+06 2.295030e+06 2.405054e+06 2.218936e+06 2.375050e+06 9.190138e+04 4.595069e+05 2.297534e+06 1.148767e+07
    7 ITA Italy 4.995968e+04 4.937785e+04 5.443679e+04 4.469902e+04 4.566914e+04 5.101547e+04 5.089759e+04 1.248992e+06 1.234446e+06 1.360920e+06 1.117475e+06 1.141729e+06 4.882850e+04 2.441425e+05 1.220712e+06 6.103562e+06
    8 JPN Japan 2.305465e+05 2.284133e+05 2.708935e+05 1.548252e+05 2.332056e+05 2.835167e+05 1.574007e+05 5.763662e+06 5.710333e+06 6.772337e+06 3.870631e+06 5.830141e+06 2.235768e+05 1.117884e+06 5.589421e+06 2.794710e+07
    9 KHM Cambodia 4.954698e+05 5.731698e+05 4.517045e+05 5.592610e+05 5.947277e+05 6.412802e+05 5.644891e+05 1.238675e+07 1.432925e+07 1.129261e+07 1.398153e+07 1.486819e+07 5.348666e+05 2.674333e+06 1.337166e+07 6.685832e+07
    10 KOR Korea (the Republic of) 1.451878e+05 1.274597e+05 1.463222e+05 1.293543e+05 1.327782e+05 1.165467e+05 1.013006e+05 3.629695e+06 3.186493e+06 3.658056e+06 3.233858e+06 3.319455e+06 1.362205e+05 6.811023e+05 3.405512e+06 1.702756e+07
    11 LAO Lao People's Democratic Republic (the) 1.661169e+04 1.696441e+04 1.168063e+04 1.009675e+04 1.461058e+04 2.136270e+04 1.475014e+04 4.152924e+05 4.241102e+05 2.920158e+05 2.524186e+05 3.652645e+05 1.399281e+04 6.996406e+04 3.498203e+05 1.749102e+06
    12 LKA Sri Lanka 8.305626e+04 1.011743e+05 5.911841e+04 9.018914e+04 8.476088e+04 9.248238e+04 8.466966e+04 2.076407e+06 2.529358e+06 1.477960e+06 2.254728e+06 2.119022e+06 8.365981e+04 4.182990e+05 2.091495e+06 1.045748e+07
    13 MMR Myanmar 1.132082e+06 1.290806e+06 1.205169e+06 1.372447e+06 1.256888e+06 1.221904e+06 1.289837e+06 2.830206e+07 3.227014e+07 3.012923e+07 3.431117e+07 3.142221e+07 1.251478e+06 6.257392e+06 3.128696e+07 1.564348e+08
    14 MYS Malaysia 1.057399e+05 1.110049e+05 1.111291e+05 1.066525e+05 1.056287e+05 1.127141e+05 1.069696e+05 2.643498e+06 2.775123e+06 2.778227e+06 2.666313e+06 2.640717e+06 1.080310e+05 5.401551e+05 2.700775e+06 1.350388e+07
    15 NPL Nepal 1.007479e+05 6.667161e+04 8.081300e+04 9.200752e+04 1.164235e+05 7.168401e+04 4.811408e+04 2.518697e+06 1.666790e+06 2.020325e+06 2.300188e+06 2.910588e+06 9.133271e+04 4.566635e+05 2.283318e+06 1.141659e+07
    16 PAK Pakistan 4.852431e+05 5.945922e+05 5.372641e+05 4.532297e+05 6.528548e+05 6.401201e+05 4.849205e+05 1.213108e+07 1.486480e+07 1.343160e+07 1.133074e+07 1.632137e+07 5.446368e+05 2.723184e+06 1.361592e+07 6.807960e+07
    17 PHL Philippines (the) 3.432021e+05 4.073554e+05 3.836830e+05 4.175210e+05 3.584550e+05 4.462836e+05 4.383270e+05 8.580052e+06 1.018389e+07 9.592074e+06 1.043803e+07 8.961374e+06 3.820433e+05 1.910216e+06 9.551082e+06 4.775541e+07
    18 PRK Korea (the Democratic People's Republic of) 1.143217e+05 9.177653e+04 1.085457e+05 8.662578e+04 9.655062e+04 8.581038e+04 7.735988e+04 2.858041e+06 2.294413e+06 2.713641e+06 2.165645e+06 2.413765e+06 9.956405e+04 4.978202e+05 2.489101e+06 1.244551e+07
    19 THA Thailand 1.393798e+06 1.780993e+06 1.164699e+06 9.166575e+05 1.305046e+06 1.520788e+06 8.528673e+05 3.484495e+07 4.452483e+07 2.911748e+07 2.291644e+07 3.262615e+07 1.312239e+06 6.561194e+06 3.280597e+07 1.640298e+08
    20 TWN Taiwan (Province of China) 7.866956e+04 8.089149e+04 8.705634e+04 8.138151e+04 8.990870e+04 8.333327e+04 6.619861e+04 1.966739e+06 2.022287e+06 2.176408e+06 2.034538e+06 2.247717e+06 8.358152e+04 4.179076e+05 2.089538e+06 1.044769e+07
    21 USA United States of America (the) 1.611324e+05 1.618576e+05 1.684799e+05 1.657254e+05 1.691351e+05 1.941455e+05 1.634842e+05 4.028310e+06 4.046440e+06 4.211999e+06 4.143136e+06 4.228377e+06 1.652661e+05 8.263305e+05 4.131652e+06 2.065826e+07
    22 VNM Viet Nam 1.346013e+06 1.483777e+06 1.406437e+06 1.317455e+06 1.269751e+06 1.374450e+06 1.502787e+06 3.365033e+07 3.709441e+07 3.516092e+07 3.293637e+07 3.174377e+07 1.364686e+06 6.823432e+06 3.411716e+07 1.705858e+08
    23 NaN Total 2.270357e+07 2.205208e+07 2.237234e+07 2.179462e+07 2.340491e+07 2.485387e+07 2.218801e+07 5.675891e+08 5.513021e+08 5.593084e+08 5.448654e+08 5.851228e+08 2.246550e+07 1.123275e+08 5.616376e+08 2.808188e+09
    24 NaN mean 9.871116e+05 9.587863e+05 9.727103e+05 9.475920e+05 1.017605e+06 1.080603e+06 9.646959e+05 2.467779e+07 2.396966e+07 2.431776e+07 2.368980e+07 2.544012e+07 9.767610e+05 4.883805e+06 2.441902e+07 1.220951e+08

    Write Data to File

    outfile = "/Users/jnapolitano/Projects/wattime-takehome/wattime-takehome/data/TRACE_DATA.csv"
    
    malaysia_emissions_df.to_csv(outfile)
    

    University of Malaysia Plots

    University of Malaysia Bar Plot

    malaysia_emissions_df.plot(kind = "barh", x = 'country_name', xlabel = "Country Name", ylabel = "CH4 Tonnes", figsize = (10,5))
    
    <AxesSubplot:ylabel='Country Name'>
    

    png

    University of Malaysia Density Plot

    malaysia_emissions_df.plot(rot = 0, kind = "density", figsize = (15,5)) 
    
    <AxesSubplot:ylabel='Density'>
    

    png

    I did not exclude totals or mean from the dataframe, but as we can see the second hump in the density graph shows the distribution of totals annualy. Interestingly the 2020 data is shifted further to the right than other years. This actually questions the validity of the study promoted by the University of malaysia

    FAOSTAT Data

    filepath = "/Users/jnapolitano/Projects/wattime-takehome/wattime-takehome/data/emissions_csv_fao_emiss_csv_ch4_fao_2015_2019_tonnes.xlsx"
    
    faostat_emissions_df = pd.read_excel(filepath)
    
    ## I didn't write the index to the csv file in the previous step.  IF time permits go back and fix this error
    faostat_emissions_df
    

    code country country_fao 2015 2016 2017 2018 2019 2020 2021
    0 BGD Bangladesh Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 NaN NaN
    1 BRA Brazil Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 NaN NaN
    2 CHN China China, mainland 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 NaN NaN
    3 ESP Spain Spain 55082.2 55073.1 54232.4 52925.0 52098.5 NaN NaN
    4 IDN Indonesia Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 NaN NaN
    5 IND India India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 NaN NaN
    6 IRN Iran (Islamic Republic of) Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 NaN NaN
    7 ITA Italy Italy 114574.8 118003.0 118003.0 109463.8 110895.1 NaN NaN
    8 JPN Japan Japan 330353.1 326403.0 323700.3 322245.0 320581.8 NaN NaN
    9 KHM Cambodia Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 NaN NaN
    10 KOR Korea (the Republic of) Republic of Korea 167862.2 163534.1 158489.7 154911.3 153260.9 NaN NaN
    11 LAO Lao People's Democratic Republic (the) Lao People's Democratic Republic 94826.8 95630.0 93940.7 83333.6 77005.5 NaN NaN
    12 LKA Sri Lanka Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 NaN NaN
    13 MMR Myanmar Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 NaN NaN
    14 MYS Malaysia Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 NaN NaN
    15 NPL Nepal Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 NaN NaN
    16 PAK Pakistan Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 NaN NaN
    17 PHL Philippines (the) Philippines 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 NaN NaN
    18 PRK Korea (the Democratic People's Republic of) Democratic People's Republic of Korea 82823.3 83442.3 84596.2 83943.3 82937.0 NaN NaN
    19 THA Thailand Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 NaN NaN
    20 TWN Taiwan (Province of China) China, Taiwan Province of 45838.7 49838.3 49991.2 49414.1 49152.0 NaN NaN
    21 USA United States of America (the) United States of America 364728.0 438662.0 336255.5 412177.5 350136.5 NaN NaN
    22 VNM Viet Nam Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 NaN NaN
    23 NaN NaN Total 23829994.0 23917225.1 24075931.9 24042561.5 23446028.9 NaN NaN

    Change code to iso3_country

    
    faostat_emissions_df.rename(columns={"code": "iso3_country"}, inplace =True)
    faostat_emissions_df.rename(columns={"country": "country_name"}, inplace =True)
    # The column title is not a string.  It is understood as an int or a datetime.  
    #faostat_emissions_df['2015']
    faostat_emissions_df
    

    iso3_country country_name country_fao 2015 2016 2017 2018 2019 2020 2021
    0 BGD Bangladesh Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 NaN NaN
    1 BRA Brazil Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 NaN NaN
    2 CHN China China, mainland 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 NaN NaN
    3 ESP Spain Spain 55082.2 55073.1 54232.4 52925.0 52098.5 NaN NaN
    4 IDN Indonesia Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 NaN NaN
    5 IND India India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 NaN NaN
    6 IRN Iran (Islamic Republic of) Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 NaN NaN
    7 ITA Italy Italy 114574.8 118003.0 118003.0 109463.8 110895.1 NaN NaN
    8 JPN Japan Japan 330353.1 326403.0 323700.3 322245.0 320581.8 NaN NaN
    9 KHM Cambodia Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 NaN NaN
    10 KOR Korea (the Republic of) Republic of Korea 167862.2 163534.1 158489.7 154911.3 153260.9 NaN NaN
    11 LAO Lao People's Democratic Republic (the) Lao People's Democratic Republic 94826.8 95630.0 93940.7 83333.6 77005.5 NaN NaN
    12 LKA Sri Lanka Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 NaN NaN
    13 MMR Myanmar Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 NaN NaN
    14 MYS Malaysia Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 NaN NaN
    15 NPL Nepal Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 NaN NaN
    16 PAK Pakistan Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 NaN NaN
    17 PHL Philippines (the) Philippines 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 NaN NaN
    18 PRK Korea (the Democratic People's Republic of) Democratic People's Republic of Korea 82823.3 83442.3 84596.2 83943.3 82937.0 NaN NaN
    19 THA Thailand Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 NaN NaN
    20 TWN Taiwan (Province of China) China, Taiwan Province of 45838.7 49838.3 49991.2 49414.1 49152.0 NaN NaN
    21 USA United States of America (the) United States of America 364728.0 438662.0 336255.5 412177.5 350136.5 NaN NaN
    22 VNM Viet Nam Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 NaN NaN
    23 NaN NaN Total 23829994.0 23917225.1 24075931.9 24042561.5 23446028.9 NaN NaN

    Set country_name total to total

    faostat_emissions_df.at[23,'country_name'] = 'Total'
    

    Drop Fao Country Code

    faostat_emissions_df.drop(labels = ['country_fao'], axis=1, inplace=True)
    faostat_emissions_df
    

    iso3_country country_name 2015 2016 2017 2018 2019 2020 2021
    0 BGD Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 NaN NaN
    1 BRA Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 NaN NaN
    2 CHN China 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 NaN NaN
    3 ESP Spain 55082.2 55073.1 54232.4 52925.0 52098.5 NaN NaN
    4 IDN Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 NaN NaN
    5 IND India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 NaN NaN
    6 IRN Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 NaN NaN
    7 ITA Italy 114574.8 118003.0 118003.0 109463.8 110895.1 NaN NaN
    8 JPN Japan 330353.1 326403.0 323700.3 322245.0 320581.8 NaN NaN
    9 KHM Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 NaN NaN
    10 KOR Korea (the Republic of) 167862.2 163534.1 158489.7 154911.3 153260.9 NaN NaN
    11 LAO Lao People's Democratic Republic (the) 94826.8 95630.0 93940.7 83333.6 77005.5 NaN NaN
    12 LKA Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 NaN NaN
    13 MMR Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 NaN NaN
    14 MYS Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 NaN NaN
    15 NPL Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 NaN NaN
    16 PAK Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 NaN NaN
    17 PHL Philippines (the) 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 NaN NaN
    18 PRK Korea (the Democratic People's Republic of) 82823.3 83442.3 84596.2 83943.3 82937.0 NaN NaN
    19 THA Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 NaN NaN
    20 TWN Taiwan (Province of China) 45838.7 49838.3 49991.2 49414.1 49152.0 NaN NaN
    21 USA United States of America (the) 364728.0 438662.0 336255.5 412177.5 350136.5 NaN NaN
    22 VNM Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 NaN NaN
    23 NaN Total 23829994.0 23917225.1 24075931.9 24042561.5 23446028.9 NaN NaN

    Calculate Co2 Equivalency

    faostat_emissions_df['tCO2_2015'] = faostat_emissions_df[2015] * 25
    faostat_emissions_df['tCO2_2016'] = faostat_emissions_df[2016] * 25
    faostat_emissions_df['tCO2_2017'] = faostat_emissions_df[2017] * 25
    faostat_emissions_df['tCO2_2018'] = faostat_emissions_df[2018] * 25
    faostat_emissions_df['tCO2_2019'] = faostat_emissions_df[2019] * 25
    

    Calculate Means

    faostat_emissions_df.loc['mean'] = faostat_emissions_df.loc[(faostat_emissions_df['country_name'] != "Total")].select_dtypes(np.number).mean()
    faostat_emissions_df.at['mean','country_name'] = 'mean'
    faostat_emissions_df.reset_index(inplace=True, drop=True)
    #faostat_emissions_df.at['mean','country_fao'] = 'mean'
    

    Calculate Means and Totals Across Rows

    mean_series = faostat_emissions_df[[2015,2016,2017,2018,2019]].select_dtypes(np.number).mean(axis=1)
    total_series = faostat_emissions_df[[2015,2016,2017,2018,2019]].select_dtypes(np.number).sum(axis=1)
    faostat_emissions_df["Mean_CH4"] = mean_series
    faostat_emissions_df['Total_CH4'] = total_series 
    
    ## the select np.number is uncecessary, but i'm including anyways as it doesnt really hurt but for a small calculation penalty
    mean_series = faostat_emissions_df[['tCO2_2015','tCO2_2016','tCO2_2017','tCO2_2018','tCO2_2019']].select_dtypes(np.number).mean(axis=1)
    total_series = faostat_emissions_df[['tCO2_2015','tCO2_2016','tCO2_2017','tCO2_2018','tCO2_2019']].select_dtypes(np.number).sum(axis=1)
    faostat_emissions_df["Mean_CO2"] = mean_series
    faostat_emissions_df['Total_CO2'] = total_series 
    
    faostat_emissions_df.reset_index(inplace=True, drop=True)
    
    faostat_emissions_df
    

    iso3_country country_name 2015 2016 2017 2018 2019 2020 2021 tCO2_2015 tCO2_2016 tCO2_2017 tCO2_2018 tCO2_2019 Mean_CH4 Total_CH4 Mean_CO2 Total_CO2
    0 BGD Bangladesh 1131293.4 1.093480e+06 1.154531e+06 1.144591e+06 1.144745e+06 NaN NaN 28282335.0 2.733701e+07 2.886328e+07 2.861478e+07 2.861863e+07 1.133728e+06 5.668641e+06 2.834321e+07 1.417160e+08
    1 BRA Brazil 138910.3 1.262782e+05 1.303229e+05 1.216152e+05 1.110848e+05 NaN NaN 3472757.5 3.156955e+06 3.258072e+06 3.040380e+06 2.777120e+06 1.256423e+05 6.282114e+05 3.141057e+06 1.570528e+07
    2 CHN China 5406593.9 5.399920e+06 5.400129e+06 5.302173e+06 5.214455e+06 NaN NaN 135164847.5 1.349980e+08 1.350032e+08 1.325543e+08 1.303614e+08 5.344654e+06 2.672327e+07 1.336164e+08 6.680818e+08
    3 ESP Spain 55082.2 5.507310e+04 5.423240e+04 5.292500e+04 5.209850e+04 NaN NaN 1377055.0 1.376828e+06 1.355810e+06 1.323125e+06 1.302462e+06 5.388224e+04 2.694112e+05 1.347056e+06 6.735280e+06
    4 IDN Indonesia 2407953.5 2.387656e+06 2.425291e+06 2.405614e+06 2.257604e+06 NaN NaN 60198837.5 5.969141e+07 6.063226e+07 6.014035e+07 5.644011e+07 2.376824e+06 1.188412e+07 5.942059e+07 2.971030e+08
    5 IND India 4580248.4 4.559136e+06 4.620791e+06 4.661155e+06 4.621417e+06 NaN NaN 114506210.0 1.139784e+08 1.155198e+08 1.165289e+08 1.155354e+08 4.608549e+06 2.304275e+07 1.152137e+08 5.760687e+08
    6 IRN Iran (Islamic Republic of) 116486.7 1.310085e+05 8.723360e+04 9.393660e+04 9.610340e+04 NaN NaN 2912167.5 3.275212e+06 2.180840e+06 2.348415e+06 2.402585e+06 1.049538e+05 5.247688e+05 2.623844e+06 1.311922e+07
    7 ITA Italy 114574.8 1.180030e+05 1.180030e+05 1.094638e+05 1.108951e+05 NaN NaN 2864370.0 2.950075e+06 2.950075e+06 2.736595e+06 2.772378e+06 1.141879e+05 5.709397e+05 2.854698e+06 1.427349e+07
    8 JPN Japan 330353.1 3.264030e+05 3.237003e+05 3.222450e+05 3.205818e+05 NaN NaN 8258827.5 8.160075e+06 8.092508e+06 8.056125e+06 8.014545e+06 3.246566e+05 1.623283e+06 8.116416e+06 4.058208e+07
    9 KHM Cambodia 436826.0 4.590031e+05 4.737453e+05 4.793627e+05 4.683789e+05 NaN NaN 10920650.0 1.147508e+07 1.184363e+07 1.198407e+07 1.170947e+07 4.634632e+05 2.317316e+06 1.158658e+07 5.793290e+07
    10 KOR Korea (the Republic of) 167862.2 1.635341e+05 1.584897e+05 1.549113e+05 1.532609e+05 NaN NaN 4196555.0 4.088352e+06 3.962243e+06 3.872783e+06 3.831522e+06 1.596116e+05 7.980582e+05 3.990291e+06 1.995146e+07
    11 LAO Lao People's Democratic Republic (the) 94826.8 9.563000e+04 9.394070e+04 8.333360e+04 7.700550e+04 NaN NaN 2370670.0 2.390750e+06 2.348518e+06 2.083340e+06 1.925138e+06 8.894732e+04 4.447366e+05 2.223683e+06 1.111842e+07
    12 LKA Sri Lanka 132640.0 1.217563e+05 8.445630e+04 1.110490e+05 1.021563e+05 NaN NaN 3316000.0 3.043908e+06 2.111408e+06 2.776225e+06 2.553908e+06 1.104116e+05 5.520579e+05 2.760290e+06 1.380145e+07
    13 MMR Myanmar 1059409.6 1.052288e+06 1.087030e+06 1.118850e+06 1.083100e+06 NaN NaN 26485240.0 2.630719e+07 2.717574e+07 2.797125e+07 2.707751e+07 1.080135e+06 5.400677e+06 2.700339e+07 1.350169e+08
    14 MYS Malaysia 121942.6 1.232328e+05 1.226563e+05 1.252385e+05 1.224538e+05 NaN NaN 3048565.0 3.080820e+06 3.066408e+06 3.130962e+06 3.061345e+06 1.231048e+05 6.155240e+05 3.077620e+06 1.538810e+07
    15 NPL Nepal 149262.2 1.427237e+05 1.625746e+05 1.538908e+05 1.562154e+05 NaN NaN 3731555.0 3.568093e+06 4.064365e+06 3.847270e+06 3.905385e+06 1.529333e+05 7.646667e+05 3.823334e+06 1.911667e+07
    16 PAK Pakistan 383529.3 3.813618e+05 4.060833e+05 3.934042e+05 4.247551e+05 NaN NaN 9588232.5 9.534045e+06 1.015208e+07 9.835105e+06 1.061888e+07 3.978267e+05 1.989134e+06 9.945668e+06 4.972834e+07
    17 PHL Philippines (the) 1557810.6 1.524292e+06 1.609862e+06 1.606048e+06 1.556226e+06 NaN NaN 38945265.0 3.810731e+07 4.024656e+07 4.015120e+07 3.890564e+07 1.570848e+06 7.854239e+06 3.927120e+07 1.963560e+08
    18 PRK Korea (the Democratic People's Republic of) 82823.3 8.344230e+04 8.459620e+04 8.394330e+04 8.293700e+04 NaN NaN 2070582.5 2.086058e+06 2.114905e+06 2.098582e+06 2.073425e+06 8.354842e+04 4.177421e+05 2.088710e+06 1.044355e+07
    19 THA Thailand 1554254.0 1.703328e+06 1.714466e+06 1.702989e+06 1.553836e+06 NaN NaN 38856350.0 4.258319e+07 4.286164e+07 4.257473e+07 3.884589e+07 1.645774e+06 8.228872e+06 4.114436e+07 2.057218e+08
    20 TWN Taiwan (Province of China) 45838.7 4.983830e+04 4.999120e+04 4.941410e+04 4.915200e+04 NaN NaN 1145967.5 1.245958e+06 1.249780e+06 1.235352e+06 1.228800e+06 4.884686e+04 2.442343e+05 1.221172e+06 6.105858e+06
    21 USA United States of America (the) 364728.0 4.386620e+05 3.362555e+05 4.121775e+05 3.501365e+05 NaN NaN 9118200.0 1.096655e+07 8.406388e+06 1.030444e+07 8.753412e+06 3.803919e+05 1.901960e+06 9.509798e+06 4.754899e+07
    22 VNM Viet Nam 1381744.4 1.365174e+06 1.360552e+06 1.336231e+06 1.318431e+06 NaN NaN 34543610.0 3.412934e+07 3.401379e+07 3.340578e+07 3.296078e+07 1.352426e+06 6.762132e+06 3.381066e+07 1.690533e+08
    23 NaN Total 23829994.0 2.391723e+07 2.407593e+07 2.404256e+07 2.344603e+07 NaN NaN 595749850.0 5.979306e+08 6.018983e+08 6.010640e+08 5.861507e+08 2.386235e+07 1.193117e+08 5.965587e+08 2.982794e+09
    24 NaN mean 948478.0 9.522272e+05 9.590840e+05 9.575896e+05 9.316100e+05 NaN NaN 23711950.0 2.380568e+07 2.397710e+07 2.393974e+07 2.329025e+07 9.497978e+05 4.748989e+06 2.374494e+07 1.187247e+08

    FAOSTAT Data to File

    outfile = "/Users/jnapolitano/Projects/wattime-takehome/wattime-takehome/data/FAOSTAT_DATA.csv"
    
    faostat_emissions_df.to_csv(outfile)
    

    FaoSTAT PLOTS

    FAOSTAT Hectare Estimates Bar Plot

    faostat_emissions_df.plot(kind = "barh", x = 'country_name', y = [2015, 2016, 2017, 2018, 2019], xlabel = "Country Name", ylabel = "Tonnes CH4", figsize = (10,5))
    
    <AxesSubplot:ylabel='Country Name'>
    

    png

    FAOSTAT Density Plot

    faostat_emissions_df.plot(rot = 90, kind = "density",y = [2015, 2016, 2017, 2018, 2019], figsize = (15,5)) 
    
    <AxesSubplot:ylabel='Density'>
    

    png

    The density plot is fairly consistent. There is nearly no variation between nations and in total. The 2020 data may show otherwise as the Malaysian data shows.

    Join Df’s by ISO3 Country

    Drop totals and means from the original df.

    Because I am joining on iso3 country country code if the totals and means are located at different indexes we may experience merge and calculation errors

    faostat_emissions_df = faostat_emissions_df[(faostat_emissions_df["country_name"] != "Total") & (faostat_emissions_df['country_name'] != 'mean')].copy()
    
    malaysia_emissions_df = malaysia_emissions_df[(malaysia_emissions_df["country_name"] != "Total") & (malaysia_emissions_df['country_name'] != 'mean')].copy()
    
    faostat_emissions_df
    

    iso3_country country_name 2015 2016 2017 2018 2019 2020 2021 tCO2_2015 tCO2_2016 tCO2_2017 tCO2_2018 tCO2_2019 Mean_CH4 Total_CH4 Mean_CO2 Total_CO2
    0 BGD Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 NaN NaN 28282335.0 27337010.0 28863275.0 28614775.0 28618635.0 1133728.24 5668641.2 28343206.0 141716030.0
    1 BRA Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 NaN NaN 3472757.5 3156955.0 3258072.5 3040380.0 2777120.0 125642.28 628211.4 3141057.0 15705285.0
    2 CHN China 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 NaN NaN 135164847.5 134998000.0 135003225.0 132554327.5 130361367.5 5344654.14 26723270.7 133616353.5 668081767.5
    3 ESP Spain 55082.2 55073.1 54232.4 52925.0 52098.5 NaN NaN 1377055.0 1376827.5 1355810.0 1323125.0 1302462.5 53882.24 269411.2 1347056.0 6735280.0
    4 IDN Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 NaN NaN 60198837.5 59691410.0 60632265.0 60140345.0 56440107.5 2376823.72 11884118.6 59420593.0 297102965.0
    5 IND India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 NaN NaN 114506210.0 113978410.0 115519770.0 116528872.5 115535420.0 4608549.46 23042747.3 115213736.5 576068682.5
    6 IRN Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 NaN NaN 2912167.5 3275212.5 2180840.0 2348415.0 2402585.0 104953.76 524768.8 2623844.0 13119220.0
    7 ITA Italy 114574.8 118003.0 118003.0 109463.8 110895.1 NaN NaN 2864370.0 2950075.0 2950075.0 2736595.0 2772377.5 114187.94 570939.7 2854698.5 14273492.5
    8 JPN Japan 330353.1 326403.0 323700.3 322245.0 320581.8 NaN NaN 8258827.5 8160075.0 8092507.5 8056125.0 8014545.0 324656.64 1623283.2 8116416.0 40582080.0
    9 KHM Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 NaN NaN 10920650.0 11475077.5 11843632.5 11984067.5 11709472.5 463463.20 2317316.0 11586580.0 57932900.0
    10 KOR Korea (the Republic of) 167862.2 163534.1 158489.7 154911.3 153260.9 NaN NaN 4196555.0 4088352.5 3962242.5 3872782.5 3831522.5 159611.64 798058.2 3990291.0 19951455.0
    11 LAO Lao People's Democratic Republic (the) 94826.8 95630.0 93940.7 83333.6 77005.5 NaN NaN 2370670.0 2390750.0 2348517.5 2083340.0 1925137.5 88947.32 444736.6 2223683.0 11118415.0
    12 LKA Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 NaN NaN 3316000.0 3043907.5 2111407.5 2776225.0 2553907.5 110411.58 552057.9 2760289.5 13801447.5
    13 MMR Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 NaN NaN 26485240.0 26307192.5 27175737.5 27971250.0 27077507.5 1080135.42 5400677.1 27003385.5 135016927.5
    14 MYS Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 NaN NaN 3048565.0 3080820.0 3066407.5 3130962.5 3061345.0 123104.80 615524.0 3077620.0 15388100.0
    15 NPL Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 NaN NaN 3731555.0 3568092.5 4064365.0 3847270.0 3905385.0 152933.34 764666.7 3823333.5 19116667.5
    16 PAK Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 NaN NaN 9588232.5 9534045.0 10152082.5 9835105.0 10618877.5 397826.74 1989133.7 9945668.5 49728342.5
    17 PHL Philippines (the) 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 NaN NaN 38945265.0 38107312.5 40246562.5 40151195.0 38905645.0 1570847.84 7854239.2 39271196.0 196355980.0
    18 PRK Korea (the Democratic People's Republic of) 82823.3 83442.3 84596.2 83943.3 82937.0 NaN NaN 2070582.5 2086057.5 2114905.0 2098582.5 2073425.0 83548.42 417742.1 2088710.5 10443552.5
    19 THA Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 NaN NaN 38856350.0 42583192.5 42861640.0 42574727.5 38845887.5 1645774.38 8228871.9 41144359.5 205721797.5
    20 TWN Taiwan (Province of China) 45838.7 49838.3 49991.2 49414.1 49152.0 NaN NaN 1145967.5 1245957.5 1249780.0 1235352.5 1228800.0 48846.86 244234.3 1221171.5 6105857.5
    21 USA United States of America (the) 364728.0 438662.0 336255.5 412177.5 350136.5 NaN NaN 9118200.0 10966550.0 8406387.5 10304437.5 8753412.5 380391.90 1901959.5 9509797.5 47548987.5
    22 VNM Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 NaN NaN 34543610.0 34129345.0 34013790.0 33405780.0 32960777.5 1352426.42 6762132.1 33810660.5 169053302.5
    malaysia_emissions_df
    

    iso3_country country_name tCH4_2015 tCH4_2016 tCH4_2017 tCH4_2018 tCH4_2019 tCH4_2020 tCH4_2021 tCO2_2015 tCO2_2016 tCO2_2017 tCO2_2018 tCO2_2019 Mean_CH4 Total_CH4 Mean_CO2 Total_CO2
    0 BGD Bangladesh 2.344420e+06 2.278158e+06 2.098958e+06 2.141231e+06 2.070985e+06 2.106781e+06 1.983974e+06 5.861049e+07 5.695395e+07 5.247394e+07 5.353076e+07 5.177463e+07 2.186750e+06 1.093375e+07 5.466875e+07 2.733438e+08
    1 BRA Brazil 3.410233e+05 3.104189e+05 3.725173e+05 3.717030e+05 3.294713e+05 4.902874e+05 4.544874e+05 8.525583e+06 7.760473e+06 9.312934e+06 9.292575e+06 8.236783e+06 3.450268e+05 1.725134e+06 8.625670e+06 4.312835e+07
    2 CHN China 6.133647e+06 5.859531e+06 6.355071e+06 5.413962e+06 5.603352e+06 6.402353e+06 6.068210e+06 1.533412e+08 1.464883e+08 1.588768e+08 1.353491e+08 1.400838e+08 5.873113e+06 2.936556e+07 1.468278e+08 7.341391e+08
    3 ESP Spain 1.141464e+04 1.334803e+04 1.217299e+04 1.405410e+04 1.148324e+04 1.305461e+04 8.531579e+03 2.853661e+05 3.337007e+05 3.043248e+05 3.513524e+05 2.870810e+05 1.249460e+04 6.247300e+04 3.123650e+05 1.561825e+06
    4 IDN Indonesia 1.283649e+06 1.023129e+06 9.615327e+05 1.176982e+06 1.266668e+06 1.188195e+06 1.009936e+06 3.209122e+07 2.557824e+07 2.403832e+07 2.942454e+07 3.166670e+07 1.142392e+06 5.711960e+06 2.855980e+07 1.427990e+08
    5 IND India 6.219887e+06 5.309413e+06 6.228451e+06 6.589798e+06 7.501556e+06 7.599764e+06 6.567960e+06 1.554972e+08 1.327353e+08 1.557113e+08 1.647450e+08 1.875389e+08 6.369821e+06 3.184910e+07 1.592455e+08 7.962276e+08
    6 IRN Iran (Islamic Republic of) 8.774407e+04 9.180121e+04 9.620217e+04 8.875744e+04 9.500199e+04 9.600254e+04 9.053525e+04 2.193602e+06 2.295030e+06 2.405054e+06 2.218936e+06 2.375050e+06 9.190138e+04 4.595069e+05 2.297534e+06 1.148767e+07
    7 ITA Italy 4.995968e+04 4.937785e+04 5.443679e+04 4.469902e+04 4.566914e+04 5.101547e+04 5.089759e+04 1.248992e+06 1.234446e+06 1.360920e+06 1.117475e+06 1.141729e+06 4.882850e+04 2.441425e+05 1.220712e+06 6.103562e+06
    8 JPN Japan 2.305465e+05 2.284133e+05 2.708935e+05 1.548252e+05 2.332056e+05 2.835167e+05 1.574007e+05 5.763662e+06 5.710333e+06 6.772337e+06 3.870631e+06 5.830141e+06 2.235768e+05 1.117884e+06 5.589421e+06 2.794710e+07
    9 KHM Cambodia 4.954698e+05 5.731698e+05 4.517045e+05 5.592610e+05 5.947277e+05 6.412802e+05 5.644891e+05 1.238675e+07 1.432925e+07 1.129261e+07 1.398153e+07 1.486819e+07 5.348666e+05 2.674333e+06 1.337166e+07 6.685832e+07
    10 KOR Korea (the Republic of) 1.451878e+05 1.274597e+05 1.463222e+05 1.293543e+05 1.327782e+05 1.165467e+05 1.013006e+05 3.629695e+06 3.186493e+06 3.658056e+06 3.233858e+06 3.319455e+06 1.362205e+05 6.811023e+05 3.405512e+06 1.702756e+07
    11 LAO Lao People's Democratic Republic (the) 1.661169e+04 1.696441e+04 1.168063e+04 1.009675e+04 1.461058e+04 2.136270e+04 1.475014e+04 4.152924e+05 4.241102e+05 2.920158e+05 2.524186e+05 3.652645e+05 1.399281e+04 6.996406e+04 3.498203e+05 1.749102e+06
    12 LKA Sri Lanka 8.305626e+04 1.011743e+05 5.911841e+04 9.018914e+04 8.476088e+04 9.248238e+04 8.466966e+04 2.076407e+06 2.529358e+06 1.477960e+06 2.254728e+06 2.119022e+06 8.365981e+04 4.182990e+05 2.091495e+06 1.045748e+07
    13 MMR Myanmar 1.132082e+06 1.290806e+06 1.205169e+06 1.372447e+06 1.256888e+06 1.221904e+06 1.289837e+06 2.830206e+07 3.227014e+07 3.012923e+07 3.431117e+07 3.142221e+07 1.251478e+06 6.257392e+06 3.128696e+07 1.564348e+08
    14 MYS Malaysia 1.057399e+05 1.110049e+05 1.111291e+05 1.066525e+05 1.056287e+05 1.127141e+05 1.069696e+05 2.643498e+06 2.775123e+06 2.778227e+06 2.666313e+06 2.640717e+06 1.080310e+05 5.401551e+05 2.700775e+06 1.350388e+07
    15 NPL Nepal 1.007479e+05 6.667161e+04 8.081300e+04 9.200752e+04 1.164235e+05 7.168401e+04 4.811408e+04 2.518697e+06 1.666790e+06 2.020325e+06 2.300188e+06 2.910588e+06 9.133271e+04 4.566635e+05 2.283318e+06 1.141659e+07
    16 PAK Pakistan 4.852431e+05 5.945922e+05 5.372641e+05 4.532297e+05 6.528548e+05 6.401201e+05 4.849205e+05 1.213108e+07 1.486480e+07 1.343160e+07 1.133074e+07 1.632137e+07 5.446368e+05 2.723184e+06 1.361592e+07 6.807960e+07
    17 PHL Philippines (the) 3.432021e+05 4.073554e+05 3.836830e+05 4.175210e+05 3.584550e+05 4.462836e+05 4.383270e+05 8.580052e+06 1.018389e+07 9.592074e+06 1.043803e+07 8.961374e+06 3.820433e+05 1.910216e+06 9.551082e+06 4.775541e+07
    18 PRK Korea (the Democratic People's Republic of) 1.143217e+05 9.177653e+04 1.085457e+05 8.662578e+04 9.655062e+04 8.581038e+04 7.735988e+04 2.858041e+06 2.294413e+06 2.713641e+06 2.165645e+06 2.413765e+06 9.956405e+04 4.978202e+05 2.489101e+06 1.244551e+07
    19 THA Thailand 1.393798e+06 1.780993e+06 1.164699e+06 9.166575e+05 1.305046e+06 1.520788e+06 8.528673e+05 3.484495e+07 4.452483e+07 2.911748e+07 2.291644e+07 3.262615e+07 1.312239e+06 6.561194e+06 3.280597e+07 1.640298e+08
    20 TWN Taiwan (Province of China) 7.866956e+04 8.089149e+04 8.705634e+04 8.138151e+04 8.990870e+04 8.333327e+04 6.619861e+04 1.966739e+06 2.022287e+06 2.176408e+06 2.034538e+06 2.247717e+06 8.358152e+04 4.179076e+05 2.089538e+06 1.044769e+07
    21 USA United States of America (the) 1.611324e+05 1.618576e+05 1.684799e+05 1.657254e+05 1.691351e+05 1.941455e+05 1.634842e+05 4.028310e+06 4.046440e+06 4.211999e+06 4.143136e+06 4.228377e+06 1.652661e+05 8.263305e+05 4.131652e+06 2.065826e+07
    22 VNM Viet Nam 1.346013e+06 1.483777e+06 1.406437e+06 1.317455e+06 1.269751e+06 1.374450e+06 1.502787e+06 3.365033e+07 3.709441e+07 3.516092e+07 3.293637e+07 3.174377e+07 1.364686e+06 6.823432e+06 3.411716e+07 1.705858e+08
    merged_df = faostat_emissions_df.merge(malaysia_emissions_df,suffixes=('_FAOSTAT', '_TRACE'), on='iso3_country', how='left', sort=False)
    

    Dropping 2020 and 2021 from the data sets

    I will only compare data compiled from the same year.

    merged_df.drop([2020, 2021, "tCH4_2020","tCH4_2021"], axis = 1, inplace = True)
    
    merged_df
    

    iso3_country country_name_FAOSTAT 2015 2016 2017 2018 2019 tCO2_2015_FAOSTAT tCO2_2016_FAOSTAT tCO2_2017_FAOSTAT ... tCH4_2019 tCO2_2015_TRACE tCO2_2016_TRACE tCO2_2017_TRACE tCO2_2018_TRACE tCO2_2019_TRACE Mean_CH4_TRACE Total_CH4_TRACE Mean_CO2_TRACE Total_CO2_TRACE
    0 BGD Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 28282335.0 27337010.0 28863275.0 ... 2.070985e+06 5.861049e+07 5.695395e+07 5.247394e+07 5.353076e+07 5.177463e+07 2.186750e+06 1.093375e+07 5.466875e+07 2.733438e+08
    1 BRA Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 3472757.5 3156955.0 3258072.5 ... 3.294713e+05 8.525583e+06 7.760473e+06 9.312934e+06 9.292575e+06 8.236783e+06 3.450268e+05 1.725134e+06 8.625670e+06 4.312835e+07
    2 CHN China 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 135164847.5 134998000.0 135003225.0 ... 5.603352e+06 1.533412e+08 1.464883e+08 1.588768e+08 1.353491e+08 1.400838e+08 5.873113e+06 2.936556e+07 1.468278e+08 7.341391e+08
    3 ESP Spain 55082.2 55073.1 54232.4 52925.0 52098.5 1377055.0 1376827.5 1355810.0 ... 1.148324e+04 2.853661e+05 3.337007e+05 3.043248e+05 3.513524e+05 2.870810e+05 1.249460e+04 6.247300e+04 3.123650e+05 1.561825e+06
    4 IDN Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 60198837.5 59691410.0 60632265.0 ... 1.266668e+06 3.209122e+07 2.557824e+07 2.403832e+07 2.942454e+07 3.166670e+07 1.142392e+06 5.711960e+06 2.855980e+07 1.427990e+08
    5 IND India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 114506210.0 113978410.0 115519770.0 ... 7.501556e+06 1.554972e+08 1.327353e+08 1.557113e+08 1.647450e+08 1.875389e+08 6.369821e+06 3.184910e+07 1.592455e+08 7.962276e+08
    6 IRN Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 2912167.5 3275212.5 2180840.0 ... 9.500199e+04 2.193602e+06 2.295030e+06 2.405054e+06 2.218936e+06 2.375050e+06 9.190138e+04 4.595069e+05 2.297534e+06 1.148767e+07
    7 ITA Italy 114574.8 118003.0 118003.0 109463.8 110895.1 2864370.0 2950075.0 2950075.0 ... 4.566914e+04 1.248992e+06 1.234446e+06 1.360920e+06 1.117475e+06 1.141729e+06 4.882850e+04 2.441425e+05 1.220712e+06 6.103562e+06
    8 JPN Japan 330353.1 326403.0 323700.3 322245.0 320581.8 8258827.5 8160075.0 8092507.5 ... 2.332056e+05 5.763662e+06 5.710333e+06 6.772337e+06 3.870631e+06 5.830141e+06 2.235768e+05 1.117884e+06 5.589421e+06 2.794710e+07
    9 KHM Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 10920650.0 11475077.5 11843632.5 ... 5.947277e+05 1.238675e+07 1.432925e+07 1.129261e+07 1.398153e+07 1.486819e+07 5.348666e+05 2.674333e+06 1.337166e+07 6.685832e+07
    10 KOR Korea (the Republic of) 167862.2 163534.1 158489.7 154911.3 153260.9 4196555.0 4088352.5 3962242.5 ... 1.327782e+05 3.629695e+06 3.186493e+06 3.658056e+06 3.233858e+06 3.319455e+06 1.362205e+05 6.811023e+05 3.405512e+06 1.702756e+07
    11 LAO Lao People's Democratic Republic (the) 94826.8 95630.0 93940.7 83333.6 77005.5 2370670.0 2390750.0 2348517.5 ... 1.461058e+04 4.152924e+05 4.241102e+05 2.920158e+05 2.524186e+05 3.652645e+05 1.399281e+04 6.996406e+04 3.498203e+05 1.749102e+06
    12 LKA Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 3316000.0 3043907.5 2111407.5 ... 8.476088e+04 2.076407e+06 2.529358e+06 1.477960e+06 2.254728e+06 2.119022e+06 8.365981e+04 4.182990e+05 2.091495e+06 1.045748e+07
    13 MMR Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 26485240.0 26307192.5 27175737.5 ... 1.256888e+06 2.830206e+07 3.227014e+07 3.012923e+07 3.431117e+07 3.142221e+07 1.251478e+06 6.257392e+06 3.128696e+07 1.564348e+08
    14 MYS Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 3048565.0 3080820.0 3066407.5 ... 1.056287e+05 2.643498e+06 2.775123e+06 2.778227e+06 2.666313e+06 2.640717e+06 1.080310e+05 5.401551e+05 2.700775e+06 1.350388e+07
    15 NPL Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 3731555.0 3568092.5 4064365.0 ... 1.164235e+05 2.518697e+06 1.666790e+06 2.020325e+06 2.300188e+06 2.910588e+06 9.133271e+04 4.566635e+05 2.283318e+06 1.141659e+07
    16 PAK Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 9588232.5 9534045.0 10152082.5 ... 6.528548e+05 1.213108e+07 1.486480e+07 1.343160e+07 1.133074e+07 1.632137e+07 5.446368e+05 2.723184e+06 1.361592e+07 6.807960e+07
    17 PHL Philippines (the) 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 38945265.0 38107312.5 40246562.5 ... 3.584550e+05 8.580052e+06 1.018389e+07 9.592074e+06 1.043803e+07 8.961374e+06 3.820433e+05 1.910216e+06 9.551082e+06 4.775541e+07
    18 PRK Korea (the Democratic People's Republic of) 82823.3 83442.3 84596.2 83943.3 82937.0 2070582.5 2086057.5 2114905.0 ... 9.655062e+04 2.858041e+06 2.294413e+06 2.713641e+06 2.165645e+06 2.413765e+06 9.956405e+04 4.978202e+05 2.489101e+06 1.244551e+07
    19 THA Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 38856350.0 42583192.5 42861640.0 ... 1.305046e+06 3.484495e+07 4.452483e+07 2.911748e+07 2.291644e+07 3.262615e+07 1.312239e+06 6.561194e+06 3.280597e+07 1.640298e+08
    20 TWN Taiwan (Province of China) 45838.7 49838.3 49991.2 49414.1 49152.0 1145967.5 1245957.5 1249780.0 ... 8.990870e+04 1.966739e+06 2.022287e+06 2.176408e+06 2.034538e+06 2.247717e+06 8.358152e+04 4.179076e+05 2.089538e+06 1.044769e+07
    21 USA United States of America (the) 364728.0 438662.0 336255.5 412177.5 350136.5 9118200.0 10966550.0 8406387.5 ... 1.691351e+05 4.028310e+06 4.046440e+06 4.211999e+06 4.143136e+06 4.228377e+06 1.652661e+05 8.263305e+05 4.131652e+06 2.065826e+07
    22 VNM Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 34543610.0 34129345.0 34013790.0 ... 1.269751e+06 3.365033e+07 3.709441e+07 3.516092e+07 3.293637e+07 3.174377e+07 1.364686e+06 6.823432e+06 3.411716e+07 1.705858e+08

    23 rows × 31 columns

    Calculate difference in Ch4 Tonnes Between the Estimates

    # Calculate Difference in tons
    merged_df['CH4_diff_2015'] = merged_df[2015] - merged_df['tCH4_2015']
    merged_df['CH4_diff_2016'] = merged_df[2016] - merged_df['tCH4_2016']
    merged_df['CH4_diff_2017'] = merged_df[2017] - merged_df['tCH4_2017']
    merged_df['CH4_diff_2018'] = merged_df[2018] - merged_df['tCH4_2018']
    merged_df['CH4_diff_2019'] = merged_df[2019] - merged_df['tCH4_2019']
    merged_df['CH4_diff_means'] = merged_df['Mean_CH4_FAOSTAT'] - merged_df['Mean_CH4_TRACE']
    merged_df['CH4_diff_totals'] = merged_df['Total_CH4_FAOSTAT'] - merged_df['Total_CH4_TRACE']
    
    merged_df
    

    iso3_country country_name_FAOSTAT 2015 2016 2017 2018 2019 tCO2_2015_FAOSTAT tCO2_2016_FAOSTAT tCO2_2017_FAOSTAT ... Total_CH4_TRACE Mean_CO2_TRACE Total_CO2_TRACE CH4_diff_2015 CH4_diff_2016 CH4_diff_2017 CH4_diff_2018 CH4_diff_2019 CH4_diff_means CH4_diff_totals
    0 BGD Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 28282335.0 27337010.0 28863275.0 ... 1.093375e+07 5.466875e+07 2.733438e+08 -1.213126e+06 -1.184678e+06 -9.444266e+05 -9.966395e+05 -9.262397e+05 -1.053022e+06 -5.265110e+06
    1 BRA Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 3472757.5 3156955.0 3258072.5 ... 1.725134e+06 8.625670e+06 4.312835e+07 -2.021130e+05 -1.841407e+05 -2.421944e+05 -2.500878e+05 -2.183865e+05 -2.193845e+05 -1.096923e+06
    2 CHN China 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 135164847.5 134998000.0 135003225.0 ... 2.936556e+07 1.468278e+08 7.341391e+08 -7.270532e+05 -4.596111e+05 -9.549420e+05 -1.117892e+05 -3.888972e+05 -5.284585e+05 -2.642293e+06
    3 ESP Spain 55082.2 55073.1 54232.4 52925.0 52098.5 1377055.0 1376827.5 1355810.0 ... 6.247300e+04 3.123650e+05 1.561825e+06 4.366756e+04 4.172507e+04 4.205941e+04 3.887090e+04 4.061526e+04 4.138764e+04 2.069382e+05
    4 IDN Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 60198837.5 59691410.0 60632265.0 ... 5.711960e+06 2.855980e+07 1.427990e+08 1.124305e+06 1.364527e+06 1.463758e+06 1.228632e+06 9.909364e+05 1.234432e+06 6.172158e+06
    5 IND India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 114506210.0 113978410.0 115519770.0 ... 3.184910e+07 1.592455e+08 7.962276e+08 -1.639638e+06 -7.502765e+05 -1.607660e+06 -1.928643e+06 -2.880139e+06 -1.761271e+06 -8.806357e+06
    6 IRN Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 2912167.5 3275212.5 2180840.0 ... 4.595069e+05 2.297534e+06 1.148767e+07 2.874263e+04 3.920729e+04 -8.968572e+03 5.179159e+03 1.101412e+03 1.305238e+04 6.526191e+04
    7 ITA Italy 114574.8 118003.0 118003.0 109463.8 110895.1 2864370.0 2950075.0 2950075.0 ... 2.441425e+05 1.220712e+06 6.103562e+06 6.461512e+04 6.862515e+04 6.356621e+04 6.476478e+04 6.522596e+04 6.535944e+04 3.267972e+05
    8 JPN Japan 330353.1 326403.0 323700.3 322245.0 320581.8 8258827.5 8160075.0 8092507.5 ... 1.117884e+06 5.589421e+06 2.794710e+07 9.980661e+04 9.798967e+04 5.280682e+04 1.674198e+05 8.737617e+04 1.010798e+05 5.053990e+05
    9 KHM Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 10920650.0 11475077.5 11843632.5 ... 2.674333e+06 1.337166e+07 6.685832e+07 -5.864384e+04 -1.141667e+05 2.204082e+04 -7.989830e+04 -1.263488e+05 -7.140337e+04 -3.570168e+05
    10 KOR Korea (the Republic of) 167862.2 163534.1 158489.7 154911.3 153260.9 4196555.0 4088352.5 3962242.5 ... 6.811023e+05 3.405512e+06 1.702756e+07 2.267438e+04 3.607436e+04 1.216747e+04 2.555698e+04 2.048269e+04 2.339118e+04 1.169559e+05
    11 LAO Lao People's Democratic Republic (the) 94826.8 95630.0 93940.7 83333.6 77005.5 2370670.0 2390750.0 2348517.5 ... 6.996406e+04 3.498203e+05 1.749102e+06 7.821511e+04 7.866559e+04 8.226007e+04 7.323685e+04 6.239492e+04 7.495451e+04 3.747725e+05
    12 LKA Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 3316000.0 3043907.5 2111407.5 ... 4.182990e+05 2.091495e+06 1.045748e+07 4.958374e+04 2.058196e+04 2.533789e+04 2.085986e+04 1.739542e+04 2.675177e+04 1.337589e+05
    13 MMR Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 26485240.0 26307192.5 27175737.5 ... 6.257392e+06 3.128696e+07 1.564348e+08 -7.267265e+04 -2.385180e+05 -1.181398e+05 -2.535969e+05 -1.737880e+05 -1.713431e+05 -8.567154e+05
    14 MYS Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 3048565.0 3080820.0 3066407.5 ... 5.401551e+05 2.700775e+06 1.350388e+07 1.620268e+04 1.222789e+04 1.152724e+04 1.858597e+04 1.682512e+04 1.507378e+04 7.536891e+04
    15 NPL Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 3731555.0 3568092.5 4064365.0 ... 4.566635e+05 2.283318e+06 1.141659e+07 4.851434e+04 7.605209e+04 8.176160e+04 6.188328e+04 3.979186e+04 6.160063e+04 3.080032e+05
    16 PAK Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 9588232.5 9534045.0 10152082.5 ... 2.723184e+06 1.361592e+07 6.807960e+07 -1.017138e+05 -2.132304e+05 -1.311808e+05 -5.982552e+04 -2.280997e+05 -1.468100e+05 -7.340502e+05
    17 PHL Philippines (the) 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 38945265.0 38107312.5 40246562.5 ... 1.910216e+06 9.551082e+06 4.775541e+07 1.214609e+06 1.116937e+06 1.226180e+06 1.188527e+06 1.197771e+06 1.188805e+06 5.944023e+06
    18 PRK Korea (the Democratic People's Republic of) 82823.3 83442.3 84596.2 83943.3 82937.0 2070582.5 2086057.5 2114905.0 ... 4.978202e+05 2.489101e+06 1.244551e+07 -3.149836e+04 -8.334231e+03 -2.394946e+04 -2.682481e+03 -1.361362e+04 -1.601563e+04 -8.007815e+04
    19 THA Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 38856350.0 42583192.5 42861640.0 ... 6.561194e+06 3.280597e+07 1.640298e+08 1.604559e+05 -7.766539e+04 5.497665e+05 7.863316e+05 2.487897e+05 3.335357e+05 1.667678e+06
    20 TWN Taiwan (Province of China) 45838.7 49838.3 49991.2 49414.1 49152.0 1145967.5 1245957.5 1249780.0 ... 4.179076e+05 2.089538e+06 1.044769e+07 -3.283086e+04 -3.105319e+04 -3.706514e+04 -3.196741e+04 -4.075670e+04 -3.473466e+04 -1.736733e+05
    21 USA United States of America (the) 364728.0 438662.0 336255.5 412177.5 350136.5 9118200.0 10966550.0 8406387.5 ... 8.263305e+05 4.131652e+06 2.065826e+07 2.035956e+05 2.768044e+05 1.677756e+05 2.464521e+05 1.810014e+05 2.151258e+05 1.075629e+06
    22 VNM Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 34543610.0 34129345.0 34013790.0 ... 6.823432e+06 3.411716e+07 1.705858e+08 3.573139e+04 -1.186027e+05 -4.588520e+04 1.877621e+04 4.868034e+04 -1.226000e+04 -6.129998e+04

    23 rows × 38 columns

    Calculate difference in C02 Tonnes Between the Estimates

    # Calculate Difference in tons
    merged_df['CO2_diff_2015'] = merged_df['tCO2_2015_FAOSTAT'] - merged_df['tCO2_2015_TRACE']
    merged_df['CO2_diff_2016'] = merged_df['tCO2_2016_FAOSTAT'] - merged_df['tCO2_2016_TRACE']
    merged_df['CO2_diff_2017'] = merged_df['tCO2_2017_FAOSTAT'] - merged_df['tCO2_2017_TRACE']
    merged_df['CO2_diff_2018'] = merged_df['tCO2_2018_FAOSTAT'] - merged_df['tCO2_2018_TRACE']
    merged_df['CO2_diff_2019'] = merged_df['tCO2_2019_FAOSTAT'] - merged_df['tCO2_2019_TRACE']
    merged_df['CO2_diff_means'] = merged_df['Mean_CO2_FAOSTAT'] - merged_df['Mean_CO2_TRACE']
    merged_df['CO2_diff_totals'] = merged_df['Total_CO2_FAOSTAT'] - merged_df['Total_CO2_TRACE']
    

    Calculating the CH4 Percent Differences Between the Estimates

    ## Calculate Percent Differnces on this data set )*100
    # With raw data i could have accomplished this with a groupby.aggregate(lambda x ), however the pivot tables given are not easy to apply #vectorized functions across time series
    merged_df['CH4_abs_percent_diff_2015'] = ((abs(merged_df[2015] - merged_df['tCH4_2015']))/((merged_df[2015] + merged_df['tCH4_2015'])/2))*100
    merged_df['CH4_abs_percent_diff_2016'] = ((abs((merged_df[2016] - merged_df['tCH4_2016']))/((merged_df[2016] + merged_df['tCH4_2016'])/2)))*100
    merged_df['CH4_abs_percent_diff_2017'] = ((abs(merged_df[2017] - merged_df['tCH4_2017']))/((merged_df[2017] + merged_df['tCH4_2017'])/2))*100
    merged_df['CH4_abs_percent_diff_2018'] = (abs((merged_df[2018] - merged_df['tCH4_2018']))/((merged_df[2018] + merged_df['tCH4_2018'])/2))*100
    merged_df['CH4_abs_percent_diff_2019'] = (abs((merged_df[2019] - merged_df['tCH4_2019']))/((merged_df[2019] + merged_df['tCH4_2019'])/2))*100
    merged_df['CH4_abs_percent_diff_means'] = (abs((merged_df['Mean_CH4_FAOSTAT'] - merged_df['Mean_CH4_TRACE']))/((merged_df['Mean_CH4_FAOSTAT'] + merged_df['Mean_CH4_TRACE'])/2))* 100
    merged_df['CH4_abs_percent_diff_totals'] = (abs((merged_df['Total_CH4_FAOSTAT'] - merged_df['Total_CH4_TRACE']))/((merged_df['Total_CH4_TRACE'] + merged_df['Total_CH4_FAOSTAT'])/2))*100
    
    
    merged_df['CH4_relative_percent_diff_2015'] = ((merged_df[2015] - merged_df['tCH4_2015'])/(merged_df[2015]))*100
    merged_df['CH4_relative_percent_diff_2016'] = ((merged_df[2016] - merged_df['tCH4_2016'])/(merged_df[2016]))*100
    merged_df['CH4_relative_percent_diff_2017'] = ((merged_df[2017] - merged_df['tCH4_2017'])/(merged_df[2017]))*100
    merged_df['CH4_relative_percent_diff_2018'] = ((merged_df[2018] - merged_df['tCH4_2018'])/(merged_df[2018]))*100
    merged_df['CH4_relative_percent_diff_2019'] = ((merged_df[2019] - merged_df['tCH4_2019'])/(merged_df[2019]))*100
    merged_df['CH4_relative_percent_diff_means'] = ((merged_df['Mean_CH4_FAOSTAT'] - merged_df['Mean_CH4_TRACE'])/(merged_df["Mean_CH4_FAOSTAT"]))*100
    merged_df['CH4_relative_percent_diff_totals'] = ((merged_df['Total_CH4_FAOSTAT'] - merged_df['Total_CH4_TRACE'])/(merged_df["Total_CH4_FAOSTAT"]))*100
    

    Calculate CO2 Differences

    ## Calculate Percent Differnces on this data set )*100
    # With raw data i could have accomplished this with a groupby.aggregate(lambda x ), however the pivot tables given are not easy to apply #vectorized functions across time series
    merged_df['CO2_abs_percent_diff_2015'] = (abs((merged_df['tCO2_2015_FAOSTAT'] - merged_df['tCO2_2015_TRACE']))/((merged_df['tCO2_2015_TRACE'] + merged_df['tCO2_2015_FAOSTAT'])/2))*100
    merged_df['CO2_abs_percent_diff_2016'] = ((abs(merged_df['tCO2_2016_FAOSTAT'] - merged_df['tCO2_2016_TRACE']))/((merged_df['tCO2_2016_TRACE'] + merged_df['tCO2_2016_FAOSTAT'])/2))*100
    merged_df['CO2_abs_percent_diff_2017'] = ((abs(merged_df['tCO2_2017_FAOSTAT'] - merged_df['tCO2_2017_TRACE']))/((merged_df['tCO2_2017_TRACE'] + merged_df['tCO2_2017_FAOSTAT'])/2))*100
    merged_df['CO2_abs_percent_diff_2018'] = ((abs(merged_df['tCO2_2018_FAOSTAT'] - merged_df['tCO2_2018_TRACE']))/((merged_df['tCO2_2018_TRACE'] + merged_df['tCO2_2018_FAOSTAT'])/2))*100
    merged_df['CO2_abs_percent_diff_2019'] = ((abs(merged_df['tCO2_2019_FAOSTAT'] - merged_df['tCO2_2019_TRACE']))/((merged_df['tCO2_2019_TRACE'] + merged_df['tCO2_2019_FAOSTAT'])/2))*100
    merged_df['CO2_abs_percent_diff_means'] = ((abs(merged_df['Mean_CO2_FAOSTAT'] - merged_df['Mean_CO2_TRACE']))/((merged_df['Mean_CO2_FAOSTAT'] + merged_df['Mean_CO2_TRACE'])/2))* 100
    merged_df['CO2_abs_percent_diff_totals'] = ((abs(merged_df['Total_CO2_FAOSTAT'] - merged_df['Total_CO2_TRACE']))/((merged_df['Total_CO2_TRACE'] + merged_df['Total_CO2_FAOSTAT'])/2))*100
    
    
    merged_df['CO2_relative_percent_diff_2015'] = ((merged_df['tCO2_2015_FAOSTAT']  - merged_df['tCO2_2015_TRACE'])/(merged_df['tCO2_2015_FAOSTAT']))*100
    merged_df['CO2_relative_percent_diff_2016'] = ((merged_df['tCO2_2016_FAOSTAT']  - merged_df['tCO2_2016_TRACE'])/(merged_df['tCO2_2016_FAOSTAT']))*100
    merged_df['CO2_relative_percent_diff_2017'] = ((merged_df['tCO2_2017_FAOSTAT']  - merged_df['tCO2_2017_TRACE'])/(merged_df['tCO2_2017_FAOSTAT']))*100
    merged_df['CO2_relative_percent_diff_2018'] = ((merged_df['tCO2_2018_FAOSTAT']  - merged_df['tCO2_2018_TRACE'])/(merged_df['tCO2_2018_FAOSTAT']))*100
    merged_df['CO2_relative_percent_diff_2019'] = ((merged_df['tCO2_2019_FAOSTAT']  - merged_df['tCO2_2019_TRACE'])/(merged_df['tCO2_2019_FAOSTAT']))*100
    merged_df['CO2_relative_percent_diff_means'] = ((merged_df["Mean_CO2_FAOSTAT"] - merged_df['Mean_CO2_TRACE'])/(merged_df["Mean_CO2_FAOSTAT"]))*100
    merged_df['CO2_relative_percent_diff_totals'] = ((merged_df['Total_CO2_FAOSTAT'] - merged_df['Total_CO2_TRACE'])/(merged_df["Total_CO2_FAOSTAT"]))*100
    
    merged_df
    

    iso3_country country_name_FAOSTAT 2015 2016 2017 2018 2019 tCO2_2015_FAOSTAT tCO2_2016_FAOSTAT tCO2_2017_FAOSTAT ... CO2_abs_percent_diff_2019 CO2_abs_percent_diff_means CO2_abs_percent_diff_totals CO2_relative_percent_diff_2015 CO2_relative_percent_diff_2016 CO2_relative_percent_diff_2017 CO2_relative_percent_diff_2018 CO2_relative_percent_diff_2019 CO2_relative_percent_diff_means CO2_relative_percent_diff_totals
    0 BGD Bangladesh 1131293.4 1093480.4 1154531.0 1144591.0 1144745.4 28282335.0 27337010.0 28863275.0 ... 57.606797 63.425917 63.425917 -107.233570 -108.340089 -81.801753 -87.073857 -80.912284 -92.881337 -92.881337
    1 BRA Brazil 138910.3 126278.2 130322.9 121615.2 111084.8 3472757.5 3156955.0 3258072.5 ... 99.141295 93.222402 93.222402 -145.498958 -145.821472 -185.841820 -205.638590 -196.594424 -174.610414 -174.610414
    2 CHN China 5406593.9 5399920.0 5400129.0 5302173.1 5214454.7 135164847.5 134998000.0 135003225.0 ... 7.189945 9.421813 9.421813 -13.447527 -8.511442 -17.683689 -2.108366 -7.458060 -9.887609 -9.887609
    3 ESP Spain 55082.2 55073.1 54232.4 52925.0 52098.5 1377055.0 1376827.5 1355810.0 ... 127.757634 124.705066 124.705066 79.277072 75.763074 77.554025 73.445258 77.958601 76.811284 76.811284
    4 IDN Indonesia 2407953.5 2387656.4 2425290.6 2405613.8 2257604.3 60198837.5 59691410.0 60632265.0 ... 56.234955 70.153790 70.153790 46.691296 57.149216 60.353916 51.073546 43.893273 51.936188 51.936188
    5 IND India 4580248.4 4559136.4 4620790.8 4661154.9 4621416.8 114506210.0 113978410.0 115519770.0 ... 47.515394 32.086209 32.086209 -35.798018 -16.456550 -34.791882 -41.376939 -62.321563 -38.217479 -38.217479
    6 IRN Iran (Islamic Republic of) 116486.7 131008.5 87233.6 93936.6 96103.4 2912167.5 3275212.5 2180840.0 ... 1.152675 13.260902 13.260902 24.674600 29.927285 -10.281098 5.513463 1.146070 12.436318 12.436318
    7 ITA Italy 114574.8 118003.0 118003.0 109463.8 110895.1 2864370.0 2950075.0 2950075.0 ... 83.321649 80.187551 80.187551 56.395576 58.155426 53.868303 59.165480 58.817707 57.238482 57.238482
    8 JPN Japan 330353.1 326403.0 323700.3 322245.0 320581.8 8258827.5 8160075.0 8092507.5 ... 31.555853 36.874729 36.874729 30.212099 30.021068 16.313490 51.954183 27.255501 31.134372 31.134372
    9 KHM Cambodia 436826.0 459003.1 473745.3 479362.7 468378.9 10920650.0 11475077.5 11843632.5 ... 23.769728 14.304565 14.304565 -13.424988 -24.872761 4.652461 -16.667610 -26.975760 -15.406481 -15.406481
    10 KOR Korea (the Republic of) 167862.2 163534.1 158489.7 154911.3 153260.9 4196555.0 4088352.5 3962242.5 ... 14.321600 15.813819 15.813819 13.507737 22.059230 7.677136 16.497815 13.364589 14.655057 14.655057
    11 LAO Lao People's Democratic Republic (the) 94826.8 95630.0 93940.7 83333.6 77005.5 2370670.0 2390750.0 2348517.5 ... 136.209544 145.627378 145.627378 82.482067 82.260369 87.565950 87.883944 81.026578 84.268427 84.268427
    12 LKA Sri Lanka 132640.0 121756.3 84456.3 111049.0 102156.3 3316000.0 3043907.5 2111407.5 ... 18.612972 27.569003 27.569003 37.382189 16.904226 30.001182 18.784377 17.028241 24.229137 24.229137
    13 MMR Myanmar 1059409.6 1052287.7 1087029.5 1118850.0 1083100.3 26485240.0 26307192.5 27175737.5 ... 14.853751 14.697380 14.697380 -6.859731 -22.666619 -10.868129 -22.665852 -16.045425 -15.863111 -15.863111
    14 MYS Malaysia 121942.6 123232.8 122656.3 125238.5 122453.8 3048565.0 3080820.0 3066407.5 ... 14.753544 13.043224 13.043224 13.287137 9.922595 9.398001 14.840462 13.739977 12.244674 12.244674
    15 NPL Nepal 149262.2 142723.7 162574.6 153890.8 156215.4 3731555.0 3568092.5 4064365.0 ... 29.190154 50.437328 50.437328 32.502761 53.286238 50.291743 40.212459 25.472433 40.279401 40.279401
    16 PAK Pakistan 383529.3 381361.8 406083.3 393404.2 424755.1 9588232.5 9534045.0 10152082.5 ... 42.334367 31.154530 31.154530 -26.520489 -55.912882 -32.303912 -15.207139 -53.701452 -36.903009 -36.903009
    17 PHL Philippines (the) 1557810.6 1524292.5 1609862.5 1606047.8 1556225.8 38945265.0 38107312.5 40246562.5 ... 125.114417 121.748165 121.748165 77.968948 73.275770 76.166725 74.003201 76.966391 75.679166 75.679166
    18 PRK Korea (the Democratic People's Republic of) 82823.3 83442.3 84596.2 83943.3 82937.0 2070582.5 2086057.5 2114905.0 ... 15.169422 17.492669 17.492669 -38.030793 -9.988016 -28.310326 -3.195586 -16.414407 -19.169278 -19.169278
    19 THA Thailand 1554254.0 1703327.7 1714465.6 1702989.1 1553835.5 38856350.0 42583192.5 42861640.0 ... 17.404685 22.551331 22.551331 10.323662 -4.559627 32.066348 46.173612 16.011325 20.266184 20.266184
    20 TWN Taiwan (Province of China) 45838.7 49838.3 49991.2 49414.1 49152.0 1145967.5 1245957.5 1249780.0 ... 58.617135 52.458032 52.458032 -71.622589 -62.307893 -74.143329 -64.692890 -82.919716 -71.109302 -71.109302
    21 USA United States of America (the) 364728.0 438662.0 336255.5 412177.5 350136.5 9118200.0 10966550.0 8406387.5 ... 69.713577 78.850049 78.850049 55.821215 63.101978 49.895259 59.792699 51.694526 56.553728 56.553728
    22 VNM Viet Nam 1381744.4 1365173.8 1360551.6 1336231.2 1318431.1 34543610.0 34129345.0 34013790.0 ... 3.761740 0.902428 0.902428 2.585962 -8.687738 -3.372544 1.405162 3.692293 -0.906518 -0.906518

    23 rows × 73 columns

    Recalculate Means

    merged_df.loc['mean'] = merged_df.select_dtypes(np.number).mean()
    
    merged_df.at['mean','country_name_FAOSTAT'] = 'mean'
    merged_df.at['mean','country_name_TRACE'] = 'mean'
    
    merged_df
    

    iso3_country country_name_FAOSTAT 2015 2016 2017 2018 2019 tCO2_2015_FAOSTAT tCO2_2016_FAOSTAT tCO2_2017_FAOSTAT ... CO2_abs_percent_diff_2019 CO2_abs_percent_diff_means CO2_abs_percent_diff_totals CO2_relative_percent_diff_2015 CO2_relative_percent_diff_2016 CO2_relative_percent_diff_2017 CO2_relative_percent_diff_2018 CO2_relative_percent_diff_2019 CO2_relative_percent_diff_means CO2_relative_percent_diff_totals
    0 BGD Bangladesh 1131293.4 1.093480e+06 1.154531e+06 1.144591e+06 1.144745e+06 28282335.0 2.733701e+07 2.886328e+07 ... 57.606797 63.425917 63.425917 -107.233570 -108.340089 -81.801753 -87.073857 -80.912284 -92.881337 -92.881337
    1 BRA Brazil 138910.3 1.262782e+05 1.303229e+05 1.216152e+05 1.110848e+05 3472757.5 3.156955e+06 3.258072e+06 ... 99.141295 93.222402 93.222402 -145.498958 -145.821472 -185.841820 -205.638590 -196.594424 -174.610414 -174.610414
    2 CHN China 5406593.9 5.399920e+06 5.400129e+06 5.302173e+06 5.214455e+06 135164847.5 1.349980e+08 1.350032e+08 ... 7.189945 9.421813 9.421813 -13.447527 -8.511442 -17.683689 -2.108366 -7.458060 -9.887609 -9.887609
    3 ESP Spain 55082.2 5.507310e+04 5.423240e+04 5.292500e+04 5.209850e+04 1377055.0 1.376828e+06 1.355810e+06 ... 127.757634 124.705066 124.705066 79.277072 75.763074 77.554025 73.445258 77.958601 76.811284 76.811284
    4 IDN Indonesia 2407953.5 2.387656e+06 2.425291e+06 2.405614e+06 2.257604e+06 60198837.5 5.969141e+07 6.063226e+07 ... 56.234955 70.153790 70.153790 46.691296 57.149216 60.353916 51.073546 43.893273 51.936188 51.936188
    5 IND India 4580248.4 4.559136e+06 4.620791e+06 4.661155e+06 4.621417e+06 114506210.0 1.139784e+08 1.155198e+08 ... 47.515394 32.086209 32.086209 -35.798018 -16.456550 -34.791882 -41.376939 -62.321563 -38.217479 -38.217479
    6 IRN Iran (Islamic Republic of) 116486.7 1.310085e+05 8.723360e+04 9.393660e+04 9.610340e+04 2912167.5 3.275212e+06 2.180840e+06 ... 1.152675 13.260902 13.260902 24.674600 29.927285 -10.281098 5.513463 1.146070 12.436318 12.436318
    7 ITA Italy 114574.8 1.180030e+05 1.180030e+05 1.094638e+05 1.108951e+05 2864370.0 2.950075e+06 2.950075e+06 ... 83.321649 80.187551 80.187551 56.395576 58.155426 53.868303 59.165480 58.817707 57.238482 57.238482
    8 JPN Japan 330353.1 3.264030e+05 3.237003e+05 3.222450e+05 3.205818e+05 8258827.5 8.160075e+06 8.092508e+06 ... 31.555853 36.874729 36.874729 30.212099 30.021068 16.313490 51.954183 27.255501 31.134372 31.134372
    9 KHM Cambodia 436826.0 4.590031e+05 4.737453e+05 4.793627e+05 4.683789e+05 10920650.0 1.147508e+07 1.184363e+07 ... 23.769728 14.304565 14.304565 -13.424988 -24.872761 4.652461 -16.667610 -26.975760 -15.406481 -15.406481
    10 KOR Korea (the Republic of) 167862.2 1.635341e+05 1.584897e+05 1.549113e+05 1.532609e+05 4196555.0 4.088352e+06 3.962243e+06 ... 14.321600 15.813819 15.813819 13.507737 22.059230 7.677136 16.497815 13.364589 14.655057 14.655057
    11 LAO Lao People's Democratic Republic (the) 94826.8 9.563000e+04 9.394070e+04 8.333360e+04 7.700550e+04 2370670.0 2.390750e+06 2.348518e+06 ... 136.209544 145.627378 145.627378 82.482067 82.260369 87.565950 87.883944 81.026578 84.268427 84.268427
    12 LKA Sri Lanka 132640.0 1.217563e+05 8.445630e+04 1.110490e+05 1.021563e+05 3316000.0 3.043908e+06 2.111408e+06 ... 18.612972 27.569003 27.569003 37.382189 16.904226 30.001182 18.784377 17.028241 24.229137 24.229137
    13 MMR Myanmar 1059409.6 1.052288e+06 1.087030e+06 1.118850e+06 1.083100e+06 26485240.0 2.630719e+07 2.717574e+07 ... 14.853751 14.697380 14.697380 -6.859731 -22.666619 -10.868129 -22.665852 -16.045425 -15.863111 -15.863111
    14 MYS Malaysia 121942.6 1.232328e+05 1.226563e+05 1.252385e+05 1.224538e+05 3048565.0 3.080820e+06 3.066408e+06 ... 14.753544 13.043224 13.043224 13.287137 9.922595 9.398001 14.840462 13.739977 12.244674 12.244674
    15 NPL Nepal 149262.2 1.427237e+05 1.625746e+05 1.538908e+05 1.562154e+05 3731555.0 3.568093e+06 4.064365e+06 ... 29.190154 50.437328 50.437328 32.502761 53.286238 50.291743 40.212459 25.472433 40.279401 40.279401
    16 PAK Pakistan 383529.3 3.813618e+05 4.060833e+05 3.934042e+05 4.247551e+05 9588232.5 9.534045e+06 1.015208e+07 ... 42.334367 31.154530 31.154530 -26.520489 -55.912882 -32.303912 -15.207139 -53.701452 -36.903009 -36.903009
    17 PHL Philippines (the) 1557810.6 1.524292e+06 1.609862e+06 1.606048e+06 1.556226e+06 38945265.0 3.810731e+07 4.024656e+07 ... 125.114417 121.748165 121.748165 77.968948 73.275770 76.166725 74.003201 76.966391 75.679166 75.679166
    18 PRK Korea (the Democratic People's Republic of) 82823.3 8.344230e+04 8.459620e+04 8.394330e+04 8.293700e+04 2070582.5 2.086058e+06 2.114905e+06 ... 15.169422 17.492669 17.492669 -38.030793 -9.988016 -28.310326 -3.195586 -16.414407 -19.169278 -19.169278
    19 THA Thailand 1554254.0 1.703328e+06 1.714466e+06 1.702989e+06 1.553836e+06 38856350.0 4.258319e+07 4.286164e+07 ... 17.404685 22.551331 22.551331 10.323662 -4.559627 32.066348 46.173612 16.011325 20.266184 20.266184
    20 TWN Taiwan (Province of China) 45838.7 4.983830e+04 4.999120e+04 4.941410e+04 4.915200e+04 1145967.5 1.245958e+06 1.249780e+06 ... 58.617135 52.458032 52.458032 -71.622589 -62.307893 -74.143329 -64.692890 -82.919716 -71.109302 -71.109302
    21 USA United States of America (the) 364728.0 4.386620e+05 3.362555e+05 4.121775e+05 3.501365e+05 9118200.0 1.096655e+07 8.406388e+06 ... 69.713577 78.850049 78.850049 55.821215 63.101978 49.895259 59.792699 51.694526 56.553728 56.553728
    22 VNM Viet Nam 1381744.4 1.365174e+06 1.360552e+06 1.336231e+06 1.318431e+06 34543610.0 3.412934e+07 3.401379e+07 ... 3.761740 0.902428 0.902428 2.585962 -8.687738 -3.372544 1.405162 3.692293 -0.906518 -0.906518
    mean NaN mean 948478.0 9.522272e+05 9.590840e+05 9.575896e+05 9.316100e+05 23711950.0 2.380568e+07 2.397710e+07 ... 47.621862 49.129925 49.129925 4.551116 4.508756 3.322003 6.179080 -1.533721 3.599038 3.599038

    24 rows × 73 columns

    Recalculate Totals

    merged_df.loc['total'] = merged_df[merged_df['country_name_FAOSTAT'] != 'mean'].select_dtypes(np.number).sum()
    
    merged_df.at['total','country_name_FAOSTAT'] = 'total'
    merged_df.at['total','country_name_TRACE'] = 'total'
    
    merged_df.reset_index(inplace=True, drop = True)
    
    merged_df
    

    iso3_country country_name_FAOSTAT 2015 2016 2017 2018 2019 tCO2_2015_FAOSTAT tCO2_2016_FAOSTAT tCO2_2017_FAOSTAT ... CO2_abs_percent_diff_2019 CO2_abs_percent_diff_means CO2_abs_percent_diff_totals CO2_relative_percent_diff_2015 CO2_relative_percent_diff_2016 CO2_relative_percent_diff_2017 CO2_relative_percent_diff_2018 CO2_relative_percent_diff_2019 CO2_relative_percent_diff_means CO2_relative_percent_diff_totals
    0 BGD Bangladesh 1131293.4 1.093480e+06 1.154531e+06 1.144591e+06 1.144745e+06 28282335.0 2.733701e+07 2.886328e+07 ... 57.606797 63.425917 63.425917 -107.233570 -108.340089 -81.801753 -87.073857 -80.912284 -92.881337 -92.881337
    1 BRA Brazil 138910.3 1.262782e+05 1.303229e+05 1.216152e+05 1.110848e+05 3472757.5 3.156955e+06 3.258072e+06 ... 99.141295 93.222402 93.222402 -145.498958 -145.821472 -185.841820 -205.638590 -196.594424 -174.610414 -174.610414
    2 CHN China 5406593.9 5.399920e+06 5.400129e+06 5.302173e+06 5.214455e+06 135164847.5 1.349980e+08 1.350032e+08 ... 7.189945 9.421813 9.421813 -13.447527 -8.511442 -17.683689 -2.108366 -7.458060 -9.887609 -9.887609
    3 ESP Spain 55082.2 5.507310e+04 5.423240e+04 5.292500e+04 5.209850e+04 1377055.0 1.376828e+06 1.355810e+06 ... 127.757634 124.705066 124.705066 79.277072 75.763074 77.554025 73.445258 77.958601 76.811284 76.811284
    4 IDN Indonesia 2407953.5 2.387656e+06 2.425291e+06 2.405614e+06 2.257604e+06 60198837.5 5.969141e+07 6.063226e+07 ... 56.234955 70.153790 70.153790 46.691296 57.149216 60.353916 51.073546 43.893273 51.936188 51.936188
    5 IND India 4580248.4 4.559136e+06 4.620791e+06 4.661155e+06 4.621417e+06 114506210.0 1.139784e+08 1.155198e+08 ... 47.515394 32.086209 32.086209 -35.798018 -16.456550 -34.791882 -41.376939 -62.321563 -38.217479 -38.217479
    6 IRN Iran (Islamic Republic of) 116486.7 1.310085e+05 8.723360e+04 9.393660e+04 9.610340e+04 2912167.5 3.275212e+06 2.180840e+06 ... 1.152675 13.260902 13.260902 24.674600 29.927285 -10.281098 5.513463 1.146070 12.436318 12.436318
    7 ITA Italy 114574.8 1.180030e+05 1.180030e+05 1.094638e+05 1.108951e+05 2864370.0 2.950075e+06 2.950075e+06 ... 83.321649 80.187551 80.187551 56.395576 58.155426 53.868303 59.165480 58.817707 57.238482 57.238482
    8 JPN Japan 330353.1 3.264030e+05 3.237003e+05 3.222450e+05 3.205818e+05 8258827.5 8.160075e+06 8.092508e+06 ... 31.555853 36.874729 36.874729 30.212099 30.021068 16.313490 51.954183 27.255501 31.134372 31.134372
    9 KHM Cambodia 436826.0 4.590031e+05 4.737453e+05 4.793627e+05 4.683789e+05 10920650.0 1.147508e+07 1.184363e+07 ... 23.769728 14.304565 14.304565 -13.424988 -24.872761 4.652461 -16.667610 -26.975760 -15.406481 -15.406481
    10 KOR Korea (the Republic of) 167862.2 1.635341e+05 1.584897e+05 1.549113e+05 1.532609e+05 4196555.0 4.088352e+06 3.962243e+06 ... 14.321600 15.813819 15.813819 13.507737 22.059230 7.677136 16.497815 13.364589 14.655057 14.655057
    11 LAO Lao People's Democratic Republic (the) 94826.8 9.563000e+04 9.394070e+04 8.333360e+04 7.700550e+04 2370670.0 2.390750e+06 2.348518e+06 ... 136.209544 145.627378 145.627378 82.482067 82.260369 87.565950 87.883944 81.026578 84.268427 84.268427
    12 LKA Sri Lanka 132640.0 1.217563e+05 8.445630e+04 1.110490e+05 1.021563e+05 3316000.0 3.043908e+06 2.111408e+06 ... 18.612972 27.569003 27.569003 37.382189 16.904226 30.001182 18.784377 17.028241 24.229137 24.229137
    13 MMR Myanmar 1059409.6 1.052288e+06 1.087030e+06 1.118850e+06 1.083100e+06 26485240.0 2.630719e+07 2.717574e+07 ... 14.853751 14.697380 14.697380 -6.859731 -22.666619 -10.868129 -22.665852 -16.045425 -15.863111 -15.863111
    14 MYS Malaysia 121942.6 1.232328e+05 1.226563e+05 1.252385e+05 1.224538e+05 3048565.0 3.080820e+06 3.066408e+06 ... 14.753544 13.043224 13.043224 13.287137 9.922595 9.398001 14.840462 13.739977 12.244674 12.244674
    15 NPL Nepal 149262.2 1.427237e+05 1.625746e+05 1.538908e+05 1.562154e+05 3731555.0 3.568093e+06 4.064365e+06 ... 29.190154 50.437328 50.437328 32.502761 53.286238 50.291743 40.212459 25.472433 40.279401 40.279401
    16 PAK Pakistan 383529.3 3.813618e+05 4.060833e+05 3.934042e+05 4.247551e+05 9588232.5 9.534045e+06 1.015208e+07 ... 42.334367 31.154530 31.154530 -26.520489 -55.912882 -32.303912 -15.207139 -53.701452 -36.903009 -36.903009
    17 PHL Philippines (the) 1557810.6 1.524292e+06 1.609862e+06 1.606048e+06 1.556226e+06 38945265.0 3.810731e+07 4.024656e+07 ... 125.114417 121.748165 121.748165 77.968948 73.275770 76.166725 74.003201 76.966391 75.679166 75.679166
    18 PRK Korea (the Democratic People's Republic of) 82823.3 8.344230e+04 8.459620e+04 8.394330e+04 8.293700e+04 2070582.5 2.086058e+06 2.114905e+06 ... 15.169422 17.492669 17.492669 -38.030793 -9.988016 -28.310326 -3.195586 -16.414407 -19.169278 -19.169278
    19 THA Thailand 1554254.0 1.703328e+06 1.714466e+06 1.702989e+06 1.553836e+06 38856350.0 4.258319e+07 4.286164e+07 ... 17.404685 22.551331 22.551331 10.323662 -4.559627 32.066348 46.173612 16.011325 20.266184 20.266184
    20 TWN Taiwan (Province of China) 45838.7 4.983830e+04 4.999120e+04 4.941410e+04 4.915200e+04 1145967.5 1.245958e+06 1.249780e+06 ... 58.617135 52.458032 52.458032 -71.622589 -62.307893 -74.143329 -64.692890 -82.919716 -71.109302 -71.109302
    21 USA United States of America (the) 364728.0 4.386620e+05 3.362555e+05 4.121775e+05 3.501365e+05 9118200.0 1.096655e+07 8.406388e+06 ... 69.713577 78.850049 78.850049 55.821215 63.101978 49.895259 59.792699 51.694526 56.553728 56.553728
    22 VNM Viet Nam 1381744.4 1.365174e+06 1.360552e+06 1.336231e+06 1.318431e+06 34543610.0 3.412934e+07 3.401379e+07 ... 3.761740 0.902428 0.902428 2.585962 -8.687738 -3.372544 1.405162 3.692293 -0.906518 -0.906518
    23 NaN mean 948478.0 9.522272e+05 9.590840e+05 9.575896e+05 9.316100e+05 23711950.0 2.380568e+07 2.397710e+07 ... 47.621862 49.129925 49.129925 4.551116 4.508756 3.322003 6.179080 -1.533721 3.599038 3.599038
    24 NaN total 21814994.0 2.190123e+07 2.205893e+07 2.202456e+07 2.142703e+07 545374850.0 5.475306e+08 5.514733e+08 ... 1095.302835 1129.988279 1129.988279 104.675658 103.701389 76.406059 142.118831 -35.275585 82.777881 82.777881

    25 rows × 73 columns

    Merged Data to File

    outfile = "/Users/jnapolitano/Projects/wattime-takehome/wattime-takehome/data/MERGED_DATA.csv"
    
    merged_df.to_csv(outfile)
    

    CO2 Difference Plots

    merged_df.plot(kind = "barh", x = 'country_name_FAOSTAT', y = ["CO2_diff_2015", "CO2_diff_2016",	"CO2_diff_2017", "CO2_diff_2018", "CO2_diff_2019"], xlabel = "Country Name", ylabel = "Tonnes CH4", figsize = (10,10))
    
    <AxesSubplot:ylabel='Country Name'>
    

    png

    Percent Difference Plot

    merged_df.plot(kind = "barh", x = 'country_name_FAOSTAT', y = ["CO2_relative_percent_diff_2015", "CO2_relative_percent_diff_2016",	"CO2_relative_percent_diff_2017", "CO2_relative_percent_diff_2018", "CO2_relative_percent_diff_2019"], xlabel = "Country Name", ylabel = "Tonnes CH4", figsize=(10,10))
    
    <AxesSubplot:ylabel='Country Name'>
    

    png


    Series

    This is a post in the Rice Paddy Methane Emissions series.
    Other posts in this series:

  • Rice Paddy Methane Emissions Estimation: Part 1

  • comments powered by Disqus