WeatherNext_Xee.ipynb

weathernext_xee.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "private_outputs": true,
      "provenance": [],
      "authorship_tag": "ABX9TyNK7QVAbbvU/y5oiaao3vXN",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/KMarkert/0bee72f72f056eee5db92cae888fb77f/weathernext_xee.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38c128d9"
      },
      "source": [
        "# Earth Engine, WeatherNext, Xee\n",
        "\n",
        "This notebook is a demonstration for how to extract WeatherNext data from Earth Engine using Xee. This will pull the data in manner that can be directly stored as netCDF for downstream use.\n",
        "\n",
        "This goes through an example to visualize weather data (e.g., temperature) for South Florida, create a simple time series graph, and then storing as netCDF."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "681821f2"
      },
      "source": [
        "%%capture\n",
        "!pip install xee cartopy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8557c575"
      },
      "source": [
        "import ee\n",
        "import xarray as xr\n",
        "import matplotlib.pyplot as plt\n",
        "import cartopy.crs as ccrs"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5e390a83"
      },
      "source": [
        "try:\n",
        "    # Trigger the authentication flow.\n",
        "    ee.Authenticate()\n",
        "\n",
        "    # Initialize the library.\n",
        "    # Update to use your Earth Engine cloud project\n",
        "    # See https://developers.google.com/earth-engine/guides/auth#python_and_command_line for details\n",
        "    ee.Initialize(project='YOUR-CLOUD-PROJECT')\n",
        "except Exception as e:\n",
        "    print(f\"An error occurred during authentication or initialization: {e}\")\n",
        "    print(\"If you need to specify a project, use: ee.Initialize(project='your-project-id')\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "646f9d89"
      },
      "source": [
        "# Define the Region of Interest (ROI) for South Florida\n",
        "# Format: [min_lon, min_lat, max_lon, max_lat]\n",
        "roi = ee.Geometry.Rectangle([-83.0, 24.5, -79.5, 29.0])\n",
        "\n",
        "# Define the forecast time\n",
        "start_date = '2025-11-22T00:00:00'\n",
        "end_date = '2025-11-22T06:00:00'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "510c3f9e"
      },
      "source": [
        "## Load WeatherNext Data with Xee\n",
        "\n",
        "Load the WeatherNext dataset from Earth Engine, filter it by the defined date range, and open it as an Xarray dataset using the `xee` engine with specified geometry, CRS, and scale.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "33b1ee54"
      },
      "source": [
        "\n",
        "# Create an Earth Engine ImageCollection object for the WeatherNext dataset.\n",
        "# Filter this collection by forecast initialization using the `start_date` and `end_date` variables.\n",
        "ic = (\n",
        "    ee.ImageCollection('projects/gcp-public-data-weathernext/assets/59572747_4_0')\n",
        "    .filterDate(start_date, end_date)\n",
        "    .filter(ee.Filter.gt('forecast_hour', 0))\n",
        ")\n",
        "\n",
        "# Update the time information to be valid forecast time\n",
        "# Adds the forecast hour to the initialization time\n",
        "ic = ic.map(\n",
        "    lambda img: img.set(\n",
        "        'system:time_start',\n",
        "        img.date().advance(img.get('forecast_hour'), 'hour').millis()\n",
        "    )\n",
        ")\n",
        "\n",
        "# Use `xarray.open_dataset` to load the data.\n",
        "# Set the `engine` parameter to 'ee' to pull data from Earth Engine.\n",
        "# Provide the `geometry` parameter as the `roi` variable.\n",
        "ds = xr.open_dataset(\n",
        "    ic,\n",
        "    engine='ee',\n",
        "    geometry=roi,\n",
        "    crs='EPSG:4326',\n",
        "    scale=0.25,\n",
        ")\n",
        "\n",
        "# Print the `ds` object to inspect the loaded dataset structure.\n",
        "print(ds)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ee798dcb"
      },
      "source": [
        "## Create a map plot from the WeatherNext data\n",
        "\n",
        "Select the `2m_temperature` variable for the first time step and generate a spatial map plot using Matplotlib and Cartopy to visualize the weather data over South Florida.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ed95a848"
      },
      "source": [
        "# Select the '2m_temperature' variable for the first time step\n",
        "# We select time=0 to plot a single snapshot\n",
        "temp_data = ds['2m_temperature'].isel(time=0)\n",
        "\n",
        "# Create a figure and axes with a PlateCarree projection (standard lat/lon)\n",
        "plt.figure(figsize=(12, 8))\n",
        "ax = plt.axes(projection=ccrs.PlateCarree())\n",
        "\n",
        "# Plot the temperature data\n",
        "# transform=ccrs.PlateCarree() ensures the data is projected correctly onto the map\n",
        "temp_plot = temp_data.plot(\n",
        "    ax=ax,\n",
        "    transform=ccrs.PlateCarree(),\n",
        "    cmap='coolwarm',\n",
        "    cbar_kwargs={'label': 'Temperature (K)'},\n",
        "    x='lon',\n",
        "    y='lat',\n",
        ")\n",
        "\n",
        "# Add coastlines for geographic context\n",
        "ax.coastlines()\n",
        "\n",
        "# Add gridlines with labels\n",
        "ax.gridlines(draw_labels=True)\n",
        "\n",
        "# Set a title for the plot\n",
        "plt.title(f'2m Temperature in South Florida\\nTime: {temp_data.time.values}')\n",
        "\n",
        "# Display the plot\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aa5fc4a5"
      },
      "source": [
        "## Calculate Time Series\n",
        "\n",
        "Compute the spatial mean of the `2m_temperature` variable over the defined Region of Interest (ROI) for each time step to generate a time series.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a858bbc1"
      },
      "source": [
        "# Select the '2m_temperature' variable\n",
        "temp_var = ds['2m_temperature']\n",
        "\n",
        "# Calculate the mean value across spatial dimensions ('lat' and 'lon')\n",
        "# This aggregates the data spatially, resulting in a single value per time step\n",
        "temp_timeseries = temp_var.mean(dim=['lat', 'lon'])\n",
        "\n",
        "# Print the resulting time series data to verify its structure\n",
        "print(temp_timeseries)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5592f022"
      },
      "source": [
        "Visualize the computed temperature time series using Matplotlib\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cb0fad68"
      },
      "source": [
        "# Create a figure for the time series plot\n",
        "plt.figure(figsize=(12, 6))\n",
        "\n",
        "# Plot the time series using the Xarray plot method (wraps Matplotlib)\n",
        "# Use markers to indicate data points\n",
        "temp_timeseries.plot(marker='o', linestyle='-')\n",
        "\n",
        "# Set the title and axis labels\n",
        "plt.title(f'Spatially Averaged 2m Temperature Forecast (Forecast init: {start_date})')\n",
        "plt.xlabel('Time')\n",
        "plt.ylabel('Temperature (K)')\n",
        "\n",
        "# Add gridlines for better readability\n",
        "plt.grid(True)\n",
        "\n",
        "# Display the plot\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Store the data as NetCDF\n",
        "\n",
        "To store the data from Xee as netCDF, it is as simple as calling `.to_netcdf()`. There are more options for how to store the final output (such as chunking and compression) but this is the easiest way to get started."
      ],
      "metadata": {
        "id": "v3eQgPdE-Jac"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ds.to_netcdf(f'weathernext_data_south_florida_{start_date}.nc')"
      ],
      "metadata": {
        "id": "BFyRoCnD80is"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "w7GSRLoW-cAo"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}