A version of YaleDH's pixplot tool & image corpus similarity visualizer that runs on python 3.12, also extended to generate network edges, nodes for a similarity graph

pixplot-using-python3-12.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/shawngraham/2882ab859a20062f0b34a7efca6f402b/pixplot-using-python3-12.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
       {
      "cell_type": "markdown",
      "source": [
        "This notebook loads a version of [PixPlot, from YaleDH](https://dhlab.yale.edu/projects/pixplot/) that has been updated to run in Google Colab, or on your own machine, using Python 3.12. This version of PixPlot has also been expanded to develop a network visualization where images are connected to n most similar. An ancillary script will output this data in edge and node csv format for visualization and further exploration in Gephi.\n",
        "\n",
        "By Shawn Graham, XLab, Carleton U; using Claude Opus 4.5 to diagnose and update the code dependency chains that PixPlot uses.\n",
        "\n",
        "Save a copy of this file, then open the copy and start playing!"
      ],
      "metadata": {
        "id": "2jq1KFLnCJjJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!python --version"
      ],
      "metadata": {
        "id": "wpzvd8S7xx0-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HYV9FPkXiSu7",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "# Cell 1: Clone and install\n",
        "!git clone https://github.com/XLabCU/pix-plot.git\n",
        "%cd pix-plot\n",
        "!python install.py\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!python install.py\n"
      ],
      "metadata": {
        "id": "Xd-L0j5GiZac",
        "collapsed": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Cell 2: Change directory to avoid import conflict, then verify\n",
        "# if you don't change back to the content folder, you get import conflicts.\n",
        "%cd /content\n",
        "import pixplot\n",
        "import numpy\n",
        "import tensorflow\n",
        "import rasterfairy\n",
        "\n",
        "print('✓ PixPlot successfully installed!')\n",
        "print(f'✓ numpy version: {numpy.__version__}')\n",
        "print(f'✓ tensorflow version: {tensorflow.__version__}')\n",
        "print('✓ rasterfairy imported successfully!')\n"
      ],
      "metadata": {
        "id": "aTWRJ0E4idZo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "drag and drop the zip of your data into the file tray, then run next cell.\n",
        "make sure there are no subfolders etc in your images folder."
      ],
      "metadata": {
        "id": "oztv1h6oAsTG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!unzip /content/AssetLibrary-20260107T185345Z-1-001.zip"
      ],
      "metadata": {
        "collapsed": true,
        "id": "YPO-Gv6GAVye"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Process a folder of images\n",
        "!pixplot --images \"AssetLibrary/*.jpg\"\n",
        "\n",
        "# this will take a while."
      ],
      "metadata": {
        "id": "PHXr2ewomNbU",
        "collapsed": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# view within notebook cell (downloading, viewing on your machine is better)\n",
        "from google.colab import output\n",
        "output.serve_kernel_port_as_iframe(5000)\n",
        "%cd output\n",
        "!python -m http.server 5000\n"
      ],
      "metadata": {
        "id": "w-4FD9VJmijO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# modifying the network visualization:\n",
        "!pixplot --images \"AssetLibrary/*.jpg\" --network_n_neighbors 5 --network_edge_threshold 0.7 --network_layout_iterations 100\n",
        "\n",
        "# If you had pixplot installed on your own machine, you could run a local web server to view the visualization:\n",
        "#pixplot --serve"
      ],
      "metadata": {
        "id": "5alr9ehCmTlI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%cd .."
      ],
      "metadata": {
        "id": "w9H3fcucDnvR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "## turn the network visualization into node, edges csv suitable for use with gephi\n",
        "\n",
        "!python pix-plot/pixplot_network_export.py \\\n",
        "  --data_dir output \\\n",
        "  --n_neighbors 5 \\\n",
        "  --output network_edges.csv \\\n",
        "  --layout umap \\\n",
        "  --include_thumbs \\\n",
        "  --include_metadata\n"
      ],
      "metadata": {
        "id": "VdOxFHuQDbhS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "cd /content"
      ],
      "metadata": {
        "id": "tVHvrbXG-lUs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!zip -r output.zip output/"
      ],
      "metadata": {
        "collapsed": true,
        "id": "SKyAJ8q8-tzO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "then right-click and download output.zip\n",
        "\n",
        "you can then drag your output.zip into the circle at https://app.netlify.com/drop and get a hosted version of your output for live exploration. Otherwise, you need to unzip on your machine, then open a terminal in that output folder. Assuming you have python installed in your machine, you'd type\n",
        "\n",
        "```python -m http.server 8000```\n",
        "\n",
        "and then in your browser you'd enter the address bar: ```localhost:8000```"
      ],
      "metadata": {
        "id": "2tsiCyEPBNJj"
      }
    }
  ]
}