Skip to content

Instantly share code, notes, and snippets.

@chunibyo-wly

chunibyo-wly/ABCStepUrl.txt

Created July 31, 2025 02:49
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save chunibyo-wly/3d08196aef4de5b77f6326814bd9e341 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save chunibyo-wly/3d08196aef4de5b77f6326814bd9e341 to your computer and use it in GitHub Desktop.
Download ZIP
ABC Dataset archive download url
Raw
ABCStepUrl.txt
https://archive.nyu.edu/bitstream/2451/44309/3/abc_0000_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44310/3/abc_0001_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44318/3/abc_0002_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44319/3/abc_0003_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44320/3/abc_0004_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44321/3/abc_0005_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44322/3/abc_0006_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44323/3/abc_0007_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44324/3/abc_0008_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44325/3/abc_0009_step_v00.7z
https://archive.nyu.edu/retrieve/88666/abc_0010_step_v00.7z
https://archive.nyu.edu/retrieve/88670/abc_0011_step_v00.7z
https://archive.nyu.edu/retrieve/88674/abc_0012_step_v00.7z
https://archive.nyu.edu/retrieve/88678/abc_0013_step_v00.7z
https://archive.nyu.edu/retrieve/88682/abc_0014_step_v00.7z
https://archive.nyu.edu/retrieve/88686/abc_0015_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44332/3/abc_0016_step_v00.7z
https://archive.nyu.edu/retrieve/88694/abc_0017_step_v00.7z
https://archive.nyu.edu/retrieve/88698/abc_0018_step_v00.7z
https://archive.nyu.edu/retrieve/88702/abc_0019_step_v00.7z
https://archive.nyu.edu/retrieve/88707/abc_0020_step_v00.7z
https://archive.nyu.edu/retrieve/88711/abc_0021_step_v00.7z
https://archive.nyu.edu/retrieve/88715/abc_0022_step_v00.7z
https://archive.nyu.edu/retrieve/88719/abc_0023_step_v00.7z
https://archive.nyu.edu/retrieve/88723/abc_0024_step_v00.7z
https://archive.nyu.edu/retrieve/88727/abc_0025_step_v00.7z
https://archive.nyu.edu/retrieve/88731/abc_0026_step_v00.7z
https://archive.nyu.edu/retrieve/88735/abc_0027_step_v00.7z
https://archive.nyu.edu/retrieve/88739/abc_0028_step_v00.7z
https://archive.nyu.edu/retrieve/88743/abc_0029_step_v00.7z
https://archive.nyu.edu/retrieve/88747/abc_0030_step_v00.7z
https://archive.nyu.edu/retrieve/88751/abc_0031_step_v00.7z
https://archive.nyu.edu/retrieve/88755/abc_0032_step_v00.7z
https://archive.nyu.edu/retrieve/88759/abc_0033_step_v00.7z
https://archive.nyu.edu/retrieve/88763/abc_0034_step_v00.7z
https://archive.nyu.edu/retrieve/88767/abc_0035_step_v00.7z
https://archive.nyu.edu/retrieve/88771/abc_0036_step_v00.7z
https://archive.nyu.edu/retrieve/88775/abc_0037_step_v00.7z
https://archive.nyu.edu/retrieve/88779/abc_0038_step_v00.7z
https://archive.nyu.edu/retrieve/88783/abc_0039_step_v00.7z
https://archive.nyu.edu/retrieve/88787/abc_0040_step_v00.7z
https://archive.nyu.edu/retrieve/88791/abc_0041_step_v00.7z
https://archive.nyu.edu/retrieve/88795/abc_0042_step_v00.7z
https://archive.nyu.edu/retrieve/88799/abc_0043_step_v00.7z
https://archive.nyu.edu/retrieve/88803/abc_0044_step_v00.7z
https://archive.nyu.edu/retrieve/88807/abc_0045_step_v00.7z
https://archive.nyu.edu/retrieve/88811/abc_0046_step_v00.7z
https://archive.nyu.edu/retrieve/88816/abc_0047_step_v00.7z
https://archive.nyu.edu/retrieve/88820/abc_0048_step_v00.7z
https://archive.nyu.edu/retrieve/88824/abc_0049_step_v00.7z
https://archive.nyu.edu/retrieve/88830/abc_0050_step_v00.7z
https://archive.nyu.edu/retrieve/88834/abc_0051_step_v00.7z
https://archive.nyu.edu/retrieve/88838/abc_0052_step_v00.7z
https://archive.nyu.edu/retrieve/88842/abc_0053_step_v00.7z
https://archive.nyu.edu/retrieve/88846/abc_0054_step_v00.7z
https://archive.nyu.edu/retrieve/88850/abc_0055_step_v00.7z
https://archive.nyu.edu/retrieve/88854/abc_0056_step_v00.7z
https://archive.nyu.edu/retrieve/88858/abc_0057_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44376/6/abc_0058_step_v00.7z
https://archive.nyu.edu/retrieve/88873/abc_0059_step_v00.7z
https://archive.nyu.edu/retrieve/88877/abc_0060_step_v00.7z
https://archive.nyu.edu/retrieve/88881/abc_0061_step_v00.7z
https://archive.nyu.edu/retrieve/88885/abc_0062_step_v00.7z
https://archive.nyu.edu/retrieve/88889/abc_0063_step_v00.7z
https://archive.nyu.edu/retrieve/88893/abc_0064_step_v00.7z
https://archive.nyu.edu/retrieve/88897/abc_0065_step_v00.7z
https://archive.nyu.edu/retrieve/88901/abc_0066_step_v00.7z
https://archive.nyu.edu/retrieve/88905/abc_0067_step_v00.7z
https://archive.nyu.edu/retrieve/88909/abc_0068_step_v00.7z
https://archive.nyu.edu/retrieve/88913/abc_0069_step_v00.7z
https://archive.nyu.edu/retrieve/88917/abc_0070_step_v00.7z
https://archive.nyu.edu/retrieve/88921/abc_0071_step_v00.7z
https://archive.nyu.edu/retrieve/88925/abc_0072_step_v00.7z
https://archive.nyu.edu/retrieve/88929/abc_0073_step_v00.7z
https://archive.nyu.edu/retrieve/88933/abc_0074_step_v00.7z
https://archive.nyu.edu/retrieve/88937/abc_0075_step_v00.7z
https://archive.nyu.edu/retrieve/88941/abc_0076_step_v00.7z
https://archive.nyu.edu/retrieve/88945/abc_0077_step_v00.7z
https://archive.nyu.edu/retrieve/88986/abc_0078_step_v00.7z
https://archive.nyu.edu/retrieve/88990/abc_0079_step_v00.7z
https://archive.nyu.edu/retrieve/88994/abc_0080_step_v00.7z
https://archive.nyu.edu/retrieve/88998/abc_0081_step_v00.7z
https://archive.nyu.edu/retrieve/89002/abc_0082_step_v00.7z
https://archive.nyu.edu/retrieve/89006/abc_0083_step_v00.7z
https://archive.nyu.edu/retrieve/89010/abc_0084_step_v00.7z
https://archive.nyu.edu/retrieve/89014/abc_0085_step_v00.7z
https://archive.nyu.edu/retrieve/89018/abc_0086_step_v00.7z
https://archive.nyu.edu/retrieve/89022/abc_0087_step_v00.7z
https://archive.nyu.edu/retrieve/89026/abc_0088_step_v00.7z
https://archive.nyu.edu/bitstream/2451/44417/3/abc_0089_step_v00.7z
https://archive.nyu.edu/retrieve/88950/abc_0090_step_v00.7z
https://archive.nyu.edu/retrieve/88954/abc_0091_step_v00.7z
https://archive.nyu.edu/retrieve/88958/abc_0092_step_v00.7z
https://archive.nyu.edu/retrieve/88962/abc_0093_step_v00.7z
https://archive.nyu.edu/retrieve/88966/abc_0094_step_v00.7z
https://archive.nyu.edu/retrieve/88970/abc_0095_step_v00.7z
https://archive.nyu.edu/retrieve/88974/abc_0096_step_v00.7z
https://archive.nyu.edu/retrieve/88978/abc_0097_step_v00.7z
https://archive.nyu.edu/retrieve/88982/abc_0098_step_v00.7z
https://archive.nyu.edu/retrieve/89034/abc_0099_step_v00.7z
@chunibyo-wly
Copy link
Author

chunibyo-wly commented Jul 31, 2025

vibe coding

import requests
from bs4 import BeautifulSoup

links = [
    "https://archive.nyu.edu/handle/2451/44309?mode=full",
    "https://archive.nyu.edu/handle/2451/44310?mode=full",
    "https://archive.nyu.edu/handle/2451/44318?mode=full",
    "https://archive.nyu.edu/handle/2451/44319?mode=full",
    "https://archive.nyu.edu/handle/2451/44320?mode=full",
    "https://archive.nyu.edu/handle/2451/44321?mode=full",
    "https://archive.nyu.edu/handle/2451/44322?mode=full",
    "https://archive.nyu.edu/handle/2451/44323?mode=full",
    "https://archive.nyu.edu/handle/2451/44324?mode=full",
    "https://archive.nyu.edu/handle/2451/44325?mode=full",
    "https://archive.nyu.edu/handle/2451/44326?mode=full",
    "https://archive.nyu.edu/handle/2451/44327?mode=full",
    "https://archive.nyu.edu/handle/2451/44328?mode=full",
    "https://archive.nyu.edu/handle/2451/44329?mode=full",
    "https://archive.nyu.edu/handle/2451/44330?mode=full",
    "https://archive.nyu.edu/handle/2451/44331?mode=full",
    "https://archive.nyu.edu/handle/2451/44332?mode=full",
    "https://archive.nyu.edu/handle/2451/44333?mode=full",
    "https://archive.nyu.edu/handle/2451/44334?mode=full",
    "https://archive.nyu.edu/handle/2451/44335?mode=full",
    "https://archive.nyu.edu/handle/2451/44337?mode=full",
    "https://archive.nyu.edu/handle/2451/44338?mode=full",
    "https://archive.nyu.edu/handle/2451/44339?mode=full",
    "https://archive.nyu.edu/handle/2451/44340?mode=full",
    "https://archive.nyu.edu/handle/2451/44341?mode=full",
    "https://archive.nyu.edu/handle/2451/44342?mode=full",
    "https://archive.nyu.edu/handle/2451/44343?mode=full",
    "https://archive.nyu.edu/handle/2451/44344?mode=full",
    "https://archive.nyu.edu/handle/2451/44345?mode=full",
    "https://archive.nyu.edu/handle/2451/44346?mode=full",
    "https://archive.nyu.edu/handle/2451/44347?mode=full",
    "https://archive.nyu.edu/handle/2451/44348?mode=full",
    "https://archive.nyu.edu/handle/2451/44349?mode=full",
    "https://archive.nyu.edu/handle/2451/44350?mode=full",
    "https://archive.nyu.edu/handle/2451/44351?mode=full",
    "https://archive.nyu.edu/handle/2451/44352?mode=full",
    "https://archive.nyu.edu/handle/2451/44353?mode=full",
    "https://archive.nyu.edu/handle/2451/44354?mode=full",
    "https://archive.nyu.edu/handle/2451/44355?mode=full",
    "https://archive.nyu.edu/handle/2451/44356?mode=full",
    "https://archive.nyu.edu/handle/2451/44357?mode=full",
    "https://archive.nyu.edu/handle/2451/44358?mode=full",
    "https://archive.nyu.edu/handle/2451/44359?mode=full",
    "https://archive.nyu.edu/handle/2451/44360?mode=full",
    "https://archive.nyu.edu/handle/2451/44361?mode=full",
    "https://archive.nyu.edu/handle/2451/44362?mode=full",
    "https://archive.nyu.edu/handle/2451/44363?mode=full",
    "https://archive.nyu.edu/handle/2451/44365?mode=full",
    "https://archive.nyu.edu/handle/2451/44366?mode=full",
    "https://archive.nyu.edu/handle/2451/44367?mode=full",
    "https://archive.nyu.edu/handle/2451/44368?mode=full",
    "https://archive.nyu.edu/handle/2451/44369?mode=full",
    "https://archive.nyu.edu/handle/2451/44370?mode=full",
    "https://archive.nyu.edu/handle/2451/44371?mode=full",
    "https://archive.nyu.edu/handle/2451/44372?mode=full",
    "https://archive.nyu.edu/handle/2451/44373?mode=full",
    "https://archive.nyu.edu/handle/2451/44374?mode=full",
    "https://archive.nyu.edu/handle/2451/44375?mode=full",
    "https://archive.nyu.edu/handle/2451/44376?mode=full",
    "https://archive.nyu.edu/handle/2451/44378?mode=full",
    "https://archive.nyu.edu/handle/2451/44379?mode=full",
    "https://archive.nyu.edu/handle/2451/44380?mode=full",
    "https://archive.nyu.edu/handle/2451/44381?mode=full",
    "https://archive.nyu.edu/handle/2451/44382?mode=full",
    "https://archive.nyu.edu/handle/2451/44383?mode=full",
    "https://archive.nyu.edu/handle/2451/44384?mode=full",
    "https://archive.nyu.edu/handle/2451/44385?mode=full",
    "https://archive.nyu.edu/handle/2451/44386?mode=full",
    "https://archive.nyu.edu/handle/2451/44387?mode=full",
    "https://archive.nyu.edu/handle/2451/44388?mode=full",
    "https://archive.nyu.edu/handle/2451/44389?mode=full",
    "https://archive.nyu.edu/handle/2451/44390?mode=full",
    "https://archive.nyu.edu/handle/2451/44391?mode=full",
    "https://archive.nyu.edu/handle/2451/44392?mode=full",
    "https://archive.nyu.edu/handle/2451/44393?mode=full",
    "https://archive.nyu.edu/handle/2451/44394?mode=full",
    "https://archive.nyu.edu/handle/2451/44395?mode=full",
    "https://archive.nyu.edu/handle/2451/44396?mode=full",
    "https://archive.nyu.edu/handle/2451/44406?mode=full",
    "https://archive.nyu.edu/handle/2451/44407?mode=full",
    "https://archive.nyu.edu/handle/2451/44408?mode=full",
    "https://archive.nyu.edu/handle/2451/44409?mode=full",
    "https://archive.nyu.edu/handle/2451/44410?mode=full",
    "https://archive.nyu.edu/handle/2451/44411?mode=full",
    "https://archive.nyu.edu/handle/2451/44412?mode=full",
    "https://archive.nyu.edu/handle/2451/44413?mode=full",
    "https://archive.nyu.edu/handle/2451/44414?mode=full",
    "https://archive.nyu.edu/handle/2451/44415?mode=full",
    "https://archive.nyu.edu/handle/2451/44416?mode=full",
    "https://archive.nyu.edu/handle/2451/44417?mode=full",
    "https://archive.nyu.edu/handle/2451/44397?mode=full",
    "https://archive.nyu.edu/handle/2451/44398?mode=full",
    "https://archive.nyu.edu/handle/2451/44399?mode=full",
    "https://archive.nyu.edu/handle/2451/44400?mode=full",
    "https://archive.nyu.edu/handle/2451/44401?mode=full",
    "https://archive.nyu.edu/handle/2451/44402?mode=full",
    "https://archive.nyu.edu/handle/2451/44403?mode=full",
    "https://archive.nyu.edu/handle/2451/44404?mode=full",
    "https://archive.nyu.edu/handle/2451/44405?mode=full",
    "https://archive.nyu.edu/handle/2451/44418?mode=full",
]


def main():
    for _idx, link in enumerate(links):
        try:
            response = requests.get(link)
            if response.status_code == 200:
                content = BeautifulSoup(response.content, "html.parser")
                sublinks = content.find_all("a", href=True)
                for sublink in sublinks:
                    sublink_url = sublink["href"]
                    if "step" in sublink.text.strip():
                        with open("abcdataseturl.txt", "a") as f:
                            f.write("https://archive.nyu.edu" + sublink_url + "\n")
                        print(_idx)
                        break
            else:
                raise requests.exceptions.HTTPError(f"Failed to access {link}: Status code {response.status_code}")
        except requests.exceptions.RequestException as e:
            exit(f"Error accessing {link}: {e}")


if __name__ == "__main__":
    main()

@chunibyo-wly
Copy link
Author

chunibyo-wly commented Jul 31, 2025

https://archive.nyu.edu/handle/2451/43778?rpp=100&value=1&data-order=asc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment