Skip to content

Instantly share code, notes, and snippets.

@siddacool
Created November 4, 2020 14:40
Show Gist options
  • Select an option

  • Save siddacool/da284adb7474c7646f1af514d00e194f to your computer and use it in GitHub Desktop.

Select an option

Save siddacool/da284adb7474c7646f1af514d00e194f to your computer and use it in GitHub Desktop.
deno-web-scraper: Bringing it all together
import { DOMParser } from 'https://deno.land/x/deno_dom/deno-dom-wasm.ts';
const url = 'http://books.toscrape.com/';
try {
const res = await fetch(url);
const html = await res.text();
const doc: any = new DOMParser().parseFromString(html, 'text/html');
const books: any = [];
const productsPods = doc.querySelectorAll('.product_pod');
productsPods.forEach((product: any) => {
const title = product.querySelector('h3').querySelector('a').getAttribute('title');
const price = product.querySelector('.price_color').textContent;
const availability = product.querySelector('.availability').textContent.trim();
books.push({
title,
price,
availability,
})
});
console.log(books);
} catch(error) {
console.log(error);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment