Latest Posts

parselcli - css and xpath selectors in your terminal!

I'd like to introduce parselcli an command line application for evaluating css and xpath selectors in your terminal.

Scrapy: How to loop a crawler

Explore ways to properly loop scrapy crawlers with twisted callbacks.

Why Do We Crawl The Web?

Why do we crawl websites? What projects are being born from gathered public data and how can that help a business or society as a whole?