Web Scraping

The list of tools, programming libraries and APIs used in web-scraping.

• Python

• PHP

• Ruby

• JavaScript

• Golang

  • Feel free to add your favourite language. Use new_language_template.md as start point.

• Proxy Services

• Web Services

• Console tools

• Books

Other Awesome List Projects

• lists - List of useful, silly and awesome lists curated on GitHub

• HeadlessBrowsers - a list of (almost) all headless web browsers in existence

Contributing

Make this list better! Your contributions are always welcome! See contributing how-to

Credits

This list partially contains data from these sources:

• awesome-python by vinta / CC BY 4.0
• awesome-php by ziadoz
• awesome-ruby by markets
• ruby-nlp by diasks2
• awesome-javascript by sorrycc