The Web Scraping Club

The Web Scraping Club

  • The newsletterOur substack with more articles
  • Tools
    • Cloudflare Turnstile Tester
  • Offers
  • About
  • Join our Discord
  • Bypassing CAPTCHA with AI: Is the game over?

    Bypassing CAPTCHA with AI: Is the game over?

    Are CAPTCHAs still a threat of web scrapers or nowadays they can be solved with AI? Let’s see in this article how AI is helping to bypass CAPTCHAs

    Read More

    September 13, 2023

  • Cloudflare Turnstile: What It Is & How It Affects Scraping

    Cloudflare Turnstile: What It Is & How It Affects Scraping

    On September 2022 Cloudflare announced its new service, called Turnstile, a “No Captcha” Captcha. Let’s see how it works and see if your scrapers can bypass it.

    Read More

    August 27, 2023

  • THE LAB #25: How to Freely Bypass PerimeterX in 2023: A Comprehensive Guide

    THE LAB #25: How to Freely Bypass PerimeterX in 2023: A Comprehensive Guide

    How to bypass PerimeterX anti-bot solution using both free and commercial solutions, explained in the latest episode of The Lab

    Read More

    August 27, 2023

  • Mastering the Art of Scraping Robots.txt & Sitemaps

    Mastering the Art of Scraping Robots.txt & Sitemaps

    Why Robots file and XML Sitemaps are important for web scraping? They’re used to index data in the web, used by search engines but also for web scraping.

    Read More

    August 27, 2023

  • Hands On #5: Oxylabs Web Unblocker – In-Depth Testing and Expert Review

    Hands On #5: Oxylabs Web Unblocker – In-Depth Testing and Expert Review

    Testing the new Oxylabs web Unblocker and all its features in this product review by The Web Scraping Club. All in one solution for bypass the most common antibot.

    Read More

    August 21, 2023

  • A Step-by-Step Beginner’s Guide: Writing Your First Scraper with Scrapy

    A Step-by-Step Beginner’s Guide: Writing Your First Scraper with Scrapy

    Write your first scraper with Scrapy with this tutorial. Bonus content: adding a rotating proxy with a simple add on.

    Read More

    August 17, 2023

  • July 2023: Monthly Web Scraping News Digest

    July 2023: Monthly Web Scraping News Digest

    In this usual end of month post we’ll see what’s happening in the web scraping industry in July 2023

    Read More

    August 17, 2023

  • Understanding Geofencing: Implications for Web Scraping

    Understanding Geofencing: Implications for Web Scraping

    The summer holiday season started and, on Tik Tok, Youtube, and other media, just like every year, there’s an article or a video regarding tricks on how to save money when booking a flight bypassing the websites’ geofencing. Common sense apart, like booking earlier and trying different days to see what’s the cheapest combination of…

    Read More

    August 3, 2023

  • How to Bypass Anti-Bot Systems with Undetected Chromedriver 3.5

    How to Bypass Anti-Bot Systems with Undetected Chromedriver 3.5

    Is the new version of undetected-chromedriver capable to bypass Cloudflare and other antibots? We tested it in this article

    Read More

    August 3, 2023

  • Mastering TikTok Scraping: Methods & Best Practices

    Mastering TikTok Scraping: Methods & Best Practices

    Tik Tok Scraping is challenging but it’s a huge source of data for finding trends and topics. With a good parner, its scraping can be easily done.

    Read More

    July 18, 2023

1 2 3 … 11
Next Page→

The Web Scraping Club