Java Web Crawling

5 Web Crawling libraries and projects

• jsoup

  9.2 9.4 L2 Java

  jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

• Crawler4j

  8.7 0.0 L2 Java

  Open Source Web Crawler for Java

TestGPT | Generating meaningful tests for busy devs

Get non-trivial tests (and trivial, too!) suggested right inside your IDE, so you can code smart, create more value, and stay confident when you push.

Promo codium.ai



• Apache Nutch

  8.0 4.6 L2 Java

  Apache Nutch is an extensible and scalable web crawler
  

• storm-crawler

  5.7 6.3 HTML

  A scalable, mature and versatile web crawler based on Apache Storm
  

• Sparkler

  4.7 0.0 Java

  Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
  

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

ONLYOFFICE Docs — document collaboration in your environment sponsored www.onlyoffice.com

Popular Comparisons

• jsoup
  vs
  Apache Nutch
• Crawler4j
  vs
  Apache Nutch
• Crawler4j
  vs
  jsoup
• Apache Nutch
  vs
  storm-crawler
• storm-crawler
  vs
  jsoup

Access the most powerful time series database as a service sponsored www.influxdata.com