Java Web Crawling

5 Web Crawling libraries and projects

• Sparkler

  4.7 7.3 Java

  Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
  

• Apache Nutch

  8.1 7.3 L2 Java

  Apache Nutch is an extensible and scalable web crawler
  

Run Linux Software Faster and Safer than Linux with Unikernels

Promo github.com/nanovms



• storm-crawler

  5.7 8.0 Java

  A scalable, mature and versatile web crawler based on Apache Storm
  

• jsoup

  9.1 9.2 L2 Java

  jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

• Crawler4j

  8.7 0.0 L2 Java

  Open Source Web Crawler for Java

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

Scout APM: A developer's best friend. Try free for 14-days sponsored scoutapm.com

Popular Comparisons

• Apache Nutch
  vs
  jsoup
• Apache Nutch
  vs
  Crawler4j
• Crawler4j
  vs
  jsoup
• storm-crawler
  vs
  Apache Nutch
• storm-crawler
  vs
  Crawler4j

SaaSHub - Software Alternatives and Reviews sponsored www.saashub.com