Java Web Crawling

5 Web Crawling libraries and projects

• Sparkler

  4.8 3.0 Java

  Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
  

• Apache Nutch

  8.1 8.0 L2 Java

  Apache Nutch is an extensible and scalable web crawler
  

InfluxDB - Power Real-Time Data Analytics at Scale

Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

Promo www.influxdata.com



• storm-crawler

  5.9 8.7 HTML

  A scalable, mature and versatile web crawler based on Apache Storm
  

• jsoup

  9.2 9.1 L2 Java

  jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

• Crawler4j

  8.7 0.0 L2 Java

  Open Source Web Crawler for Java

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

WorkOS - The modern identity platform for B2B SaaS sponsored workos.com

Popular Comparisons

• jsoup
  vs
  Apache Nutch
• Crawler4j
  vs
  Apache Nutch
• Crawler4j
  vs
  jsoup
• Apache Nutch
  vs
  storm-crawler
• storm-crawler
  vs
  jsoup

SaaSHub - Software Alternatives and Reviews sponsored www.saashub.com