Hazelcast Jet v4.3 Release Notes

Release Date: 2020-10-20 // about 1 year ago
    • @MohamedMandouh implemented distributed sorting: BatchStage.sort().
    • @guenter-hesse added a high-throughput source with precisely controlled
      ๐Ÿšค timing, to be used to benchmark Jet's throughput and latency.
    • ๐Ÿ‘ท We added the ability for Jet to suspend an unbounded, fault-tolerant job
      ๐Ÿ‘ป when it fails due to an exception in the pipeline code. This allows you
      ๐Ÿšš to diagnose the problem, remove its cause, and resume the pipeline without
      data loss.
    • We have improved the consistency of reconnect behaviour across CDC sources and
      ๐Ÿ”ง introduced a uniform API to configure the reconnect strategy.
    • We improved the behavior of streaming pipelines based on ingestion time: a
      low event rate in any partition of the source used to stall the progress,
      now the watermark advances even without events.
    • ๐Ÿšš We moved the Pipeline to DAG transformation from the client to the server,
      in preparation for the upcoming feature of preventing event reordering in the
      pipeline.

Previous changes from v4.2

    • ๐Ÿ‘ We added Change Data Capture (CDC) support for PostgreSQL and MySql. New
      connectors use Debezium to provide a high-level API for processing the
      stream of changes from relational databases. A new map sink can be used to
      ๐Ÿ”€ automatically keep the map in-sync with changes from the database.
    • We added new source and sink connectors for Elasticsearch 5, 6 and 7
      ๐ŸŽ with support for slicing and co-located reading for maximum read performance
      and extended Elasticsearch's client retry mechanism for reliable writing.
    • ๐Ÿ‘ We improved JSON support through a new file source which can read JSON files
      ๐Ÿ“œ directly and also added built-in Jet methods for parsing and mapping JSON strings.
    • We introduced .relabance() operator which can be used for distributing items
      across the cluster.
    • We made some improvements to Jet startup scripts:
      • You can now use the PROMETHEUS_PORT to start Jet with prometheus metrics enabled.
      • You can use the jvm.options file to edit the startup JVM parameters.
      • You can now use the JET_MODULES environment variable to add modules when using docker
        or jet-start script.
    • ๐Ÿ“š We did several rounds of improvements to the documentation:
      • Improvements to Docker and Kubernetes instructions
      • New guidance on Garbage Collector choice and settings
      • New and improved tutorials for CDC
      • New documentation for Python module and early results support in windowing