Apache Pulsar v2.7.0 Release Notes

Release Date: 2020-11-30 // 11 months ago

Previous changes from v2.6.2

  • ๐Ÿš€ The following lists fixes and enhancements in the 2.6.2 release.

    Broker

    [Broker] Catch throwable when start pulsar 7221
    โšก๏ธ [Broker] Protobuf-shaded package can not update version 7228
    [Broker] Check for null arguments in Namespaces Rest API 7247
    [Broker] Handle SubscriptionBusyException in resetCursor api 7335
    [Broker] Converted the namespace bundle unload into async operation 7364
    โšก๏ธ [Broker] Update Jersey to 2.31 7515
    [Broker] Stop to dispatch when skip message temporally since Key_Shared consumer stuck on delivery 7553
    [Broker] Fix bug where producer for geo-replication is not closed when topic is unloaded 7735
    [Broker] Make resetting cursor in REST API asynchronous 7744
    [Broker] Reestablish namespace bundle ownership from false negative releasing and false positive acquiring 7773
    ๐Ÿ”ง [Broker] make pulsar executor pool size configurable 7782
    [Broker] Redirect Get message by id request when broker not serve for the topic 7786
    ๐Ÿ”ง [Broker] Make zk cache executor thread pool size configurable 7794
    [Broker] Implement toString() method for TopicMessageIdImpl class 7807
    [Broker] Fix pending batchIndexAcks bitSet batchSize in PersistentAcknowledgmentsGroupingTracker 7828
    [Broker] Fix deadlock when adding consumer 7841
    [Broker] Split message ranges by ledger ID and store them in individualDeletedMessages 7861
    [Broker] Fix pulsar metrics providing wrong information 7905
    [Broker] Don't fail the health check request when trying to delete the previous subscription 7906
    ๐Ÿ”ง [Broker] Add configuration to set number of channels per bookie 7910
    [Broker] Fix publish buffer of one ServerCnx calculated multi-times when check broker's publish buffer 7926
    โœ๏ธ [Broker] Add some logging to improve Authentication debugging and Fix typos in code "occured" -> "occurred" 7934
    [Broker] Fix NPE when acknowledge messages at the broker side 7937
    [Broker] Fix the wrong issuer url concatenate 7980
    โฌ†๏ธ [Broker] Upgrade the snakeyaml version to 1.26 7994
    ๐Ÿ“ฆ [Broker] Exclude vertx from bookkeeper-http package 7997
    [Broker] Check null point before setting auto read 7999
    [Broker] Fix IndexOutOfBoundsException in the KeyShared subscription when dispatching messages to consumers 8024
    โฌ†๏ธ [Broker] Upgrade jetty-util version to 9.4.31 8035
    [Broker] Add replicated check to checkInactiveSubscriptions 8066
    [Broker] Add get-last-message-id admin for v1 api 8081
    [Broker] Fix client lookup hangs when broker restarts 8101
    [Broker] Should not cache the owner that does not belong to current server 8111
    ๐Ÿ‘ [Broker] Support to specify multi ipv6 hosts in brokerServiceUrl 8120
    ๐Ÿ‘ป [Broker] Intercept messages to consumers and add intercept exception 8129
    [Broker] Add ChannelFutures utility class to pulsar-common 8137
    ๐Ÿ‘ [Broker] Support Disable Replicated Subscriptions 8144
    [Broker] Fix error code returned to client when service unit is not ready 8147
    [Broker] Skip intercepting multipart requests 8156
    ๐Ÿ”ง [Broker] Enable intercept filters only when interceptors are configured 8157
    [Broker] Clean inactive non-persistent subscriptions 8166
    [Broker] Add a new state for namespace-level TTL 8178
    [Broker] Fix peek messages failed with subscriptionName not exist 8182
    ๐Ÿ‘ป [Broker] Fix pulsar service close exception 8197
    [Broker] Use ThreadPoolExecutor instead of EventLoop 8208
    ๐Ÿ“‡ [Broker] Close ZK connections at end of metadata setup 8228
    ๐Ÿ“‡ [Broker] Delete associated ledgers before deleting cluster metadata 8244
    [Broker] Fix stuck lookup operations when the broker is starting up 8273
    [Broker] Fix Broker enters an infinite loop in ManagedLedgerImpl.asyncReadEntries 8284
    [Broker] Fix message TTL on Key_Shared subscription and Fix ordering issue when replay messages 8292
    โšก๏ธ [Broker] Fix race condition in updating readPosition in ManagedCursorImpl 8299
    [Broker] Refresh ZooKeeper-data cache in background to avoid deadlock and blocking IO on ZK thread 8304
    โฌ†๏ธ [Broker] Upgrade hdfs2 version to 2.8.5 8319
    โฌ†๏ธ [Broker] Upgrade solr version to 8.6.3 8328
    [Broker] Fix deadlock that occurred during topic ownership check 8406

    Proxy

    [Proxy] Add advertisedAddress config field to ProxyConfiguration 7542
    [Proxy] Fix deadlock in pulsar proxy 7690
    โšก๏ธ [Proxy] Handle NPE while updating proxy stats 7766
    ๐Ÿ‘ป [Proxy] Fix the null exception when starting the proxy service 8019
    ๐Ÿ”Œ [Proxy] Add proxy plugin interface to support user defined additional servlet 8067

    Pulsar SQL

    โฌ†๏ธ [Pulsar SQL] Upgrade Presto version to 332 7194
    [Pulsar SQL] Replace com.ning.asynchttpclient with org.asynchttpclient 8099

    Java Client

    ๐Ÿ‘ [Java Client] Support input-stream for trustStore cert 7442
    [Java Client] Avoid subscribing the same topic again 7823
    โšก๏ธ [java Client] Add autoPartitionsUpdateInterval for producer and consumer 7840
    [Java Client] Avoid resolving address for sni-host + thread-safe connection creation 8062
    [Java Client] Websocket interface decode URL encoding 8072
    [Java Client] Always use SNI for TLS enabled Pulsar Java broker client 8117
    โฑ [Java Client] Improve timeout handling in ClientCnx to cover all remaining request types (GetLastMessageId, GetTopics, GetSchema, GetOrCreateSchema) 8149
    [Java Client] Fix ConsumerImpl memory leaks 8160
    [Java Client] Fix issue where paused consumer receives new message when reconnecting 8165
    โ™ป๏ธ [Java Client] Improve refactored client connection code 8177
    ๐Ÿ”ง [Java Client] Add log level configuration in pulsar-client 8195
    ๐Ÿšš [Java Client] Remove unnecessary locks 8207
    โšก๏ธ [Java Client] Fix AutoUpdatePartitionsInterval setting problem 8227
    [Java Client] Add read position when joining in the consumer stats 8274
    ๐Ÿ‘ [Java Client] Support reset cursor to a batch index of the batching message 8285
    ๐Ÿ‘ [Java Client] Support exclude the message when reset cursor by message ID 8306
    โฑ [Java Client] Increasing timeout for pulsar client io threads to shutdown 8316
    ๐Ÿ‘ [Java Client] Support cancelling message & batch futures returned from Reader & Consumer 8326
    [Java Client] Disable batch receive timer for Readers 8381
    [Java Client] Fix pause does not work for new created consumer 8387

    CPP Client

    ๐Ÿ‘€ [CPP Client] Wait for all seek operations completed 7216
    [CPP Client] Ensure parallel invocations of MultiTopicsConsumerImpl::subscribeAsync with the same topic name do not produce an error. 7691
    ๐Ÿ‘ป [CPP Client] Throw std::exception types 7798
    [CPP Client] Make clear() thread-safe 7862
    ๐Ÿ‘ [CPP Client] Support key based batching 7996
    ๐Ÿ”ง [CPP Client] The token endpoint should get from the well-known configuration 8006
    ๐Ÿ— [CPP Client] Add Snappy library to Docker images for building C++ packages 8086
    โฑ [CPP Client] Add epoch for C++ client HandleBase to handle create producer timeout 8191
    [CPP Client] Fix some pending requests may never complete when broker's down 8232
    [CPP Client] Client is allocating buffer bigger than needed 8283
    [CPP Client] Client back-pressure is done on batches rather than number of messages 8331
    [CPP Client] Fix message id error when subscribing a single partition 8341

    Python Client

    0๏ธโƒฃ [Python Client] Add python schema field default value 8122
    [Python Client] Expose schema version (of writerSchema) in Message 8173

    Pulsar Functions

    โšก๏ธ [Pulsar Functions] During Function update, cleanup should only happen for temp files that were generated 7201
    ๐Ÿ“‡ [Pulsar Functions] Have metadata tailer use its own thread for processing 7211
    [Pulsar Functions] Allow kubernetes runtime to customize function instance class path 7844
    [Pulsar Functions] SinkRecord adds an overridden method 8038
    [Pulsar Functions] Set dryrun of KubernetesRuntime is null 8064
    [Pulsar Functions] Allow disabling forwarding source message properties 8158
    ๐Ÿšง [Pulsar Functions] Missed dryRun on maintenance of secrets 8286

    Pulsar Perf

    ๐Ÿ‘ [Pulsar Perf] Support setting message key 7989
    ๐Ÿ”ง [Pulsar Perf] Make pulsar-perf ioThread number configurable 8090

    Pulsar Admin

    ๐Ÿ‘ [Pulsar Admin] Support initial namespace of the cluster without startup the broker 7434
    [Pulsar Admin] Fix some params on consumer broken by #4400 (regex, initialSouscriptionPosition) 7795
    [Pulsar Admin] Return more informative error message when trying to create subscription on non-persistent through Rest API or pulsar-admin CLI 7831
    [Pulsar Admin] Add cli command to get last message Id 8082
    ๐Ÿ‘ [Pulsar Admin] Support delete all data associated with a cluster 8133
    ๐Ÿ‘ [Pulsar Admin] Support delete schema ledgers when delete topics 8167
    ๐Ÿ“‡ [Pulsar Admin] Add command to delete a cluster's metadata from ZK 8169
    ๐Ÿ‘ [Pulsar Admin] Support reset cursor to a batch index for Pulsar Admin 8329

    Tiered Storage

    โ™ป๏ธ [Tiered Storage] Refactored JCloud Tiered Storage 6335
    โšก๏ธ [Tiered Storage] Remove duplicate updates 8198
    [Tiered Storage] Make the field name in OffloadPolicies match with config file 8310