Concourse v5.4.1 Release Notes

Release Date: 2019-08-07 // 11 days ago
  • This release is about fixing some unintended consequences of switching to Zstandard compression in v5.4.0. More specifically, having our Go code depend on C libraries caused a few failure modes.

    🍱 🔗 fix

    • 🍎 When we first switched to using cgo to build fly (which was necessary in order to make use of Zstandard compression), unbeknownst to us, fly started to depend on a system library for DNS resolution. This meant that on systems that lacked this library (macOS in particular), fly would segfault. Furthermore, the system library has different features than the native Go version, so even on systems that had the library, fly would fail to resolve certain names. We fixed these problems by building fly in a way that would still use cgo but depend on the native Go DNS resolver concourse/ci#59.

    🍱 🔗 fix

    • On the server side, it turns out we had been a bit lax in freeing resources associated with volume streams, but because we had been using pure Go streaming and compression code, the Go garbage collector had helped us out. In switching to a C-based compression library, failing to manually release memory via Go APIs meant that the underlying memory allocated by the C code was never freed, and several folks have observed server-side memory leaks when using v5.4.0. We plugged those holes #4182, #4199.

Previous changes from v5.4.0

  • known Issues

    🚀 > We've observed a memory leak with this release. Please avoid upgrading to this release if you are using a multi-tenant concourse with heavy load. You can follow along with our investigation in #4150.

    🐧 > fly may segfault on execution when running on certain versions of Linux. Workaround steps can be found here: #4141 (comment)

    🍱 🔗 feature, breaking

    • We have switched to a more efficient algorithm for streaming volumes between various components of the system - the web and worker nodes now transmit volumes compressed with the Zstandard algorithm rather than Gzip - fly is affected, too (when uploading/downloading artifacts during fly execute). We're hoping this will resolve some general complaints of steps being slow to start or appearing to "hang" in clustered environments. note: this will require upgrading your workers #3880.

    🍱 🔗 feature

    • 🏗 Because of the efficiency gained from employing Zstandard compression to volume streaming, the CPU on the web and worker nodes is no longer a limiting factor. Now volumes will be streamed over multiple parallel connections, and we have seen some very exciting improvements in the time taken to initialize build steps. You can see some of our measurements in the discussion on this issue #3993.

    🍱 🔗 feature, breaking

    • 🏗 On the build page all steps are now collapsed by default. This means that the build page loads faster, but if you want to watch the logs in real time, you will need to click on the header of the running step #4058.

    🍱 🔗 feature

    • Improved the appearance of the sidebar (re-introduced in v5.2.0) in response to user feedback. Teams and pipelines are now easier to distinguish, and there are nicely styled tooltips for long items #4032.

    🍱 🔗 feature

    🍱 🔗 feature

    • 🏗 When you manually trigger a build through the web UI, the 'prep' section at the top of the build page now shows a spinner on the entries marked "discovering new versions of..." while the resource checks are in progress. As the checks resolve, the spinners will become checkmarks, so you'll have a better sense of what might be holding up your build #3970.

    🍱 🔗 feature

    🍱 🔗 feature

    • ⏱ When you abort a pending build before it gets scheduled, the build page will now report it as 'cancelled' instead of showing a loading spinner forever #3922.

    🍱 🔗 feature

    • ⚡️ The CSS optimizer used in building the web UI has been upgraded, so the resulting CSS is slightly smaller, which should result in some faster page loads #4004.

    🍱 🔗 fix

    🍱 🔗 fix

    • 📇 @robwhitby fixed an issue where re-checking a resource would cause the metadata from its latest version to be erased #4028.

    🍱 🔗 fix

    • 🖨 @cappyzawa fixed up fly to behave like a sensible Unix tool when passing the -h or --help flag: it now prints the usage text on stdout and returns a successful status code #4064.

    🍱 🔗 fix

    🍱 🔗 fix

    • 🛠 Fixed a regression found recently involving momentum based scrolling on build pages on iOS browsers The fix was also applied to the sidebar and the dashboard #4050

    🍱 🔗 fix

    • 🚀 The BOSH Backup and Restore job for the database now works properly if you provide your database configuration via a BOSH link. Before, it would always use a hardcoded default if you didn't explicitly specify, causing BOSH links to be ignored concourse/concourse-bosh-release#41.

    🍱 🔗 fix

    • 🛠 Fixed a bug where, if your local machine is running Mac or Linux and you run fly execute with --input against a cluster with a mix of Linux and Windows workers, your files could get uploaded to a Windows worker, where ther permission metadata would be stripped off. Now fly execute makes sure only to upload user artifacts to workers matching the platform of the task being executed #4066.