Changelog History
Page 4
-
v5.5.0 Changes
September 06, 2019🍱 🔗 feature
- 👷 There is a new container placement strategy,
limit-active-tasks
. If you specify this strategy, the cluster will maintain a counter of the number of task containers currently running on each worker. Whenever it is time to run a new container, when this strategy is in use, the worker with the fewest active tasks containers will be chosen to run it.
⏱ There is also an optional 'max active tasks per worker' configuration. If this is set to a positive integer, you will see the following behaviour: If all workers are at their active task limit, you will see the messageAll workers are busy at the moment, please stand-by.
and the scheduler will re-try a minute later. This pattern will repeat each minute indefinitely, until a worker is available.
Thanks to @aledeganopix4d for all their hard work on this feature! #4118, #4148, #4208, #4277, #4142, #4221, #4293, #4161, #4315
🍱 🔗 feature
- 🚀 We have changed our release notes flow! Now, contributors can add draft release notes right in their PRs, by modifying the
release-notes/latest.md
file in theconcourse/concourse
repo #4312.
🍱 🔗 feature
- 🏗 In the past, admins (owners of the
main
team) had permission to modify the auth configuration for other teams in the same cluster. Now, admins also have full control over pipelines, jobs, resources, builds, etc for all teams. Usingfly
, they can log in to any team on the cluster as though they are an owner #4238, #4273.
🍱 🔗 feature
- 🏗 We noticed after #4058 (where build steps are collapsed by default) that it wasn't very easy to see failing steps.
Now a failing step has a red border around its header, an erroring step has an orange border, and a running step has a yellow border. #4164, #4250
🍱 🔗 feature
- 🔧 On particularly busy clusters, users have observed metrics events being dropped due to a full queue #3937. @rudolfv added a configurable buffer size for metrics emission, regardless of your configured emitter. This should allow operators to trade memory pressure on the web nodes for reliability of metric transmission.
🍱 🔗 feature
- 🔧 @rudolfv also added some features for the special case of InfluxDB metrics. To decrease the request load on InfluxDB, you can configure the number of events to batch into a single InfluxDB request, or you can specify a hardcoded interval at which to emit events, regardless of how many have accumulated #3937.
🍱 🔗 feature
- @nazrhom improved the output of
fly targets
to show an error message in the table if your token for a given target is invalid #4181, #4228.
🍱 🔗 feature
- 0️⃣ Concourse admins can now run
fly active-users
and get a summary of all the users on the cluster, filtering by their last login time (the last 2 months by default) #4096.
🍱 🔗 feature
- 📇 @SimonXming fixed up the worker containers and worker volumes metrics to include metadata for the tag(s) on the worker and/or the team the worker belongs to (if any) #3037.
🍱 🔗 feature, breaking
- 🏗 Now requests from the web nodes to the garden servers inside workers (i.e. when creating containers) will time out after 5 minutes. This means that if you have a lot of latency or your garden servers are slow, your builds and/or resource checks will start to fail. In general, we have found that failing earlier in cases like this is preferable to hanging forever or failing silently #4137.
🍱 🔗 feature
- 🔧 Fly has the ability to autocomplete certain configurations for certain shells. @cappyzawa added some more, allowing team and worker names to be autocompleted in fly commands #4087.
🍱 🔗 feature
- @brunocous added an option to skip SSL verification for the git driver on the semver-resource concourse/semver-resource#97.
🍱 🔗 feature
- @mgsolid added a
pre_without_version
param to theget
andput
steps associated to the semver resource. This allows for notating pre-releases without an associated "RC number" (like the-SNAPSHOT
suffix often seen on pre-release builds managed by Maven) concourse/semver-resource#94.
🍱 🔗 feature
- A
depth
parameter was added by @maryamklabib to theget
steps associated with the pool resource. This allows you to get a specified number of commits when fetching a repository concourse/pool-resource#50.
🍱 🔗 feature
- 🚀 @ThePhilderbeast added proxy support to the Github release resource concourse/github-release-resource#84.
🍱 🔗 feature
- 🚀 The number of assets listed per release was raised by @oppegard from 30 to 100 on the Github release resource. This will hopefully prevent most issues where the resource would not have the full list of assets to download concourse/github-release-resource#85.
🍱 🔗 feature
- 🚀 @dprotaso added annotated tag support to the Github release resource concourse/github-release-resource#88, where previously only lightweight tags were supported.
🍱 🔗 feature
- The
evolve
extension has been added to the Mercurial resource by @xavierzwirtz concourse/hg-resource#12.
🍱 🔗 feature
- 🏗 Build step headers are now sticky, staying at the top of the body on the build page instead of scrolling out of sight as you navigate long logs. This should make it easier to keep track of which step you're inspecting in complicated jobs. #4230
🍱 🔗 fix
- When using the overlay volume driver, you would have to be cautious about preserving your mount table. If it had been cleared, you're volumes would appear empty. Baggageclaim now has the capability to recover missing mounts when the process restarts. #4264
🍱 🔗 fix
- @evanchaoli improved
fly
- when outputting sample commands to the terminal, the CLI is aware of the path from which it is being executed #4284.
🍱 🔗 fix
- 💻 The web UI used to silently break when your token (which includes a potentially-long JSON-encoded string detailing all the teams you are part of and what roles you have on them) was longer than the size of a single cookie (4096 bytes on most browsers!). This limit has been increased 15-fold, which should unblock most users on clusters with a lot of teams #4280.
🍱 🔗 fix
- 🚀 For the past few releases, the web nodes have allowed themselves to make up to 64 parallel connections to the database, to allow for parallelizing work like GC and scheduling within a single node. @ebilling has configured the web node's tolerance for idle connections to be more lenient: If a node has been using more than 32 of its available connections, up to 32 connections will be allowed to stay idly open. Anecdotally, CPU savings (resulting from less opening/closing of connections) of up to 30% have been observed on web nodes because of this change. Furthermore, the total max connection pool size has been made configurable - this should allow operators to avoid overloading the max connection limit on the database side #4232.
🍱 🔗 fix
- 🛠 @josecv found and fixed a subtle bug where, if you had a
try
step and you aborted while the hooked step was running, your whole web node would crash! Good catch #4252.
🍱 🔗 fix
- 🏁 @aledeganopix4d fixed an issue where the logs for a Windows or Darwin worker get populated with irrelevant error messages #4167.
🍱 🔗 fix
- 🚀 Since introducing Zstandard compression for volume streaming, we noticed a new class of baggageclaim errors saying
http: unexpected EOF reading trailer
cropping up in our own CI environment, so we updated our http clients to retry requests on this error #4233.
🍱 🔗 fix
- 🚀 Since the pipeline-operator role was introduced in v5.2.0, the tag that appears on the dashboard pages has been improperly formatted. Thanks to @paroxp for noticing and fixing this #4170.
🍱 🔗 fix
- 🏗 Thanks to @osis for noticing that the sidebar becomes unclickable when viewing a build for which you are unauthorized to see the logs, and fixing it too! #4201
🍱 🔗 fix
- The
get-team
command for fly will now taketeam-name
as an argument for the team name instead of justteam
. Thanks to @cappyzawa for finding this inconsistency! #4086
🍱 🔗 fix
- 🛠 @xavierzwirtz also fixed
hg.pull
by passing thesourceUri
in the Mercurial resource. This could fix some issues where Mercurial wasn't saving the full uri in thehgrc
file concourse/hg-resource#13.
🍱 🔗 fix
- 0️⃣ Scrollbars have been given a touch of the Concourse design language. It should no longer use browser defaults. #4355
🍱 🔗 fix
- 🛠 Fixed a (probably-long-standing) issue where the pipelines in the sidebar could randomly rearrange #4375.
- 👷 There is a new container placement strategy,
-
v5.4.1 Changes
August 07, 2019This release is about fixing some unintended consequences of switching to Zstandard compression in v5.4.0. More specifically, having our Go code depend on C libraries caused a few failure modes.
🍱 🔗 fix
- 🍎 When we first switched to using
cgo
to buildfly
(which was necessary in order to make use of Zstandard compression), unbeknownst to us,fly
started to depend on a system library for DNS resolution. This meant that on systems that lacked this library (macOS in particular),fly
would segfault. Furthermore, the system library has different features than the native Go version, so even on systems that had the library, fly would fail to resolve certain names. We fixed these problems by buildingfly
in a way that would still usecgo
but depend on the native Go DNS resolver concourse/ci#59.
🍱 🔗 fix
- On the server side, it turns out we had been a bit lax in freeing resources associated with volume streams, but because we had been using pure Go streaming and compression code, the Go garbage collector had helped us out. In switching to a C-based compression library, failing to manually release memory via Go APIs meant that the underlying memory allocated by the C code was never freed, and several folks have observed server-side memory leaks when using v5.4.0. We plugged those holes #4182, #4199.
- 🍎 When we first switched to using
-
v5.4.0 Changes
July 19, 2019known Issues
🚀 > We've observed a memory leak with this release. Please avoid upgrading to this release if you are using a multi-tenant concourse with heavy load. You can follow along with our investigation in #4150.
🐧 >
fly
may segfault on execution when running on certain versions of Linux. Workaround steps can be found here: #4141 (comment)🍱 🔗 feature, breaking
- We have switched to a more efficient algorithm for streaming volumes between various components of the system - the
web
andworker
nodes now transmit volumes compressed with the Zstandard algorithm rather thanGzip
-fly
is affected, too (when uploading/downloading artifacts duringfly execute
). We're hoping this will resolve some general complaints of steps being slow to start or appearing to "hang" in clustered environments. note: this will require upgrading your workers #3880.
🍱 🔗 feature
- 🏗 Because of the efficiency gained from employing Zstandard compression to volume streaming, the CPU on the web and worker nodes is no longer a limiting factor. Now volumes will be streamed over multiple parallel connections, and we have seen some very exciting improvements in the time taken to initialize build steps. You can see some of our measurements in the discussion on this issue #3993.
🍱 🔗 feature, breaking
- 🏗 On the build page all steps are now collapsed by default. This means that the build page loads faster, but if you want to watch the logs in real time, you will need to click on the header of the running step #4058.
🍱 🔗 feature
- Improved the appearance of the sidebar (re-introduced in v5.2.0) in response to user feedback. Teams and pipelines are now easier to distinguish, and there are nicely styled tooltips for long items #4032.
🍱 🔗 feature
- ⚡️ @maryamklabib optimized the
pool
resource -check
andin
scripts will run faster by using shallow cloning concourse/pool-resource#49.
🍱 🔗 feature
- 🏗 When you manually trigger a build through the web UI, the 'prep' section at the top of the build page now shows a spinner on the entries marked "discovering new versions of..." while the resource checks are in progress. As the checks resolve, the spinners will become checkmarks, so you'll have a better sense of what might be holding up your build #3970.
🍱 🔗 feature
- @deadNightTiger added a
fly completion
subcommand that allowsfly
to output autocompletion for thebash
andzsh
shells #4012.
🍱 🔗 feature
- ⏱ When you abort a pending build before it gets scheduled, the build page will now report it as 'cancelled' instead of showing a loading spinner forever #3922.
🍱 🔗 feature
- ⚡️ The CSS optimizer used in building the web UI has been upgraded, so the resulting CSS is slightly smaller, which should result in some faster page loads #4004.
🍱 🔗 fix
- 🔧 @cappyzawa fixed an issue where the
s3
resource would break whendisable_multipart: true
was set in the source configuration, and a tracked file was smaller than the minimum upload part size for the configured s3-compatible blobstore concourse/s3-resource#131.
🍱 🔗 fix
- 📇 @robwhitby fixed an issue where re-checking a resource would cause the metadata from its latest version to be erased #4028.
🍱 🔗 fix
- 🖨 @cappyzawa fixed up
fly
to behave like a sensible Unix tool when passing the-h
or--help
flag: it now prints the usage text onstdout
and returns a successful status code #4064.
🍱 🔗 fix
- 🛠 @cappyzawa fixed a small issue where the
-j
or--json
flag was not honored when runningfly get-team
#4046.
🍱 🔗 fix
- 🛠 Fixed a regression found recently involving momentum based scrolling on build pages on iOS browsers The fix was also applied to the sidebar and the dashboard #4050
🍱 🔗 fix
- 🚀 The BOSH Backup and Restore job for the database now works properly if you provide your database configuration via a BOSH link. Before, it would always use a hardcoded default if you didn't explicitly specify, causing BOSH links to be ignored concourse/concourse-bosh-release#41.
🍱 🔗 fix
- 🛠 Fixed a bug where, if your local machine is running Mac or Linux and you run
fly execute
with--input
against a cluster with a mix of Linux and Windows workers, your files could get uploaded to a Windows worker, where ther permission metadata would be stripped off. Nowfly execute
makes sure only to upload user artifacts to workers matching theplatform
of the task being executed #4066.
- We have switched to a more efficient algorithm for streaming volumes between various components of the system - the
-
v5.3.0 Changes
June 11, 2019🍱 🔗 feature
- 🔧 @rliebz added the ability for
put
steps togit
resources with the parametermerge: true
to be further configured with the parameterreturning: unmerged
, which will ensure that concurrent changes to the same remote branch do not affect downstream jobs concourse/git-resource#262.
🍱 🔗 feature
- ➕ Added the option for the BOSH Backup and Restore database backup job in the BOSH release to be configured via a BOSH link concourse/concourse-bosh-release#29, concourse/concourse-bosh-release#35.
🍱 🔗 feature
- 🏗 @stigtermichiel tuned our build process so that the size of the compressed frontend code is reduced by a factor of 3 - page loads should be accordingly faster #3915.
🍱 🔗 feature
If you want your web node to accept encrypted connections over HTTPS, you now have the option of connecting to an ACME server to automatically retrieve a certificate from an ACME server of your choice. By default it will reach the free public ACME service provided by Let's Encrypt.
Thanks to @henderjm for the PR #3878!
🍱 🔗 feature
@agurney improved the info api endpoint (
/api/v1/info
) to return both the external URL and cluster name #3862.💻 Hopefully this will enable future improvements to
fly
and the web UI!🍱 🔗 feature
- @jmccann enabled Concourse to authenticate with Vault as a credential manager via an LDAP backend by setting
CONCOURSE_VAULT_AUTH_PARAM="username:<your ldap username>,password:<your ldap password>"
#3884.
🍱 🔗 feature
- 📌 When you pin a resource through the web UI, your name and current local time will now be automatically added as the pin comment #3743.
🍱 🔗 feature
- 🔒 The 'password' field on the login page now has an HTML property that will be ignored by most browsers, but helps Concourse pass some automated security scans #3839.
🍱 🔗 feature
- 🏗 @tvon gave the highlighted lines on the build page a slightly darker color so they really pop #3866.
🍱 🔗 fix
- 🛠 @palixthepalalix fixed a bug in the
git
resource where commits from extraneous branches could appear in the version history if a tag filter was specified concourse/git-resource#261.
🍱 🔗 fix
Made the Concourse worker process systemd-aware. This fixed a subtle bug introduced in v5.0.0 that could, on rare occasions, cause container limits not to be enforced or prevent containers from being killed concourse/concourse-bosh-release#24, concourse/concourse-bosh-release#31.
Thanks to @bodymindarts for contributing to this effort.
🍱 🔗 fix
- ➕ Addressed a bug where in rare circumstances,
fly intercept -c
would find multiple check containers for the same resource #3983.
🍱 🔗 fix
- 🚀 In service of a refactor, task caches are now decoupled from the worker(s) they are stored on. As a result, you may find that in multi-worker deployments, pipelines making use of task caches may consume duplicate storage across workers #3830, #3965.
🍱 🔗 fix
- Corrected a surprising behaviour where, when a pipeline was paused for an extended period of time, many of its resources' versions would disappear #3963.
🍱 🔗 fix
- There was a long-standing bug on the dashboard, where the existence of circular pipelines would cause the browser to crash. This was fixed in v5.2.0, but the fix caused a performance regression, which is now also fixed #3870, #3901.
🍱 🔗 fix
- Fixed a bug, introduced in v5.1.0, where, on a build step with
attempts:
specified, only one attempt would be possible to view on the build page #3898.
🍱 🔗 fix
- @gaelL Fixed a bug, which had existed since at least v4.2.3, where operators were prevented from retiring team-scoped workers with the
concourse retire-worker
command #3929.
🍱 🔗 fix
📚 A bug on the reaped build logs screen was fixed, where a link was pointing to deprecated documentation #3931.
🏗 This screen is visible when you visit a build that is older than its job's configured retention policy.
🍱 🔗 fix
- Fixed a problem with one of the database migrations introduced in v5.0.0 which could cause the web node startup to fail in some cases. This won't affect users who have kept up to date until now, but if this is the first version of Concourse you are deploying since before v5.0.0, the upgrade should be a little more resilient #3996.
🍱 🔗 fix
- 🏗 @mockersf fixed a problem where timestamps weren't being properly returned when a get step or a put step finished. This caused some wacky durations to be displayed on build pages #3871.
🍱 🔗 fix
- 🛠 Fixed an issue with our build process that caused
fly
to stop working for some folks making use of a VPN with split DNS.
🍱 🔗 fix
- 🔧 @rliebz added the ability for
-
v5.3.0-rc.9
May 30, 2019 -
v5.3.0-rc.8
May 30, 2019 -
v5.2.8 Changes
March 24, 2020🍱 🔗 fix
- 🛠 Fix an edge case of CVE-2018-15798 where redirect URI during login flow could be embedded with a malicious host.
🍱 🔗 fix
- ➕ Added a flag,
--disable-list-all-jobs
. When this flag is passed, the /api/v1/jobs endpoint (which is known to have performance issues) will always return an empty JSON array instead of making complex and expensive database operations. The most significant end-user impact of this change should be that the dashboard will no longer display pipeline previews. #5340
-
v5.2.7 Changes
February 27, 2020🍱 🔗 fix
- ⬆️ Bump golang.org/x/crypto module from
v0.0.0-20190313024323-a1f597ede03a
tov0.0.0-20200220183623-bac4c82f6975
to address vulnerability in ssh package.
- ⬆️ Bump golang.org/x/crypto module from
-
v5.2.6 Changes
December 18, 2019🔒 🔗 security
- 🚀 Updates the git resource to v1.6.3 to address a recently reported security vulnerability:
- CVE-2019-19604:
- Arbitrary command execution is possible in Git before 2.20.2, 2.21.x before 2.21.1, 2.22.x before 2.22.2, 2.23.x before 2.23.1, and 2.24.x before 2.24.1 because a "git submodule update" operation can run commands found in the .gitmodules file of a malicious repository.
- 🚀 Updates the git resource to v1.6.3 to address a recently reported security vulnerability:
-
v5.2.5 Changes
November 15, 2019🍱 🔗 feature
- 💻 API endpoints have been changed to use a single transaction per request, so that they become "all or nothing" instead of holding data in memory while waiting for another connection from the pool. This could lead to snowballing and increased memory usage as requests from the web UI (polling every 5 seconds) piled up. #4494