Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-run of release pipeline results in loss of release notes information #440

Closed
wpschaub opened this issue Feb 25, 2019 · 25 comments
Closed

Comments

@wpschaub
Copy link

Azure DevOps Extension you are using

Generate Release Notes (Crossplatform)

Where are you running it?

Azure DevOps Service (VSTS)

Version of Extension/Task

2.8.1

Expected behaviour and actual behaviour

During a successful deployment the release notes are generated based on current and previous release, resulting in a release note as expected. If user re-deploys an environment, the release notes are re-generated (and potentially overwritten), but without any changes as extension now compares release to itself.

Steps to reproduce the problem

  1. Complete a pull request with valid changes, deploy an environment, and generate the release notes.
  2. Generated release notes are healthy and reflect the changes introduced by latest artefacts.
  3. Re-deploy the environment, which generates the release notes again.
  4. Changes introduced by the latest artefacts are not reflected in the release notes.
@wpschaub wpschaub changed the title Re-run of release pipeline results in loss of information Re-run of release pipeline results in loss of release notes information Feb 25, 2019
@rfennell
Copy link
Owner

I think this might be a limitation of the API call that is being made, as it compared between simiple numeric releases, ignoring and rerelease complexities. I need to investigate

@wpschaub
Copy link
Author

@rfennell I did the following test:

  1. Cloned pipeline and re-ran release.
  2. Run 1: Release = Release 1, Compare = Release 1, changes none.
  3. Run 2: Release = Release 2, Compare = Release 1
  4. Re-Run 2: Release = Release 2, Compare = Release 2

In step 1, should the Compare not be "None" and in step 4 be the last successful, but itself, i.e. "Release 1"? When it compares itself, there will be no differences.

@rfennell
Copy link
Owner

rfennell commented Feb 26, 2019

  1. Cloned pipeline and re-ran release.
  2. Run 1: Release = Release 1, Compare = Release 1, changes none. - I would expect all wi/cs associated with the trigger build (this is new feature), the detailed logs show a special route has been taken
  3. Run 2: Release = Release 2, Compare = Release 1 - Normal behavior
  4. Re-Run 2: Release = Release 2, Compare = Release 2 - this is the problem, I need to look into this this, but it won't be straight away

@rfennell
Copy link
Owner

I think the question is how is the task meant to know that step 4 (above) is really step 3.

It just looks for the last good release to compare against, and the step 3 one was good.

Any thoughts, how can we signal that step 4 is a redeploy?

@rfennell rfennell self-assigned this Mar 20, 2019
@wpschaub
Copy link
Author

@rfennell is there any way that the task can compare itself with the most recent successful build?
image

For example, release 31 compared itself with release 30. Release 30 compared itself with release 29, which was failed in post=deploy approvals.

I'm thinking aloud ... if we can configure the task to compare itself with the last successful release, for example Release 30 would compare itself with Release 26, we can fail a release, before re-deploying to fix this #440 issue. A bonus would be that the release notes would show the same results as what we see in the release summary when we manually compare with the last successful release.

Thoughts?

@rfennell
Copy link
Owner

Yes, I think that is the only way

I think the task needs to check that this release does match the last successful release's artifacts. if it does in your example run the compare for 30 > 26 as opposed to 31 >30

Think that works?

@wpschaub
Copy link
Author

wpschaub commented Mar 20, 2019

@rfennell, sorry, my image is adding confusion. Ignore release 31 at the top. When Release 30 runs, it compares itself with Release 29 (which was failed in post-approval) according to the release notes. This is default behavior, even for the release summary view, so I would leave it "by default".

It would, however, be "cool" if we could configure the task not to process default behavior, but to compare itself with the most recent successful release. So, if we set this option, release notes would show that Release 31 compared itself with Release 26.

This behavior would unblock a few branching and release reporting issues we 're experiencing.

@wpschaub
Copy link
Author

@rfennell if we could take it one step further. If configured to NON-DEFAULT, release 30 looks for the last successful release (other than itself), which is 26. When Release 31 runs it finds 30. When release 31 is re-deployed, it still looks for the last (other than itself) successful release, which is 30.

@rfennell
Copy link
Owner

Seems reasonable, I will have a look

@rfennell
Copy link
Owner

Checked the code, it should do the compare against the last successful release. If you run with system.debug=true it lists the last successful release, seems Ok from my tests

@wpschaub
Copy link
Author

wpschaub commented Mar 20, 2019 via email

@wpschaub
Copy link
Author

@rfennell you're correct. release summary, by default, compares with the previous release, even if failed. Your extension skips the failed releases 👍 I'm stoked and cannot wait to go back to the office tomorrow :) That means the only thing that is still an issue is the re-deploy, which compares the release with itself.

@rfennell
Copy link
Owner

So my bug is good !

I won't fix it then, just the rerun issue and I have plan for that

@wpschaub
Copy link
Author

wpschaub commented Mar 21, 2019 via email

@wpschaub
Copy link
Author

@rfennell Test was successful. I failed 3 PR and 3 releases, then ran another release and the extension compared Current to Current - 6. Spot on!

In terms of a re-deploy a colleague had a great idea. How about the task skips release notes processing if Release.AttemptNumber > 1?

@rfennell
Copy link
Owner

@wpschaub been a while but I have been thinking on this one.

  • On the Release.AttemptNumber > 1 option where you thinking output a message somewhere?
  • I was looking at trying to detect of a release was off the same artifacts as the previous one and using the changes from the previous release, but this is confusing.

I like tracking the Release.AttemptNumber number , but not sure what to do with it.

@wpschaub
Copy link
Author

@rfennell I was thinking of allowing user to decide whether to re-run the release note generation if ReleaseAttemptNumber>1. By default I would not re-generate release notes, instead place a warning in the log that release notes were not regenerated due to the release being re-deployed.

@rfennell
Copy link
Owner

That sounds like I need both, if we are a re-run then need to go back and get the old artifact number for the checks

@wpschaub
Copy link
Author

wpschaub commented Apr 15, 2019 via email

rfennell pushed a commit that referenced this issue Apr 16, 2019
@rfennell
Copy link
Owner

Give 2..9.6 a try, it has a new advanced option that will skip generation of release notes on a re-deploy (of the stage the task is in). It completes with a 'pass with warnings'

The default is the old behavior, so it is a non-breaking change

Does that work for you?

@wpschaub
Copy link
Author

rfennell added a commit that referenced this issue Apr 18, 2019
* Stops generation if a redeploy
* Added the parameter to the task.json
* Updated docs and fixed return type
@scuba6388
Copy link

scuba6388 commented Oct 4, 2019

@rfennell It sounds like you were attempting to find a way to compare to latest successful deployment (not the current) when redeploying. Did you determine that this wasn't going to be possible?

Sometimes if I don't like what was in the notes, or a work item gets missed, I'd like to go back and regenerate the release notes. Since redeploying won't recreate the notes, and since DevOps won't let you delete a release once it is completed, I don't really have any options if something went wrong.

@scuba6388
Copy link

@rfennell It sounds like you were attempting to find a way to compare to latest successful deployment (not the current) when redeploying. Did you determine that this wasn't going to be possible?

Sometimes if I don't like what was in the notes, or a work item gets missed, I'd like to go back and regenerate the release notes. Since redeploying won't recreate the notes, and since DevOps won't let you delete a release once it is completed, I don't really have any options if something went wrong.

@rfennell Any tips for situations like this? The main issue is that DevOps doesn't let you delete your last release. Still trying to figure out how I could regenerate the correct notes.

@rfennell
Copy link
Owner

The only option I have found is to use the command line test harness I recently wrote and inject the values for the release you wish to re-run.

However even this is problemic if the release ran to completion as the task looks for the last success release to compare against, less if an issue if using multistage YAML.

@scuba6388
Copy link

scuba6388 commented Aug 10, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants