[Wonder-disc] integration and master divergence

Hi Kieran,

Post by Kieran Kelleher
The original plan with integration was that commits that survived for at least a month in integration without
breakage or bugs would be deemed worthy of merging to master.
If all goes well, this merge will be shifted to master and then we can from then on ONLY merge integration commits that are a month old PERIODICALLY (1-4 weeks intervals).

As I see it, master should be a stable thing that production apps can rely on. That requires two premises:

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

2. Some sort of version number to tag each release, so that it will be possible for everyone who experiences problems in production can easily switch back to the previous master release. Every other framework out there has version numbers that make it possible to refer to a specific release.

What do you think?

Maik

Pascal Robert

2012-07-18 10:12:11 UTC

Post by Maik Musall
Hi Kieran,

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

Post by Maik Musall
2. Some sort of version number to tag each release, so that it will be possible for everyone who experiences problems in production can easily switch back to the previous master release. Every other framework out there has version numbers that make it possible to refer to a specific release.

Already done. All commits done in master, except a couple of small ones, coming from integration have been tagged.

https://github.com/projectwonder/wonder/tags

We even have a change log for master:

https://github.com/projectwonder/wonder/wiki/Changelog

Post by Maik Musall
What do you think?
Maik
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Maik Musall

2012-07-18 10:21:52 UTC

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

Already done. All commits done in master, except a couple of small ones, coming from integration have been tagged.
https://github.com/projectwonder/wonder/tags
https://github.com/projectwonder/wonder/wiki/Changelog

Oh, I somehow missed that, thanks.

Maik

Kieran Kelleher

2012-07-18 12:25:46 UTC

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

It is about one month better than we had in the old days of svn trunk.

If a few people want to have master lag integration by two months, or some other time period, instead of one month, that's fine by me too.

So the premise right now is (1) active testing is done by the original feature/change contributor/committer, followed by (2) passive small-crowd testing happens in integration branch, and followed by (3) passive large-crowd testing in master branch users. When bugs are found by someone, they are quickly fixed. Any user has the opportunity to be completely conservative and stick to specific master release tags, running *their own* extensive functional and/or unit tests of their applications when they upgrade and sending an email saying, "hey, something is broken in release x.y.x" if something is indeed broken. Such is the dynamic nature of the crowd-built wonder :-)

Ideally we would have an extensive set of unit tests and a high measured percentage of test coverage, but we don't. It is what it is. Anyone who wants to spend their time building and maintaining such extensive unit tests and measuring test coverage is welcome to work on it if they have the time to dedicate to it.... I am sure they will earn a few beers at the next WOWODC :-)

As long as stuff keeps coming into integration, we need to keep master moving along in some organized way. The main purpose of this merge is to "catch up" one time on all the integration commits since Feb 29 that have *not* been cherry-picked, and give us a more recent "merge-base common ancestor" ready for fast-forward merge catchup's in the future since I think our recent experience has shown that cherry-picking a lot of commits over time leads to divergence that is difficult to reconcile with confidence. 'master' needs to follow 'integration', not diverge from it.

Maik, if you have a better suggestion that fits the expedient nature of keeping wonder master moving, now is the time to propose it.

Oh, I somehow missed that, thanks.
Maik
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Maik Musall

2012-07-18 13:21:53 UTC

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

It is about one month better than we had in the old days of svn trunk.
If a few people want to have master lag integration by two months, or some other time period, instead of one month, that's fine by me too.
So the premise right now is (1) active testing is done by the original feature/change contributor/committer, followed by (2) passive small-crowd testing happens in integration branch, and followed by (3) passive large-crowd testing in master branch users. When bugs are found by someone, they are quickly fixed. Any user has the opportunity to be completely conservative and stick to specific master release tags, running *their own* extensive functional and/or unit tests of their applications when they upgrade and sending an email saying, "hey, something is broken in release x.y.x" if something is indeed broken. Such is the dynamic nature of the crowd-built wonder :-)
Ideally we would have an extensive set of unit tests and a high measured percentage of test coverage, but we don't. It is what it is. Anyone who wants to spend their time building and maintaining such extensive unit tests and measuring test coverage is welcome to work on it if they have the time to dedicate to it.... I am sure they will earn a few beers at the next WOWODC :-)
As long as stuff keeps coming into integration, we need to keep master moving along in some organized way. The main purpose of this merge is to "catch up" one time on all the integration commits since Feb 29 that have *not* been cherry-picked, and give us a more recent "merge-base common ancestor" ready for fast-forward merge catchup's in the future since I think our recent experience has shown that cherry-picking a lot of commits over time leads to divergence that is difficult to reconcile with confidence. 'master' needs to follow 'integration', not diverge from it.
Maik, if you have a better suggestion that fits the expedient nature of keeping wonder master moving, now is the time to propose it.

My concern is: The periodic merge into master results in a version of Wonder that might not have been used by anybody at all in that specific combination. As a result, for example some inter-framework dependency might be broken without showing up as compile error. As it is, this can only be discovered by field tests, meaning that Wonder master candidate has to be used by as many developers as possible *before* it becomes the new master, aiming for master to be actually as production-ready as possible.

A way to prevent that would be to make that merge not directly into master, but into a stage or release-candidate branch. As many as possible of us should then use and test that for a week or two, apply fixes there, and when it's ready, that one is merged to master.

Using that strategy I don't even see the need to start that branch with an integration version that's already a month old. Could be even newer then.

Maik

Travis Britt

2012-07-18 15:13:05 UTC

My gut / experience says the community isn't large or formal enough to absorb any more layers than integration and master -- which is a great sanity check that we've never had before, if used properly...

Kieran is right that we can't regularly cherry pick stuff from integration and put it in master. I'd actually say, never do it. Unless you must. Then still don't do it. :)

(1) It encourages a mess.

(2) It means master has diverged from integration. The point of integration/master is that master is a time-lagged snapshot of integration, which will have been used by committers (and others on the bleeding edge) for a month. If master diverges via cherry picking, it results in a combination of code that hasn't been used before.

tb

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

It is about one month better than we had in the old days of svn trunk.
If a few people want to have master lag integration by two months, or some other time period, instead of one month, that's fine by me too.
So the premise right now is (1) active testing is done by the original feature/change contributor/committer, followed by (2) passive small-crowd testing happens in integration branch, and followed by (3) passive large-crowd testing in master branch users. When bugs are found by someone, they are quickly fixed. Any user has the opportunity to be completely conservative and stick to specific master release tags, running *their own* extensive functional and/or unit tests of their applications when they upgrade and sending an email saying, "hey, something is broken in release x.y.x" if something is indeed broken. Such is the dynamic nature of the crowd-built wonder :-)
Ideally we would have an extensive set of unit tests and a high measured percentage of test coverage, but we don't. It is what it is. Anyone who wants to spend their time building and maintaining such extensive unit tests and measuring test coverage is welcome to work on it if they have the time to dedicate to it.... I am sure they will earn a few beers at the next WOWODC :-)
As long as stuff keeps coming into integration, we need to keep master moving along in some organized way. The main purpose of this merge is to "catch up" one time on all the integration commits since Feb 29 that have *not* been cherry-picked, and give us a more recent "merge-base common ancestor" ready for fast-forward merge catchup's in the future since I think our recent experience has shown that cherry-picking a lot of commits over time leads to divergence that is difficult to reconcile with confidence. 'master' needs to follow 'integration', not diverge from it.
Maik, if you have a better suggestion that fits the expedient nature of keeping wonder master moving, now is the time to propose it.

My concern is: The periodic merge into master results in a version of Wonder that might not have been used by anybody at all in that specific combination. As a result, for example some inter-framework dependency might be broken without showing up as compile error. As it is, this can only be discovered by field tests, meaning that Wonder master candidate has to be used by as many developers as possible *before* it becomes the new master, aiming for master to be actually as production-ready as possible.
A way to prevent that would be to make that merge not directly into master, but into a stage or release-candidate branch. As many as possible of us should then use and test that for a week or two, apply fixes there, and when it's ready, that one is merged to master.
Using that strategy I don't even see the need to start that branch with an integration version that's already a month old. Could be even newer then.
Maik

Pascal Robert

2012-07-18 15:35:20 UTC

Post by Travis Britt
My gut / experience says the community isn't large or formal enough to absorb any more layers than integration and master -- which is a great sanity check that we've never had before, if used properly...
Kieran is right that we can't regularly cherry pick stuff from integration and put it in master. I'd actually say, never do it. Unless you must. Then still don't do it. :)

I must add that cherry picking looked good when we started the integration branch, but that was based on the number of contributions that we had before. I didn't expect contributions to sky roof like now. It was ok at the beginning because I could cherry pick one month old commits + newer and smaller commits (like JavaDoc changes) into master, but now it's just plain impossible.

Post by Travis Britt
(1) It encourages a mess.
(2) It means master has diverged from integration. The point of integration/master is that master is a time-lagged snapshot of integration, which will have been used by committers (and others on the bleeding edge) for a month. If master diverges via cherry picking, it results in a combination of code that hasn't been used before.
tb

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

It is about one month better than we had in the old days of svn trunk.
If a few people want to have master lag integration by two months, or some other time period, instead of one month, that's fine by me too.
So the premise right now is (1) active testing is done by the original feature/change contributor/committer, followed by (2) passive small-crowd testing happens in integration branch, and followed by (3) passive large-crowd testing in master branch users. When bugs are found by someone, they are quickly fixed. Any user has the opportunity to be completely conservative and stick to specific master release tags, running *their own* extensive functional and/or unit tests of their applications when they upgrade and sending an email saying, "hey, something is broken in release x.y.x" if something is indeed broken. Such is the dynamic nature of the crowd-built wonder :-)
Ideally we would have an extensive set of unit tests and a high measured percentage of test coverage, but we don't. It is what it is. Anyone who wants to spend their time building and maintaining such extensive unit tests and measuring test coverage is welcome to work on it if they have the time to dedicate to it.... I am sure they will earn a few beers at the next WOWODC :-)
As long as stuff keeps coming into integration, we need to keep master moving along in some organized way. The main purpose of this merge is to "catch up" one time on all the integration commits since Feb 29 that have *not* been cherry-picked, and give us a more recent "merge-base common ancestor" ready for fast-forward merge catchup's in the future since I think our recent experience has shown that cherry-picking a lot of commits over time leads to divergence that is difficult to reconcile with confidence. 'master' needs to follow 'integration', not diverge from it.
Maik, if you have a better suggestion that fits the expedient nature of keeping wonder master moving, now is the time to propose it.

My concern is: The periodic merge into master results in a version of Wonder that might not have been used by anybody at all in that specific combination. As a result, for example some inter-framework dependency might be broken without showing up as compile error. As it is, this can only be discovered by field tests, meaning that Wonder master candidate has to be used by as many developers as possible *before* it becomes the new master, aiming for master to be actually as production-ready as possible.
A way to prevent that would be to make that merge not directly into master, but into a stage or release-candidate branch. As many as possible of us should then use and test that for a week or two, apply fixes there, and when it's ready, that one is merged to master.
Using that strategy I don't even see the need to start that branch with an integration version that's already a month old. Could be even newer then.
Maik

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Kieran Kelleher

2012-07-18 15:38:00 UTC

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

It is about one month better than we had in the old days of svn trunk.
If a few people want to have master lag integration by two months, or some other time period, instead of one month, that's fine by me too.
So the premise right now is (1) active testing is done by the original feature/change contributor/committer, followed by (2) passive small-crowd testing happens in integration branch, and followed by (3) passive large-crowd testing in master branch users. When bugs are found by someone, they are quickly fixed. Any user has the opportunity to be completely conservative and stick to specific master release tags, running *their own* extensive functional and/or unit tests of their applications when they upgrade and sending an email saying, "hey, something is broken in release x.y.x" if something is indeed broken. Such is the dynamic nature of the crowd-built wonder :-)
Ideally we would have an extensive set of unit tests and a high measured percentage of test coverage, but we don't. It is what it is. Anyone who wants to spend their time building and maintaining such extensive unit tests and measuring test coverage is welcome to work on it if they have the time to dedicate to it.... I am sure they will earn a few beers at the next WOWODC :-)
As long as stuff keeps coming into integration, we need to keep master moving along in some organized way. The main purpose of this merge is to "catch up" one time on all the integration commits since Feb 29 that have *not* been cherry-picked, and give us a more recent "merge-base common ancestor" ready for fast-forward merge catchup's in the future since I think our recent experience has shown that cherry-picking a lot of commits over time leads to divergence that is difficult to reconcile with confidence. 'master' needs to follow 'integration', not diverge from it.
Maik, if you have a better suggestion that fits the expedient nature of keeping wonder master moving, now is the time to propose it.

The concern is somewhat valid, however if we are fast-forwarding from integration and most (many?) are using integration in their everyday project work, then integration is being field tested to some extent.

Post by Maik Musall
A way to prevent that would be to make that merge not directly into master, but into a stage or release-candidate branch. As many as possible of us should then use and test that for a week or two, apply fixes there, and when it's ready, that one is merged to master.

That is not an unreasonable request, and is kind of what I did anyway with the 'temp/MergeOneMonthIntegrationToMaster' branch.

OK, so ...... if it is OK, we can loosely (since we tend to be that way anyway ;-) ) follow this process:

periodically merge one-month-old+ commits from integration and current master into a new temporary "Release Candidate" branch, example 'rc-wonder-5.7.0'. Then issue pull request on the release candidate branch to merge to master. Allow 2 weeks of no bug-fixes to pass. then merge it to master.

Also, pull request now re-submitted as a release candidate pull request (https://github.com/projectwonder/wonder/pull/243) and a 2-week no-bug-fix window before it can be merged.

Post by Maik Musall
Using that strategy I don't even see the need to start that branch with an integration version that's already a month old. Could be even newer then.
Maik

Ramsey Gurley

2012-07-18 17:54:13 UTC

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

Chuck Hill

2012-07-18 18:10:27 UTC

Post by Ramsey Gurley
FWIW, I don't use master in my personal projects anymore. One reason is cherry picking. I can't tell what's in master anymore. It sounds like we're fixing this.
The other reason is master is a month behind integration. I'm not sure how we fix that, but master is useless to me when it's that far behind current patches.

I can see myself using master in two cases:
1. I am getting close to the end of a project, don't need changes in Wonder, and want to "freeze" the version

2. I am working on a large team and any breakage would be very costly and I don't need any recent changes

Chuck

Maik Musall

2012-07-19 06:55:05 UTC

Post by Ramsey Gurley
The other reason is master is a month behind integration. I'm not sure how we fix that, but master is useless to me when it's that far behind current patches.

1. I am getting close to the end of a project, don't need changes in Wonder, and want to "freeze" the version

End of a project? What do you mean, end? :~|

Maik

Kieran Kelleher

2012-07-19 11:28:44 UTC

Post by Ramsey Gurley
The other reason is master is a month behind integration. I'm not sure how we fix that, but master is useless to me when it's that far behind current patches.

1. I am getting close to the end of a project, don't need changes in Wonder, and want to "freeze" the version

End of a project? What do you mean, end? :~|

The period of time just before a project is released to production for the first time.

Post by Maik Musall
Maik
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Kieran Kelleher

2012-07-18 18:44:22 UTC

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

FWIW, I don't use master in my personal projects anymore. One reason is cherry picking. I can't tell what's in master anymore. It sounds like we're fixing this.

That's the objective .... we can just put it down to our 6 months experience that wholesale cherry-picking makes it hard to see the cherries left in the tree :-)

Post by Ramsey Gurley
The other reason is master is a month behind integration. I'm not sure how we fix that, but master is useless to me when it's that far behind current patches.

And that is no problem ...... the more people that are absolutely pegged to using integration, then the more trust we can have in moving mature integration commits to master. So, no worries.

Post by Ramsey Gurley
Ramsey
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Henrique Prange

2012-07-18 19:24:31 UTC

Hi all,

I agree that the separation of master/integration is pointless as soon as we keep creating tags every month. The integration branch is intended for people that want the latest changes. Anyone wanting a more "reliable" approach should point to a tag. The master branch adds no value.

Just my two cents.

Cheers,

Henrique

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

FWIW, I don't use master in my personal projects anymore. One reason is cherry picking. I can't tell what's in master anymore. It sounds like we're fixing this.
The other reason is master is a month behind integration. I'm not sure how we fix that, but master is useless to me when it's that far behind current patches.
Ramsey
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Kieran Kelleher

2012-07-18 20:10:20 UTC

Good point, nevertheless, it is just a workflow. Branches and tags are very similar in the sense that they are both generally just "post-it note" references to commits in the object graph, however it is easier for less adventurous users who want to work with source code and still want stable source to have a stable "master" since checking out a tag (detached HEAD and warning message) is not a good user experience, especially for those not living on bleeding edge.

So, IMHO, we can keep everyone happy by continuing with two branches (just labels on commits) and someone (anyone with commit access) can periodically make a release candidate branch based on master, integrate one-month+ old integration, push it and announce the release candidate via pull request of it to master.

Wonder "Users" who want to work with the source (who doesn't want to learn form wonder source?!) can clone and will, by default, get a stable master checkout instead of having to *know* that they need to clone, then go find a stable tag, then make a personal branch based on that and then check that out. Let's keep it easy for Wonder "users" and those less familiar with the layout.

Wonder "Developers/Contributors" can clone and use integration or whatever they want in between.

So for the sake of simplicity and less confusion for many users, cloning and defaulting to a stable (master) snapshot makes most sense.

Post by Kieran Kelleher
Hi all,
I agree that the separation of master/integration is pointless as soon as we keep creating tags every month. The integration branch is intended for people that want the latest changes. Anyone wanting a more "reliable" approach should point to a tag. The master branch adds no value.
Just my two cents.
Cheers,
Henrique

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Chuck Hill

2012-07-18 20:19:25 UTC

Hi Kieran,

What about the case where that point in time had a bug that was fixed in a later commit, perhaps the next week? Merging a point in time does not guarantee a working version, just that any bugs in that merge were fixed in a subsequent commit. Addressing that seems to lead us back to cherry picking the commit. What am I missing?

Chuck

Post by Kieran Kelleher
Wonder "Users" who want to work with the source (who doesn't want to learn form wonder source?!) can clone and will, by default, get a stable master checkout instead of having to *know* that they need to clone, then go find a stable tag, then make a personal branch based on that and then check that out. Let's keep it easy for Wonder "users" and those less familiar with the layout.
Wonder "Developers/Contributors" can clone and use integration or whatever they want in between.
So for the sake of simplicity and less confusion for many users, cloning and defaulting to a stable (master) snapshot makes most sense.

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Ramsey Gurley

2012-07-18 20:28:11 UTC

Post by Maik Musall
Hi Kieran,

Just another branch of course! We shall call it hotfix :3

http://nvie.com/posts/a-successful-git-branching-model/

See, everything can be fixed by a new branch... except the part where it takes half my day managing them all ;-)

Ramsey

Chuck Hill

2012-07-18 20:47:38 UTC

Just another branch of course! We shall call it hotfix :3
http://nvie.com/posts/a-successful-git-branching-model/
See, everything can be fixed by a new branch... except the part where it takes half my day managing them all ;-)

Is there an app for that?

Kieran Kelleher

2012-07-18 23:39:39 UTC

One interesting thing is that the current stable (?) tip of master from 3 weeks ago has no tag, so I just labeled it wonder-5.6.1 just now and pushed that tag so we have a reference point.

Just another branch of course! We shall call it hotfix :3
http://nvie.com/posts/a-successful-git-branching-model/
See, everything can be fixed by a new branch... except the part where it takes half my day managing them all ;-)

Is there an app for that?
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
http://www.global-village.net/gvc/practical_webobjects

Henrique Prange

2012-07-21 23:49:39 UTC

Hi Kieran,

I've deployed the 5.6.1 version to the WOCommunity Repository. I'll deploy the 5.7.0 as soon as it's approved.

Cheers,

Henrique

Post by Kieran Kelleher
One interesting thing is that the current stable (?) tip of master from 3 weeks ago has no tag, so I just labeled it wonder-5.6.1 just now and pushed that tag so we have a reference point.

Just another branch of course! We shall call it hotfix :3
http://nvie.com/posts/a-successful-git-branching-model/
See, everything can be fixed by a new branch... except the part where it takes half my day managing them all ;-)

Kieran Kelleher

2012-07-18 23:10:40 UTC

:D ... yep, it can get out of hand .... that's a good article, but IMHO is too complex for our little wonder group.

Post by Maik Musall
Hi Kieran,

Just another branch of course! We shall call it hotfix :3
http://nvie.com/posts/a-successful-git-branching-model/
See, everything can be fixed by a new branch... except the part where it takes half my day managing them all ;-)
Ramsey

Kieran Kelleher

2012-07-18 21:09:19 UTC

Post by Maik Musall
Hi Kieran,

A good point. Same can be said for just picking a point in time to tag a commit, and as we know, there are always bugs that we have not found yet.

Post by Maik Musall
Addressing that seems to lead us back to cherry picking the commit. What am I missing?

The peer review of the release candidate branch pull request to master gives people a opportunity to fix and/or cherry-pick important bug fixes on the release candidate branch. A small few cherry-picked commits are easy to manage in each release candidate cycle. It's the hundreds of cherry-picks over 5 months, some of which were modified during the cherry-pick process, that makes maintaining a non-divergent master a challenge.... right now master and integration are 5 months apart on a genuine merge-base.

If you think of Henrique's idea of just tagging stable snapshots in the integration history, then master reference will just always be fast-forwarding to that latest stable snapshot. In essence the master reference "post-it note" just moves along integration's history to the latest snapshot tag in an ideal world of no cherry-picking at all. Going through the release candidate hoopla just gives us the opportunity to bug fix (with more recent integration commit "essential" cherry-picking or otherwise).

Anyway, there is more than one way to skin a cat - so (1) if any committer feels strongly enough that we should not do the monthly release candidate dance, then just just go to https://github.com/projectwonder/wonder/pull/243, type a comment "This sucks" and press "Close" :-P ;-) .... or (2) we can try it and if it sucks abandon the process, or (3) do nothing and let master flounder away into divergence oblivion :D

Kieran

PS. More importantly, did you ever find the Redbreast 12 Cask Strength up there in Vancouver?

Post by Maik Musall
Chuck

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Chuck Hill

2012-07-18 23:26:42 UTC

Hi Kieran,

A good point. Same can be said for just picking a point in time to tag a commit, and as we know, there are always bugs that we have not found yet.

Post by Chuck Hill
Addressing that seems to lead us back to cherry picking the commit. What am I missing?

The peer review of the release candidate branch pull request to master gives people a opportunity to fix and/or cherry-pick important bug fixes on the release candidate branch. A small few cherry-picked commits are easy to manage in each release candidate cycle. It's the hundreds of cherry-picks over 5 months, some of which were modified during the cherry-pick process, that makes maintaining a non-divergent master a challenge.... right now master and integration are 5 months apart on a genuine merge-base.
If you think of Henrique's idea of just tagging stable snapshots in the integration history, then master reference will just always be fast-forwarding to that latest stable snapshot. In essence the master reference "post-it note" just moves along integration's history to the latest snapshot tag in an ideal world of no cherry-picking at all. Going through the release candidate hoopla just gives us the opportunity to bug fix (with more recent integration commit "essential" cherry-picking or otherwise).
Anyway, there is more than one way to skin a cat - so (1) if any committer feels strongly enough that we should not do the monthly release candidate dance, then just just go to https://github.com/projectwonder/wonder/pull/243, type a comment "This sucks" and press "Close" :-P ;-) .... or (2) we can try it and if it sucks abandon the process, or (3) do nothing and let master flounder away into divergence oblivion :D

I don't have any good solutions given that we lack any semblance of comprehensive tests or a even a QA group. I was just pointing out a potential problem. But it seems you are still willing to pick some cherries so as long as we somehow remember which ones to pick, we are good.

Post by Kieran Kelleher
PS. More importantly, did you ever find the Redbreast 12 Cask Strength up there in Vancouver?

No such luck. I may have found a place that has the regular 12 year old.

Chuck

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Kieran Kelleher

2012-07-18 23:43:52 UTC

Post by Maik Musall
Hi Kieran,

A good point. Same can be said for just picking a point in time to tag a commit, and as we know, there are always bugs that we have not found yet.

Post by Chuck Hill
Addressing that seems to lead us back to cherry picking the commit. What am I missing?

The peer review of the release candidate branch pull request to master gives people a opportunity to fix and/or cherry-pick important bug fixes on the release candidate branch. A small few cherry-picked commits are easy to manage in each release candidate cycle. It's the hundreds of cherry-picks over 5 months, some of which were modified during the cherry-pick process, that makes maintaining a non-divergent master a challenge.... right now master and integration are 5 months apart on a genuine merge-base.
If you think of Henrique's idea of just tagging stable snapshots in the integration history, then master reference will just always be fast-forwarding to that latest stable snapshot. In essence the master reference "post-it note" just moves along integration's history to the latest snapshot tag in an ideal world of no cherry-picking at all. Going through the release candidate hoopla just gives us the opportunity to bug fix (with more recent integration commit "essential" cherry-picking or otherwise).
Anyway, there is more than one way to skin a cat - so (1) if any committer feels strongly enough that we should not do the monthly release candidate dance, then just just go to https://github.com/projectwonder/wonder/pull/243, type a comment "This sucks" and press "Close" :-P ;-) .... or (2) we can try it and if it sucks abandon the process, or (3) do nothing and let master flounder away into divergence oblivion :D

Sounds good. :-)

Cheers, Kieran

Post by Kieran Kelleher
PS. More importantly, did you ever find the Redbreast 12 Cask Strength up there in Vancouver?

No such luck. I may have found a place that has the regular 12 year old.
Chuck

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Kieran Kelleher

2012-07-19 02:38:41 UTC

FYI,

This command will list the range of commits on integration that are *NOT* being included in the rc-wonder-5.7.0 release candidate.

$ git log --oneline upstream/integration ^6154e3593

If anyone recognizes any critical/essential bug fixes there that they want brought over, then go ahead and cherry-pick those to rc-wonder-5.7.0 please.

Regards, Kieran

Post by Maik Musall
Hi Kieran,

A good point. Same can be said for just picking a point in time to tag a commit, and as we know, there are always bugs that we have not found yet.

Post by Chuck Hill
Addressing that seems to lead us back to cherry picking the commit. What am I missing?

The peer review of the release candidate branch pull request to master gives people a opportunity to fix and/or cherry-pick important bug fixes on the release candidate branch. A small few cherry-picked commits are easy to manage in each release candidate cycle. It's the hundreds of cherry-picks over 5 months, some of which were modified during the cherry-pick process, that makes maintaining a non-divergent master a challenge.... right now master and integration are 5 months apart on a genuine merge-base.
If you think of Henrique's idea of just tagging stable snapshots in the integration history, then master reference will just always be fast-forwarding to that latest stable snapshot. In essence the master reference "post-it note" just moves along integration's history to the latest snapshot tag in an ideal world of no cherry-picking at all. Going through the release candidate hoopla just gives us the opportunity to bug fix (with more recent integration commit "essential" cherry-picking or otherwise).
Anyway, there is more than one way to skin a cat - so (1) if any committer feels strongly enough that we should not do the monthly release candidate dance, then just just go to https://github.com/projectwonder/wonder/pull/243, type a comment "This sucks" and press "Close" :-P ;-) .... or (2) we can try it and if it sucks abandon the process, or (3) do nothing and let master flounder away into divergence oblivion :D

Post by Kieran Kelleher
PS. More importantly, did you ever find the Redbreast 12 Cask Strength up there in Vancouver?

No such luck. I may have found a place that has the regular 12 year old.
Chuck

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Tim Worman

2012-07-19 02:17:54 UTC

Post by Maik Musall
Hi Kieran,

Post by Chuck Hill
Addressing that seems to lead us back to cherry picking the commit. What am I missing?

The peer review of the release candidate branch pull request to master gives people a opportunity to fix and/or cherry-pick important bug fixes on the release candidate branch. A small few cherry-picked commits are easy to manage in each release candidate cycle. It's the hundreds of cherry-picks over 5 months, some of which were modified during the cherry-pick process, that makes maintaining a non-divergent master a challenge.... right now master and integration are 5 months apart on a genuine merge-base.
If you think of Henrique's idea of just tagging stable snapshots in the integration history, then master reference will just always be fast-forwarding to that latest stable snapshot. In essence the master reference "post-it note" just moves along integration's history to the latest snapshot tag in an ideal world of no cherry-picking at all. Going through the release candidate hoopla just gives us the opportunity to bug fix (with more recent integration commit "essential" cherry-picking or otherwise).
Anyway, there is more than one way to skin a cat - so (1) if any committer feels strongly enough that we should not do the monthly release candidate dance, then just just go to https://github.com/projectwonder/wonder/pull/243, type a comment "This sucks" and press "Close" :-P ;-) .... or (2) we can try it and if it sucks abandon the process, or (3) do nothing and let master flounder away into divergence oblivion :D

I don't have any good solutions given that we lack any semblance of comprehensive tests or a even a QA group.

I'm comfortable with that - sounds just like my work!! What are tests?

Tim

Post by Maik Musall
I was just pointing out a potential problem. But it seems you are still willing to pick some cherries so as long as we somehow remember which ones to pick, we are good.

Post by Kieran Kelleher
PS. More importantly, did you ever find the Redbreast 12 Cask Strength up there in Vancouver?

No such luck. I may have found a place that has the regular 12 year old.
Chuck

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

That also means that nobody has used exactly that one month old integration commit for longer than until the next commit came in, right? Is that a sufficient basis for stability?

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems.
http://www.global-village.net/gvc/practical_webobjects
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Wonder-disc mailing list
https://lists.sourceforge.net/lists/listinfo/wonder-disc

Maik Musall

2012-07-19 06:53:51 UTC

Hi Chuck,

Post by Kieran Kelleher
So, IMHO, we can keep everyone happy by continuing with two branches (just labels on commits) and someone (anyone with commit access) can periodically make a release candidate branch based on master, integrate one-month+ old integration, push it and announce the release candidate via pull request of it to master.

Well, the whole point of the release candidate is being able to test it, which only makes sense if found bugs are fixed before it becomes master. Now those fixes can either be new, in which case the fix commit would get applied to both rc and integration, or it exists already in integration as a later commit, in which case it is pushed to rc. You can call the latter cherry picking, but it sounds reasonable and natural to me...

Maik

John Huss

2012-07-18 20:19:37 UTC

Post by Kieran Kelleher
Good point, nevertheless, it is just a workflow. Branches and tags are
very similar in the sense that they are both generally just "post-it note"
references to commits in the object graph, however it is easier for less
adventurous users who want to work with source code and still want stable
source to have a stable "master" since checking out a tag (detached HEAD
and warning message) is not a good user experience, especially for those
not living on bleeding edge.
So, IMHO, we can keep everyone happy by continuing with two branches (just
labels on commits) and someone (anyone with commit access) can periodically
make a release candidate branch based on master, integrate one-month+ old
integration, push it and announce the release candidate via pull request of
it to master.
Wonder "Users" who want to work with the source (who doesn't want to learn
form wonder source?!) can clone and will, by default, get a stable master
checkout instead of having to *know* that they need to clone, then go find
a stable tag, then make a personal branch based on that and then check that
out. Let's keep it easy for Wonder "users" and those less familiar with the
layout.

If you're just USING Wonder there is no reason to checkout the source from
the Git repo. The source is already included in the binaries that are
available on jenkins, which make it available to be viewed in Eclipse,
debugged, and provides javadoc. Going back to the other post about
installation problems - he had a lot of problem related to trying to get
the wonder source code - when there was probably no need to do so! Unless
someone is planning to actual make changes to Wonder there is no need to
clone the repo with git.

Jesse Tayler

2012-07-18 15:12:01 UTC

so, we'll use a few volunteer guinea pig users as a live-regression test suite!

seriously, we need to push harder on updates, and getting changes pushed through to all the sites and developers.

clearly there are risks involved with this, everyone is going to have to update along with their own sense of safety.

I know we have put some tests in place, but of course - tests never pick up the issues you really want.

not to mention, every project has different safety concerns and most projects make use of only parts of WOnder.

Plus, our resources will never be so perfect as to just spend time on these things.

I heartily second the notion that we might produce a time-based automated stage (two weeks seems fine to me) if there were a few low level projects, some developer systems and various users willing to be on that trunk - it would likely weed out any significant bugs or errors before the code reaches those who really don't want their build to break in ways they don't understand.

my two cents.

Post by Maik Musall
Hi Kieran,

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

Kieran Kelleher

2012-07-18 16:25:32 UTC

Release Candidate branch is named 'rc-wonder-5.7.0'. Associated pull request (for discussion, etc.) is here https://github.com/projectwonder/wonder/pull/243

Those who are interested can check out the 'rc-wonder-5.7.0' branch. Test it with your apps/frameworks. If you find a bug, fix it. No committed bug fixes by Aug 1st will result in it getting merged to master :-)

And yeah, contributors without direct commit access to wonder can do a pull request for bug fixes with the rc-wonder-5.7.0 branch as the target - just make sure the bug fix is based on that branch.

This is a temporary branch and it will be deleted once merged to master and the final merge commit will be tagged as wonder-5.7.0

Cheers, Kieran

Post by Jesse Tayler
so, we'll use a few volunteer guinea pig users as a live-regression test suite!
seriously, we need to push harder on updates, and getting changes pushed through to all the sites and developers.
clearly there are risks involved with this, everyone is going to have to update along with their own sense of safety.
I know we have put some tests in place, but of course - tests never pick up the issues you really want.
not to mention, every project has different safety concerns and most projects make use of only parts of WOnder.
Plus, our resources will never be so perfect as to just spend time on these things.
I heartily second the notion that we might produce a time-based automated stage (two weeks seems fine to me) if there were a few low level projects, some developer systems and various users willing to be on that trunk - it would likely weed out any significant bugs or errors before the code reaches those who really don't want their build to break in ways they don't understand.
my two cents.

Post by Maik Musall
Hi Kieran,

1. An integration test of the periodic merge to make sure everything works fine with everything else after merging stuff that nobody was running in exactly that combination before.

Right now, the test is that most of us are using integration branch. Most Wonder committers use integration as their primary branch, and I know that other people use it to.

Pascal Robert

2012-07-18 17:16:37 UTC

For those who prefer to test it against the compiled frameworks:

http://jenkins.wocommunity.org/view/Wonder/job/WonderReleaseCandidate/lastSuccessfulBuild/artifact/Root/Roots/

Post by Kieran Kelleher
Release Candidate branch is named 'rc-wonder-5.7.0'. Associated pull request (for discussion, etc.) is here https://github.com/projectwonder/wonder/pull/243
Those who are interested can check out the 'rc-wonder-5.7.0' branch. Test it with your apps/frameworks. If you find a bug, fix it. No committed bug fixes by Aug 1st will result in it getting merged to master :-)
And yeah, contributors without direct commit access to wonder can do a pull request for bug fixes with the rc-wonder-5.7.0 branch as the target - just make sure the bug fix is based on that branch.
This is a temporary branch and it will be deleted once merged to master and the final merge commit will be tagged as wonder-5.7.0
Cheers, Kieran