On Wed, Feb 29, 2012 at 10:46:43AM -0500, George Greer wrote:
> On Wed, 29 Feb 2012, Nicholas Clark wrote:
>
> > The thing that *would* have got it is a "build all of CPAN", which we don't
> > have the resources to do for every change. So it gets down to - how do we
> > spot which changes need that sort of thing?
>
> If I can get a copy of the "build all of CPAN" script that the comparisons
> from a while ago were done with, I can try to set it up on a rolling
> basis. That is, it would compare blead from the last pass with whatever
> was blead at the start of the next pass. So it wouldn't be every change
> but it would at least be more than never.
I think you'd need to ask Steffen where that is
> >> Should then *all* changes go through a smoke-me change? I treat all of
> >
> > I'd actually be happy if (nearly) all non-doc changes *did*. But it wouldn't
> > have caught this one.
> >
> > (We'd need to improve the smoke-me reporting infrastructure to make this
> > useful though)
>
> What's on your wishlist?
I fear that this isn't complete, as I think I've forgotten something.
It's partly that (as best I can tell) the code you're running locally has
diverged from the "upstream" code, so it's unclear whether bugs fixes and
other improvements are getting made in more than one place, which is a
duplication of effort.
In particular, I'd like everyone else to run your code, because of a couple of
minor things:
* the subject using the branch name is terser
Smoke [blead] v5.15.9-20-g15d94df
vs the tautological
Smoke [5.15.9] v5.15.9-20-g15d94df
Also those 1 or 2 characters can make a difference when the most important
bit is actually the detail of PASS(...) or FAIL(...), which can fall off the
right
* the smoke-me code mails me directly if the branch fails
* I can get the logs
but also I'd like a couple of visibility bugs in your setup to be fixed:
X X O X X X O X -Uusenm -Duseithreads -Dmad
| | | | | +- LC_ALL = en_US.utf8 -DDEBUGGING
| | | | +--- PERLIO = perlio -DDEBUGGING
| | | +----- PERLIO = stdio -DDEBUGGING
| | +------- LC_ALL = en_US.utf8
| +--------- PERLIO = perlio
+----------- PERLIO = stdio
8 results vs 6 annotations, or 4 results vs 2 annotations:
O F F F
O F F F -Duseithreads
| +--------- -DDEBUGGING
+----------- no debugging
after which I guess that there are more general skimming issues with the
smoke output. I'm familiar with it, and I find it easy to read, but others
are not paying attention to the smoke output because they perceive it as
impenetrable noise. It would be useful force those people to explain what
they find most obnoxious about it, fix that, iterate until they run out of
complaints.
First off, I'm not sure whether the line "Summary: PASS" (or FAIL...) should
be the first line, with 2 (or 3) blank lines beneath it. But I'm not the
target for such improvements - really the monthly release managers are the
people whose input we should be getting.
Nicholas Clark
> On Wed, 29 Feb 2012, Nicholas Clark wrote:
>
> > The thing that *would* have got it is a "build all of CPAN", which we don't
> > have the resources to do for every change. So it gets down to - how do we
> > spot which changes need that sort of thing?
>
> If I can get a copy of the "build all of CPAN" script that the comparisons
> from a while ago were done with, I can try to set it up on a rolling
> basis. That is, it would compare blead from the last pass with whatever
> was blead at the start of the next pass. So it wouldn't be every change
> but it would at least be more than never.
I think you'd need to ask Steffen where that is
> >> Should then *all* changes go through a smoke-me change? I treat all of
> >
> > I'd actually be happy if (nearly) all non-doc changes *did*. But it wouldn't
> > have caught this one.
> >
> > (We'd need to improve the smoke-me reporting infrastructure to make this
> > useful though)
>
> What's on your wishlist?
I fear that this isn't complete, as I think I've forgotten something.
It's partly that (as best I can tell) the code you're running locally has
diverged from the "upstream" code, so it's unclear whether bugs fixes and
other improvements are getting made in more than one place, which is a
duplication of effort.
In particular, I'd like everyone else to run your code, because of a couple of
minor things:
* the subject using the branch name is terser
Smoke [blead] v5.15.9-20-g15d94df
vs the tautological
Smoke [5.15.9] v5.15.9-20-g15d94df
Also those 1 or 2 characters can make a difference when the most important
bit is actually the detail of PASS(...) or FAIL(...), which can fall off the
right
* the smoke-me code mails me directly if the branch fails
* I can get the logs
but also I'd like a couple of visibility bugs in your setup to be fixed:
X X O X X X O X -Uusenm -Duseithreads -Dmad
| | | | | +- LC_ALL = en_US.utf8 -DDEBUGGING
| | | | +--- PERLIO = perlio -DDEBUGGING
| | | +----- PERLIO = stdio -DDEBUGGING
| | +------- LC_ALL = en_US.utf8
| +--------- PERLIO = perlio
+----------- PERLIO = stdio
8 results vs 6 annotations, or 4 results vs 2 annotations:
O F F F
O F F F -Duseithreads
| +--------- -DDEBUGGING
+----------- no debugging
after which I guess that there are more general skimming issues with the
smoke output. I'm familiar with it, and I find it easy to read, but others
are not paying attention to the smoke output because they perceive it as
impenetrable noise. It would be useful force those people to explain what
they find most obnoxious about it, fix that, iterate until they run out of
complaints.
First off, I'm not sure whether the line "Summary: PASS" (or FAIL...) should
be the first line, with 2 (or 3) blank lines beneath it. But I'm not the
target for such improvements - really the monthly release managers are the
people whose input we should be getting.
Nicholas Clark