Mailing List Archive

httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e))
On Mon, Feb 17, 2020 at 10:52:10AM -0500, Eric Covener wrote:
> On Mon, Feb 17, 2020 at 10:44 AM Joe Orton <jorton@redhat.com> wrote:
> >
> > On Mon, Feb 17, 2020 at 04:30:40PM +0100, Stefan Eissing wrote:
> > > Does this look like a Travis problem? Just red on some "Linux Ubuntu"
> > > combinations, but I do not really see the cause.
> >
> > Yep, it's a timeout doing the "svn checkout" because the test suite was
> > updated, you can ignore.
>
> Could we point travis at GitHub for that?

The Perl framework isn't mirrored to GitHub at the moment so we have to
get it from SVN. We could:

a) ask infra to mirror httpd/test/framework to some new github repo name
(httpd-test-framework? httpd-perl-tests? httpd-tests?)

b) move httpd/test/framework under trunk/test/framework or something

c) work out the underlying issue which makes Travis/SVN fail, my
assuption is that since a Travis build will hit SVN with 30+
near-simultaneous new connections which looks a bit DoS-like so it
gets rate-limited/banned by the ASF infra somehow, but it's a guess

I have a vague preference for (b) since it would be nice to be able to
commit tests and code at the same time but it'd be quite a big change.
(a) might be easier

Regards, Joe
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On 02/17/2020 05:31 PM, Joe Orton wrote:
> On Mon, Feb 17, 2020 at 10:52:10AM -0500, Eric Covener wrote:
>> On Mon, Feb 17, 2020 at 10:44 AM Joe Orton <jorton@redhat.com> wrote:
>>>
>>> On Mon, Feb 17, 2020 at 04:30:40PM +0100, Stefan Eissing wrote:
>>>> Does this look like a Travis problem? Just red on some "Linux Ubuntu"
>>>> combinations, but I do not really see the cause.
>>>
>>> Yep, it's a timeout doing the "svn checkout" because the test suite was
>>> updated, you can ignore.
>>
>> Could we point travis at GitHub for that?
>
> The Perl framework isn't mirrored to GitHub at the moment so we have to
> get it from SVN. We could:
>
> a) ask infra to mirror httpd/test/framework to some new github repo name
> (httpd-test-framework? httpd-perl-tests? httpd-tests?)
>
> b) move httpd/test/framework under trunk/test/framework or something
>
> c) work out the underlying issue which makes Travis/SVN fail, my
> assuption is that since a Travis build will hit SVN with 30+
> near-simultaneous new connections which looks a bit DoS-like so it
> gets rate-limited/banned by the ASF infra somehow, but it's a guess
>
> I have a vague preference for (b) since it would be nice to be able to
> commit tests and code at the same time but it'd be quite a big change.
> (a) might be easier
>

I am +1 on a) and +0 on b). Regarding c) I guess we need to discuss with infra if your assumption is true.
OTOH I guess once we have a) or b) in place c) becomes academic :-).

Regards

Rüdiger
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On Mon, Feb 17, 2020 at 2:40 PM Ruediger Pluem <rpluem@apache.org> wrote:
>
>
>
> On 02/17/2020 05:31 PM, Joe Orton wrote:
> > On Mon, Feb 17, 2020 at 10:52:10AM -0500, Eric Covener wrote:
> >> On Mon, Feb 17, 2020 at 10:44 AM Joe Orton <jorton@redhat.com> wrote:
> >>>
> >>> On Mon, Feb 17, 2020 at 04:30:40PM +0100, Stefan Eissing wrote:
> >>>> Does this look like a Travis problem? Just red on some "Linux Ubuntu"
> >>>> combinations, but I do not really see the cause.
> >>>
> >>> Yep, it's a timeout doing the "svn checkout" because the test suite was
> >>> updated, you can ignore.
> >>
> >> Could we point travis at GitHub for that?
> >
> > The Perl framework isn't mirrored to GitHub at the moment so we have to
> > get it from SVN. We could:
> >
> > a) ask infra to mirror httpd/test/framework to some new github repo name
> > (httpd-test-framework? httpd-perl-tests? httpd-tests?)
> >
> > b) move httpd/test/framework under trunk/test/framework or something
> >
> > c) work out the underlying issue which makes Travis/SVN fail, my
> > assuption is that since a Travis build will hit SVN with 30+
> > near-simultaneous new connections which looks a bit DoS-like so it
> > gets rate-limited/banned by the ASF infra somehow, but it's a guess
> >
> > I have a vague preference for (b) since it would be nice to be able to
> > commit tests and code at the same time but it'd be quite a big change.
> > (a) might be easier
> >
>
> I am +1 on a) and +0 on b). Regarding c) I guess we need to discuss with infra if your assumption is true.
> OTOH I guess once we have a) or b) in place c) becomes academic :-).

There is also the complicating factor of the svn:external for
https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On 02/17/2020 08:44 PM, Eric Covener wrote:
> On Mon, Feb 17, 2020 at 2:40 PM Ruediger Pluem <rpluem@apache.org> wrote:
>>
>>
>>
>> On 02/17/2020 05:31 PM, Joe Orton wrote:
>>> On Mon, Feb 17, 2020 at 10:52:10AM -0500, Eric Covener wrote:
>>>> On Mon, Feb 17, 2020 at 10:44 AM Joe Orton <jorton@redhat.com> wrote:
>>>>>
>>>>> On Mon, Feb 17, 2020 at 04:30:40PM +0100, Stefan Eissing wrote:
>>>>>> Does this look like a Travis problem? Just red on some "Linux Ubuntu"
>>>>>> combinations, but I do not really see the cause.
>>>>>
>>>>> Yep, it's a timeout doing the "svn checkout" because the test suite was
>>>>> updated, you can ignore.
>>>>
>>>> Could we point travis at GitHub for that?
>>>
>>> The Perl framework isn't mirrored to GitHub at the moment so we have to
>>> get it from SVN. We could:
>>>
>>> a) ask infra to mirror httpd/test/framework to some new github repo name
>>> (httpd-test-framework? httpd-perl-tests? httpd-tests?)
>>>
>>> b) move httpd/test/framework under trunk/test/framework or something
>>>
>>> c) work out the underlying issue which makes Travis/SVN fail, my
>>> assuption is that since a Travis build will hit SVN with 30+
>>> near-simultaneous new connections which looks a bit DoS-like so it
>>> gets rate-limited/banned by the ASF infra somehow, but it's a guess
>>>
>>> I have a vague preference for (b) since it would be nice to be able to
>>> commit tests and code at the same time but it'd be quite a big change.
>>> (a) might be easier
>>>
>>
>> I am +1 on a) and +0 on b). Regarding c) I guess we need to discuss with infra if your assumption is true.
>> OTOH I guess once we have a) or b) in place c) becomes academic :-).
>
> There is also the complicating factor of the svn:external for
> https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
>

Good point. Not sure how to map svn:external in the git world.

Regards

Rüdiger
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On 02/17/2020 09:02 PM, Ruediger Pluem wrote:
>
>
> On 02/17/2020 08:44 PM, Eric Covener wrote:
>> On Mon, Feb 17, 2020 at 2:40 PM Ruediger Pluem <rpluem@apache.org> wrote:
>>>
>>>
>>>
>>> On 02/17/2020 05:31 PM, Joe Orton wrote:
>>>> On Mon, Feb 17, 2020 at 10:52:10AM -0500, Eric Covener wrote:
>>>>> On Mon, Feb 17, 2020 at 10:44 AM Joe Orton <jorton@redhat.com> wrote:
>>>>>>
>>>>>> On Mon, Feb 17, 2020 at 04:30:40PM +0100, Stefan Eissing wrote:
>>>>>>> Does this look like a Travis problem? Just red on some "Linux Ubuntu"
>>>>>>> combinations, but I do not really see the cause.
>>>>>>
>>>>>> Yep, it's a timeout doing the "svn checkout" because the test suite was
>>>>>> updated, you can ignore.
>>>>>
>>>>> Could we point travis at GitHub for that?
>>>>
>>>> The Perl framework isn't mirrored to GitHub at the moment so we have to
>>>> get it from SVN. We could:
>>>>
>>>> a) ask infra to mirror httpd/test/framework to some new github repo name
>>>> (httpd-test-framework? httpd-perl-tests? httpd-tests?)
>>>>
>>>> b) move httpd/test/framework under trunk/test/framework or something
>>>>
>>>> c) work out the underlying issue which makes Travis/SVN fail, my
>>>> assuption is that since a Travis build will hit SVN with 30+
>>>> near-simultaneous new connections which looks a bit DoS-like so it
>>>> gets rate-limited/banned by the ASF infra somehow, but it's a guess
>>>>
>>>> I have a vague preference for (b) since it would be nice to be able to
>>>> commit tests and code at the same time but it'd be quite a big change.
>>>> (a) might be easier
>>>>
>>>
>>> I am +1 on a) and +0 on b). Regarding c) I guess we need to discuss with infra if your assumption is true.
>>> OTOH I guess once we have a) or b) in place c) becomes academic :-).
>>
>> There is also the complicating factor of the svn:external for
>> https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
>>
>
> Good point. Not sure how to map svn:external in the git world.

https://help.github.com/en/github/using-git/about-git-subtree-merges

seems to be an approach, but not sure how this works with our svn to git mirroring.

Regards

Rüdiger
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
> >> There is also the complicating factor of the svn:external for
> >> https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
> >>
> >
> > Good point. Not sure how to map svn:external in the git world.
>
> https://help.github.com/en/github/using-git/about-git-subtree-merges
>
> seems to be an approach, but not sure how this works with our svn to git mirroring.

IIUC We could also just use it from CPAN with a README about dropping
in an Apache-Test checkout if you need to debug the framework itself?
Other than that, we could use a vendor/ like approach and see what's
new around release time.
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On Mon, Feb 17, 2020 at 05:32:48PM -0500, Eric Covener wrote:
> > >> There is also the complicating factor of the svn:external for
> > >> https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
> > >>
> > >
> > > Good point. Not sure how to map svn:external in the git world.
> >
> > https://help.github.com/en/github/using-git/about-git-subtree-merges
> >
> > seems to be an approach, but not sure how this works with our svn to git mirroring.
>
> IIUC We could also just use it from CPAN with a README about dropping
> in an Apache-Test checkout if you need to debug the framework itself?
> Other than that, we could use a vendor/ like approach and see what's
> new around release time.

Yeah, so long as the existence of the svn:externals doesn't preclude
mirroring, I think we can safely ignore it. The test suite does seem to
work with an external install of Apache::Test just fine.

OK so I will ask for httpd/test/framework to be mirrored to github, but
first the really hard question - what to name it? Assuming we can pick
an arbitrary name, "httpd-tests" makes sense to me.

Regards, Joe
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On Wed, Feb 19, 2020 at 8:17 AM Joe Orton <jorton@redhat.com> wrote:
>
> On Mon, Feb 17, 2020 at 05:32:48PM -0500, Eric Covener wrote:
> > > >> There is also the complicating factor of the svn:external for
> > > >> https://svn.apache.org/repos/asf/perl/Apache-Test/trunk
> > > >>
> > > >
> > > > Good point. Not sure how to map svn:external in the git world.
> > >
> > > https://help.github.com/en/github/using-git/about-git-subtree-merges
> > >
> > > seems to be an approach, but not sure how this works with our svn to git mirroring.
> >
> > IIUC We could also just use it from CPAN with a README about dropping
> > in an Apache-Test checkout if you need to debug the framework itself?
> > Other than that, we could use a vendor/ like approach and see what's
> > new around release time.
>
> Yeah, so long as the existence of the svn:externals doesn't preclude
> mirroring, I think we can safely ignore it. The test suite does seem to
> work with an external install of Apache::Test just fine.
>
> OK so I will ask for httpd/test/framework to be mirrored to github, but
> first the really hard question - what to name it? Assuming we can pick
> an arbitrary name, "httpd-tests" makes sense to me.

works for me
AW: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
C2 General

> -----Ursprüngliche Nachricht-----
> Von: Eric Covener <covener@gmail.com>
> Gesendet: Mittwoch, 19. Februar 2020 14:20
> An: Apache HTTP Server Development List <dev@httpd.apache.org>
> Betreff: Re: httpd/test/framework location/mirroring (was Re: Errored:
> apache/httpd#357 (2.4.x - d56819e))
>
> On Wed, Feb 19, 2020 at 8:17 AM Joe Orton <jorton@redhat.com> wrote:
> >
> >
> > Yeah, so long as the existence of the svn:externals doesn't preclude
> > mirroring, I think we can safely ignore it. The test suite does seem
> to
> > work with an external install of Apache::Test just fine.
> >
> > OK so I will ask for httpd/test/framework to be mirrored to github,
> but
> > first the really hard question - what to name it? Assuming we can
> pick
> > an arbitrary name, "httpd-tests" makes sense to me.
>
> works for me

+1

Regards

Rüdiger
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On Wed, Feb 19, 2020 at 08:20:20AM -0500, Eric Covener wrote:
> On Wed, Feb 19, 2020 at 8:17 AM Joe Orton <jorton@redhat.com> wrote:
> > OK so I will ask for httpd/test/framework to be mirrored to github, but
> > first the really hard question - what to name it? Assuming we can pick
> > an arbitrary name, "httpd-tests" makes sense to me.
>
> works for me

Thanks folks. -> https://issues.apache.org/jira/browse/INFRA-19872
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On Thu, Feb 20, 2020 at 09:04:21AM +0000, Joe Orton wrote:
> On Wed, Feb 19, 2020 at 08:20:20AM -0500, Eric Covener wrote:
> > On Wed, Feb 19, 2020 at 8:17 AM Joe Orton <jorton@redhat.com> wrote:
> > > OK so I will ask for httpd/test/framework to be mirrored to github, but
> > > first the really hard question - what to name it? Assuming we can pick
> > > an arbitrary name, "httpd-tests" makes sense to me.
> >
> > works for me
>
> Thanks folks. -> https://issues.apache.org/jira/browse/INFRA-19872

OK Infra set this up at https://github.com/apache/httpd-tests

I srarted using this in a PR
https://travis-ci.org/apache/httpd/builds/655723094

and weirdly the tests failing everywhere in the same places:

Test Summary Report
-------------------
t/modules/allowmethods.t (Wstat: 0 Tests: 10 Failed: 1)
Failed test: 10
t/modules/proxy_fcgi.t (Wstat: 0 Tests: 26 Failed: 1)
Failed test: 26
Parse errors: Bad plan. You planned 42 tests but ran 26.

this seems extremely mysterious. The arm build is still using verbose
logging and the failures are:

# testing : Get - When Post/reset is allowed.
# expected: 200
# received: '405'
not ok 10
Failed 1/10 subtests

and I guess all the php-fpm failures are because it is failing to run
php-fpm properly, but, what has changed to make that happen?

[27-Feb-2020 10:16:12] ERROR: failed to open error_log (/home/travis/build/apache/httpd/test/perl-framework/t/php-fpm/log/php-fpm.log): No such file or directory (2)
[27-Feb-2020 10:16:12] ERROR: failed to post process the configuration
[27-Feb-2020 10:16:12] ERROR: FPM initialization failed
# Failed test 26 in t/modules/proxy_fcgi.t at line 245
Re: httpd/test/framework location/mirroring (was Re: Errored: apache/httpd#357 (2.4.x - d56819e)) [ In reply to ]
On Thu, Feb 27, 2020 at 11:30:58AM +0000, Joe Orton wrote:
> and weirdly the tests failing everywhere in the same places:

Obviously worked it out just after writing this, duh. It's because the
svn2git process drops empty directories, which we need preserved. Will
see if infra can fix this in the mirroring, otherwise we could work
around it.