AMD64 Arch Testers needed urgently

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

AMD64 Arch Testers needed urgently

Michał Górny-5
Hi, everyone.

It seems that we've started lacking arch testers for AMD64 architecture.
At this moment, there are already 159 bugs in amd64 backlog, and there
is no noticeable progress. New stabilization requests are usually
handled much faster by x86, sparc and hppa teams!

If you have a stable AMD64 system and would like to help arch testing,
please do! I don't know what exact rules for that are but I think [1]
could be helpful. If someone knows better, then please share.

[1]:https://wiki.gentoo.org/wiki/Project:AMD64_Arch_Testers

--
Best regards,
Michał Górny


Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Rich Freeman
On Tue, Dec 12, 2017 at 1:15 PM, Michał Górny <[hidden email]> wrote:

>
> It seems that we've started lacking arch testers for AMD64 architecture.
> At this moment, there are already 159 bugs in amd64 backlog, and there
> is no noticeable progress. New stabilization requests are usually
> handled much faster by x86, sparc and hppa teams!
>
> If you have a stable AMD64 system and would like to help arch testing,
> please do! I don't know what exact rules for that are but I think [1]
> could be helpful. If someone knows better, then please share.
>

As far as I'm aware the standing policy already exists that
maintainers can stabilize their own packages on amd64.

That said, if somebody wants to organize an AT program for amd64 they
should feel free to do so.  I could explain how things used to work at
least if that is helpful.

The old way it used to work is that ATs had to pass the ebuild quiz
and they would get editbugs privs to tag bugs as tested.  I won't
elaborate here unless there is interest...

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Francesco Riosa-3


On 12/12/2017 19:24, Rich Freeman wrote:

> On Tue, Dec 12, 2017 at 1:15 PM, Michał Górny <[hidden email]> wrote:
>> It seems that we've started lacking arch testers for AMD64 architecture.
>> At this moment, there are already 159 bugs in amd64 backlog, and there
>> is no noticeable progress. New stabilization requests are usually
>> handled much faster by x86, sparc and hppa teams!
>>
>> If you have a stable AMD64 system and would like to help arch testing,
>> please do! I don't know what exact rules for that are but I think [1]
>> could be helpful. If someone knows better, then please share.
>>
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.

It would be interesting to discuss if proxy maintainers should be able
to stabilize too.


>
> That said, if somebody wants to organize an AT program for amd64 they
> should feel free to do so.  I could explain how things used to work at
> least if that is helpful.
>
> The old way it used to work is that ATs had to pass the ebuild quiz
> and they would get editbugs privs to tag bugs as tested.  I won't
> elaborate here unless there is interest...
>


Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

R0b0t1
In reply to this post by Rich Freeman
On Tue, Dec 12, 2017 at 12:24 PM, Rich Freeman <[hidden email]> wrote:

> On Tue, Dec 12, 2017 at 1:15 PM, Michał Górny <[hidden email]> wrote:
>>
>> It seems that we've started lacking arch testers for AMD64 architecture.
>> At this moment, there are already 159 bugs in amd64 backlog, and there
>> is no noticeable progress. New stabilization requests are usually
>> handled much faster by x86, sparc and hppa teams!
>>
>> If you have a stable AMD64 system and would like to help arch testing,
>> please do! I don't know what exact rules for that are but I think [1]
>> could be helpful. If someone knows better, then please share.
>>
>
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.
>
> That said, if somebody wants to organize an AT program for amd64 they
> should feel free to do so.  I could explain how things used to work at
> least if that is helpful.
>
> The old way it used to work is that ATs had to pass the ebuild quiz
> and they would get editbugs privs to tag bugs as tested.  I won't
> elaborate here unless there is interest...
>

I would like to know. But on the other hand, anyone interested in
contributing to packages they work with is likely already doing so. On
the third (and final?) hand, it may also be that there are people
looking for direction.

Cheers,
    R0b0t1

Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Thomas Deutschmann
In reply to this post by Rich Freeman
On 2017-12-12 19:24, Rich Freeman wrote:
> As far as I'm aware the standing policy already exists that
> maintainers can stabilize their own packages on amd64.

That's right but keep in mind that nevertheless you need a stable
system. Marking a package stable because it works on your ~arch box you
use for your daily dev work would lead the whole process ad absurdum.

And in general maintainer stabilization should be the last resort. The
person who wrote the ebuild maybe doesn't notice that the ebuild is
doing something wrong (doesn't honor CFLAGS, calls compiler directly,
not working with /bin/sh not /bin/bash ...).


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (1000 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

William Hubbs
On Wed, Dec 13, 2017 at 01:22:04PM +0100, Thomas Deutschmann wrote:
> On 2017-12-12 19:24, Rich Freeman wrote:
> > As far as I'm aware the standing policy already exists that
> > maintainers can stabilize their own packages on amd64.
>
> That's right but keep in mind that nevertheless you need a stable
> system. Marking a package stable because it works on your ~arch box you
> use for your daily dev work would lead the whole process ad absurdum.

In  my discussions with other developers, I've found that this is the
biggest concern. Most devs are runnning ~amd64, so they don't feel that
they can mark things stable.

> And in general maintainer stabilization should be the last resort. The
> person who wrote the ebuild maybe doesn't notice that the ebuild is
> doing something wrong (doesn't honor CFLAGS, calls compiler directly,
> not working with /bin/sh not /bin/bash ...).

In theory, this is correct. However, when maintainers don't stabilize
packages and no one else does either, our stable tree suffers.

William

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Lucas Ramage
In  my discussions with other developers, I've found that this is the
​> ​
biggest concern. Most devs are runnning ~amd64, so they don't feel that

​> ​
they can mark things stable.

W
​hat about running a stable chroot?​ Are there any tools that can be used to automate this process?

On Wed, Dec 13, 2017 at 12:51 PM, William Hubbs <[hidden email]> wrote:
On Wed, Dec 13, 2017 at 01:22:04PM +0100, Thomas Deutschmann wrote:
> On 2017-12-12 19:24, Rich Freeman wrote:
> > As far as I'm aware the standing policy already exists that
> > maintainers can stabilize their own packages on amd64.
>
> That's right but keep in mind that nevertheless you need a stable
> system. Marking a package stable because it works on your ~arch box you
> use for your daily dev work would lead the whole process ad absurdum.

In  my discussions with other developers, I've found that this is the
biggest concern. Most devs are runnning ~amd64, so they don't feel that
they can mark things stable.

> And in general maintainer stabilization should be the last resort. The
> person who wrote the ebuild maybe doesn't notice that the ebuild is
> doing something wrong (doesn't honor CFLAGS, calls compiler directly,
> not working with /bin/sh not /bin/bash ...).

In theory, this is correct. However, when maintainers don't stabilize
packages and no one else does either, our stable tree suffers.

William



--
Regards,

Visit online journal

Lucas Ramage / Software Engineer
[hidden email] / (941) 404-6794

PGP Fingerprint / Learn More
EAE7 45DF 818D 4948 DDA7 0F44 F52A 5A96 7B9B 6FB7

Visit online journal
http://lramage94.github.io

Github Linkedin

Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Aaron W. Swenson-2
On 2017-12-13 13:20, Lucas Ramage wrote:
> > In  my discussions with other developers, I've found that this is the
> ​> ​
> biggest concern. Most devs are runnning ~amd64, so they don't feel that
> ​> ​
> they can mark things stable.
>
> W
> ​hat about running a stable chroot?​ Are there any tools that can be used
> to automate this process?

Yes, a stable chroot is okay, and there is app-portage/tatt that can run
through the various USE flag combinations for you.

It’s output isn’t terribly helpful, so it takes a little while to
understand what it’s trying to tell you.

signature.asc (386 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Lucas Ramage
I see, well I can setup buildbot to do that. Is there some place in particular that I should send my test results?

On Wed, Dec 13, 2017 at 1:28 PM, Aaron W. Swenson <[hidden email]> wrote:
On 2017-12-13 13:20, Lucas Ramage wrote:
> > In  my discussions with other developers, I've found that this is the
> ​> ​
> biggest concern. Most devs are runnning ~amd64, so they don't feel that
> ​> ​
> they can mark things stable.
>
> W
> ​hat about running a stable chroot?​ Are there any tools that can be used
> to automate this process?

Yes, a stable chroot is okay, and there is app-portage/tatt that can run
through the various USE flag combinations for you.

It’s output isn’t terribly helpful, so it takes a little while to
understand what it’s trying to tell you.



--
Regards,

Visit online journal

Lucas Ramage / Software Engineer
[hidden email] / (941) 404-6794

PGP Fingerprint / Learn More
EAE7 45DF 818D 4948 DDA7 0F44 F52A 5A96 7B9B 6FB7

Visit online journal
http://lramage94.github.io

Github Linkedin

Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Thomas Deutschmann
In reply to this post by William Hubbs
On 2017-12-13 18:51, William Hubbs wrote:
> In theory, this is correct. However, when maintainers don't stabilize
> packages and no one else does either, our stable tree suffers.

I agree but we have to pay attention that we don't stabilize packages at
all costs because otherwise they would never go stable.

If this is the problem then we should discuss stabilization at all. What
do people expect from something marked stable vs. reality. ;)

And in this case I would prefer a system like Debian SID -> Testing
supported by build bots. I can think of 2 variants:

a) Once maintainer files a stabilization request, a build bot will pick
up the bug and try building the package in a chroot per architecture. If
everything passes build bot will mark the version stable for the tested
architecture.

A flag or a blacklist could prevent build bot stabilization.


b) Because not all devs care about stable Gentoo, I would recommend
auto-stabilization: I.e. if a package is in the repository for x days
build bot would try to build the package and mark the package stable if
everything passes. If for some reason maintainer want to block a
specific version they could create a bug or set a flag in an already
existing bug which will cause build bot to ignore this version.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (1000 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Kent Fredric-2
On Wed, 13 Dec 2017 21:58:05 +0100
Thomas Deutschmann <[hidden email]> wrote:

> b) Because not all devs care about stable Gentoo, I would recommend
> auto-stabilization: I.e. if a package is in the repository for x days
> build bot would try to build the package and mark the package stable
> if everything passes. If for some reason maintainer want to block a
> specific version they could create a bug or set a flag in an already
> existing bug which will cause build bot to ignore this version.

Slightly modified suggestion:

Add a flag called "autostabilize" with [unset], [y], [n]

Default is 'unset', and if found unset after a given time, it flips to
y and the stable bot gets queued up.

If its set to 'n', then stable bot never does anything.

This way maintainers who want to rush the stablebot on things they
consider "safe" can get ahead of the queue.


[nb: unsigned mail, pinentry currently broken due to -fPIC profile
migration]


Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Thomas Deutschmann
On 2017-12-14 01:58, Kent Fredric wrote:

> On Wed, 13 Dec 2017 21:58:05 +0100
> Thomas Deutschmann <[hidden email]> wrote:
>
>> b) Because not all devs care about stable Gentoo, I would recommend
>> auto-stabilization: I.e. if a package is in the repository for x days
>> build bot would try to build the package and mark the package stable
>> if everything passes. If for some reason maintainer want to block a
>> specific version they could create a bug or set a flag in an already
>> existing bug which will cause build bot to ignore this version.
>
> Slightly modified suggestion:
>
> Add a flag called "autostabilize" with [unset], [y], [n]
Flag in Bugzilla?


> Default is 'unset', and if found unset after a given time, it flips to
> y and the stable bot gets queued up.
>
> If its set to 'n', then stable bot never does anything.
>
> This way maintainers who want to rush the stablebot on things they
> consider "safe" can get ahead of the queue.

Sounds good but the variant b was about full auto-stabilization. I.e.
the idea that we even don't require stabilization bugs anymore (like
Debian, where a package will move from SID to testing after some time if
nobody created a blocker bug).

We could auto-generate bugs but I am not sure if this is a good idea.
When mgorny set up autolinking of Github PRs there was some "bug spam"
when people lost control over Git and messed up the rebase (now
prevented via limits).

Also I am not sure how we should handle things like Gnome/KDE
stabilization which requires that a bunch of packages will be stabilized
at once. I.e. if bot detects a new ebuild of kde-frameworks/kservice
auto-stabilization is only allowed to kick in if it is a rev bump of an
already stable ebuild but shouldn't auto-start stabilization of next KDE
version on its own (at least I guess this is what we want).

But well, for the beginning we don't need the perfect solution. We can
start with an easy mode and blacklist most packages. So devs interested
can remove their packages from blacklist. And like said, build bot would
still handle stabilization bugs.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (1000 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Rich Freeman
On Thu, Dec 14, 2017 at 5:39 AM, Thomas Deutschmann <[hidden email]> wrote:
>
> But well, for the beginning we don't need the perfect solution. We can
> start with an easy mode and blacklist most packages. So devs interested
> can remove their packages from blacklist. And like said, build bot would
> still handle stabilization bugs.
>

I think this has all been gone over in a list post previously, but it
seems like the most straightforward solution here is flags in metadata
where individual packages can be flagged for auto-stabilization.

In the beginning the system would be opt-in.  Then once we have
confidence that it is working well the flag could potentially be made
opt-out.

Maybe initially for testing you could keep the whitelist outside of
the repository, but then you need to make it straightforward for
maintainers to have their packages added to the whitelist if they
wish.  Long-term if this becomes the standard workflow it would make
sense for the flags to go inside the repository.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Aaron W. Swenson-2
In reply to this post by Kent Fredric-2
On 2017-12-14 13:58, Kent Fredric wrote:

> On Wed, 13 Dec 2017 21:58:05 +0100
> Slightly modified suggestion:
>
> Add a flag called "autostabilize" with [unset], [y], [n]
>
> Default is 'unset', and if found unset after a given time, it flips to
> y and the stable bot gets queued up.
>
> If its set to 'n', then stable bot never does anything.
>
> This way maintainers who want to rush the stablebot on things they
> consider "safe" can get ahead of the queue.
Well, we kind of have that already with “Runtime testing required” (RTR).

RTR=no stablereqs are good candidates for automation.

If everything has been properly handled in the ebuild, then an emerge
should succeed. And, RTR being set to “No” indicates that there aren’t any
tests to perform other than those defined and handled within the ebuild.

Restricting the set to RTR=no would eliminate special cases needing to
be taken into consideration.

Of course, RTR being unset or set to yes should be skipped.

We could initially start with stablebot only adding a comment to a bug
that it thinks it’s safe to stabilize the subject. This would give us
some time to gain confidence in it and/or weed out the bugs.

signature.asc (386 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Rich Freeman
On Thu, Dec 14, 2017 at 7:07 AM, Aaron W. Swenson <[hidden email]> wrote:

> On 2017-12-14 13:58, Kent Fredric wrote:
>> On Wed, 13 Dec 2017 21:58:05 +0100
>> Slightly modified suggestion:
>>
>> Add a flag called "autostabilize" with [unset], [y], [n]
>>
>> Default is 'unset', and if found unset after a given time, it flips to
>> y and the stable bot gets queued up.
>>
>> If its set to 'n', then stable bot never does anything.
>>
>> This way maintainers who want to rush the stablebot on things they
>> consider "safe" can get ahead of the queue.
>
> Well, we kind of have that already with “Runtime testing required” (RTR).
>
> RTR=no stablereqs are good candidates for automation.
>
> If everything has been properly handled in the ebuild, then an emerge
> should succeed. And, RTR being set to “No” indicates that there aren’t any
> tests to perform other than those defined and handled within the ebuild.

I'm not sure runtime testing is the only use case for preventing
auto-stabilization.

The example of KDE/Gnome was brought up where large number of packages
need to be stabilized at the same time.  I'm not sure this is actually
a factor that should prevent auto-stabilization, since these packages
should have the correct dependencies and the stabilization system
would definitely need to be dependency-aware (which would lead to them
being stabilized as a group automatically).  If these require runtime
testing then they fall into the existing use case.

Does anybody actually have a reason to prevent stabilization other
than for runtime testing, assuming that stabilization is done in a
manner where all dependencies are satisfied at all times?

Maybe runtime testing really ought to be the only reason to prevent
auto stabilization...

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Harald Alfred Weiner
In reply to this post by Thomas Deutschmann
Dear Thomas and everyone else interested!

Before re-inventing the wheel, you might take a closer look at this Google summer of code project in 2016:
https://wiki.gentoo.org/wiki/Google_Summer_of_Code/2016/Ideas/Continuous_Stabilization

I do not know how far the author got but it might be a good starting point...

Best wishes,

Harald.

>>> Thomas Deutschmann <[hidden email]> 12/13/17 9:59 PM >>>


> auto-stabilization: I.e. if a package is in the repository for x days
> build bot would try to build the package and mark the package stable if
> everything passes. If for some reason maintainer want to block a
> specific version they could create a bug or set a flag in an already
> existing bug which will cause build bot to ignore this version.


Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

William Hubbs
In reply to this post by Thomas Deutschmann
On Wed, Dec 13, 2017 at 09:58:05PM +0100, Thomas Deutschmann wrote:

*snip*

> b) Because not all devs care about stable Gentoo, I would recommend
> auto-stabilization: I.e. if a package is in the repository for x days
> build bot would try to build the package and mark the package stable if
> everything passes. If for some reason maintainer want to block a
> specific version they could create a bug or set a flag in an already
> existing bug which will cause build bot to ignore this version.

I tend to like this better. Let's try to move away from filing stable
requests for new versions of packages once an old version is stable and
have a way to block newer versions from going stable. Maybe buildbot
could check to see if there is a bug open against the version it is
looking at, then check the keywords or severity of that bug and  use one
or the other of those to decide whether or not to skip stabilizing that version
of the package.

I think something else we should add to this is that buildbot should do
nothing if there isn't already a stable version of the package on the
architecture it is testing.

In other words, the first stabilization for a package on an architecture
should be done the way we currently do them, by filing a stable request
then using the current stabilization process.

William


signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

R0b0t1
In reply to this post by Lucas Ramage
On Wed, Dec 13, 2017 at 2:55 PM, Lucas Ramage <[hidden email]> wrote:
I see, well I can setup buildbot to do that. Is there some place in particular that I should send my test results?

Is this part of the point of a Tinderbox? The only problem I can see is that the configurations being tested can be extremely hard to replicate and lead to sporadic errors.

In response to the concerns about stability: If I run a lot of unstable packages, would that preclude my system from being able to help?

Cheers,
     R0b0t1
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

Thomas Deutschmann
On 2017-12-14 21:06, R0b0t1 wrote:
> In response to the concerns about stability: If I run a lot of unstable
> packages, would that preclude my system from being able to help?

Yes. Only clean stable systems are eligible for arch testing. That's the
whole idea of arch testing... ;)

Remember that mixed systems aren't officially supported.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (1000 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMD64 Arch Testers needed urgently

R0b0t1
On Thu, Dec 14, 2017 at 2:13 PM, Thomas Deutschmann <[hidden email]> wrote:
> On 2017-12-14 21:06, R0b0t1 wrote:
>> In response to the concerns about stability: If I run a lot of unstable
>> packages, would that preclude my system from being able to help?
>
> Yes. Only clean stable systems are eligible for arch testing. That's the
> whole idea of arch testing... ;)
>
> Remember that mixed systems aren't officially supported.
>

It seems like lagging stability is due to a lack of resources. I do
not know a single person who would be able to run only stable
packages. They seem to move too slowly, and people switch to unstable
packages because they contain bugfixes and sometimes new features.

Could the criteria for stability be reconsidered? Mixed systems might
not be supported, but save for cases of ABI/API breakage (which can
happen when transitioning from stable->stable) I do not know why the
packages would not play well with each other. I am sure there are
examples where things have blown up, but it seems like expecting that
to be the case isn't helping.

Cheers,
     R0b0t1

12