rfc: noarch keyword

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

rfc: noarch keyword

William Hubbs
All,

this came up again on the recent thread about dropping non x86/amd64
support for python packages, and I want to bring it up again on its own
thread.

How often do architecture specific bugs really exist in languages like
perl, python etc? From what I've seen they are pretty rare. Not to mention,
if we found one somewhere, we could adjust keywords as necessary.

Also, if someone did inadvertently keyword a package with noarch that didn't
work everywhere, it would be a matter of adjusting the keywords for that
package.

So, my question is, why can't we add a noarch/~noarch keyword and see
how things go? If it gets abused we can always nuke it later.

Thanks,

William


signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Jaco Kroon
Hi,

I'd be in support.  Especially for "data only" kind of packages, like:

net-misc/asterisk-moh-opsound
net-misc/asterisk-extra-sounds
net-misc/asterisk-core-sounds

For all three these I've already dropped the DEPEND on net-misc/asterisk
anyway, and upgraded the PDEPEND on net-misc/asterisk back onto these to
RDEPEND.  Other than that they really only install a bunch of audio
files in various formats.

One could even mark the various acct-{user,group}/* packages this way
(although this is a simple eclass change).

One challenge I foresee is that one could have a perl, python or php
(script language of choice) package depend on a specific version of
interpreter, which may not be stable for given arch.  So may require
some special handling in terms of dependencies.

Kind Regards,
Jaco


On 2020/03/18 16:54, William Hubbs wrote:

> All,
>
> this came up again on the recent thread about dropping non x86/amd64
> support for python packages, and I want to bring it up again on its own
> thread.
>
> How often do architecture specific bugs really exist in languages like
> perl, python etc? From what I've seen they are pretty rare. Not to mention,
> if we found one somewhere, we could adjust keywords as necessary.
>
> Also, if someone did inadvertently keyword a package with noarch that didn't
> work everywhere, it would be a matter of adjusting the keywords for that
> package.
>
> So, my question is, why can't we add a noarch/~noarch keyword and see
> how things go? If it gets abused we can always nuke it later.
>
> Thanks,
>
> William
>

Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Rolf Eike Beer
In reply to this post by William Hubbs
Am 2020-03-18 15:54, schrieb William Hubbs:

> All,
>
> this came up again on the recent thread about dropping non x86/amd64
> support for python packages, and I want to bring it up again on its own
> thread.
>
> How often do architecture specific bugs really exist in languages like
> perl, python etc? From what I've seen they are pretty rare. Not to
> mention,
> if we found one somewhere, we could adjust keywords as necessary.

I'm not judging this proposal, just a data point: packages that e.g.
read from /proc, especially /proc/cpuinfo, get easily blow up on
architecture changes. See https://bugs.gentoo.org/663424 for an example.

Eike

Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Rolf Eike Beer
In reply to this post by Jaco Kroon
Am 2020-03-18 16:10, schrieb Jaco Kroon:
> Hi,
>
> I'd be in support.  Especially for "data only" kind of packages, like:
>
> net-misc/asterisk-moh-opsound
> net-misc/asterisk-extra-sounds
> net-misc/asterisk-core-sounds

My immediate target was aspell dictionaries and fonts.

Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

William Hubbs
In reply to this post by Rolf Eike Beer
On Wed, Mar 18, 2020 at 05:11:17PM +0100, Rolf Eike Beer wrote:

> Am 2020-03-18 15:54, schrieb William Hubbs:
> > All,
> >
> > this came up again on the recent thread about dropping non x86/amd64
> > support for python packages, and I want to bring it up again on its own
> > thread.
> >
> > How often do architecture specific bugs really exist in languages like
> > perl, python etc? From what I've seen they are pretty rare. Not to
> > mention,
> > if we found one somewhere, we could adjust keywords as necessary.
>
> I'm not judging this proposal, just a data point: packages that e.g.
> read from /proc, especially /proc/cpuinfo, get easily blow up on
> architecture changes. See https://bugs.gentoo.org/663424 for an example.
 
 Sure, but if you run into something like that you just don't use the
 noarch keyword for those packages.

 Thanks,

 William

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Michał Górny-5
In reply to this post by William Hubbs
On Wed, 2020-03-18 at 09:54 -0500, William Hubbs wrote:

> this came up again on the recent thread about dropping non x86/amd64
> support for python packages, and I want to bring it up again on its own
> thread.
>
> How often do architecture specific bugs really exist in languages like
> perl, python etc? From what I've seen they are pretty rare. Not to mention,
> if we found one somewhere, we could adjust keywords as necessary.
>
> Also, if someone did inadvertently keyword a package with noarch that didn't
> work everywhere, it would be a matter of adjusting the keywords for that
> package.
>
> So, my question is, why can't we add a noarch/~noarch keyword and see
> how things go? If it gets abused we can always nuke it later.
>
1. How is this going to work when noarch package depends on non-nonarch
package?  I mean, will all the package managers actually work?  Have you
did some minimal testing before bringing this up?

2. Who will be responsible for handling noarch stablereqs?  Will there
be a noarch arch team?

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

William Hubbs
On Wed, Mar 18, 2020 at 06:12:37PM +0100, Michał Górny wrote:

> On Wed, 2020-03-18 at 09:54 -0500, William Hubbs wrote:
> > this came up again on the recent thread about dropping non x86/amd64
> > support for python packages, and I want to bring it up again on its own
> > thread.
> >
> > How often do architecture specific bugs really exist in languages like
> > perl, python etc? From what I've seen they are pretty rare. Not to mention,
> > if we found one somewhere, we could adjust keywords as necessary.
> >
> > Also, if someone did inadvertently keyword a package with noarch that didn't
> > work everywhere, it would be a matter of adjusting the keywords for that
> > package.
> >
> > So, my question is, why can't we add a noarch/~noarch keyword and see
> > how things go? If it gets abused we can always nuke it later.
> >
>
> 1. How is this going to work when noarch package depends on non-nonarch
> package?  I mean, will all the package managers actually work?  Have you
> did some minimal testing before bringing this up?
 
Can you have multiple ACCEPT_KEYWORDS values in make.conf or
make.defaults like this?

 ACCEPT_KEYWORDS="amd64 noarch"

If so, things should just work.

Currently I don't know of any arch/package combos to test this with.

> 2. Who will be responsible for handling noarch stablereqs?  Will there
> be a noarch arch team?

The maintainer would be able to add the "~noarch" keyword. I'm not sure
there needs to be a noarch arch team. We could just say that all arch
team members can stabilize these or maybe the maintainers can afterh the
timeout.

William

> --
> Best regards,
> Michał Górny
>



signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

James Le Cuirot
On Wed, 18 Mar 2020 12:47:53 -0500
William Hubbs <[hidden email]> wrote:

> On Wed, Mar 18, 2020 at 06:12:37PM +0100, Michał Górny wrote:
> > On Wed, 2020-03-18 at 09:54 -0500, William Hubbs wrote:  
> > > this came up again on the recent thread about dropping non x86/amd64
> > > support for python packages, and I want to bring it up again on its own
> > > thread.
> > >
> > > How often do architecture specific bugs really exist in languages like
> > > perl, python etc? From what I've seen they are pretty rare. Not to mention,
> > > if we found one somewhere, we could adjust keywords as necessary.
> > >
> > > Also, if someone did inadvertently keyword a package with noarch that didn't
> > > work everywhere, it would be a matter of adjusting the keywords for that
> > > package.
> > >
> > > So, my question is, why can't we add a noarch/~noarch keyword and see
> > > how things go? If it gets abused we can always nuke it later.
> > >  
> >
> > 1. How is this going to work when noarch package depends on non-nonarch
> > package?  I mean, will all the package managers actually work?  Have you
> > did some minimal testing before bringing this up?  
>  
> Can you have multiple ACCEPT_KEYWORDS values in make.conf or
> make.defaults like this?
>
>  ACCEPT_KEYWORDS="amd64 noarch"
>
> If so, things should just work.
Not quite. Tools like repoman will need to be updated to understand
that an ebuild with KEYWORDS="amd64" can depend on another ebuild with
only KEYWORDS="noarch". I do think the idea has merit though.

--
James Le Cuirot (chewi)
Gentoo Linux Developer

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Michał Górny-5
In reply to this post by William Hubbs
On Wed, 2020-03-18 at 12:47 -0500, William Hubbs wrote:

> On Wed, Mar 18, 2020 at 06:12:37PM +0100, Michał Górny wrote:
> > On Wed, 2020-03-18 at 09:54 -0500, William Hubbs wrote:
> > > this came up again on the recent thread about dropping non x86/amd64
> > > support for python packages, and I want to bring it up again on its own
> > > thread.
> > >
> > > How often do architecture specific bugs really exist in languages like
> > > perl, python etc? From what I've seen they are pretty rare. Not to mention,
> > > if we found one somewhere, we could adjust keywords as necessary.
> > >
> > > Also, if someone did inadvertently keyword a package with noarch that didn't
> > > work everywhere, it would be a matter of adjusting the keywords for that
> > > package.
> > >
> > > So, my question is, why can't we add a noarch/~noarch keyword and see
> > > how things go? If it gets abused we can always nuke it later.
> > >
> >
> > 1. How is this going to work when noarch package depends on non-nonarch
> > package?  I mean, will all the package managers actually work?  Have you
> > did some minimal testing before bringing this up?
>  
> Can you have multiple ACCEPT_KEYWORDS values in make.conf or
> make.defaults like this?
>
>  ACCEPT_KEYWORDS="amd64 noarch"
>
> If so, things should just work.
>
> Currently I don't know of any arch/package combos to test this with.
I'm talking about repoman/pkgcheck.

> > 2. Who will be responsible for handling noarch stablereqs?  Will there
> > be a noarch arch team?
>
> The maintainer would be able to add the "~noarch" keyword. I'm not sure
> there needs to be a noarch arch team. We could just say that all arch
> team members can stabilize these or maybe the maintainers can afterh the
> timeout.
>

Would you CC all arch teams on the bug then?

We have ALLARCHES already, and to my experience most arch teams fail to
handle that.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

William Hubbs
In reply to this post by James Le Cuirot
On Wed, Mar 18, 2020 at 05:52:25PM +0000, James Le Cuirot wrote:

> On Wed, 18 Mar 2020 12:47:53 -0500
> William Hubbs <[hidden email]> wrote:
>
> > On Wed, Mar 18, 2020 at 06:12:37PM +0100, Michał Górny wrote:
> > > On Wed, 2020-03-18 at 09:54 -0500, William Hubbs wrote:  
> > > > this came up again on the recent thread about dropping non x86/amd64
> > > > support for python packages, and I want to bring it up again on its own
> > > > thread.
> > > >
> > > > How often do architecture specific bugs really exist in languages like
> > > > perl, python etc? From what I've seen they are pretty rare. Not to mention,
> > > > if we found one somewhere, we could adjust keywords as necessary.
> > > >
> > > > Also, if someone did inadvertently keyword a package with noarch that didn't
> > > > work everywhere, it would be a matter of adjusting the keywords for that
> > > > package.
> > > >
> > > > So, my question is, why can't we add a noarch/~noarch keyword and see
> > > > how things go? If it gets abused we can always nuke it later.
> > > >  
> > >
> > > 1. How is this going to work when noarch package depends on non-nonarch
> > > package?  I mean, will all the package managers actually work?  Have you
> > > did some minimal testing before bringing this up?  
> >  
> > Can you have multiple ACCEPT_KEYWORDS values in make.conf or
> > make.defaults like this?
> >
> >  ACCEPT_KEYWORDS="amd64 noarch"
> >
> > If so, things should just work.
>
> Not quite. Tools like repoman will need to be updated to understand
> that an ebuild with KEYWORDS="amd64" can depend on another ebuild with
> only KEYWORDS="noarch". I do think the idea has merit though.
Ok, that's reasonable and generates more questions.

How much work would it be to update the tools to take that into account?

In the meantime, is it worth adding the noarch keyword but not dropping
other keywords until the tools are fixed?

William


signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

William Hubbs
In reply to this post by Michał Górny-5
On Wed, Mar 18, 2020 at 07:12:08PM +0100, Michał Górny wrote:

> On Wed, 2020-03-18 at 12:47 -0500, William Hubbs wrote:
> > On Wed, Mar 18, 2020 at 06:12:37PM +0100, Michał Górny wrote:
> > > On Wed, 2020-03-18 at 09:54 -0500, William Hubbs wrote:
> > > > this came up again on the recent thread about dropping non x86/amd64
> > > > support for python packages, and I want to bring it up again on its own
> > > > thread.
> > > >
> > > > How often do architecture specific bugs really exist in languages like
> > > > perl, python etc? From what I've seen they are pretty rare. Not to mention,
> > > > if we found one somewhere, we could adjust keywords as necessary.
> > > >
> > > > Also, if someone did inadvertently keyword a package with noarch that didn't
> > > > work everywhere, it would be a matter of adjusting the keywords for that
> > > > package.
> > > >
> > > > So, my question is, why can't we add a noarch/~noarch keyword and see
> > > > how things go? If it gets abused we can always nuke it later.
> > > >
> > >
> > > 1. How is this going to work when noarch package depends on non-nonarch
> > > package?  I mean, will all the package managers actually work?  Have you
> > > did some minimal testing before bringing this up?
> >  
> > Can you have multiple ACCEPT_KEYWORDS values in make.conf or
> > make.defaults like this?
> >
> >  ACCEPT_KEYWORDS="amd64 noarch"
> >
> > If so, things should just work.
> >
> > Currently I don't know of any arch/package combos to test this with.
>
> I'm talking about repoman/pkgcheck.
See my response to chewi about this part. I have no idea how
much work would be involved in making this work.

>
> > > 2. Who will be responsible for handling noarch stablereqs?  Will there
> > > be a noarch arch team?
> >
> > The maintainer would be able to add the "~noarch" keyword. I'm not sure
> > there needs to be a noarch arch team. We could just say that all arch
> > team members can stabilize these or maybe the maintainers can afterh the
> > timeout.
> >
>
> Would you CC all arch teams on the bug then?
>
> We have ALLARCHES already, and to my experience most arch teams fail to
> handle that.
There would be no need to cc all arches on the bug, just make [hidden email]
an alias that emails to all arch teams.

Thanks,

William


signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Andreas K. Huettel
Am Mittwoch, 18. März 2020, 19:40:57 CET schrieb William Hubbs:
> There would be no need to cc all arches on the bug, just make [hidden email]
> an alias that emails to all arch teams.

We might as well just make an allarches@... alias.

--
Andreas K. Hüttel
[hidden email]
Gentoo Linux developer
(council, qa, toolchain, base-system, perl, libreoffice)

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Andreas K. Huettel
In reply to this post by William Hubbs
> So, my question is, why can't we add a noarch/~noarch keyword and see
> how things go? If it gets abused we can always nuke it later.

I'm pretty sure we already discussed this in very much detail in the past at
least once, and came to the conclusion that there are problems with that
approach. What's different now?

Sorry, but for the moment your mail is a bit big on fluffy ideas and a bit thin
on details how it's going to work... as unsorted examples,
* how is allarches supposed to interact with use.stable.mask?
* who is doing allarches stabilizations?
* what are the allowed dependencies? obviously an allarches package can only
depend on other allarches packages...
* what happens if an allarches package gets, e.g., masked on one arch?

--
Andreas K. Hüttel
[hidden email]
Gentoo Linux developer
(council, qa, toolchain, base-system, perl, libreoffice)

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Kent Fredric-2
In reply to this post by William Hubbs
On Wed, 18 Mar 2020 11:59:25 -0500
William Hubbs <[hidden email]> wrote:

>  Sure, but if you run into something like that you just don't use the
>  noarch keyword for those packages.

But as soon as this happens, all dependent packages that are noarch
will need to also transition to not using no-arch.

So it turns into a lot of busywork without benefit.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Kent Fredric-2
In reply to this post by James Le Cuirot
On Wed, 18 Mar 2020 17:52:25 +0000
James Le Cuirot <[hidden email]> wrote:

> Not quite. Tools like repoman will need to be updated to understand
> that an ebuild with KEYWORDS="amd64" can depend on another ebuild with
> only KEYWORDS="noarch". I do think the idea has merit though.

But the inverse is _not_ true, an ebuild with KEYWORDS="noarch"
*cannot* depend on another ebuild with only KEYWORDS="amd64".

Otherwise it breaks the entire stabilization graph.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Thomas Deutschmann
In reply to this post by William Hubbs
Hi,

please don't introduce another keyword.

Why can't we use "ALLARCHES" stabilization for that?

However, this will getting more complicated than it will help.
Any Python package which compiles something can fail. During my x86 work
I have seen a lot of problems when it comes to anything math related (no
SSE2, -mfpmath=387...). So as long as we want that a package keyworded
for x86 really works on old x86 hardware, we have to go the long route
an test it.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Kent Fredric-2
In reply to this post by William Hubbs
On Wed, 18 Mar 2020 09:54:42 -0500
William Hubbs <[hidden email]> wrote:

> So, my question is, why can't we add a noarch/~noarch keyword and see
> how things go? If it gets abused we can always nuke it later.
>
> Thanks,

I'm just gonna say I disagree with this proposal as stated.

Stability and arch support, for the purposes of QA, should be based on
demonstrated evidence.

Not speculation ( which noarch inherently is ).

I'd be "OK" with a provisional flag that indicates a package /should/
be architecture independent, but it should be as an optional
enhancement for users on minor arches who want to minimize their
fussing with keywords.

But KEYWORDS imo, should be a graph of demonstrated evidence, otherwise
it pretty much makes the purpose of arch testing redundant.

And yes, even vanilla perl code can have arch dependent behaviour.

( I myself fell prey to pack() having some behavioural changes based on
the users 32 vs 64bitness )

However, what I propose isn't something you can "hack in" to an
existing EAPI's tree.

You'd likely need an EAPI enhancement to implement this the way I'd
imagine.

In short:

- An Ebuild should still have KEYWORDS that indicate
  - What it has been tested to work on
  - What it has been evidentially supported on
- But an Ebuild *could* have a variable that indicates
  - That in the absence of good testing and evidence, it is
    *expected* to work on all arches.
- End users could be provided tools to *permit* the latter class
  to be installed on their system based on the speculation
- But it would still be "out of scope" for users who want and demand
  a tested, predictable, quality, stable system.

Anything else is just weakening our quality assurance for our users,
in order to pander to developer ease.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Kent Fredric-2
In reply to this post by Thomas Deutschmann
On Thu, 19 Mar 2020 03:07:21 +0100
Thomas Deutschmann <[hidden email]> wrote:

> Why can't we use "ALLARCHES" stabilization for that?

Because that experiment basically failed.

Bugs with that flag, basically were treated (repeatedly) like that flag
wasn't there.

And that approach still has the weakness of it being a conjecture of
stability, not an evidence of stability.

But worse, it /hides/ this distinction, so an end user can't
differentiate between "stable by conjecture" and "stable by evidence"


attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

David Seifert
In reply to this post by Kent Fredric-2
On Thu, 2020-03-19 at 14:57 +1300, Kent Fredric wrote:

> On Wed, 18 Mar 2020 11:59:25 -0500
> William Hubbs <[hidden email]> wrote:
>
> >  Sure, but if you run into something like that you just don't use the
> >  noarch keyword for those packages.
>
> But as soon as this happens, all dependent packages that are noarch
> will need to also transition to not using no-arch.
>
> So it turns into a lot of busywork without benefit.

Second this, it doesn't sound well though out at this point.


Reply | Threaded
Open this post in threaded view
|

Re: rfc: noarch keyword

Michael Orlitzky
In reply to this post by William Hubbs
On 3/18/20 10:54 AM, William Hubbs wrote:
>
> So, my question is, why can't we add a noarch/~noarch keyword and see
> how things go? If it gets abused we can always nuke it later.
>

This is a good goal, but as others have pointed out, adding a new magic
keyword poses some workflow problems.

We already have the ability to fake this. Instead of KEYWORDS="~noarch",
if you really believe that your package is architecture-independent, you
can just add all ~arch keywords to it. If you're right, no one will ever
notice, and if you're wrong, you'll get shit for it -- just like you
would have if you marked it ~noarch incorrectly.

Maintainers can do their own stabilization (on all arches) for those
packages. I hesitate to invoke "common sense," but in cases where common
sense truly does prevail, you should have no problem keywording and
stabilizing a package on all arches that e.g. downloads a PDF of API
documentation.

12