Eclass vs EAPI For Utility Functions (Patching/etc)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Eclass vs EAPI For Utility Functions (Patching/etc)

Rich Freeman
I debated where to post this, but the topic is fairly dev-oriented and
has big long-term impact so I landed here.  This really isn't
organizational in nature.

During the council meeting there was a bit of a philosophical debate
over the proper role of EAPI vs implementing functions in eclasses.  I
felt that it was important enough to at least get more community input
before we continue voting on features like user patching/etc which
tend to favor an EAPI-based approach.

I'll try to fairly frame the arguments, though I personally lean in
the EAPI direction and I'll leave it to somebody else to fix my straw
man.


The Eclass argument goes like this:
Eclasses already work in every PM.  Half of what we're debating is
already in eutils.  Why move this code into the PM, where it has to be
re-implemented everywhere?  If anything we should be moving more PM
functionality out and into eclasses where we can have competing
implementations and more flexibility.


The EAPI argument goes like this:
Sure, you can have 14 different implementations of epatch which lets
each maintainer use the one that makes the most sense.  However, who
wants to edit an ebuild which uses a "bad" epatch implementation and
re-learn how to add a patch?  Adding patches is a common thing, so why
not have a standard way to do it?  We can still have eclasses for
one-offs.  And how can you ever do something like user patches when
they will only work if the maintainer cares to support them?


I view this as not being unlike dealing with programs that encourage
the use of Turing-complete configuration files.  Sure, they give you a
lot more power, but that power comes at a cost.  Sendmail config files
are a running joke, and if you want to control the niceness or
ioniceness of an OpenRC service then you're going to have to read the
script and possibly patch it.

When there is no value in everybody doing things differently, why not
just do them the same way?

Besides, I think user-patches are a GREAT feature to have, and one I
use all the time (without even thinking about it if a package with a
patch gets rebuilt).  As I said in the meeting, if we were selling
Gentoo to make money, it would be the sort of feature that you'd
advertise.  Why make it hard to use such a distinctive advantage of a
source-based distro?

Since this month is the last one for this Council term as an added
bonus you stack the next Council with folks who agree with you on this
one...  :)

Rich

Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

Ciaran McCreesh-4
On Sun, 15 Jun 2014 07:00:15 -0400
Rich Freeman <[hidden email]> wrote:
> The Eclass argument goes like this:
> Eclasses already work in every PM.  Half of what we're debating is
> already in eutils.  Why move this code into the PM, where it has to be
> re-implemented everywhere?  If anything we should be moving more PM
> functionality out and into eclasses where we can have competing
> implementations and more flexibility.

The big problem with eclasses is that they're far too messy and
complicated. Sure, you can *technically* express (say) ABI dependencies
using a complicated eclass which translates them into a convoluted
series of use dependencies, nested || dependencies etc (more or less
correctly most of the time). But the package mangler is being given less
information that way, which means it has to have all sorts of dodgy
heuristics to deal with them, and can't give good error messages when
it breaks.

--
Ciaran McCreesh

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

Michał Górny-5
In reply to this post by Rich Freeman
Dnia 2014-06-15, o godz. 07:00:15
Rich Freeman <[hidden email]> napisał(a):

> The Eclass argument goes like this:
> Eclasses already work in every PM.  Half of what we're debating is
> already in eutils.  Why move this code into the PM, where it has to be
> re-implemented everywhere?  If anything we should be moving more PM
> functionality out and into eclasses where we can have competing
> implementations and more flexibility.

I think this point is a bit tangential. While eclasses are the obvious
way of having code common between package managers, I don't think
sharing code should be an argument for putting something in an eclass.

Please remember that you can create your own base repository without
having 'gentoo' as master. Then, putting code in eclasses actually
requires you to duplicate it -- unlike code in PM which is shared
between all repos.


Now, for the overall point. I think there are a few cases when you want
something in the PM:

1. when it's a common thing to do that doesn't require repository-
specific details like relying on some out-of-@system packages,

2. when it requires specific interaction with the package manager,

3. when it is required by some other PM-specific code. Keeping it
'internal' just means having duplicate code between PM and eclasses.


I think the only debatable thing here is (1). If we ever decide that
having common stuff in an eclass is feasible, we ought to move all
the common stuff that's possible -- including econf, emake, possibly
some install helpers.

This may even have more benefits than that. For example, you would fit
the implementation to specific system -- like 'gentoo' repository
eclass would be fit for Gentoo-specific tools. Instead of putting
requirements for a system in PMS and trying to fit PM and profiles
together, we'd just use whatever the repository provides.

Of course this brings another argument -- every single ebuild would
have to source a specific eclass. For EAPI 6, I've focused on going
the opposite way -- putting more commonly used stuff to EAPI so that
many of eutils inherits could be removed. I don't have the numbers but
I'd dare guess inheriting eutils everywhere does come with some
overhead on metadata cache generation.


As far as both eapply & eapply_user are concerned, I believe they both
belong to the group of commonly used functions, so they ought to go
into PM wrt (1).

Moreover, eapply_user pretty much requires you to provide a location
for the patches -- and putting /etc/portage into an eclass is just
wrong. Of course, we could try some new, fancy location but well... I'd
rather keep it as-is and consider it PM-specific, so argument (2).

And if put eapply_user anyway, argument (3) says we gotta add eapply
as well, or otherwise we'll be using two similar functions all the time
-- one in the PM, the other in the eclass.

--
Best regards,
Michał Górny

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Auto-patching ebuilds themselves Was: Eclass vs EAPI For Utility Functions (Patching/etc)

Duncan-42
In reply to this post by Rich Freeman
Rich Freeman posted on Sun, 15 Jun 2014 07:00:15 -0400 as excerpted:

> Besides, I think user-patches are a GREAT feature to have, and one I use
> all the time (without even thinking about it if a package with a patch
> gets rebuilt).  As I said in the meeting, if we were selling Gentoo to
> make money, it would be the sort of feature that you'd advertise.  Why
> make it hard to use such a distinctive advantage of a source-based
> distro?

Agreed with everything, but focusing on the above as a jump-off point
for...

A similarly great feature to have would be epatch-ebuild.  Back when
gentoo/kde was trying to dump IUSE=semantic-desktop[1], I was doing
enough ebuild patching with enough churn in the ebuilds[2] that manual
per-ebuild patching was simply no longer feasible.  It can be noted that
normal sources patching (as epatch) would NOT work here, at least not
easily, because it was the ebuilds themselves that were forcing the
various semantic-desktop related config options on and forcing
dependencies to match, so it was the ebuilds themselves that needed that
patched back out.

I ended up with an /etc/portage/patches.ebuild that mirrored
/etc/portage/patches in functionality, using a sync script that
download-updated both the gentoo tree and the layman-managed overlays,
then _automatically_ applied the patches from the patches.ebuild tree and
redigested the ebuilds[3], then regenerating cache to account for the
changes.  The _automatic_ bit being critical as at that point I was
handling enough of them that doing it manually would not have been fun.

Talk about a feature worth advertising, the user-level ability to
automatically live-patch ebuilds by simply dropping a patch in the
appropriate location just like we do sources was and remains great,
definitely something I'd love to see in the PMs at some point. =:^)

As we've found with user-epatched sources, this should greatly simplify
deployment of experimental patches.  Just attach the patch to the bug and
tell the user what directory to drop it in, and no more worries about
having to tell users what to edit or how to do manual patching, in
ordered to test a fix. =:^)

---
[1] Dropping IUSE=semantic-desktop: Something I'm /ever/ so glad gentoo/
kde changed their minds on. =:^)

[2] The ebuilds in question were from the gentoo/kde overlay, before they
reached the tree.  That overlay and the ebuilds in it get a *LOT* of
churn!

[3] Redigesting:  As I run live-sources kde there were few source
tarballs to redigest, only the ebuilds.

--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman


Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

Pacho Ramos
In reply to this post by Rich Freeman
El dom, 15-06-2014 a las 07:00 -0400, Rich Freeman escribió:

> I debated where to post this, but the topic is fairly dev-oriented and
> has big long-term impact so I landed here.  This really isn't
> organizational in nature.
>
> During the council meeting there was a bit of a philosophical debate
> over the proper role of EAPI vs implementing functions in eclasses.  I
> felt that it was important enough to at least get more community input
> before we continue voting on features like user patching/etc which
> tend to favor an EAPI-based approach.
>
> I'll try to fairly frame the arguments, though I personally lean in
> the EAPI direction and I'll leave it to somebody else to fix my straw
> man.
>
>
> The Eclass argument goes like this:
> Eclasses already work in every PM.  Half of what we're debating is
> already in eutils.  Why move this code into the PM, where it has to be
> re-implemented everywhere?  If anything we should be moving more PM
> functionality out and into eclasses where we can have competing
> implementations and more flexibility.
>
>
> The EAPI argument goes like this:
> Sure, you can have 14 different implementations of epatch which lets
> each maintainer use the one that makes the most sense.  However, who
> wants to edit an ebuild which uses a "bad" epatch implementation and
> re-learn how to add a patch?  Adding patches is a common thing, so why
> not have a standard way to do it?  We can still have eclasses for
> one-offs.  And how can you ever do something like user patches when
> they will only work if the maintainer cares to support them?
>
>
> I view this as not being unlike dealing with programs that encourage
> the use of Turing-complete configuration files.  Sure, they give you a
> lot more power, but that power comes at a cost.  Sendmail config files
> are a running joke, and if you want to control the niceness or
> ioniceness of an OpenRC service then you're going to have to read the
> script and possibly patch it.
>
> When there is no value in everybody doing things differently, why not
> just do them the same way?
>
> Besides, I think user-patches are a GREAT feature to have, and one I
> use all the time (without even thinking about it if a package with a
> patch gets rebuilt).  As I said in the meeting, if we were selling
> Gentoo to make money, it would be the sort of feature that you'd
> advertise.  Why make it hard to use such a distinctive advantage of a
> source-based distro?
>
> Since this month is the last one for this Council term as an added
> bonus you stack the next Council with folks who agree with you on this
> one...  :)
>
> Rich
>

In this concrete case we the benefit I see for having support for
epatch_user/eautoreconf at EAPI level is that we won't need to implement
that support on each ebuild/eclass or need to manually overwrite default
phases for them inheriting, for example, autotools-utils.eclass to reuse
its patches handling.

Other option would be to have two kinds of eclasses, one of them would
be inherited *always* and always being used, but I am not sure if adding
this new "layer" could complicate things a bit more :/. This kind of
eclasses would be used always and would allow to backport some features
to older eapis.


Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

William Hubbs
In reply to this post by Rich Freeman
Hi all:

On Sun, Jun 15, 2014 at 07:00:15AM -0400, Rich Freeman wrote:
> During the council meeting there was a bit of a philosophical debate
> over the proper role of EAPI vs implementing functions in eclasses.  I
> felt that it was important enough to at least get more community input
> before we continue voting on features like user patching/etc which
> tend to favor an EAPI-based approach.

I am strongly in favor of the eapi-based approach as well, for all of
the reasons mentioned in the thread so far.

Eclasses can and should be used for functions, imo, that are used by
some ebuilds, but once it is determined that functionality in an eclass has
potential for very wide use, that functionality should be moved into an
eapi. The eutils functions are a prime example of this. These are
general purpose functions, so there is no reason for them to be kept in
an eclass.

William


signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

Rich Freeman
On Thu, Jun 19, 2014 at 1:03 PM, William Hubbs <[hidden email]> wrote:
> Hi all:
>
> I am strongly in favor of the eapi-based approach as well, for all of
> the reasons mentioned in the thread so far.

Good thing your proxy got it right then!  :)

http://www.gentoo.org/proj/en/council/meeting-logs/20140617-summary.txt

Rich

Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

Steven J. Long
In reply to this post by Michał Górny-5
On Sun, Jun 15, 2014 at 03:30:10PM +0200, Michał Górny wrote:

> Dnia 2014-06-15, o godz. 07:00:15
> Rich Freeman napisał(a):
>
> > The Eclass argument goes like this:
> > Eclasses already work in every PM.  Half of what we're debating is
> > already in eutils.  Why move this code into the PM, where it has to be
> > re-implemented everywhere?  If anything we should be moving more PM
> > functionality out and into eclasses where we can have competing
> > implementations and more flexibility.
>
> I think this point is a bit tangential. While eclasses are the obvious
> way of having code common between package managers, I don't think
> sharing code should be an argument for putting something in an eclass.

Er, it's the most fundamental argument for using an eclass.

Thought not between package managers, since they don't factor into the
equation, so long as they provide EAPI conformance (theoretically.)

> Please remember that you can create your own base repository without
> having 'gentoo' as master. Then, putting code in eclasses actually
> requires you to duplicate it -- unlike code in PM which is shared
> between all repos.

Well that's an argument for certain base functionality being in the PM
although I'd note that "in the PM" in most cases, and this one, means
"in ebuild.sh or one of its associated BASH modules."

It's not an argument for sharing script across ebuilds in general, by
putting it in the the context for every ebuild.

As for different master repos, that's an appropriate discussion at
at the package-manager code level, but not at the distro level, imo.

> Now, for the overall point. I think there are a few cases when you want
> something in the PM:
>
> 1. when it's a common thing to do that doesn't require repository-
> specific details like relying on some out-of-@system packages,
>
> 2. when it requires specific interaction with the package manager,
>
> 3. when it is required by some other PM-specific code.

That's enforced by the requirement.

> Keeping it
> 'internal' just means having duplicate code between PM and eclasses.

No: it also means you don't expose it until it's needed. The worst
thing in the world is having to rely on variant behaviour according
to the phase of the moon, which is about what you get when internal
APIs are exposed without cause and consensus.

Since you're speaking in general terms.

> I think the only debatable thing here is (1). If we ever decide that
> having common stuff in an eclass is feasible, we ought to move all
> the common stuff that's possible -- including econf, emake, possibly
> some install helpers.

I'm not sure what that's got to with out-of-@system packages, but it
sounds like a bad idea, and counter to your argument about sharing
code via the PM, and not eclasses.

> This may even have more benefits than that. For example, you would fit
> the implementation to specific system -- like 'gentoo' repository
> eclass would be fit for Gentoo-specific tools. Instead of putting
> requirements for a system in PMS and trying to fit PM and profiles
> together, we'd just use whatever the repository provides.

That sounds insane to me. What happened to reproducible builds? Let
alone a robust user experience.

Perhaps at the package-manager code level, but not at the distro level.
And it's the Gentoo PMS, so I don't see any conflict.

Though this appears to be the same "tangent" Rich brought up.

> Of course this brings another argument -- every single ebuild would
> have to source a specific eclass. For EAPI 6, I've focused on going
> the opposite way

So which way do you actually prefer?

You appear to be arguing for, and implementing, common code by EAPI,
in the rest of your mail. Since that's always been the point of
them, based on developer consensus about what is truly essential,
and what is only needed for a subset of the tree, that's fine by me.

Just so long as they are essential, and you do have consensus on
that.

> As far as both eapply & eapply_user are concerned

Why can't we just have epatch and upatch?


=
I'm on record as wanting this in the PM, btw, back when we were
discussing how it should deal with patches not being applied.

--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

Reply | Threaded
Open this post in threaded view
|

Re: Eclass vs EAPI For Utility Functions (Patching/etc)

Rich Freeman
On Thu, Jun 19, 2014 at 6:05 PM, Steven J. Long
<[hidden email]> wrote:
> So which way do you actually prefer?
>
> You appear to be arguing for, and implementing, common code by EAPI,
> in the rest of your mail. Since that's always been the point of
> them, based on developer consensus about what is truly essential,
> and what is only needed for a subset of the tree, that's fine by me.

This is all becoming moot since the Council just voted on this earlier
this week, nearly unanimously bringing in user patches.  However, I
believe he was advocating going with an EAPI in this case, but
offering alternatives.

While brainstorming the options I was thinking that you could almost
have an EAPI-like eclass.  That is, instead of declaring an EAPI every
ebuild could just source an EAPI-foo eclass which basically does the
same thing.  Of course, this would be somewhat less flexible than what
we do today - if we went as far as being able to determine EAPI
without sourcing an ebuild (no, I'm not bringing back that debate now)
then having EAPI in the PM lets you change the ebuild format in almost
entirely arbitrary ways over time.

There is nothing wrong with playing devil's advocate in a thread like
this.  Hopefully the Council members can weigh the arguments on their
own.  :)

Rich