pkg_pretend USE validation and VALID_USE alternative

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

pkg_pretend USE validation and VALID_USE alternative

Brian Harring-2
Hola all-

For those who aren't familiar, pkg_pretend is in EAPI4- the main usage
of it is will be use dep checking- this email is specifically
regarding an alternative to it that *should* be superior for that use
case, but I'm looking for feedback.

Basically, we use the original VALID_USE proposal from way back in
'05- if you're familiar w/ MYOPTIONS, they're reasonably similar.

Roughly, VALID_USE is a list of constraints stating what the allowed
use flag combinations are for this pkg.  If you think of normal
depdencies (I must have openssl and python merged prior), it's the
same machinery.

Examples:

# if build is set, python and openssl must be unset
VALID_USE="build? ( !python !openssl )"

# if mysql is set, sqlite must not be, and vice versa.
# note mysql/sqlite do *not* have to be set also.
VALID_USE="mysql? ( !sqlite ) !sqlite? ( mysql )"

# mysql or sqlite must be set; exclusive or.
VALID_USE="mysql? ( !sqlite ) !mysql ( sqlite )"

# alternative syntax, adding an xor group operator
# note xor isn't required- you can do the same thing
# via spelling it out, it's just a convenient thing to have.
VALID_USE="^^ ( mysql sqlite )"

# if gui is enabled, a widget set must be specified- can build
# multiple widget bindings
VALID_USE="X? ( || ( gtk qt motif ) )"


Note that like dependencies, these are assertions.  More importantly,
they're also data rather then executable code.  Via it being data, up
front GUI's can tell you exactly what conflicts arise, and what needs
to be adjusted.

Doing it as executable, can, but it's iterative and reliant on the dev
writing a clear message every time (rather then the UI tool getting
it right once).  Clarifying iterative, consider

USE="build X"
w/ valid use states being-
VALID_USE="build? ( !X !python ) X? ( ^^ ( gtk qt ) ) gtk? ( ssl )"

The user first flips off build.  They rerun emerge- next they're told
"you must choose gtk or qt, not both".  They change to USE="X gtk".  
Next they're told "you need ssl turned on if you want gtk".

At this point, the user goes and gets a beer because aparently Murphy
hates them and it's going to be a long, long night.


There also is one major issue with relying on pkg_pretend
(executable) for use state validation vs doing it as VALID_USE
(data)- the package manager cannot know what states are valid thus
limiting the things it can do.  Literally the pkg manager is screwed
when it comes to use cycle breaking.  If the pkg manager doesn't know
what states are valid, when it encounters a use cycle it doesn't know
what intermediate builds it can do to break that cycle- literally if
USE state validation is in pkg_pretend, the *user* will have to walk
the PM through breaking the cycle instead of the PM figuring out the
proper steps to break it.

Executive summary: if use validation is implemented via pkg_pretend,
1) it still has the iterative issue
2) it's harder for configuration tools to deal with (iterative issues
above, plus the fact they get a blob of text they're stuck trying to
parse)
3) it pretty much eliminates the possibility of doing use cycle
breaking properly (which is already an issue, only going to get worse-
hence why this was proposed in '04/'05 when we started playing w/ use
deps in portage 3 at the time).


Comments desired; assuming no significant blowback, I'll be pushing
this to the council level since eapi4 is annoying feature locked right
now.

Thanks-
~harring

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pkg_pretend USE validation and VALID_USE alternative

Ulrich Mueller-2
>>>>> On Wed, 31 Mar 2010, Brian Harring wrote:

> Roughly, VALID_USE is a list of constraints stating what the allowed
> use flag combinations are for this pkg. If you think of normal
> depdencies (I must have openssl and python merged prior), it's the
> same machinery.

Maybe we should first discuss if we want to drop the following
rule [1] which your proposal seems to contradict:

| Occasionally, ebuilds will have conflicting USE flags for
| functionality. Checking for them and returning an error is not a
| viable solution. Instead, you must pick one of the USE flags in
| conflict to favour.

Ulrich

[1] <http://devmanual.gentoo.org/general-concepts/use-flags/>

Reply | Threaded
Open this post in threaded view
|

Re: pkg_pretend USE validation and VALID_USE alternative

Alistair Bush-2
In reply to this post by Brian Harring-2
> Hola all-
>  
> Comments desired; assuming no significant blowback, I'll be pushing
> this to the council level since eapi4 is annoying feature locked right
> now.
>

I think this solution is far better,  until someone smarter than me tells me
otherwise.

Don't know whether I like the VALID_USE var name,  so please at least think
about something a little better if you can ;).  ( I like green,  but not too
green if you know what I mean )

Will we still have to define the use flags in IUSE?

I'm guessing we can't use IUSE to store this information because of the  whole
glep-55 thing.  IUSE would be my preferred var.  both defining the
"relationships" and the use flags in only one go.

> Thanks-
> ~harring

Reply | Threaded
Open this post in threaded view
|

Re: pkg_pretend USE validation and VALID_USE alternative

Brian Harring-2
In reply to this post by Ulrich Mueller-2
Note that while I inadvertantly cross posted (I was intending on
cc'ing [hidden email], not the ml), doubt they need to be cc'd
further- my original attention was to effectively ensure they were
paying aware of the details of this so that when I took it to them
folk were informed.

CC'ing gentoo-council so folk following it there know it moved
over to -dev.  Your discussion of devmanual relevance needs some -dev
consensus anyways before the council should be deciding on it.

Also the cross posting is making betelgeuse cry anyways (and pissing
off my procmail setup) ;)


On Wed, Mar 31, 2010 at 11:48:37AM +0200, Ulrich Mueller wrote:
> >>>>> On Wed, 31 Mar 2010, Brian Harring wrote:
>
> > Roughly, VALID_USE is a list of constraints stating what the allowed
> > use flag combinations are for this pkg. If you think of normal
> > depdencies (I must have openssl and python merged prior), it's the
> > same machinery.
>
> Maybe we should first discuss if we want to drop the following
> rule [1] which your proposal seems to contradict:

Not just my proposal- council contradicted it via even letting
pkg_pretend into EAPI3 (now EAPI4):

http://www.mail-archive.com/gentoo-council@.../msg00493.html


> | Occasionally, ebuilds will have conflicting USE flags for
> | functionality. Checking for them and returning an error is not a
> | viable solution. Instead, you must pick one of the USE flags in
> | conflict to favour.
>
> [1] <http://devmanual.gentoo.org/general-concepts/use-flags/>

I honestly consider the ebuild silently making decisions on the user's
behalf *worse*.  Consider if openoffice silently made decisions like
that- 4 hours later it'll wind up choosing the option you didn't
really want and you'll be in a foul mood.

Frankly is the devmanual even relevant on at this point beyond good
practices btw?  Last I looked through it, there was a rather unhealthy
mix of good policy that we follow, and policy that isn't relevant
anymore- in need of some cleanup at the very least.


~harring

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: pkg_pretend USE validation and VALID_USE alternative

Brian Harring-2
In reply to this post by Alistair Bush-2
Note I inadvertantly cross posted, I was intending on cc'ing
[hidden email].

As such one final cc to that ml to end this subthread while pulling
this back to -dev.


On Wed, Mar 31, 2010 at 11:16:22PM +1300, Alistair Bush wrote:

> > Hola all-
> >  
> > Comments desired; assuming no significant blowback, I'll be pushing
> > this to the council level since eapi4 is annoying feature locked right
>
> I think this solution is far better,  until someone smarter than me tells me
> otherwise.
>
> Don't know whether I like the VALID_USE var name,  so please at least think
> about something a little better if you can ;).  ( I like green,  but not too
> green if you know what I mean )
>
> Will we still have to define the use flags in IUSE?
Yes, although if folks have a better proposal that incorporates
VALID_USE into IUSE I'm definitely open to it- the original proposal
for VALID_USE tried to inline it into IUSE, but it got ugly (hence it
mutating it this form).

The problem w/ trying to reuse IUSE is the following (sorry, I like
lists of assertions)-

*) IUSE currently serves as a list of valid USE flags, just that.  No
repeat specification of a flag (which means the dev in question is
unlikely to typo a flag).

*) having a single specified list of valid use flags is the basis for
doing validation of use flags used in all other metadata.  In other
words, that list of valid use flags *really* needs to go out of it's
way to make human error hard.

*) VALID_USE is a set of assertions on the allowed state of USE; IUSE
is just a list of flags.  Intermixing the two in a way that is still
readable is really ugly

*) given a library that has optional perl and python bindings
(which can be toggled freely, they're standalone flags) I've nfc how
one would sanely specify that w/in IUSE while adding VALID_USE
semantics.  Possibly, you could include use conditionals into IUSE,
and treat the () contents as assertions- but that makes adding xor in
hard, is rather ugly/hard on the eyes, and violates the DRY (Don't
Repeat Yourself) principle from above.

Definitely open to counterproposals that address those
concerns however...


> I'm guessing we can't use IUSE to store this information because of the  whole
> glep-55 thing.

glep-55 is unrelated to this as far as I can tell- if you think
otherwise please clarify.

Thanks
~harring

attachment0 (205 bytes) Download Attachment