Changes made by acct-* ebuilds

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Changes made by acct-* ebuilds

Christopher Head
Hi all,
Yesterday something surprised me. I updated my system and got the acct-{user,group}/lighttpd for the first time. Because lighttpd was running, package installation failed to change the home directory—fine, it printed an error message, I stopped the server, changed the home directory by hand, and started the server back up.

What I didn’t realize was that it also, successfully, removed the lighttpd user from a couple of auxiliary groups I had put it in. It did this without telling me, without printing any messages. I only noticed because I happened to look at syslog and discovered that usermod or gpasswd or whatever it called had logged the changes. Presumably this has broken a service or two (nothing too critical) since now Lighttpd won’t be able to connect to SCGI sockets any more.

Does it make sense for these ebuilds to print out all the changes they make to existing users and groups, so that the sysadmin can see what happened and immediately look into alternative solutions if it breaks something, rather than silently changing things? Maybe this could even be limited to cases where the package is being newly installed (not upgraded) and the user or group already exists, to ease migration from the old world where sysadmins are easily able to do anything we want with our users and groups to the new world where we’re expected to leave them alone as the ebuilds make them, or worst case make out changes in an overlay.

Thoughts?
--
Christopher Head

signature.asc (251 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Alec Warner-2


On Wed, Feb 12, 2020 at 10:02 AM Christopher Head <[hidden email]> wrote:
Hi all,
Yesterday something surprised me. I updated my system and got the acct-{user,group}/lighttpd for the first time. Because lighttpd was running, package installation failed to change the home directory—fine, it printed an error message, I stopped the server, changed the home directory by hand, and started the server back up.

What I didn’t realize was that it also, successfully, removed the lighttpd user from a couple of auxiliary groups I had put it in. It did this without telling me, without printing any messages. I only noticed because I happened to look at syslog and discovered that usermod or gpasswd or whatever it called had logged the changes. Presumably this has broken a service or two (nothing too critical) since now Lighttpd won’t be able to connect to SCGI sockets any more.

I'm not convinced this behavior is correct, so we may be able to just fix it.

-A
 

Does it make sense for these ebuilds to print out all the changes they make to existing users and groups, so that the sysadmin can see what happened and immediately look into alternative solutions if it breaks something, rather than silently changing things? Maybe this could even be limited to cases where the package is being newly installed (not upgraded) and the user or group already exists, to ease migration from the old world where sysadmins are easily able to do anything we want with our users and groups to the new world where we’re expected to leave them alone as the ebuilds make them, or worst case make out changes in an overlay.

Thoughts?
--
Christopher Head
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
In reply to this post by Christopher Head
On Wed, 2020-02-12 at 10:02 -0800, Christopher Head wrote:
> Hi all,
> Yesterday something surprised me. I updated my system and got the acct-{user,group}/lighttpd for the first time. Because lighttpd was running, package installation failed to change the home directory—fine, it printed an error message, I stopped the server, changed the home directory by hand, and started the server back up.
>
> What I didn’t realize was that it also, successfully, removed the lighttpd user from a couple of auxiliary groups I had put it in. It did this without telling me, without printing any messages. I only noticed because I happened to look at syslog and discovered that usermod or gpasswd or whatever it called had logged the changes. Presumably this has broken a service or two (nothing too critical) since now Lighttpd won’t be able to connect to SCGI sockets any more.

I'm pretty sure user.eclass does print whatever changes it is doing.  It
isn't elog-ged though, I admit.  This is probably worth fixing.


--  
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Thomas Deutschmann
In reply to this post by Christopher Head
Hi,

thank you for bringing this to the list.

I have the same experience which is the reason why I haven't migrated
most of my packages yet (which is not a good move because I also didn't
post the problem to the list like I wanted *yet*, I only talked to
people via private mail, chat or at FOSDEM about that and was working on
a proposal I wanted to show next week when I am hopefully healthy again).

In short: It was a very bad decision that acct-* stuff is *changing*
existing stuff. This must be turned of *by default*. Maybe provide a
setting a user can put into make.conf to opt into current, still new,
behavior but by default, a package should never ever make changes to
*existing* user (unless it knows for sure it was the only source
creating that user and nothing was changed since creation which isn't
easy to track).

My example is a little bit different:

Please think about all the systems which are managed using some kind
configuration management tool (Puppet, Ansible, Salt, Chef...). It's
really common and there's is nothing wrong to make use of usermod for
example to alter users. All of the tools I mentioned have 'templates'
(=ready to use scripts to set up common software) which will make use of
things like usermod. The problem: You never expect that something in the
OS will get rid of your changes and will revert to something the package
manager believes should be the default.

In environments where you only have to deal with Gentoo, we maybe have
created something *new* which allows for new possibilities, see
https://wiki.gentoo.org/wiki/OpenDKIM#The_new_way

However, this is very bad: Configuration management tools are common
these days. While we also have systemd which helps at least to provide
some kind of interface allowing user to set timezone, time
sychonization, hostname and other general settings the same way across
different distribution, configuration management tools are also an
abstraction layer: You write 'recipes' or 'playbooks', describe 'states'
in a general language and the used cfm tool will know all the
implementation details. So in the end it doesn't matter if you apply
your configuration against Debian, Fedora, RHEL or Gentoo -- the system
you run your code against should be in the described state after all.
That's at least the idea ;-)

Thanks to the new way how we manage user in Gentoo that's no longer the
case: Suddenly you cannot use basic tools like usermod to make changes
to users anymore because you cannot be sure that these settings won't be
reverted.

Also, the idea to create *packages* to represent user configuration
doesn't scale. I already outlined that issue in [1]. Simple example:

You have two services (SerivceA and ServiceB) both using the same
package (say www-servers/nginx). We are talking about something like
'real application server'. While you can overwrite user/group via ebuild
once, you can't do it multiple times for *different* roles. Especially
when you have to deal with some kind of 'dynamic' stuff (see the
Jenkins' discussion). Creating your own acct-* repository *per role*
can't scale (aside the fact that it will be impossible to tell user,
"Yeah... for Gentoo you just cannot use 'append user X to group sudo'
syntax you use in your cfm tool. Instead you have to fork
acct-group/sudo and put user into that ebuild and ensure that this
version is installed). Also, don't forget about servers executing
multiple roles at the same time: It's easier to describe something like
"Make sure user X is in group Y" vs. making sure you have that single
acct-* ebuild creating user X or group Y with everything you ever need
right from the beginning.

tl;dr
We need to change current acct-* eclasses to not change things (i.e. not
changing home, groups...). At least if user/group were created/modified
outside of PM.


See also:
=========
[1]
https://archives.gentoo.org/gentoo-dev/message/05c9b211eb18012d16302194a7bc37e6


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Christopher Head
Hmmmm. Thinking about it more, this feels a lot like “configuration”, and we have a well-established tool for handling sysadmins customizing configuration while packages land new changes: CONFIG_PROTECT. Is that possibly relevant here?
--
Christopher Head

signature.asc (251 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
In reply to this post by Thomas Deutschmann
On Thu, 2020-02-13 at 02:32 +0100, Thomas Deutschmann wrote:

> Hi,
>
> thank you for bringing this to the list.
>
> I have the same experience which is the reason why I haven't migrated
> most of my packages yet (which is not a good move because I also didn't
> post the problem to the list like I wanted *yet*, I only talked to
> people via private mail, chat or at FOSDEM about that and was working on
> a proposal I wanted to show next week when I am hopefully healthy again).
>
In other words, instead of bringing the problem up to the person who
created the GLEP and the eclasses and/or community at large, you've been
conspiring behind their back.  Yes, that's really the procommunity
attitude we expect from Council members.  Thanks for showing it again.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Alec Warner-2
On Wed, Feb 12, 2020 at 9:59 PM Michał Górny <[hidden email]> wrote:
On Thu, 2020-02-13 at 02:32 +0100, Thomas Deutschmann wrote:
> Hi,
>
> thank you for bringing this to the list.
>
> I have the same experience which is the reason why I haven't migrated
> most of my packages yet (which is not a good move because I also didn't
> post the problem to the list like I wanted *yet*, I only talked to
> people via private mail, chat or at FOSDEM about that and was working on
> a proposal I wanted to show next week when I am hopefully healthy again).
>

In other words, instead of bringing the problem up to the person who
created the GLEP and the eclasses and/or community at large, you've been
conspiring behind their back.  Yes, that's really the procommunity
attitude we expect from Council members.  Thanks for showing it again.

This is a bit of a double standard. Either people are 'conspiring behind your back' or they 'are gathering data for a counterproposal.' There is no need to paint a negative narrative here.

-A
 

--
Best regards,
Michał Górny

Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
On Wed, 2020-02-12 at 23:03 -0800, Alec Warner wrote:

> On Wed, Feb 12, 2020 at 9:59 PM Michał Górny <[hidden email]> wrote:
>
> > On Thu, 2020-02-13 at 02:32 +0100, Thomas Deutschmann wrote:
> > > Hi,
> > >
> > > thank you for bringing this to the list.
> > >
> > > I have the same experience which is the reason why I haven't migrated
> > > most of my packages yet (which is not a good move because I also didn't
> > > post the problem to the list like I wanted *yet*, I only talked to
> > > people via private mail, chat or at FOSDEM about that and was working on
> > > a proposal I wanted to show next week when I am hopefully healthy again).
> > >
> >
> > In other words, instead of bringing the problem up to the person who
> > created the GLEP and the eclasses and/or community at large, you've been
> > conspiring behind their back.  Yes, that's really the procommunity
> > attitude we expect from Council members.  Thanks for showing it again.
> >
>
> This is a bit of a double standard. Either people are 'conspiring behind
> your back' or they 'are gathering data for a counterproposal.' There is no
> need to paint a negative narrative here.
>
Yes, I certainly do have a reason to assume positive from someone who
apparently mounts a counterproposal without bothering to consider
the original reasons.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Christopher Head
In reply to this post by Michał Górny-5
On Wed, 12 Feb 2020 19:14:45 +0100
Michał Górny <[hidden email]> wrote:

> I'm pretty sure user.eclass does print whatever changes it is doing.
> It isn't elog-ged though, I admit.  This is probably worth fixing.

Yes, I meant elog; sorry about the unclear wording. I generally don’t
even see ordinary print output—most of my machines run emerge -jN where
it isn’t shown at all, and even on the serial ones, there’s so much
build output from a few dozen packages that I’m not going to scroll up
looking for specific things which may have been lost from scrollback
anyway.
--
Christopher Head

attachment0 (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
On Thu, 2020-02-13 at 08:09 -0800, Christopher Head wrote:

> On Wed, 12 Feb 2020 19:14:45 +0100
> Michał Górny <[hidden email]> wrote:
>
> > I'm pretty sure user.eclass does print whatever changes it is doing.
> > It isn't elog-ged though, I admit.  This is probably worth fixing.
>
> Yes, I meant elog; sorry about the unclear wording. I generally don’t
> even see ordinary print output—most of my machines run emerge -jN where
> it isn’t shown at all, and even on the serial ones, there’s so much
> build output from a few dozen packages that I’m not going to scroll up
> looking for specific things which may have been lost from scrollback
> anyway.
Just sent a patch to -dev.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Mike Gilbert-2
In reply to this post by Thomas Deutschmann
On Wed, Feb 12, 2020 at 8:32 PM Thomas Deutschmann <[hidden email]> wrote:
> In short: It was a very bad decision that acct-* stuff is *changing*
> existing stuff. This must be turned of *by default*. Maybe provide a
> setting a user can put into make.conf to opt into current, still new,
> behavior but by default, a package should never ever make changes to
> *existing* user (unless it knows for sure it was the only source
> creating that user and nothing was changed since creation which isn't
> easy to track).

I think it would make sense to add some eclass variables that would
turn user.eclass functions into no-ops.

I don't agree that this should be happen by default. I suspect the
majority of users do not wish to manage system users/groups
themselves.

Reply | Threaded
Open this post in threaded view
|

Changes made by acct-* ebuilds

Michael 'veremitz' Everitt
On 13/02/20 16:17, Mike Gilbert wrote:

> On Wed, Feb 12, 2020 at 8:32 PM Thomas Deutschmann <[hidden email]> wrote:
>> In short: It was a very bad decision that acct-* stuff is *changing*
>> existing stuff. This must be turned of *by default*. Maybe provide a
>> setting a user can put into make.conf to opt into current, still new,
>> behavior but by default, a package should never ever make changes to
>> *existing* user (unless it knows for sure it was the only source
>> creating that user and nothing was changed since creation which isn't
>> easy to track).
> I think it would make sense to add some eclass variables that would
> turn user.eclass functions into no-ops.
>
> I don't agree that this should be happen by default. I suspect the
> majority of users do not wish to manage system users/groups
> themselves.
>
I would suggest anyone competent enough to build a kernel from scratch
(genkernel users, I'm ignoring you) should be equally at-home managing
system users and groups and associated permissions? Or am I perhaps
overestimating the average Genbuntu users here ... >,<



signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Mike Gilbert-2
On Thu, Feb 13, 2020 at 6:24 PM Michael 'veremitz' Everitt <[hidden email]> wrote:
On 13/02/20 16:17, Mike Gilbert wrote:
> On Wed, Feb 12, 2020 at 8:32 PM Thomas Deutschmann <[hidden email]> wrote:
>> In short: It was a very bad decision that acct-* stuff is *changing*
>> existing stuff. This must be turned of *by default*. Maybe provide a
>> setting a user can put into make.conf to opt into current, still new,
>> behavior but by default, a package should never ever make changes to
>> *existing* user (unless it knows for sure it was the only source
>> creating that user and nothing was changed since creation which isn't
>> easy to track).
> I think it would make sense to add some eclass variables that would
> turn user.eclass functions into no-ops.
>
> I don't agree that this should be happen by default. I suspect the
> majority of users do not wish to manage system users/groups
> themselves.
>
I would suggest anyone competent enough to build a kernel from scratch
(genkernel users, I'm ignoring you) should be equally at-home managing
system users and groups and associated permissions? Or am I perhaps
overestimating the average Genbuntu users here ... >,<

I said nothing of capability. Most people don’t care to micromanage accounts. 
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Thomas Deutschmann
In reply to this post by Mike Gilbert-2
On 2020-02-13 17:17, Mike Gilbert wrote:
> I don't agree that this should be happen by default. I suspect the
> majority of users do not wish to manage system users/groups
> themselves.

Follow handbook to get a working system and rebuild entire world. You
will get surprised.

This is really a bad default and it's breaking with existing principles
you can find in most distributions: Don't touch stuff which were changed
by the user.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
On Fri, 2020-02-14 at 15:12 +0100, Thomas Deutschmann wrote:

> On 2020-02-13 17:17, Mike Gilbert wrote:
> > I don't agree that this should be happen by default. I suspect the
> > majority of users do not wish to manage system users/groups
> > themselves.
>
> Follow handbook to get a working system and rebuild entire world. You
> will get surprised.
>
> This is really a bad default and it's breaking with existing principles
> you can find in most distributions: Don't touch stuff which were changed
> by the user.
>
So we're apparently dealing with a major breakage where nobody bothered
to report a bug, and it's so urgent that instead of opening a bug you
choose to try to push things your way behind the scenes causing more
users to be hurt by this apparent non-reported bug just so you can have
a better chance of pushing things your way and ignoring the other side
of the resulting breakage.  Makes sense.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Mike Gilbert-2
In reply to this post by Thomas Deutschmann
On Fri, Feb 14, 2020 at 9:12 AM Thomas Deutschmann <[hidden email]> wrote:
On 2020-02-13 17:17, Mike Gilbert wrote:
> I don't agree that this should be happen by default. I suspect the
> majority of users do not wish to manage system users/groups
> themselves.

Follow handbook to get a working system and rebuild entire world. You
will get surprised.

Could you just explain please? I’m not inclined to read through the handbook and rebuild a system to guess what this horrible breakage might be. 
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Thomas Deutschmann
On 2020-02-14 16:38, Mike Gilbert wrote:
> Could you just explain please? I’m not inclined to read through the
> handbook and rebuild a system to guess what this horrible breakage might
> be.

My point is, that even the handbook tells user to modify groups. I.e.
you should add your user to group X for being able to do Y...

That's OK. That's normal.

But if everything in Gentoo has migrated to acct-* stuff, these changes
will get reverted once the user will re-emerge world including acct-*
package or such a package will get updated for some reason.

This is unexpected. Also, I hope nobody expects that every user using
sudo for example needs to maintain an own acct-*/sudo fork in his/her
overlay in future just because in Gentoo, you cannot use normal Linux
tools like usermod anymore because your changes might get resetted
during some upgrade.

That's what currently might happen with the current implementation which
tries to keep user/group state like described in package. Something you
will only see in Gentoo and no other distribution.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
On Fri, 2020-02-14 at 18:09 +0100, Thomas Deutschmann wrote:

> On 2020-02-14 16:38, Mike Gilbert wrote:
> > Could you just explain please? I’m not inclined to read through the
> > handbook and rebuild a system to guess what this horrible breakage might
> > be.
>
> My point is, that even the handbook tells user to modify groups. I.e.
> you should add your user to group X for being able to do Y...
>
> That's OK. That's normal.
>
> But if everything in Gentoo has migrated to acct-* stuff, these changes
> will get reverted once the user will re-emerge world including acct-*
> package or such a package will get updated for some reason.
Why would that happen?  We don't have acct-user/youruser.

> This is unexpected. Also, I hope nobody expects that every user using
> sudo for example needs to maintain an own acct-*/sudo fork in his/her
> overlay in future just because in Gentoo, you cannot use normal Linux
> tools like usermod anymore because your changes might get resetted
> during some upgrade.

Nope, this isn't true.  You're getting things the other way around.

> That's what currently might happen with the current implementation which
> tries to keep user/group state like described in package. Something you
> will only see in Gentoo and no other distribution.

Have you really verified your claims?  Because I really start feeling
like you've voted for the GLEP without reading it, and then started
recruiting people to change it based on guesses, still without reading
it or understanding the implementation.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Thomas Deutschmann
> # usermod -aG thomas postfix
> # id postfix
> uid=207(postfix) gid=207(postfix)
groups=207(postfix),12(mail),1004(thomas)

> # emerge -a1 acct-user/postfix
> [...]
>
> >>> Installing (1 of 1) acct-user/postfix-0::gentoo
>  * checking 0 files for package collisions
> >>> Merging acct-user/postfix-0 to /
> >>> Safely unmerging already-installed instance...
> No package files given... Grabbing a set.
> >>> Regenerating /etc/ld.so.cache...
> >>> Original instance of package unmerged safely.
>  * Updating groups for user 'postfix' ...
>  *  - Groups: postfix,mail
> >>> acct-user/postfix-0 merged.
> >>> Auto-cleaning packages...
>
> [...]
>
> # id postfix
> uid=207(postfix) gid=207(postfix) groups=207(postfix),12(mail)
>

...and if you read that council meeting log and even the mail discussion
before you will read that I always shared concerns about touching
existing user. I was only fine because I was told "We are aware, what
you described won't happen" and I didn't make a secret that I didn't had
the time to fully review and test myself at this time. Now I had the
time to 'play' with this and it looks like all my previous concerns are
true.


--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


signature.asc (631 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changes made by acct-* ebuilds

Michał Górny-5
On Fri, 2020-02-14 at 18:42 +0100, Thomas Deutschmann wrote:
> > # usermod -aG thomas postfix
> > # id postfix
> > uid=207(postfix) gid=207(postfix)
> groups=207(postfix),12(mail),1004(thomas)
> >

And now you're changing the subject.  You've just claimed that *your*
user's group ownership will be overwritten and when challenged you
present the case of *system* user's group ownership being overwritten.

Also, I'd like to point out that adding postfix user to group thomas is
a counter-intuitive example.  Someone could mistakenly assume you're
adding user thomas to group postfix.

--
Best regards,
Michał Górny


signature.asc (631 bytes) Download Attachment
12