[PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Brian Dolbec-3
From 926c42c264e2fdb2987e145ae848fccfaa51be39 Mon Sep 17 00:00:00 2001
From: Brian Dolbec <[hidden email]>
Date: Thu, 9 Mar 2017 01:24:22 -0800
Subject: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR
 settings to new /var paths
To: [hidden email]

All settings are still configurable in catalyst.conf.
These are new reasonable settings for a relocated tree system.
A user can still set their own locations during install, or at a later time.

Default catalyst.conf settings will be changed at a later time after additional testing and
the automatic stage building scripts have been updated.
---
 catalyst/defaults.py | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/catalyst/defaults.py b/catalyst/defaults.py
index 84ed2822..947e236c 100644
--- a/catalyst/defaults.py
+++ b/catalyst/defaults.py
@@ -39,24 +39,24 @@ confdefaults={
  "compressor_options": XATTRS_OPTIONS[TAR],
  "decomp_opt": DECOMPRESSOR_PROGRAM_OPTIONS[TAR],
  "decompressor_search_order": DECOMPRESSOR_SEARCH_ORDER,
- "distdir": "/usr/portage/distfiles",
+ "distdir": "/var/portage/distfiles",
  "hash_function": "crc32",
  "icecream": "/var/cache/icecream",
  'list_xattrs_opt': LIST_XATTRS_OPTIONS[TAR],
- "local_overlay": "/usr/local/portage",
+ "local_overlay": "/var/portage/repos/local",
  "port_conf": "/etc/portage",
  "make_conf": "%(port_conf)s/make.conf",
  "options": set(),
- "packagedir": "/usr/portage/packages",
- "portdir": "/usr/portage",
+ "packagedir": "/var/portage/packages",
+ "portdir": "/var/portage/repos",
  "port_tmpdir": "/var/tmp/portage",
  "PythonDir": "./catalyst",
- "repo_basedir": "/usr",
- "repo_name": "portage",
+ "repo_basedir": "/var/portage/repos",
+ "repo_name": "gentoo",
  "sharedir": "/usr/share/catalyst",
  "shdir": "/usr/share/catalyst/targets/",
  "snapshot_cache": "/var/tmp/catalyst/snapshot_cache",
- "snapshot_name": "portage-",
+ "snapshot_name": "%(repo_name)s-",
  "source_matching": "strict",
  "storedir": "/var/tmp/catalyst",
  "target_distdir": "/var/portage/distfiles",
@@ -75,8 +75,8 @@ TARGET_MOUNT_DEFAULTS = {
  "distdir": "/usr/portage/distfiles",
  "icecream": "/usr/lib/icecc/bin",
  "kerncache": "/tmp/kerncache",
- "packagedir": "/usr/portage/packages",
- "portdir": "/usr/portage",
+ "packagedir": "/var/portage/packages",
+ "portdir": "/var/portage/repos",
  "port_tmpdir": "/var/tmp/portage",
  "port_logdir": "/var/log/portage",
  "proc": "/proc",
@@ -86,8 +86,8 @@ TARGET_MOUNT_DEFAULTS = {
 SOURCE_MOUNT_DEFAULTS = {
  "dev": "/dev",
  "devpts": "/dev/pts",
- "distdir": "/usr/portage/distfiles",
- "portdir": "/usr/portage",
+ "distdir": "/var/portage/distfiles",
+ "portdir": "/var/portage/repos",
  "port_tmpdir": "tmpfs",
  "proc": "/proc",
  "shm": "shmfs",
--
2.12.0



--
Brian Dolbec <dolsen>


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Zac Medico-2
On Fri, Mar 10, 2017 at 11:23 AM, Brian Dolbec <[hidden email]> wrote:

> @@ -75,8 +75,8 @@ TARGET_MOUNT_DEFAULTS = {
>         "distdir": "/usr/portage/distfiles",
>         "icecream": "/usr/lib/icecc/bin",
>         "kerncache": "/tmp/kerncache",
> -       "packagedir": "/usr/portage/packages",
> -       "portdir": "/usr/portage",
> +       "packagedir": "/var/portage/packages",
> +       "portdir": "/var/portage/repos",
>         "port_tmpdir": "/var/tmp/portage",
>         "port_logdir": "/var/log/portage",
>         "proc": "/proc",
> @@ -86,8 +86,8 @@ TARGET_MOUNT_DEFAULTS = {
>  SOURCE_MOUNT_DEFAULTS = {
>         "dev": "/dev",
>         "devpts": "/dev/pts",
> -       "distdir": "/usr/portage/distfiles",
> -       "portdir": "/usr/portage",
> +       "distdir": "/var/portage/distfiles",
> +       "portdir": "/var/portage/repos",
>         "port_tmpdir": "tmpfs",
>         "proc": "/proc",
>         "shm": "shmfs",

Shouldn't those portdir settings be /var/portage/repos/gentoo or
something like that?
--
Thanks,
Zac

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Brian Dolbec-3
On Fri, 10 Mar 2017 11:51:07 -0800
Zac Medico <[hidden email]> wrote:

> On Fri, Mar 10, 2017 at 11:23 AM, Brian Dolbec <[hidden email]>
> wrote:
> > @@ -75,8 +75,8 @@ TARGET_MOUNT_DEFAULTS = {
> >         "distdir": "/usr/portage/distfiles",
> >         "icecream": "/usr/lib/icecc/bin",
> >         "kerncache": "/tmp/kerncache",
> > -       "packagedir": "/usr/portage/packages",
> > -       "portdir": "/usr/portage",
> > +       "packagedir": "/var/portage/packages",
> > +       "portdir": "/var/portage/repos",
> >         "port_tmpdir": "/var/tmp/portage",
> >         "port_logdir": "/var/log/portage",
> >         "proc": "/proc",
> > @@ -86,8 +86,8 @@ TARGET_MOUNT_DEFAULTS = {
> >  SOURCE_MOUNT_DEFAULTS = {
> >         "dev": "/dev",
> >         "devpts": "/dev/pts",
> > -       "distdir": "/usr/portage/distfiles",
> > -       "portdir": "/usr/portage",
> > +       "distdir": "/var/portage/distfiles",
> > +       "portdir": "/var/portage/repos",
> >         "port_tmpdir": "tmpfs",
> >         "proc": "/proc",
> >         "shm": "shmfs",  
>
> Shouldn't those portdir settings be /var/portage/repos/gentoo or
> something like that?

No, that is the base directory for it to unpack or rsync the repo to.
It is a little confusing because of the way the code was written before.
I'll have to get in there and rename that to 'reposdir' to better
reflect the way it is used.

Note, those settings were only temporary for testing that the code was
ready for relocation.  I think I'll change them to /var/gentoo as the
base for now.  Then we can change them when the council has decreed the
final location.

--
Brian Dolbec <dolsen>


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Brian Dolbec-3
On Fri, 10 Mar 2017 12:36:06 -0800
Brian Dolbec <[hidden email]> wrote:

> On Fri, 10 Mar 2017 11:51:07 -0800
> Zac Medico <[hidden email]> wrote:
>
> > On Fri, Mar 10, 2017 at 11:23 AM, Brian Dolbec <[hidden email]>
> > wrote:  
> > > @@ -75,8 +75,8 @@ TARGET_MOUNT_DEFAULTS = {
> > >         "distdir": "/usr/portage/distfiles",
> > >         "icecream": "/usr/lib/icecc/bin",
> > >         "kerncache": "/tmp/kerncache",
> > > -       "packagedir": "/usr/portage/packages",
> > > -       "portdir": "/usr/portage",
> > > +       "packagedir": "/var/portage/packages",
> > > +       "portdir": "/var/portage/repos",
> > >         "port_tmpdir": "/var/tmp/portage",
> > >         "port_logdir": "/var/log/portage",
> > >         "proc": "/proc",
> > > @@ -86,8 +86,8 @@ TARGET_MOUNT_DEFAULTS = {
> > >  SOURCE_MOUNT_DEFAULTS = {
> > >         "dev": "/dev",
> > >         "devpts": "/dev/pts",
> > > -       "distdir": "/usr/portage/distfiles",
> > > -       "portdir": "/usr/portage",
> > > +       "distdir": "/var/portage/distfiles",
> > > +       "portdir": "/var/portage/repos",
> > >         "port_tmpdir": "tmpfs",
> > >         "proc": "/proc",
> > >         "shm": "shmfs",    
> >
> > Shouldn't those portdir settings be /var/portage/repos/gentoo or
> > something like that?  
>
> No, that is the base directory for it to unpack or rsync the repo to.
> It is a little confusing because of the way the code was written
> before. I'll have to get in there and rename that to 'reposdir' to
> better reflect the way it is used.
>
> Note, those settings were only temporary for testing that the code was
> ready for relocation.  I think I'll change them to /var/gentoo as the
> base for now.  Then we can change them when the council has decreed
> the final location.
>

Also, with these code changes, it completely separates the source host
system source tree used to generate the snapshot and the target
location and name of the snapshot tree and it's new location during
stage generation.  The new stage will end up with the new settings in
its make.conf.  This makes it possible for a host with
traditional /usr/portage or any other location or name to generate a
snapshot with a completely different base repo name.  This will make it
possible for the infra severs generating the snapshot and stages to not
need to be migrated to the new defaults in order to generate the
snapshots and stages with the new settings.  In fact, using an
alternate config, the same servers could generate both old (portage) and
new (gentoo) snapshots for an interim if desired.  Well, I think the
snapshots should be to give time for users to migrate their systems for
the new tree directory name.  That is if this code is used to generate
the daily tree snapshots.

One thing to note is that you must first edit the catalyst.conf to
add/edit the new config variables, then generate a new snapshot before
doing stage runs.

repo_basedir="/usr"
repo_name="portage"
target_distdir="/usr/portage/distfiles"
target_pkgdir="/usr/portage/packages"

Using these settings, creates a snapshot and make.conf with the
traditional locations directory names.  And completely overrides the
new defaults.

Changing the reponame to "gentoo" requires you first generate a
snapshot (the repo_name is used as the target dir and in the final
tarball name).  The stage building process will look for the snapshot
with that same name and specified version.

So, derivative distros could more easily use catalyst to generate their
own stages and snapshots.  Not to mention gentoo change locations and
repo names to something new without needing code changes to do so.  It
would be a simple config edit.
--
Brian Dolbec <dolsen>


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Mike Frysinger
In reply to this post by Brian Dolbec-3
On 10 Mar 2017 11:23, Brian Dolbec wrote:

> - "distdir": "/usr/portage/distfiles",
> + "distdir": "/var/portage/distfiles",
> - "local_overlay": "/usr/local/portage",
> + "local_overlay": "/var/portage/repos/local",
> - "packagedir": "/usr/portage/packages",
> + "packagedir": "/var/portage/packages",
> - "portdir": "/usr/portage",
> + "portdir": "/var/portage/repos",
> - "repo_basedir": "/usr",
> + "repo_basedir": "/var/portage/repos",
sticking everything under /var/portage doesn't seem like an improvement
to me.  where was the discussion that ended up with these values ?  seems
like it should be:
        repo_basedir:  /var/lib/portage/repos/
        portdir:       /var/lib/portage/repos/gentoo/
        local_overlay: /var/lib/portage/repos/local/
        packagedir:    /var/lib/portage/packages/
        distdir:       /var/cache/portage/distfiles/
-mike

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Brian Dolbec-3
On Mon, 20 Mar 2017 05:02:18 -0400
Mike Frysinger <[hidden email]> wrote:

> On 10 Mar 2017 11:23, Brian Dolbec wrote:
> > - "distdir": "/usr/portage/distfiles",
> > + "distdir": "/var/portage/distfiles",
> > - "local_overlay": "/usr/local/portage",
> > + "local_overlay": "/var/portage/repos/local",
> > - "packagedir": "/usr/portage/packages",
> > + "packagedir": "/var/portage/packages",
> > - "portdir": "/usr/portage",
> > + "portdir": "/var/portage/repos",
> > - "repo_basedir": "/usr",
> > + "repo_basedir": "/var/portage/repos",  
>
> sticking everything under /var/portage doesn't seem like an
> improvement to me.  where was the discussion that ended up with these
> values ?  seems like it should be:
> repo_basedir:  /var/lib/portage/repos/
> portdir:       /var/lib/portage/repos/gentoo/
> local_overlay: /var/lib/portage/repos/local/
> packagedir:    /var/lib/portage/packages/
> distdir:       /var/cache/portage/distfiles/
> -mike

your behind the times, I changed it to /var/gentoo as the base in the
pending branch.  The main idea is to keep the path short as possible I
think

They were just temporary anyway.  But that is also what work sets them
under.  Personally I put them under /var/db on my machine, where the
vardb is stored.  But lately, I pretty much just use teh git tree wich
is in a subdir of my user home directory.

The original email thread in -dev a few years ago never could come to
any kind of consensus where the new default should be.  It is going to
have to be a council decision to set the new default.

We'll have to  send a new email to the list with the main options, try
to get them to vote on their preference, just to satisfy council before
they make a choice.

I need to send a couple more patches to the list.  But I need to get a
couple things cleared up first.
--
Brian Dolbec <dolsen>


attachment0 (1000 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH 6/7] defaults.py: Update all repository, DISTDIR, PKGDIR settings to new /var paths

Mike Frysinger
On 20 Mar 2017 02:21, Brian Dolbec wrote:

> On Mon, 20 Mar 2017 05:02:18 -0400 Mike Frysinger wrote:
> > On 10 Mar 2017 11:23, Brian Dolbec wrote:
> > > - "distdir": "/usr/portage/distfiles",
> > > + "distdir": "/var/portage/distfiles",
> > > - "local_overlay": "/usr/local/portage",
> > > + "local_overlay": "/var/portage/repos/local",
> > > - "packagedir": "/usr/portage/packages",
> > > + "packagedir": "/var/portage/packages",
> > > - "portdir": "/usr/portage",
> > > + "portdir": "/var/portage/repos",
> > > - "repo_basedir": "/usr",
> > > + "repo_basedir": "/var/portage/repos",  
> >
> > sticking everything under /var/portage doesn't seem like an
> > improvement to me.  where was the discussion that ended up with these
> > values ?  seems like it should be:
> > repo_basedir:  /var/lib/portage/repos/
> > portdir:       /var/lib/portage/repos/gentoo/
> > local_overlay: /var/lib/portage/repos/local/
> > packagedir:    /var/lib/portage/packages/
> > distdir:       /var/cache/portage/distfiles/
>
> your behind the times, I changed it to /var/gentoo as the base in the
> pending branch.  The main idea is to keep the path short as possible I
> think
>
> They were just temporary anyway.  But that is also what work sets them
> under.  Personally I put them under /var/db on my machine, where the
> vardb is stored.  But lately, I pretty much just use teh git tree wich
> is in a subdir of my user home directory.
>
> The original email thread in -dev a few years ago never could come to
> any kind of consensus where the new default should be.  It is going to
> have to be a council decision to set the new default.
>
> We'll have to  send a new email to the list with the main options, try
> to get them to vote on their preference, just to satisfy council before
> they make a choice.
>
> I need to send a couple more patches to the list.  But I need to get a
> couple things cleared up first.
OK, as long as we don't go merging these patches into master until we've
settled on the desired defaults.  it doesn't make sense to have catalyst
do one thing since it is effectively the default install state.
-mike

signature.asc (849 bytes) Download Attachment