RAID-1 on secondary disks how?

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

RAID-1 on secondary disks how?

Peter Humphrey-3
Hello list,

(I've been off-line for ten days and I haven't yet caught up with the list. I
had to send my machine to its maker to have a cooling-system hardware fault
fixed.)

I've added two SSDs to my workstation, intending to create a RAID-1 array on
them to store backups (which may be another question when I've got the setup
right), but I cannot create the array to create LVM.

Each disk has an 80GiB NTFS partition in case I want to install Windows, then
a 500GiB ext2 partition.

When I run "mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda2 /dev/
sdb2", this is what I get:

# mdadm --stop /dev/md0
mdadm: stopped /dev/md0
# mdadm: /dev/sda2 appears to contain an ext2fs file system
       size=524288000K  mtime=Thu Jan  1 01:00:00 1970
mdadm: /dev/sdb2 appears to contain an ext2fs file system
       size=524288000K  mtime=Thu Jan  1 01:00:00 1970
Continue creating array? y
mdadm: RUN_ARRAY failed: Invalid argument

If I boot from SysRescCD I also get "device /dev/sda2 exists but is not an md
device." I also had to "mdadm --stop /dev/md127" since that OS calls the first
array that.

I've tried with a GPT disk header and with an MSDOS one, with similar results.
Also with /etc/init.d/mdraid not running, with it started on my command and
with it in the boot runlevel. Each time I changed anything I rebooted before
trying anything else.

I must be missing something, in spite of following the wiki instructions. Can
someone help an old duffer out?

--
Regards,
Peter.




Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Mick-10
Hello Peter,

On Monday, 28 January 2019 16:56:57 GMT Peter Humphrey wrote:
> Hello list,

> When I run "mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda2
> /dev/ sdb2", this is what I get:
>
> # mdadm --stop /dev/md0
> mdadm: stopped /dev/md0
> # mdadm: /dev/sda2 appears to contain an ext2fs file system
>        size=524288000K  mtime=Thu Jan  1 01:00:00 1970
> mdadm: /dev/sdb2 appears to contain an ext2fs file system
>        size=524288000K  mtime=Thu Jan  1 01:00:00 1970
> Continue creating array? y
> mdadm: RUN_ARRAY failed: Invalid argument
Do you have CONFIG_MD_RAID1 (or whatever it should be these days) built in
your kernel?


> If I boot from SysRescCD I also get "device /dev/sda2 exists but is not an
> md device." I also had to "mdadm --stop /dev/md127" since that OS calls the
> first array that.

The SysRescueCD kernel you boot with does not (yet) have access to an
mdadm.conf and is automatically compiling the RAID array with default
settings.


> I've tried with a GPT disk header and with an MSDOS one, with similar
> results. Also with /etc/init.d/mdraid not running, with it started on my
> command and with it in the boot runlevel. Each time I changed anything I
> rebooted before trying anything else.
>
> I must be missing something, in spite of following the wiki instructions.
> Can someone help an old duffer out?

You need to update your initramfs after you configure your array, so your
kernel knows what to assemble at boot time when it doesn't yet have access to
your mdadm.conf.

It could help if you examined/posted the contents of your:

cat /etc/mdadm/mdadm.conf
cat /proc/mdstat
dmesg |grep md
mdadm --detail --scan
ls -l /dev/md*

I haven't worked with RAID for some years now, but going from memory the above
should reveal any discrepancies.

--
Regards,
Mick

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Peter Humphrey-3
On Tuesday, 29 January 2019 09:20:46 GMT Mick wrote:

Hello Mick,

--->8

> Do you have CONFIG_MD_RAID1 (or whatever it should be these days) built in
> your kernel?

Yes, I have, but something else was missing: CONFIG_DM_RAID=y. This is in the
SCSI section, which I'd overlooked (I hadn't needed it before because the main
storage is an NVMe drive). After setting that and rebooting, mdadm --create is
working as expected.

The wiki needs a small addition. I submitted a bug against it but was told not
to use bugs for this purpose: I should use the wiki page's discussion page. I
see more head-scratchery coming on...

--->8

> You need to update your initramfs after you configure your array, so your
> kernel knows what to assemble at boot time when it doesn't yet have access
> to your mdadm.conf.

I'd rather not have to create an initramfs if I can avoid it. Would it be
sensible to start the raid volume by putting an mdadm --assemble command into,
say, /etc/local.d/raid.start? The machine doesn't boot from /dev/md0.

--
Regards,
Peter.




Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Mick-10
On Tuesday, 29 January 2019 16:08:27 GMT Peter Humphrey wrote:

> On Tuesday, 29 January 2019 09:20:46 GMT Mick wrote:
>
> Hello Mick,
>
> --->8
>
> > Do you have CONFIG_MD_RAID1 (or whatever it should be these days) built in
> > your kernel?
>
> Yes, I have, but something else was missing: CONFIG_DM_RAID=y. This is in
> the SCSI section, which I'd overlooked (I hadn't needed it before because
> the main storage is an NVMe drive). After setting that and rebooting, mdadm
> --create is working as expected.
Good!  I had assumed this was already selected.  ;-)


> > You need to update your initramfs after you configure your array, so your
> > kernel knows what to assemble at boot time when it doesn't yet have access
> > to your mdadm.conf.
>
> I'd rather not have to create an initramfs if I can avoid it. Would it be
> sensible to start the raid volume by putting an mdadm --assemble command
> into, say, /etc/local.d/raid.start? The machine doesn't boot from /dev/md0.

I think yes, as long as OS filesystem(s) are not on the array, or not mounted
until the array has been assembled with the correct mdadm.config.

--
Regards,
Mick

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
In reply to this post by Peter Humphrey-3
On 01/29/2019 09:08 AM, Peter Humphrey wrote:
> I'd rather not have to create an initramfs if I can avoid it. Would it
> be sensible to start the raid volume by putting an mdadm --assemble
> command into, say, /etc/local.d/raid.start? The machine doesn't boot
> from /dev/md0.

Drive by comment.

I thought there was a kernel option / command line parameter that
enabled the kernel to automatically assemble arrays as it's
initializing.  Would something like that work for you?

I have no idea where that is in the context of what you're working on.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Alan Mackenzie
Hello, All.

On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote:
> On 01/29/2019 09:08 AM, Peter Humphrey wrote:
> > I'd rather not have to create an initramfs if I can avoid it. Would it
> > be sensible to start the raid volume by putting an mdadm --assemble
> > command into, say, /etc/local.d/raid.start? The machine doesn't boot
> > from /dev/md0.

> Drive by comment.

> I thought there was a kernel option / command line parameter that
> enabled the kernel to automatically assemble arrays as it's
> initializing.  Would something like that work for you?

> I have no idea where that is in the context of what you're working on.

I use mdadm with a RAID-1 pair of SSDs, without an initramfs (YUCK!).
My root partition is on the RAID.

For this, the kernel needs to be able to assemble the drives into the
raid at booting up time, and for that you need version 0.90 metadata.
(Or, at least, you did back in 2017.)

My command for building my array was:

    # mdadm --create /dev/md2 --level=1 --raid-devices=2 \
    --metadata=0.90 /dev/nvme0n1p2 /dev/nvme1n1p2.

However, there's another quirk which bit me: something in the Gentoo
installation disk took it upon itself to renumber my /dev/md2 to
/dev/md127.  I raised bug #539162 for this, but it was decided not to
fix it.  (This was back in February 2015.)

> --
> Grant. . . .
> unix || die

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
On 01/29/2019 09:48 AM, Alan Mackenzie wrote:
> However, there's another quirk which bit me: something in the Gentoo
> installation disk took it upon itself to renumber my /dev/md2 to
> /dev/md127.  I raised bug #539162 for this, but it was decided not to
> fix it.  (This was back in February 2015.)

I tend to treat the /dev/md<number(s)> to be somewhat fluid, much like I
do /dev/sd<letter(s)>.  I prefer to use UUID for raw file systems or LV
names when using LVM for this reason.

$WORK uses sym-links from persistent names to the actual disk that is
currently the desired disk.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Rich Freeman
In reply to this post by Alan Mackenzie
On Tue, Jan 29, 2019 at 11:48 AM Alan Mackenzie <[hidden email]> wrote:

>
> On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote:
> > On 01/29/2019 09:08 AM, Peter Humphrey wrote:
> > > I'd rather not have to create an initramfs if I can avoid it. Would it
> > > be sensible to start the raid volume by putting an mdadm --assemble
> > > command into, say, /etc/local.d/raid.start? The machine doesn't boot
> > > from /dev/md0.
>
>
> For this, the kernel needs to be able to assemble the drives into the
> raid at booting up time, and for that you need version 0.90 metadata.
> (Or, at least, you did back in 2017.)
>

Can't say I've tried it recently, but I'd be shocked if it changed
much.  The linux kernel guys generally consider this somewhat
deprecated behavior, and prefer that users use an initramfs for this
sort of thing.  It is exactly the sort of problem an initramfs was
created to fix.

Honestly, I'd just bite the bullet and use dracut if you want your OS
on RAID/etc.  It is basically a one-liner at this point to install and
a relatively small tweak to your GRUB config (automatic if using
mkconfig).  Dracut will respect your mdadm.conf, and just about all
your other config info in /etc.  The only gotcha is rebuilding your
initramfs if it drastically changes (but, drastically changing your
root filesystem is something that requires care anyway).

But, if you're not using an initramfs you can get the kernel to handle
this.  Just don't be surprised when it changes your device name or
whatever.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Alan Mackenzie
Hello, Rich.

On Tue, Jan 29, 2019 at 12:58:38 -0500, Rich Freeman wrote:
> On Tue, Jan 29, 2019 at 11:48 AM Alan Mackenzie <[hidden email]> wrote:

> > On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote:
> > > On 01/29/2019 09:08 AM, Peter Humphrey wrote:
> > > > I'd rather not have to create an initramfs if I can avoid it. Would it
> > > > be sensible to start the raid volume by putting an mdadm --assemble
> > > > command into, say, /etc/local.d/raid.start? The machine doesn't boot
> > > > from /dev/md0.


> > For this, the kernel needs to be able to assemble the drives into the
> > raid at booting up time, and for that you need version 0.90 metadata.
> > (Or, at least, you did back in 2017.)


> Can't say I've tried it recently, but I'd be shocked if it changed
> much.  The linux kernel guys generally consider this somewhat
> deprecated behavior, and prefer that users use an initramfs for this
> sort of thing.  It is exactly the sort of problem an initramfs was
> created to fix.

An initramfs is conceptually so ugly that I view it as a workaround, not
a fix, to whatever problem it's applied to.  It would surely be a bug if
the kernel were capable of manipulating RAIDs, but not of initialising
and mounting them.

> Honestly, I'd just bite the bullet and use dracut if you want your OS
> on RAID/etc.  It is basically a one-liner at this point to install and
> a relatively small tweak to your GRUB config (automatic if using
> mkconfig).  Dracut will respect your mdadm.conf, and just about all
> your other config info in /etc.  The only gotcha is rebuilding your
> initramfs if it drastically changes (but, drastically changing your
> root filesystem is something that requires care anyway).

Well, at the moment my system's not broken, hence doesn't need fixing.
Last time I looked at Dracut, it would only work in a kernel built with
modules enabled, ruling out my setup.

Also, without putting in a LOT of time and study, dracut is a massive,
opaque mystery.  I've got a pretty good mental picture of how my system
works, and introducing an initramfs would degrade that picture
enormously.  That means if any problems happened with the initramfs, I'd
be faced with many days study to get to grips with it.

> But, if you're not using an initramfs you can get the kernel to handle
> this.  Just don't be surprised when it changes your device name or
> whatever.

The kernel seems to leave it alone.  Any Gentoo installation CD I've
used has corrupted the setup, changing all the names to /dev/md127,
/dev/md126, ...., leaving the victim PC unbootable.  Hence my root
partition is /dev/md127, despite me originally creating it as something
like /dev/md4.

> --
> Rich

--
Alan Mackenzie (Nuremberg, Germany).

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
In reply to this post by Rich Freeman
On 01/29/2019 10:58 AM, Rich Freeman wrote:
> Can't say I've tried it recently, but I'd be shocked if it changed much.
> The linux kernel guys generally consider this somewhat deprecated
> behavior, and prefer that users use an initramfs for this sort of thing.
> It is exactly the sort of problem an initramfs was created to fix.

I see no reason to use an initramfs (swingroot) if the kernel can do
what is needed by itself.

> Honestly, I'd just bite the bullet and use dracut if you want your OS
> on RAID/etc.

You obviously have a different opinion than Alan and I do.  I dislike
using an initramfs (swingroot) without a specific reason to actually
/need/ one.  As in the kernel is unable to do what is necessary by
itself and /requires/ assistance of an initramfs (swingroot).  (An
encrypted root device or iSCSI connected root device comes to mind as a
legitimate /need/ for an initramfs (swingroot).)

> It is basically a one-liner at this point to install and a relatively
> small tweak to your GRUB config (automatic if using mkconfig).

The dracut command may be a one-liner.  But the alteration to the system
and it's boot & mount process is CONSIDERABLY more significant.

> Dracut will respect your mdadm.conf, and just about all your other
> config info in /etc.  The only gotcha is rebuilding your initramfs if
> it drastically changes (but, drastically changing your root filesystem
> is something that requires care anyway).

I can think of some drastic changes to the root file system that would
not require changing the kernel, boot loader, or command line options.

> But, if you're not using an initramfs you can get the kernel to
> handle this.  Just don't be surprised when it changes your device name
> or whatever.

There are ways to get around the device naming issue.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Rich Freeman
In reply to this post by Alan Mackenzie
On Tue, Jan 29, 2019 at 1:39 PM Alan Mackenzie <[hidden email]> wrote:

>
> On Tue, Jan 29, 2019 at 12:58:38 -0500, Rich Freeman wrote:
> > Can't say I've tried it recently, but I'd be shocked if it changed
> > much.  The linux kernel guys generally consider this somewhat
> > deprecated behavior, and prefer that users use an initramfs for this
> > sort of thing.  It is exactly the sort of problem an initramfs was
> > created to fix.
>
> An initramfs is conceptually so ugly that I view it as a workaround, not
> a fix, to whatever problem it's applied to.

Not sure why you would think this.  It is just a cpio archive of a
root filesystem that the kernel runs as a generic bootstrap.

This means that your bootstrap for initializing your root and
everything else can use any userspace tool that exists for linux.

A similar concept lies at the heart of coreboot - using a generic
kernel/userpace as a firmware bootloader making it far more flexible.

An initramfs is basically just a fairly compact linux distro.  It
works the same as any distro.  The kernel runs init, and init does its
thing.  By convention that init will mount the real root and then exec
the init inside, but it doesn't have to work that way.  Heck, you can
run a system with nothing but an initramfs and no other root
filesystem.

> It would surely be a bug if the kernel were capable of manipulating RAIDs, but not of initialising
> and mounting them.

Linus would disagree with you there, and has said as much publicly.
He does not consider initialization to be the responsibility of kernel
space long-term, and prefers that this happen in user-space.

Some of the lvm and mdadm support remains for legacy reasons, but you
probably won't see initialization of newer volume/etc managers
supported directly in the kernel.

> > Honestly, I'd just bite the bullet and use dracut if you want your OS
> > on RAID/etc.  It is basically a one-liner at this point to install and
> > a relatively small tweak to your GRUB config (automatic if using
> > mkconfig).  Dracut will respect your mdadm.conf, and just about all
> > your other config info in /etc.  The only gotcha is rebuilding your
> > initramfs if it drastically changes (but, drastically changing your
> > root filesystem is something that requires care anyway).
>
> Well, at the moment my system's not broken, hence doesn't need fixing.
> Last time I looked at Dracut, it would only work in a kernel built with
> modules enabled, ruling out my setup.

That is news to me.  Obviously it needs whatever drivers it needs, but
I don't see why it would care if they are built in-kernel vs
in-module.

> Also, without putting in a LOT of time and study, dracut is a massive,
> opaque mystery.  I've got a pretty good mental picture of how my system
> works, and introducing an initramfs would degrade that picture
> enormously.  That means if any problems happened with the initramfs, I'd
> be faced with many days study to get to grips with it.

Sure, if you want to know exactly how it works that is true, but the
same is true of openrc or any other piece of software on your system.

Dracut is highly modular and phase/hook-driven so it isn't too hard to
grok.  Most of it is just bash/dash scripts.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Rich Freeman
In reply to this post by Grant Taylor-2
On Tue, Jan 29, 2019 at 1:54 PM Grant Taylor
<[hidden email]> wrote:
>
> On 01/29/2019 10:58 AM, Rich Freeman wrote:
> > Can't say I've tried it recently, but I'd be shocked if it changed much.
> > The linux kernel guys generally consider this somewhat deprecated
> > behavior, and prefer that users use an initramfs for this sort of thing.
> > It is exactly the sort of problem an initramfs was created to fix.
>
> I see no reason to use an initramfs (swingroot) if the kernel can do
> what is needed by itself.

Personally I use dracut on boxes with a single ext4 partition...  To
each his own.  I don't see the value in using a different
configuration on a box simply because it happens to work on that
particular box.  Dracut is a more generic solution that allows me to
keep hosts the same.

> > Honestly, I'd just bite the bullet and use dracut if you want your OS
> > on RAID/etc.
>
> You obviously have a different opinion than Alan and I do.

Thank you!  :)

> > It is basically a one-liner at this point to install and a relatively
> > small tweak to your GRUB config (automatic if using mkconfig).
>
> The dracut command may be a one-liner.  But the alteration to the system
> and it's boot & mount process is CONSIDERABLY more significant.

Kinda sorta.  The kernel boots one distro which then chroots and execs
another.  The initramfs follows the exact same rules as any other
userspace rootfs.

> > Dracut will respect your mdadm.conf, and just about all your other
> > config info in /etc.  The only gotcha is rebuilding your initramfs if
> > it drastically changes (but, drastically changing your root filesystem
> > is something that requires care anyway).
>
> I can think of some drastic changes to the root file system that would
> not require changing the kernel, boot loader, or command line options.

Sure, and I wouldn't expect them to require rebuilding your initramfs
either.  I was speaking generally.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
On 01/29/2019 12:04 PM, Rich Freeman wrote:
> I don't see the value in using a different configuration on a box simply
> because it happens to work on that particular box.  Dracut is a more
> generic solution that allows me to keep hosts the same.

And if all the boxes in the fleet can function without an initramfs?
Then why have it?  Why not apply Occam's Razor & Parsimony and use the
simpler solution.  Especially if more complex solutions introduce
additional things that need to be updated.

> Kinda sorta.  The kernel boots one distro which then chroots and execs
> another.  The initramfs follows the exact same rules as any other
> userspace rootfs.

There's no "kinda" to it.  Booting one distro (kernel & set of init
scripts), then chrooting and execing a new kernel and the booting
another distro is at least twice as complex as booting a single distro.

This is even more complicated if the first initramfs distro is not
identical to the main installed distro.  Which is quite likely to be the
case, or at least a subset of the main installed distro.

IMHO an initramfs is usually, but not always, an unnecessary complication.

> Sure, and I wouldn't expect them to require rebuilding your initramfs
> either.  I was speaking generally.

Modifying things like crypttab and / or adding / removing file systems
from the kernel that are required for boot have caused me to need to
rebuild an initramfs in the past.  But that was not necessarily Gentoo,
so it may not be a fair comparison.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Rich Freeman
On Tue, Jan 29, 2019 at 2:22 PM Grant Taylor
<[hidden email]> wrote:

>
> On 01/29/2019 12:04 PM, Rich Freeman wrote:
> > I don't see the value in using a different configuration on a box simply
> > because it happens to work on that particular box.  Dracut is a more
> > generic solution that allows me to keep hosts the same.
>
> And if all the boxes in the fleet can function without an initramfs?
> Then why have it?  Why not apply Occam's Razor & Parsimony and use the
> simpler solution.  Especially if more complex solutions introduce
> additional things that need to be updated.

If all my boxes could function reliably without an initramfs I
probably would do it that way.  However, as soon as you throw so much
as a second hard drive in a system that becomes unreliable.

I'm not saying you can't use linux without an initramfs.  I'm just
questioning why most normal people would want to.  I bet that 98% of
people who use Linux run an initramfs, and there is a reason for
that...

> > Sure, and I wouldn't expect them to require rebuilding your initramfs
> > either.  I was speaking generally.
>
> Modifying things like crypttab and / or adding / removing file systems
> from the kernel that are required for boot have caused me to need to
> rebuild an initramfs in the past.  But that was not necessarily Gentoo,
> so it may not be a fair comparison.

A lot of that is situational.  If you have a kernel without btrfs
support, and you build btrfs as a module and switch your root
filesystem to btrfs, then obviously you'll need to rebuild your
initramfs since the one you have can't do btrfs.  But, most people
would just rebuild their initramfs anytime they rebuild a kernel just
to be safe.  If you added btrfs support to the kernel (built-in) then
it is more of a toss-up, though in the case of btrfs specifically you
might still need to regenerate the initramfs to add the btrfs
userspace tools to it if you didn't already have them in /usr when you
generated it the first time.

But, if you're running btrfs you're probably forced to use an
initramfs in any case.

In any case, it isn't some kind of automatic thing.  Just as some
things require rebuilding a kernel, some things require rebuilding an
initramfs.  I just find it simplest to build an initramfs anytime I
build a kernel, and use the make install naming convention so that
grub-mkconfig just does its thing automatically.

IMO Dracut is one of the most robust solutions for these sorts of
situations.  It is highly modular, easy to extend, and it really tries
hard to respect your existing config in /etc.  In fact, not only does
it put a copy of fstab in the initramfs to help it find your root, but
after it mounts the root it checks that version of fstab to see if it
is different and then remounts things accordingly.

If you haven't guessed I'm a bit of a Dracut fan.  :)

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
In reply to this post by Rich Freeman
On 01/29/2019 12:01 PM, Rich Freeman wrote:
> Not sure why you would think this.  It is just a cpio archive of a root
> filesystem that the kernel runs as a generic bootstrap.

IMHO the simple fact that such is used when it is not needed is ugly part.

> This means that your bootstrap for initializing your root and everything
> else can use any userspace tool that exists for linux.

Why would I want to do that if I don't /need/ to do that?

I can use IPs on VXLAN VTEPs to communicate between two hosts in the
same L2 broadcast domain too.  But why would I want to do that when the
simple IP address on the Ethernet interface will suffice?

> A similar concept lies at the heart of coreboot - using a generic
> kernel/userpace as a firmware bootloader making it far more flexible.

Coreboot is not part of the operating system.

If you want to talk about the kernel in coreboot taking over the
kernel's job and removing the boot loader + (2nd) kernel, I'm interested
in discussing.

> An initramfs is basically just a fairly compact linux distro.  It works
> the same as any distro.

IP over the VXLAN VTEP works the same as IP over Ethernet too.

The simple fact that there are two distros (kernel & init scripts) that
run in succession when there is no /need/ for them is the ugly bit.

Why stop at two distros?  Why not three or four or more?

> The kernel runs init, and init does its thing.  By convention that init
> will mount the real root and then exec the init inside, but it doesn't
> have to work that way.  Heck, you can run a system with nothing but an
> initramfs and no other root filesystem.

You can also run a system in the halt run level with everything mounted
read-only.  I used to run firewalls this way.  Makes them really hard to
modify without rebooting and altering how they boot.  }:-)

> Linus would disagree with you there, and has said as much publicly.
> He does not consider initialization to be the responsibility of kernel
> space long-term, and prefers that this happen in user-space.

~chuckle~  That wouldn't be the first or last time that Linus disagreed
with someone.

> Some of the lvm and mdadm support remains for legacy reasons, but you
> probably won't see initialization of newer volume/etc managers supported
> directly in the kernel.
>
>
> That is news to me.  Obviously it needs whatever drivers it needs, but
> I don't see why it would care if they are built in-kernel vs in-module.

O.o‽

How is a kernel going to be able to mount the root file system if it
doesn't have the driver (and can't load) for said root file system type?
  Or how is it going to extract the initramfs if it doesn't have the
driver for a ram disk?

The kernel /must/ have (at least) the minimum drivers (and dependencies)
to be able to boot strap.  It doesn't matter if it's boot strapping an
initramfs or otherwise.

All of these issues about lack of a driver are avoided by having the
driver statically compiled into the kernel.

IMHO a kernel and a machine is quite a bit more secure if it doesn't
support modules.  Almost all of the root kits that I've seen require
modules.  So the root kits are SIGNIFICANTLY impeded if not completely
broken if the kernel doesn't support modules.

> Sure, if you want to know exactly how it works that is true, but the
> same is true of openrc or any other piece of software on your system.

Yep.

I think it's a LOT easier to administer a system that I understand how
and why it works.

> Dracut is highly modular and phase/hook-driven so it isn't too hard
> to grok.  Most of it is just bash/dash scripts.

That may be the case.  But why even spend time with it if it's not
actually /needed/.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Rich Freeman
On Tue, Jan 29, 2019 at 2:41 PM Grant Taylor
<[hidden email]> wrote:
>
> On 01/29/2019 12:01 PM, Rich Freeman wrote:
> >
> > That is news to me.  Obviously it needs whatever drivers it needs, but
> > I don't see why it would care if they are built in-kernel vs in-module.
>
> How is a kernel going to be able to mount the root file system if it
> doesn't have the driver (and can't load) for said root file system type?

It couldn't.  Hence the reason I said, "obviously it needs whatever
drivers it needs, but I don't see why it would care if they are built
-in-kernel vs in-module."

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
In reply to this post by Rich Freeman
On 01/29/2019 12:33 PM, Rich Freeman wrote:
> If all my boxes could function reliably without an initramfs I probably
> would do it that way.

;-)

> However, as soon as you throw so much as a second hard drive in a system
> that becomes unreliable.

I disagree.

I've been reliably booting and running systems with multiple drives for
almost two decades.  Including various combinations of PATA, SATA, SCSI,
USB, SAN, drives.

Mounting the root based on UUID (or labels) is *WONDERFUL*.  It makes
the system MUCH MORE resilient.  Even if a device somehow gets inserted
before your root device in the /dev/sd* order.

> I'm not saying you can't use linux without an initramfs.  I'm just
> questioning why most normal people would want to.  I bet that 98% of
> people who use Linux run an initramfs, and there is a reason for that...

I don't doubt your numbers.

But I do question the viability of them.  How many people that are
running Linux even know that they have an option?

I suspect the answer to that question is less extreme than you are
wanting.  I also suspect that it's more in your favor than I want.  But
70 / 30 (I pulled those from you know where) is significantly different
than 98 / 2.

> A lot of that is situational.  If you have a kernel without btrfs support,
> and you build btrfs as a module and switch your root filesystem to btrfs,
> then obviously you'll need to rebuild your initramfs since the one you
> have can't do btrfs.  But, most people would just rebuild their initramfs
> anytime they rebuild a kernel just to be safe.  If you added btrfs support
> to the kernel (built-in) then it is more of a toss-up, though in the case
> of btrfs specifically you might still need to regenerate the initramfs
> to add the btrfs userspace tools to it if you didn't already have them
> in /usr when you generated it the first time.
>
> But, if you're running btrfs you're probably forced to use an initramfs
> in any case.

That's one of many reasons that I'm not using btrfs.

> In any case, it isn't some kind of automatic thing.  Just as some things
> require rebuilding a kernel, some things require rebuilding an initramfs.
> I just find it simplest to build an initramfs anytime I build a kernel,
> and use the make install naming convention so that grub-mkconfig just
> does its thing automatically.

Simplicity is completely independent of necessity.

You have made a choice.  That's your choice to make.

> IMO Dracut is one of the most robust solutions for these sorts of
> situations.  It is highly modular, easy to extend, and it really tries
> hard to respect your existing config in /etc.  In fact, not only does
> it put a copy of fstab in the initramfs to help it find your root, but
> after it mounts the root it checks that version of fstab to see if it
> is different and then remounts things accordingly.

I find it simpler and more robust to remove the initramfs complexity if
I have no /need/ for it.

> If you haven't guessed I'm a bit of a Dracut fan.  :)

And I'm a fan of simplicity.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Wols Lists
In reply to this post by Alan Mackenzie
On 29/01/2019 16:48, Alan Mackenzie wrote:

> Hello, All.
>
> On Tue, Jan 29, 2019 at 09:32:19 -0700, Grant Taylor wrote:
>> On 01/29/2019 09:08 AM, Peter Humphrey wrote:
>>> I'd rather not have to create an initramfs if I can avoid it. Would it
>>> be sensible to start the raid volume by putting an mdadm --assemble
>>> command into, say, /etc/local.d/raid.start? The machine doesn't boot
>>> from /dev/md0.
>
>> Drive by comment.
>
>> I thought there was a kernel option / command line parameter that
>> enabled the kernel to automatically assemble arrays as it's
>> initializing.  Would something like that work for you?
>
>> I have no idea where that is in the context of what you're working on.
>
> I use mdadm with a RAID-1 pair of SSDs, without an initramfs (YUCK!).
> My root partition is on the RAID.
>
> For this, the kernel needs to be able to assemble the drives into the
> raid at booting up time, and for that you need version 0.90 metadata.
> (Or, at least, you did back in 2017.)

You still do. 0.9 is deprecated and bit-rotting. If  it breaks, nobody
is going to fix it!!!

>
> My command for building my array was:
>
>      # mdadm --create /dev/md2 --level=1 --raid-devices=2 \
>      --metadata=0.90 /dev/nvme0n1p2 /dev/nvme1n1p2.
>
> However, there's another quirk which bit me: something in the Gentoo
> installation disk took it upon itself to renumber my /dev/md2 to
> /dev/md127.  I raised bug #539162 for this, but it was decided not to
> fix it.  (This was back in February 2015.)
>
This is nothing to do with gentoo - it's mdadm. And like with sdX for
drives, it's explicitly not guaranteed that the number remains
consistent. You're supposed to use names now, eg /dev/md/root, or
/dev/md/home for example.

Cheers,
Wol

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Grant Taylor-2
In reply to this post by Rich Freeman
On 01/29/2019 12:47 PM, Rich Freeman wrote:
> It couldn't.  Hence the reason I said, "obviously it needs whatever
> drivers it needs, but I don't see why it would care if they are built
> -in-kernel vs in-module."

You are missing what I'm saying.

Even the kernel the initramfs uses MUST have support for the file
systems and devices that the initramfs uses.  You can't load the module
if you can't get to where the module is or have a place to write it to
load it.



--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: RAID-1 on secondary disks how?

Rich Freeman
In reply to this post by Grant Taylor-2
On Tue, Jan 29, 2019 at 2:52 PM Grant Taylor
<[hidden email]> wrote:
>
> On 01/29/2019 12:33 PM, Rich Freeman wrote:
>
> > However, as soon as you throw so much as a second hard drive in a system
> > that becomes unreliable.
>
> Mounting the root based on UUID (or labels) is *WONDERFUL*.  It makes
> the system MUCH MORE resilient.  Even if a device somehow gets inserted
> before your root device in the /dev/sd* order.

Interesting.  I didn't realize that linux supported PARTUUID natively.
I'll agree that addresses many more use cases.  I was under the
impression that it required an initramfs - maybe that is a recent
change...

>
> > I'm not saying you can't use linux without an initramfs.  I'm just
> > questioning why most normal people would want to.  I bet that 98% of
> > people who use Linux run an initramfs, and there is a reason for that...
>
> I don't doubt your numbers.
>
> But I do question the viability of them.  How many people that are
> running Linux even know that they have an option?

Most of them probably don't even realize they're running Linux.  :)

People design systems with an initramfs because it is more robust and
covers more use cases, including the use cases that don't require an
initramfs.  It also allows a fully modular kernel, which means less
RAM use/etc on distros that use one-size-fits-all kernels (which is
basically all of them but Gentoo).

--
Rich

12