qemu / nbd

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

qemu / nbd

n952162
Hello,

do I understand this correctly?  In order to run qemu-nbd, you emerge
app-emulation/qemu

but that isn't all, you've also got to emerge sys-block/nbd?

Why doesn't qemu have a dependency on nbd?  I don't find any relevant
USE flags.


Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Walter Dnes
On Wed, Dec 04, 2019 at 04:28:26PM +0100, n952162 wrote

> do I understand this correctly?  In order to run qemu-nbd, you emerge
> app-emulation/qemu
>
> but that isn't all, you've also got to emerge sys-block/nbd?

  nbd is a "Network Block Device" driver along the lines of NFS, but it
doesn't handle concurrency.  https://nbd.sourceforge.io/  But it's
generic, and can handle any *REGULAR* file system, not just NFS.  QCOW2,
or raw, or whatever, is a special QEMU format.  So it requires QEMU libs
(i.e. qemu-nbd) to decode QCOW2/RAW.

> Why doesn't qemu have a dependency on nbd?

  Why doesn't qemu have a dependency on NFS?  Same answer; they're both
remote network block device systems that most linux users don't need,
and they're both unrelated to the core functionality of QEMU.

--
Walter Dnes <[hidden email]>
I don't run "desktop environments"; I run useful applications

Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

n952162
But qemu includes qemu-nbd, and it seems that qemu-nbd requires nbd.ko,
which is presumably provided by sys-block/nbd.

In other words, qemu provides a facility which seems to only work with
nbd - or is that a wrong assumption?


On 12/05/19 07:03, Walter Dnes wrote:

> On Wed, Dec 04, 2019 at 04:28:26PM +0100, n952162 wrote
>
>> do I understand this correctly?  In order to run qemu-nbd, you emerge
>> app-emulation/qemu
>>
>> but that isn't all, you've also got to emerge sys-block/nbd?
>    nbd is a "Network Block Device" driver along the lines of NFS, but it
> doesn't handle concurrency.  https://nbd.sourceforge.io/  But it's
> generic, and can handle any *REGULAR* file system, not just NFS.  QCOW2,
> or raw, or whatever, is a special QEMU format.  So it requires QEMU libs
> (i.e. qemu-nbd) to decode QCOW2/RAW.
>
>> Why doesn't qemu have a dependency on nbd?
>    Why doesn't qemu have a dependency on NFS?  Same answer; they're both
> remote network block device systems that most linux users don't need,
> and they're both unrelated to the core functionality of QEMU.
>


Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

n952162

Or maybe the assumption is wrong - after emerging nbd, I still get this when I try to modprobe nbd, which is required for running qemu-nbd:

modprobe: FATAL: Module ndb not found in directory /lib/modules/4.19.72-gentoo

Can anyone explain how to run qemu-nbd on gentoo?
On ubuntu, one would do this:
sudo modprobe nbd
qemu-nbd -c /dev/nbd0 drive.vdi

On 12/05/19 07:34, n952162 wrote:
But qemu includes qemu-nbd, and it seems that qemu-nbd requires nbd.ko,
which is presumably provided by sys-block/nbd.

In other words, qemu provides a facility which seems to only work with
nbd - or is that a wrong assumption?


On 12/05/19 07:03, Walter Dnes wrote:
On Wed, Dec 04, 2019 at 04:28:26PM +0100, n952162 wrote

do I understand this correctly?  In order to run qemu-nbd, you emerge
app-emulation/qemu

but that isn't all, you've also got to emerge sys-block/nbd?
   nbd is a "Network Block Device" driver along the lines of NFS, but it
doesn't handle concurrency.  https://nbd.sourceforge.io/  But it's
generic, and can handle any *REGULAR* file system, not just NFS.  QCOW2,
or raw, or whatever, is a special QEMU format.  So it requires QEMU libs
(i.e. qemu-nbd) to decode QCOW2/RAW.

Why doesn't qemu have a dependency on nbd?
   Why doesn't qemu have a dependency on NFS?  Same answer; they're both
remote network block device systems that most linux users don't need,
and they're both unrelated to the core functionality of QEMU.




Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Arve Barsnes
On Thu, 5 Dec 2019 at 08:11, n952162 <[hidden email]> wrote:

>
> Or maybe the assumption is wrong - after emerging nbd, I still get this when I try to modprobe nbd, which is required for running qemu-nbd:
>
> modprobe: FATAL: Module ndb not found in directory /lib/modules/4.19.72-gentoo
>
> Can anyone explain how to run qemu-nbd on gentoo?
> On ubuntu, one would do this:
>
> sudo modprobe nbd
> qemu-nbd -c /dev/nbd0 drive.vdi

You can only modprobe kernel modules, so you need to have nbd compiled
as a module to your kernel to make this work.

You could also compile it directly into your kernel, then you could
skip directly to step 2.

Regards,
Arve

Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

n952162
On 12/05/19 08:17, Arve Barsnes wrote:

> On Thu, 5 Dec 2019 at 08:11, n952162 <[hidden email]> wrote:
>> Or maybe the assumption is wrong - after emerging nbd, I still get this when I try to modprobe nbd, which is required for running qemu-nbd:
>>
>> modprobe: FATAL: Module ndb not found in directory /lib/modules/4.19.72-gentoo
>>
>> Can anyone explain how to run qemu-nbd on gentoo?
>> On ubuntu, one would do this:
>>
>> sudo modprobe nbd
>> qemu-nbd -c /dev/nbd0 drive.vdi
> You can only modprobe kernel modules, so you need to have nbd compiled
> as a module to your kernel to make this work.

"to [my] kernel" ... why doesn't the ebuild do that?  That shouldn't
require rebuilding the kernel or touching /boot




>
> You could also compile it directly into your kernel, then you could
> skip directly to step 2.
>
> Regards,
> Arve
>


Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Neil Bothwick
In reply to this post by n952162
On Thu, 5 Dec 2019 08:05:46 +0100, n952162 wrote:

> Or maybe the assumption is wrong - after emerging *nbd*, I still get
> this when I try to modprobe nbd, which is required for running
> *qemu-nbd*:
>
> modprobe: FATAL: Module ndb not found in directory
> /lib/modules/4.19.72-gentoo

The assumption is wrong, sys-block/nbd contains the userland stuff, to
get the module you need CONFIG_BLK_DEV_NBD=m in yor kernel config.

The qemu ebuild already checks kernel configs for required settings and it
sounds like this should be added to the list. I'd suggest filing a bug
report, if there isn't already one.


--
Neil Bothwick

Capt'n! The spellchecker kinna take this abuse!

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Grant Taylor-2
In reply to this post by Walter Dnes
On 12/4/19 11:03 PM, Walter Dnes wrote:
> nbd is a "Network Block Device" driver along the lines of NFS, but
> it doesn't handle concurrency.  https://nbd.sourceforge.io/

I think I'd liken NBD to iSCSI more so than NFS.  Primarily because both
NBD and iSCSI provide local block devices that are backed by something
across the network.  Conversely, NFS provides access to regular files
from across the network.

You can put a file system on an NBD / iSCSI block device if you want,
but you don't have to.  Conversely, NFS is a file system and doesn't
require putting a file system on top of it.

> But it's generic, and can handle any *REGULAR* file system, not just NFS.

NFS is not a file system in the /typical/ sense.  There is no mkfs.nfs.
NFS is a file system in the sense that it is mounted and provides access
to files.

> QCOW2, or raw, or whatever, is a special QEMU format.  So it requires
> QEMU libs (i.e. qemu-nbd) to decode QCOW2/RAW.
>
> Why doesn't qemu have a dependency on NFS?

It doesn't, nor does it need to.

> Same answer; they're both remote network block device systems that most
> linux users don't need,

NFS is not a block device.

> and they're both unrelated to the core functionality of QEMU.

QEMU can use image files (qcow(2) or raw) on any mounted file system.

NFS qualifies as a mounted file system, but that is completely outside
of QEMU.

Normal file systems can be put on NBD & iSCSI devices and mounted, but
that is also completely outside of QEMU.

QEMU proper will not use NBD devices (directly) itself.

qemu-nbd is a utility to act as a NBD server to allow the Linux kernel
to be an NBD client to access qcow(2) image files.

qemu-nbd is not /needed/ for normal QEMU operation.



--
Grant. . . .
unix || die





--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

n952162
Very informative explanation.


On 12/05/19 20:01, Grant Taylor wrote:

> On 12/4/19 11:03 PM, Walter Dnes wrote:
>> nbd is a "Network Block Device" driver along the lines of NFS, but it
>> doesn't handle concurrency. https://nbd.sourceforge.io/
>
> I think I'd liken NBD to iSCSI more so than NFS.  Primarily because
> both NBD and iSCSI provide local block devices that are backed by
> something across the network.  Conversely, NFS provides access to
> regular files from across the network.
>
> You can put a file system on an NBD / iSCSI block device if you want,
> but you don't have to.  Conversely, NFS is a file system and doesn't
> require putting a file system on top of it.
>
>> But it's generic, and can handle any *REGULAR* file system, not just
>> NFS.
>
> NFS is not a file system in the /typical/ sense.  There is no
> mkfs.nfs. NFS is a file system in the sense that it is mounted and
> provides access to files.
>
>> QCOW2, or raw, or whatever, is a special QEMU format.  So it requires
>> QEMU libs (i.e. qemu-nbd) to decode QCOW2/RAW.
>>
>> Why doesn't qemu have a dependency on NFS?
>
> It doesn't, nor does it need to.
>
>> Same answer; they're both remote network block device systems that
>> most linux users don't need,
>
> NFS is not a block device.
>
>> and they're both unrelated to the core functionality of QEMU.
>
> QEMU can use image files (qcow(2) or raw) on any mounted file system.
>
> NFS qualifies as a mounted file system, but that is completely outside
> of QEMU.
>
> Normal file systems can be put on NBD & iSCSI devices and mounted, but
> that is also completely outside of QEMU.
>
> QEMU proper will not use NBD devices (directly) itself.
>
> qemu-nbd is a utility to act as a NBD server to allow the Linux kernel
> to be an NBD client to access qcow(2) image files.
>
> qemu-nbd is not /needed/ for normal QEMU operation.
>
>


But, since it's included in the package, and apparently (from the name)
will use a NBD device, then I think the dependency is logical



Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Neil Bothwick
In reply to this post by Grant Taylor-2
On Thu, 5 Dec 2019 12:01:33 -0700, Grant Taylor wrote:

> qemu-nbd is a utility to act as a NBD server to allow the Linux kernel
> to be an NBD client to access qcow(2) image files.
>
> qemu-nbd is not /needed/ for normal QEMU operation.

No, but since it is provided by the ebuild, the ebuild should check that
the target system is capable of supporting it. The qemu ebuild already
spits out warnings about missing kernel options, not all of them
essential, so why not this one too?


--
Neil Bothwick

I thought I saw the light at the end of the tunnel...
but it was just some sod with a torch bringing me more work!

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Grant Taylor-2
On 12/5/19 12:50 PM, Neil Bothwick wrote:
> No, but since it is provided by the ebuild, the ebuild should check
> that the target system is capable of supporting it. The qemu ebuild
> already spits out warnings about missing kernel options, not all of them
> essential, so why not this one too?

I think a warning for missing NBD support is absolutely the correct way
to go.

I think blocking / failing the ebuild is the wrong thing to do.

IMHO warning ≠ failure



--
Grant. . . .
unix || die





--
Grant. . . .
unix || die

Reply | Threaded
Open this post in threaded view
|

Re: qemu / nbd

Grant Taylor-2
In reply to this post by n952162
On 12/5/19 12:22 PM, n952162 wrote:
> But, since it's included in the package, and apparently (from the name)
> will use a NBD device, then I think the dependency is logical

I disagree.

QEMU itself does not use NBD.  Thus QEMU does not need to depend on
qemu-nbd.  QEMU uses files on mounted file systems.

qemu-nbd is a — perhaps questionably named — utility that allows QEMU
disk images / files to be accessed as if they were NBDs.  Not the other
way around.  qemu-nbd does not allow QEMU to use NBDs.

As I understand it, nothing about /QEMU/ is dependent on any NBD support.



--
Grant. . . .
unix || die