Network Block Devices Using Hardware Over a Network

[Lisa Kachold]

	Most
Linux administrators today are familiar with protocols such as the
Network File System (NFS) and the Server Message Block/Common Internet
File System (SMB/CIFS, the protocol used by Samba). These protocols
enable you to share filesystems across a network — the Linux computer
sitting on your desk can access files stored on another Linux system, a
Windows computer, or whatnot across the office (or on the other side of
the planet!).
While NFS, SMB/CIFS, and similar protocols are handy, they aren’t
always ideally suited to what you need to do. One obscure Linux tool
can sometimes help on this score: Network block devices (NBDs). This
tool is a way to enable one computer to provide another computer with
direct low-level access to block device hardware. (Block devices
exchange data in fixed-size multi-byte blocks, whereas character
devices communicate in byte-size chunks. Disks, including hard disks,
floppy disks, and CD-ROM/DVD drives, are the most common block
devices.) 
Why Use NBDs?
Why would you want to use an NBD rather than a more common file-sharing protocol? Several scenarios come to mind:
If client can provide better tools for low-level maintenance of disks (newer versions of fsck, for instance) than the server, providing NBD access may make sense.The client may have need of expanded network disk space, but a
traditional network filesystem may not be adequate. For instance, you
might be testing new filesystem drivers or need features of a
particular filesystem and those features might not be properly handled
by NFS or SMB/CIFS.The server might not support the filesystem or data structures on
the device you wish to export. Note that NBD server software is
available for Windows and other non-Linux OSes, so in principle a
Windows system could host a ReiserFS or XFS partition for a Linux
client, or you could export CD-ROMs burned with Rock Ridge extensions
from a Windows server.Under some circumstances, using NBDs can provide superior
performance compared to using traditional network filesystems. This can
be true if diskless clients need to boot simultaneously, for instance.
You may be able to find workarounds or superior alternatives to NBDs
in many of these cases. For instance, you could upgrade your server’s
low-level disk utilities or expand the disk space on the client.
Sometimes, though, NBDs may make more sense. You’ll need to be the
judge of which is true in your case.
You should be aware that NBDs have their limitations, too. The most
important of these is that providing read/write access to NBDs to more
than one client is extremely dangerous. Ordinarily, you’ll either limit
the NBD access to a single client or use NBDs to provide read-only
access to data (as is possible for a diskless workstation’s root
filesystem).
NBDs can also encounter deadlock conditions, particularly if you try
to access the NBD from the server. If you need to update the NBD’s
partition from the server, you should do so directly — and when no
client is using it, particularly if read/write client access is
desirable.
Obtaining NBD Software
NBD software for Linux includes three components:
NBD user-space server softwareNBD user-space client softwareNBD kernel-space client software
The user-space software can be obtained from http://sourceforge.net/projects/nbd. The nbd package includes both client and server software. Some distributions provide NBD packages; for instance, Ubuntu calls them nbd-client and nbd-server.
It’s usually easiest to install a ready-made package, but if you can’t
find one for your distribution, the installation process is
conventional:
Download the source tarball, unpack it in a convenient location, and cd into the resulting directory.Type ./configureType makeAs root, type make install
The result will be the installation of the nbd-server and nbd-client programs, along with associated man pages. As of version 2.99, the NBD source package includes a subdirectory called winnbd, which includes a Windows NBD server. I didn’t test it for this column, though.
Your NBD client requires kernel support to use NBDs, but the kernel support is not needed on the server. To add this support, you must compile it into your kernel (or as a module). The module is called nbd, so you can look for it on your system before you recompile your kernel. (On my systems, it’s stored in /lib/modules/version/kernel/drivers/block, where version is the kernel version number.)
Completely describing the process of recompiling your kernel is
beyond the scope of this column. Assuming you know how to do it, you
can find the NBD option under Device Drivers -> Block Devices, as Network block device support.
Activate this option (either compiling it directly into the kernel or
building it as a module) and recompile your kernel. (In some cases,
recompiling just your kernel modules will do.) You’ll also need to
install your kernel modules, and probably the kernel. If you changed
the kernel proper, you may need to modify your LILO or GRUB
configuration and reboot.
Preparing an NBD Server
The first step in preparing an NBD server is to set aside a block
device for use by the server. Typically, this will be a hard disk
partition, although it could be a logical volume manager (LVM) volume,
redundant array of independent disks (RAID) device, CD-ROM/DVD-ROM
drive, or some other block device. You may also create a regular file
and export it as a block device. This approach is similar to using a
loopback device locally, but you give a remote system access to the
file as if it were a block device. This can be handy if you want to
provide clients with access to a variety of image files (CD-ROM images,
say).
Depending on how you want to use the NBD, you might want or need to
create a filesystem on the device and populate it with files
beforehand. This would be necessary, for instance, if you intend to use
the NBD as a read-only root filesystem for diskless network clients. In
other cases, you might leave the preparation of the storage space to
the client. This would be necessary if the server doesn’t support the
filesystem or other low-level format the client uses, as for instance
when using a Windows NBD server.
Whether or not you prepare the NBD storage space using the server computer, you should be sure to never access the NBD space using the server when the clients do so!
In the case of read/write client access, having two clients write to a
filesystem simultaneously is almost certain to produce data corruption.
In the case of read-only access, unexpected writes might confuse the
client. You’ll need to be careful about how you prepare and use your
disk space on the server side to avoid such problems.
With the disk space prepared, you can run the NBD server:
# nbd-server 2000 /dev/sdb1

This command exports /dev/sdb1 using port 2000. Note that
the device filename must use an absolute path, not a relative path. The
version of NBD I tested issues a warning message about being unable to
open its config file. This message seems to be harmless, so you can
ignore it. In principle, you can run nbd-server as a non-root
user; however, the user must have read (and perhaps write) access to
whatever file or device you export. Note that both the port number and
block device are required parameters to nbd-server. You can read the nbd-server man page to learn about its options; some highlights include:
You can precede the port number with an IP address, as in
192.168.17.2:2000 to listen to port 2000 on IP address 192.168.17.2.
This is most useful if your server has multiple network cards and you
only want to export the device using one interface.-r exports the device in read-only mode, which is a
highly recommended precaution if clients should not be able to write to
the device.-c creates a copy-on-write export, meaning that client
write operations are performed on a temporary file, rather than on the
original file. The temporary file is discarded when the client
disconnects.-a timeout specifies a timeout period (in seconds), after which the NBD server terminates the connection.-l host_list_file specifies a file that includes the IP addresses of hosts that may connect to the NBD server. The default value is nbd_server.allow. If the file is missing, any host may connect. This tool is obviously useful for controlling access to your NBD server.-C config_file tells nbd-server where to find its configuration file (described shortly).
The -c (copy-on-write) option deserves a few more comments. Because most modern Linux distributions use udev to create a dynamic /dev directory tree on a virtual filesystem, -c
can produce write errors after a while. To avoid this problem, try
creating symbolic links on a conventional filesystem to point to the
real device file and then export the symbolic links rather than the
device files to which they point. The -c option also has the drawback of reducing performance. 
On the plus side, it’s a way around the danger associated with
providing write access to multiple clients; each client gets a unique
diff file on the server, so each client can safely write to the
“shared” file without damaging other clients’ files.
As an example of some of nbd-server’s options in action, consider the following extended command:
# nbd-server 2000 /dev/sdb1 -r -l /etc/nbd.allow

This command provides read-only access to /dev/sdb1 to those clients whose IP addresses appear in /etc/nbd.allow.
Preparing a Server Configuration File
Instead of specifying options on the command line, you can create an
NBD server configuration file. This file consists of named sections,
each section name enclosed in square brackets ([]). Within each section, lines contain parameter/value pairs, with parameters and values separated by equal signs (=). The [generic] section is required and sets global options. Each subsequent named section sets options for particular devices.  Listing One presents an example, which demonstrates two NBD exports, one for /dev/sdb1 and one for /dev/fd0.
Listing One: Sample NBD Configuration File
[generic]
  listenaddr = 0.0.0.0
  authfile = /etc/nbd-server/allow

[diskfile]
  exportname = /dev/sdb1
  copyonwrite = true
  port = 2000

[floppy]
  exportname = /dev/fd0
  port = 2001

The easiest way to create a configuration file is to specify the options you want and use the -o section_name option to nbd-server, where section_name
is the name you want to give to the section. You can then cut-and-paste
the output into your configuration file. Be sure each section specifies
a unique port number! You may also want to peruse the README
file that comes with the NBD source package; it provides examples of a
few additional options that the configuration file supports.
With configuration file created, you can pass it to nbd-server via its -C
option. The server will then share all the listed block devices, each
on its specified port and with its specified options, with just one
call to nbd-server.
Preparing an NBD Client
With an NBD-enabled kernel running, you can use the nbd-client program on the client computer to map a remote NBD to a local device file:
# nbd-client nbdserver 2000 /dev/nbd0

In this example, nbdserver is the hostname or IP address of the server computer, 2000 is the port number associated with the device you want to use, and /dev/nbd0 is the local device filename you want to link to the remote server. A few caveats and tricks require attention:
You may need to explicitly load the nbd module before running nbd-client.Once the nbd module is loaded, some systems (including Fedora and Ubuntu systems I’ve checked) automatically create /dev/nbd# devices, where # is a number from 0 up. Upon creation, these device files are not assigned to any server, but you should specify one of these devices in your nbd-client command line.The nbd-client program fails silently if it can’t find
an NBD server on the specified computer and port. If it does find an
NBD server, it displays a couple of lines of text summarizing the size
of the device it finds. If something goes wrong during this process, it
may hang with the word Negotiation on the screen. This
looks like a prompt, but it isn’t. If you see this, chances are there’s
something wrong with your NBD server configuration.The nbd-client program must be run as root.
Although the basic nbd-client command specified earlier works fine in many cases, the program supports some additional options, including:
-swap tells the system that the device should be used as swap space. This helps prevent deadlocks.timeout=seconds specifies the timeout period for NBD operations.
Using NBDs
With your NBD client pointing to a network-accessible NBD server, you can begin using NBD. You can treat your /dev/nbd#
device just as you would any local block device. Typically, you’ll
mount it as a filesystem (perhaps first creating a filesystem on it)
with mount:
# mount /dev/nbd0 /mnt/nbd

This command mounts the device at /mnt/nbd. If the device
was exported for read/write operations, you’ll be able to write to it,
with the caveat that two clients should not connect to the same
read/write device simultaneously unless you use the -c option on the server!
Of course, you can also perform other operations on an NBD, such as use mkfs to create a filesystem, check a filesystem with fsck, and so on. Many of these actions require that you have read/write access to the device, though.
When you’re done using an NBD that you’ve mounted, you can unmount it using umount, just as you would a local filesystem. You can then type nbd-client -d /dev/nbd0
(substituting the correct device filename) to terminate the
client/server NBD network link. Although this last step may not always
be strictly necessary, it can help you avoid confusion over what
devices are active, and it causes the server to delete diff files it
created if you used the -c option on the server.
One of the problems with NBDs is that, if a network connection goes
down or a server crashes, clients will have a hard time recovering.
Data may be lost, much as in a local disk crash. To guard against such
problems, I recommend using NBDs for brief periods, if possible; if you
don’t need access to the NBD now, unmount it. Try to avoid
using NBDs over anything but local networks. NBD traffic is
unencrypted, so passing it over wireless networks or the Internet can
be risky from a security point of view.
Overall, NBDs can fill some specialized needs in a Linux network. If
NFS or SMB/CIFS just doesn’t seem to fill your needs, give NBDs a try.
	
		
	
	Roderick W. Smith is the author or coauthor of more than a dozen books, including Linux in a Windows World and Linux Power Tools. He can be reached at rodsmith at rodsbooks.com.

Released into public domain.


http://wapedia.mobi/en/Obnosis || http://en.wiktionary.org/wiki/obnosis | Obnosis.com (503)754-4452
Laugh at this MSN Footer




_________________________________________________________________
See how Windows connects the people, information, and fun that are part of your life.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.PLUG.phoenix.az.us/pipermail/plug-discuss/attachments/20081006/11fe488e/attachment.htm