5.1. Setup

This is a description of the salient features of the setup used to develop the procedures described below. All the procedures have been tried and tested, also for Linux-2.6. The screen-like displays are precise copies of what appeared on my screen.

• Hardware: Intel (R) Celeron (TM) 1100 MHz
• Distribution: RedHat Linux 7.0 (extensively modified)
• Kernel: Linux-2.4.20 (from www.kernel.org). See also Section 12 for Linux-2.6.x.
• Tools: util-linux-2.11z (mount, umount, fdisk); e2fsprogs-1.32 (mke2fs, dumpe2fs, fsck.ext2); mkdosfs-2.2

5.2. Kernel options

It is uncertain if USB-support is sufficient in kernels earlier than 2.4.xx. The following support, relevant to this document, was compiled into the kernel. A modular approach may also be followed.

5.3. Notes

In the lists above the entities in uppercase refer to the variable names to be found in the .config file in the upper level directory of the kernel source (/usr/src/linux/). The entities xxx.o refer to the modules created when a modular approach is followed. When there is no reference to a module, the option can only be hard-compiled into the kernel.

Different kernel versions may have different indications of options when, for example, make menuconfig or make xconfig are run. Variables such as CONFIG_USB, which can be gleaned from the various help options, may be a more reliable indication.

Very recent Linux distributions such as RedHat and SuSE probably have the appropriate kernel options compiled in.

Under USB-support, options for a number of digital cameras are available.

Please consult the relevant texts as set out in Section 4 if you consider (re)compiling your kernel.

6.1. The /proc filesystem

The /proc filesystem serves as a window through which we can see the workings of a Linux setup. The objects of most interest for this document are the directories/proc/bus/usb/ and /proc/scsi/. These will be used to verify that the kernel had been set up properly for our purpose (Section 7.2).

6.2. SCSI devices

Your memory stick will be considered as a USB mass storage device posing as a removable SCSI disk (sd). SCSI disk devices are mapped to the /dev (devices) directory under /dev/sda , /dev/sdb , ... When different disk devices are present, they will be mapped to /dev/sda, /dev/sdb, etc. If, for example, a memory stick and a digital camera are plugged in, the one would be mapped to /dev/sda and the other to /dev/sdb. Tests have indicated that the first device to be detected is mapped to sda and represented in the directory /proc/scsi/usb-storage-0. The second will be mapped to sdb and represented in /proc/scsi/usb-storage-0. What will happen when both devices are present at boot-up, is not known. An overview of the partitions relevant to this document, the file /proc/partitions may be consulted. Look for entries like sda, sda1, sdb. Under the Linux-2.6 series of kernels this problem may be very neatly resolved (Section 12).

In the descriptions further on, it will be assumed that only one memory stick is present, and therefore mapped to /dev/sda.

In most distributions these points exist in abundance. You should check that this is the case in your setup (ls /dev/sda*. If there are none you can create them by doing (as root, indicated by the hash).

             # mknod /dev/sda  b 8 0
             # mknod /dev/sda1 b 8 1
             # mknod /dev/sda2 b 8 2
   

etc, up to /dev/sda15 if necessary. The device as a whole is mapped to /dev/sda, while /dev/sdax (x = 1 ... 15) represent the various partitions of the device. We shall describe, later on, how to create different partitions for different file systems (see Section 8 and Section 9 ). If you intend to use your memory stick with a single partition (filesystem), /dev/sda1 is sufficient.

6.3. The USB device filesystem

This dynamically generated filesystem should be mounted at /proc/bus/usb/. It is therefore essential that this mount point exists. Once it is mounted, there should be more to be seen in /proc/bus/usb/ and in /proc/scsi/ (Section 7.3). To find out which USB devices are present, read the file /proc/bus/usb/devices ( less /proc/bus/usb/devices). It takes a small effort to interpret the writing on the screen, but it is not difficult. A memory stick is indicated by Mass Storage Device.

6.4. The ext2 filesystem

The ext2 (second extended) filesystem still is the most prevalent in Linux setups. It is most versatile and sophisticated, carrying with it permissions (read-write-execute, who is permitted to do what), ownership (user, group, others), a timestamp (when last modified), etc. Moreover, tools for the proper maintenance of a device endowed with ext2, exist (Section 11). If a flash memory device is to be used exclusively within Linux setups, it is therefore desirable to format it in ext2 (see Section 8).

6.5. The vfat filesystem

Enabling vfat in the kernel makes it possible to mount filesystems created under Dos/Windows to be mounted in a Linux system. Most memory sticks are formatted for direct use on Windows and should therefore be considered as vfat-formatted. The vfat filesystem is less sophisticated than ext2 and will therefore be more economic as far as memory space is concerned. It does not carry permissions in the same way as ext2 with the result that an ext2-file saved in vfat will re-appear with its permissions changed. Nonetheless, if a memory stick is to be used for the purpose of carrying data between Linux and Windows machines, it is best to keep it in the vfat format. A compromise is to partition the memory stick in two parts: one in vfat and the other in ext2. At least the Linux machine should be able to handle both. This is described in Section 9

6.6. Explanation of command options

Here is a short list of command options that occur in this document:

• mount -t ext2 /dev/sda1 /mnt/memstick Mount in filetype ext2, device sda1 at mountpoint /mnt/memstick
• ls -l Make a complete list (modes, ownerships, etc.)
• mkdosfs -F 32 /dev/sda1 Create an ms-dos filesystem, FAT-size 32, on the partition /dev/sda1
• ln -s /dev/sda1 /dev/flash link, symbolically, the existing /dev/sda1 to the symbolic /dev/flash
• mkdir -m 777 /mnt/memstick/superdir create a new directory with mode rwx for user, group and others
• dumpe2fs -h /dev/sda1 Show the header of the ext2 partition /dev/sda1

7.1. A quick check

It is possible that your system is already set up for handling a USB-flash memory device. To check this, simply issue the command $ mount from an X-terminal. If something like

           none on /proc/bus/usb type usbfs (rw)
      

appears among the output, you are ready to go to Section 7.4 and proceed from there. It may be virtuous, however, to glance through the intervening sections. If the test is unsuccessful, all is not lost. Please read on.

7.2. Probing the /proc filesystem

A few things in the /proc directory can be checked so see if the kernel options have been properly included or the appropriate modules properly inserted. As a first check, see if the directory /proc/bus/usb exists. If it does, your kernel supports the USB-subsystem correctly. If this is not the case the kernel has to be re-compiled with appropriate USB support (see Section 5.2.3) or the kernel has to be updated. Next, check if the directory /proc/scsi exists. If it does, you're well-away. If it doesn't, SCSI support has not been compiled into your kernel (see Section 5.2.1).

In /proc there should be provision for a mount point for the USB filesystem. That point is /proc/bus/usb. If it exists, the kernel is correctly set up.

7.3. Mounting the USB filesystem

If the verifications of the previous sections were positive, the next step is to mount the USB filesystem. This is issued from root as follows:

             # mount -t usbfs none /proc/bus/usb 
   

In older kernel versions the mount command above may have to be changed by replacing usbfs with usbdevfs. In Linux-2.4.20 both forms work.

If there are no complaints, do some more tests. The first is the quick test (Section 7.1). A deeper probe would be to issue the command # ls -l /proc/bus/usb . This should give something similar to

             dr-xr-xr-x 1 root root 0  Sep 19  14:21  001
             dr-xr-xr-x 1 root root 0  Sep 19  14:21  002
             -r--r--r-- 1 root root 0  Sep 19  22:30  devices 
             -r--r--r-- 1 root root 0  Sep 19  22:30  drivers
   

The path /proc/scsi/usb-storage-0/ should now exist and one of the files at the end will show this. In my setup the command $ less /proc/scsi/usb-storage-0/1 gives

                Host scsi1: usb-storage  
                    Vendor: Generic  
                   Product: Mass Storage Device  
             Serial Number: None  
                  Protocol: Transparent SCSI  
                 Transport: Bulk  
                      GUID: 0ed166800000000000000000  
                  Attached: Yes/No  
   

If the flashdrive is present, the last line will have the entry "Yes". If it is not plugged in, the entry will be "No".

7.4. Tests -- vfat

You are now ready to find out if the memory stick is working. Let us suppose that you bought it over the counter. Its instruction leaflet will most likely tell you how to set it up for Windows. That is an indication that it is formatted in vfat. Before you try to mount it, create a mount point for it. Do something like # mkdir -m 777 /mnt/memstick. The mounting command would most likely be

    # mount -t vfat /dev/sda1 /mnt/memstick
   

If everything went smoothly, you should be able to see the device: # ls /mnt/memstick.

Now try some standard things like making a directory on the device and copying a favourite text file to it:

             # mkdir /mnt/memstick/apollo 
             # cp /home/myname/myfavourite_file /mnt/memstick/apollo/.
   

List again (# ls -l /mnt/memstick ) and pay attention to the permissions.

Unmount the device (# umount /dev/sda1 ) and mount it again as above. List again and check the permissions. Most likely your favourite text file will now have an x-permission. It became executable. That is normal in the vfat filesystem. If you are happy with that, unmount the device and skip to Section 10 .

8.1. Partitioning

It is assumed that the flash memory is mapped to /dev/sda. In this section we treat the situation where the whole flash memory device will become devoted to a single ext2 partition. In the example procedure shown below, a 128MB flash memory was formatted for ext2. That is why figures like 131MB and 888 cylinders appear. The fdisk tool, which is used, simply reads that from the device.

All operations are done by root. The single partition will be created on /dev/sda (please note: not /dev/sda1 ) The procedure is described in a series of steps with comments. The standard prompt of fdisk is Command (m for help): and you can, at any stage enter m to see the available commands. If you do that the result would be

             Command   action 
                a      toggle a bootable flag 
                b      edit bsd disklabel 
                c      toggle the dos compatibility flag 
                d      delete a partition 
                l      list known partition types 
                m      print this menu 
                n      add a new partition 
                o      create a new empty DOS partition table 
                p      print the partition table 
                q      quit without saving changes 
                s      create a new empty Sun disklabel 
                t      change a partition's system id 
                u      change display/entry units 
                v      verify the partition table 
                w      write table to disk and exit 
                x      extra functionality (experts only) 
   

Your memory stick must be plugged in, but not mounted. Take care that write protect is off.

Here goes:

             # fdisk /dev/sda 
             Command (m for help):d {enter }
             Selected partition 1  
             Command (m for help):n {enter} 
             Command action  
                e  extended  
                p  primary partition (1-4)  
             p {enter}  
             Partition number (1-4):1 {enter} 
             First cylinder (1-888, default 1): {press enter} 
             Using default value 1  
             Last cylinder ... (1-888, default 888): {press enter} 
             Using default value 888  
   

Now is a good time to verify that all went well by printing out the partition details and table.

             Command (m for help): p 
             Disk /dev/sda: 131 MB, 131072000 bytes  
             9 heads, 32 sectors/track, 888 cylinders  
             Units = cylinders of 288 * 512 = 147456 bytes 

             Device     Boot Start End Blocks Id System 
	  
             /dev/sda1         1   888 127856 83 Linux 
   

Make sure that the partition is not set up as bootable. There should not be an asterisk under the heading "Boot" in the partition table. Furthermore, the headings "Id" and "System" should be as they are in the printout shown above. That indicates that you can format the device in ext2 (next section). These are the defaults. If they are not, they can be changed by

             Command (m for help): a [toggle a bootable flag] 
             Command (m for help): t [change a partition's system id] 
   

If (or when) the partition table is correct, you can conclude the procedure with

             Command (m for help): w [write table to disk and exit] 
    

That's it!

8.2. Making an ext2 device

Having completed the partition part, we go straight on to "formatting" the device in ext2. For this we use the tool mke2fs. This is straightforward.

             # mke2fs /dev/sda1 
   

You might like to watch the led on your memory stick flicker while this is happening. When it stops, the job is done.

8.3. Tests -- ext2

To see if you were successful, essentially repeat the procedures described in Section 7.3 and Section 7.4 with two exceptions of detail. In the first place the mount command should be

             # mount -t ext2 /dev/sda1 /mnt/memstick 
   

The second exception is that the permissions on your favourite text file should not change anymore.

9.1. Partitioning

The procedures described here will partition the flash memory device in two pieces -- one of the partitions will be formatted in ext2 and the other in vfat. This may be handy when the device is to be used to transport data between two (or more) Linux machines (ext2) and between Linux and Windows machines (vfat)

The device mapped to /dev/sda will be re-formatted. Any data present on it will be destroyed. To make sure that the device you work with is the one intended, remove similar devices from the USB-bus before starting these procedures.

In this section we describe how to set up a dual partition for the memory stick: One half of the device will be devoted to the vfat filesystem, and the other will be ext2. In this way your device will be more versatile, but also a little shattered. The procedure will be given in somewhat less detailed than in Section 8.1 . Note that the procedure of the previous section is repeated and that the hex code for each file type is explicitly entered (lines following the entry "t": Win95 = b; Linux = 83). When the partition will only be for Linux, this is unnecessary -- Linux is the default. Also note that the vfat partition goes on first -- that is the way Windows would have it.

             # fdisk /dev/sda
	    
             Command (m for help): d
             Selected partition 1

             Command (m for help): n
             Command action
                e   extended
                p   primary partition (1-4)
             p
             Partition number (1-4): 1
             First cylinder (1-888, default 1): 
             Using default value 1
             Last cylinder or +size or +sizeM or +sizeK (1-888,
                   default 888):444
             Command (m for help): t
             Selected partition 1
             Hex code (type L to list codes): b
             
              Changed system type of partition 1 to b (Win95 FAT32)
             
             Command (m for help): n
             Command action
                e   extended
                p   primary partition (1-4)
             p
             Partition number (1-4): 2
             First cylinder (445-888, default 445): 
             Using default value 445
             
              Last cylinder or +size or +sizeM or +sizeK (445-888,
              default 888): 
             
             Using default value 888

             Command (m for help): t
             Partition number (1-4): 2
             Hex code (type L to list codes): 83

             Command (m for help): p

             
				 Disk /dev/sda: 131 MB, 131072000 bytes
             9 heads, 32 sectors/track, 888 cylinders
             Units = cylinders of 288 * 512 = 147456 bytes

                Device Boot    Start       End    Blocks   Id  System
             /dev/sda1             1       444     63920    b  Win95 FAT32
             /dev/sda2           445       888     63936   83  Linux
				 

             Command (m for help):w

   

Note that we went on with the second partition before writing.

The choice of splitting the device right in the middle (1--444; 445--888 for a 128 MB memory stick) is arbitrary. Any other well-conceived splitting would be fine.

If you had more than one partition to begin with, the "delete" part of the procedure would have to be repeated.

9.2. Making a dual ext2/vfat system

This step is a repetition of Section 8.2. The first step makes the dos partition, the second makes the Linux ext2 partition.

             # mkdosfs -F 32 /dev/sda1
             # mke2fs /dev/sda2
   

9.3. Tests -- ext2 and vfat

Simply carry out the tests of Section 7.4 and Section 8.3 on the two devices separately. The mount commands would be

             # mount -t vfat /dev/sda1 /mnt/fatstick 
             # mount -t ext2 /dev/sda2 /mnt/memstick
   

to mount both partitions on your device.

9.4. Configuring Linux for the dual partition

The Linux machine you did the partitioning on, will be configured for the dual partition. Evidently you need to carry the keychain to another Linux machine. There you will find that it only sees /dev/sda1. This is, perhaps, a shortcoming of the kernel. The workaround is as follows: On the new host machine do (keychain plugged in but not mounted)

             # fdisk /dev/sda
                   Choose p; check partition table
                   If satisfied, choose w
             # less /proc/partitions
  

Your device with its partitions should be there. Data on your keychain should be untouched. For a single partition this is not necessary.

11.1. Filesystem inspection

It is a virtuous habit to inspect your ext2 filesystem on the flash memory regularly. To do this, the tool dumpe2fs may be used in the following way: # dumpe2fs -h /dev/sda1 (you must be root and the device should not be mounted). The result should be similar to the following:

             Filesystem volume name:   <none>
             Last mounted on:          <not available>
             Filesystem UUID:          c42a6963-5e6a-4cd2-b7d7-c8f09dca6c52
             Filesystem magic number:  0xEF53
             Filesystem revision #:    1 (dynamic)
             Filesystem features:      dir_index filetype sparse_super
             Default mount options:    (none)
             Filesystem state:         clean
             Errors behavior:          Continue
             Filesystem OS type:       Linux
             Inode count:              32000
             Block count:              127856
             Reserved block count:     6392
             Free blocks:              116456
             Free inodes:              31922
             First block:              1
             Block size:               1024
             Fragment size:            1024
             Blocks per group:         8192
             Fragments per group:      8192
             Inodes per group:         2000
             Inode blocks per group:   250
             Filesystem created:       Sat Sep 20 12:43:00 2003
             Last mount time:          Tue Oct 28 14:13:03 2003
             Last write time:          Tue Oct 28 14:28:27 2003
             Mount count:              13
             Maximum mount count:      35
             Last checked:             Sat Oct 18 11:28:26 2003
             Check interval:           15552000 (6 months)
             Next check after:         Thu Apr 15 11:28:26 2004
             Reserved blocks uid:      0 (user root)
             Reserved blocks gid:      0 (group root)
             First inode:              11
             Inode size:               128
             Default directory hash:   tea
             Directory Hash Seed:      118bee0a-efa5-4771-967e-41a0badd0355
   

A few important aspects need to be pointed out.

• When the ext2 filesystem is created, it is by default given maximal usage before it has to be checked. These can be seen Maximum mount count (35) and Check interval (expiry date).

• The usage so far: Mount count and Last checked .

• The existence of corrupted files (bad blocks): Filesystem state .

You might get warnings about these things when you mount the device or when you try the read files from the device.

11.2. And then?

When the usage allocation has been spent, or there is evidence of file corruption, the thing to do is to run # fsck.ext2 /dev/sda1 with the device unmounted. After that, usage parameters will be freshly allocated and bad blocks will be gone.

When dealing with the vfat system, the dump does not seem to exist. The command # dumpe2fs -f /dev/sda1 for filesystems other than ext2 does not work for vfat. The tool dosfsck exists (it is still Alpha), and may be risky to use on a device you have not formatted yourself.

12.1. What is new?

Setting up your system as described above for the Linux-2.4.x kernels will be perfectly good, but with the Linux-2.6.x kernels (x = 0, 1, 2, 3 at the time of this writing) much more is possible:

• USB Hotplugging of devices has been improved immensely. This means that it has become possible to attach and remove a device from the USB port with full cognisance of the system. If a device is attached, it will show in the /proc filesystem. Once it is removed, it will not show anymore.

• The sysfs system has been introduced. By using this facility, a very tight grip on attached devices, their attributes and nodes of attachment is possible.

• Utilities for handling hotplugging have been improved or became available. With the aid of these it is possible to easily distinguish between attached devices irrespective of the order in which they were attached.

12.2. Kernel options

Kernel configuration options are very much the same as described in Section 5.2. The configuration menu (# make menuconfig) has been organized differently and more systematically. Here are the relevant choices:

12.3. Notes

It may be in your interest to consult Migrating to Linux Kernel 2.6, but it is not necessary to follow steps that cannot be followed.

12.4. Hotplugging and Naming

In Section 6.2 the situation of two USB devices plugged in at the same time was discussed. The difficulty is that it becomes hard for the user to distinguish the two devices. Progress recently made in the Linux Hotplug Project (search for hotplug) and implemented in the 2.6-series of kernels, has made it possible to address this problem efficiently. For implementation some utilities are needed:

• sysfs -- compiled by default into Linux-2.6.x
• sysfsutils-0.4.0 or later
• hotplug-2004_01_05 or later
• udev-016 or later

              # cd /
              # mkdir sys
              # mount -t sysfs none /sys
            

              none  /sys   sysfs  defaults  0  0
            

              ## Flash Memory 1
              BUS="scsi", SYSFS_vendor="UFD*", NAME="namib%n"

              ## Flash Memory 2
              BUS="scsi", SYSFS_vendor="STORAGE*", NAME="kalahari%n"
           

             /udev/namib1     /mnt/namib     vfat,ext2  user,noauto,rw  0 0 
             /udev/kalahari1  /mnt/kalahari  vfat,ext2  user,noauto,rw  0 0         
            

13.1. Copyright Information

This document is copyrighted (c) 2003 Niko Sauer and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <[email protected] >

13.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

13.3. Credits

I wish to express my deep gratitude towards my sons Paul and Philip who in 1995 persuaded me to move to Linux. Philip gave me profound advice and insights which made this project a pleasant learning experience.

Emma Jane Hogbin, in her review, raised a number of important questions which served to improve this document immensely. My sincerest thanks to her.

13.4. Feedback

Users of this document are most welcome to email me with suggestions on the improvement of the presentation and possible additions to the scope of it.