Device Drivers: A Sample Device Driver

The best way to learn something is to do it, so let's explore the design of a simple device driver. This section will lead you through the process of creating a device driver for a fictional PCI card called a BeGadget. Although this sample examines the creation of a PCI device driver, the basic concepts are similar for all types of devices.

The BeGadget card supports four logical devices, each of which can be individually opened and accessed. The card has a single base register, which serves as a pointer to the memory-mapped control registers for the four logical devices. Each of these four logical devices is capable of both reading from and writing to system memory via DMA.

The card has two special-purpose registers, both of which are 32 bits wide. The first is a special self-test register, which can be examined to determine whether or not the card is functioning properly. It is located at offset 0x80 in PCI configuration space on the card. The value in the register is zero if the card is functioning properly and non-zero otherwise. The other special-purpose register is a reset register, located at 0x84. Writing to this register resets the BeGadget card to its default status. These registers will be referred to as GADGET_CONFIG_STATUS and GADGET_CONFIG_RESET, respectively, and are defined with these names in the source code for the driver.

Reads from and writes to these devices are initiated using a 32-bit command register. There is one command register for each of the four logical devices on the card. To perform I/O, the BeGadget card requires that we write to this register three times. The first write specifies the command to issue (read or write). The second write to the register specifies the address of the buffer in memory. The third write specifies the number of bytes to read or write.

In addition, each of the virtual devices has a speed register, which controls the rate at which data transfers between the BeGadget card and the computer.

This sample driver will allow us to investigate the use of interrupts for managing device I/O operations, coping with memory-mapped I/O in virtual memory space, and using semaphores to synchronize access to the hardware.

The complete source for the BeGadget sample driver is available from the Be Web site. <<Location forthcoming>> It's recommended that you download the code, which will make this discussion easier to follow; additionally, the downloadable source code is more heavily commented than the snippets included here.

Because the BeGadget device is fictional, there is no actual hardware on which the BeGadget driver can be tested. The DEBUG_NO_HW #define is provided to allow us to build and run the driver without a BeGadget card; if DEBUG_NO_HW is 1, extra code is compiled which allows the driver to simulate functioning without a card available.

Let's examine the implementations of each individual component of the BeGadget driver.


Entry Points

These entry points are called by the kernel to manage your driver as a whole. Functions by these names must exist in your driver, since the kernel calls them explicitly. These entry points are:

Entry Point Description
init_hardware() Called once by the kernel during system startup to allow the driver to reset the hardware to a known default state. If your device doesn't need initialization, you don't have to provide this function.
init_driver() Called whenever the driver is loaded into memory. This allows the driver to set up any memory space needed, as well as to initialize any global variables it uses. If your driver doesn't need to be initialized, you don't have to provide this function.
uninit_driver() Called whenever the driver is unloaded from memory. This can be used to release any system resources allocated by init_driver(). If your driver doesn't need to be uninitialized, you don't have to provide this function.
publish_devices() Called by the kernel to obtain a list of device names supported by the driver.
find_device() Called by the kernel when it needs to obtain a list of the hook functions for a particular device.


init_hardware()

The init_hardware() function is called by the kernel when the BeOS first starts up, and is only called once. This allows the driver to put the device into a known initial state. The init_hardware() function should verify that the device is attached to the computer, then initialize it to a stable state.

This function should return B_OK if the hardware was successfully initialized or B_ERROR if an error occurred or the device is not present; if B_OK is not returned, the kernel will not use the driver.

In the case of the BeGadget card, there are three steps to initializing the device:

Before we look at the init_hardware() function, let's look at the lookup_pci_device() function, which is used by the BeGadget driver to search the PCI bus for specific devices.

   static bool lookup_pci_device (short vendor, short device, pci_info *returned_info)
   {
      int   i;
      
      for (i = 0; ; i++) {
         if (get_nth_pci_info (i, returned_info) != B_OK)
            return false;      /* Error or end of device list */
         
         /* Compare the vendor ID and device ID of the scanned
            device with the desired information.  If it is a
            match, break out of the loop so we can return it. */
         
         if (returned_info->vendor_id == vendor &&
             returned_info->device_id == device)
                break;
      }
      return true;            /* Device was found */
   }

This code iterates through all available PCI devices by calling the get_nth_pci_info() function provided by the kernel, each time passing in the index number into the device list and a pointer to the pci_info structure called returned_info. If an error is reported by get_nth_pci_info(), lookup_pci_device() returns false, thereby indicating that no matching device was found. Each device's vendor and device ID numbers are compared with those specified by the input parameters vendor_id and device_id. When a match is found, true is returned, with the pci_info structure filled out with the information about the BeGadget card.

Now let's have a look at the init_hardware() handler.

   status_t init_hardware (void)
   {
      pci_info   info;
      int32      val;
   
      /* Get the device information */   
   
      if (!lookup_pci_device (GADGET_DEV_VENDOR_ID, 
                              GADGET_DEV_DEVICE_ID, &info))
         return ENODEV;
   
   #if DEBUG_NO_HW
      return true;      /* for debugging purposes */
   #endif
      
      /* Check the self-test register */
   
      val = read_pci_config (info.bus, info.device, info.function, 
         GADGET_CONFIG_STATUS,   /* offset in config space */
         4                  /* size of register */
      );
      
      if (val)                /* Check the self-test result */
         return ENODEV;         /* Return no device found if failed */
      
      /* Reset the BeGadget card */
   
      write_pci_config (info.bus, info.device, info.function, 
         GADGET_CONFIG_RESET,   /* offset in config space */
         4,                  /* size of register */
         0                  /* initialization value */
      );
      
      return B_OK;         /* We've successfully initialized the device */
   }

This function starts by calling the lookup_pci_device() function to obtain the PCI device information record about the BeGadget card. As a helpful side effect, we know that if it reports an error, the device was not found on the PCI bus, so we can immediately return the ENODEV return code, which tells the kernel there is no BeGadget installed in the computer.

Once it has been determined that the BeGadget device is installed, read_pci_config() is used to read the value of the BeGadget's self-test register. If the self-test register reports a non-zero result, which indicates that the card is malfunctioning, we again report an ENODEV result code.

Finally, the device is reset by using the write_pci_config() function to write to the 32-bit reset register on the BeGadget card. Then the result code B_OK is returned to tell the kernel that the device has been initialized.


init_driver()

The init_driver() function is called by the kernel whenever the driver is loaded from disk into memory. This function is used to initialize global variables used by the driver and to allocate any system resources required by the driver as a whole, such as semaphores or memory. This function should return B_OK if the driver is safely initialized, or B_ERROR if the driver should not be used for whatever reason.

If individual logical devices have system resource needs of their own, it is usually better to allocate those resources when the device is opened, to preserve system resources when they aren't needed.

For devices that utilize memory-mapped I/O, the driver must map the physical memory locations into the virtual memory system. This should be done when initializing the driver, and the memory should be unmapped when uninit_driver() is called.

The BeGadget init_driver() function will perform the following three tasks:

Let's have a look at the code.

   status_t
   init_driver (void)
   {
      pci_info         info;
      int32         base;
      int32         size;
      
      /* Get the info record for the device */
   
      if (!lookup_pci_device (GADGET_DEV_VENDOR_ID, 
                              GADGET_DEV_DEVICE_ID, &info))
         return ENODEV;
      
      /* Get the base address and size of the registers */   
   
      base = info.u.h0.base_registers[0];      /* get address of the reg */
      size = info.u.h0.base_register_sizes[0];   /* get reg size */
      
      /* Round the base address of the register down to
         the nearest page boundary. */
      base = base & ~(B_PAGE_SIZE - 1);
      
      /* Adjust the size of our register space based on
         the rounding we just did. */
      size += info.u.h0.base_registers[0] - base;
      
      /* And round up to the nearest page size boundary,
         so that we occupy only complete pages, and not
         any partial pages. */
      size = (size + (B_PAGE_SIZE - 1)) & ~(B_PAGE_SIZE - 1);
      
      /* Now we ask the kernel to create a memory area
         which locks this I/O space in memory. */
      
       reg_area = map_physical_memory (
          "gadget_pci_device_regs",
          (void *) base,
          size,
          B_ANY_KERNEL_ADDRESS,
          B_READ_AREA + B_WRITE_AREA,
          &reg_base
       );
       
       /* Negative results from map_physical_memory() are
          errors.  If we got an error, return that. */
       
       if (reg_area < 0)
           return reg_area;
   
      /* Note that there are no open devices on the card and
         return B_OK */
   
      is_open = 0;      /* no devices open yet */
      return B_OK;
   }

The lookup_pci_device() function is used to obtain the pci_info record for the BeGadget device. Again, it has the beneficial side effect of verifying the presence of the BeGadget card. We then look into the pci_info record for the device to obtain the physical address assigned to the card's base registers in memory as well as the size of the physical memory range into which the registers are mapped.

Then we round the physical address down to the beginning of the page on which the registers lie, and round the size up to the nearest full page, so that we know exactly which pages of memory need to be mapped.

Once we know which pages need to be mapped, we use the map_physical_memory() call to map the I/O registers into the virtual address space. This function creates a new area named "gadget_pci_device_regs." The physical address is specified by base, and the address as mapped into virtual memory is returned in reg_base. We specify through flags that the memory is readable and writable (B_READ_AREA + B_WRITE_AREA) and that it can be located anywhere within the kernel's address space (B_ANY_KERNEL_ADDRESS).

If map_physical_memory() returns a negative number, it is an error code, and we return that error code. Otherwise we zero the is_open variable, which is a bitfield representing which logical devices we control are open; at this point, none of them are open. Finally, we return the B_OK result code, since we have successfully initialized the driver.


uninit_driver()

When your driver is about to be unloaded from memory by the kernel, your uninit_driver() function is called. This function should dispose of any system resources allocated by your init_driver() function, including memory areas used to manage your I/O memory.

The BeGadget uninit_driver() function doesn't need to do much; the I/O memory which was mapped by init_driver() must be released.

   void
   uninit_driver (void)
   {
      delete_area (reg_area);      /* Dispose of I/O space memory area */
   }

The BeGadget driver simply deletes the memory area created in init_driver() for the purpose of mapping the I/O memory into virtual memory. This is the only system resource allocated by our init_driver() function, so that's all we have to release here.


publish_devices()

The publish_devices() function is called by the kernel when the kernel needs to know the names of the devices your driver supports. This function's responsibilities are very simple:

The publish_devices() function in the BeGadget driver simply returns a pointer to a static list of four device names. The code looks like this:

   const char   **
   publish_devices(void)
   {
      return name_list;         /* Return a pointer to the device name
                              list */
   }

The name_list, which contains the names of the devices, is:

   static char      *name_list[] = {
      "begadget/gadget1",
      "begadget/gadget2",
      "begadget/gadget3",
      "begadget/gadget4",
      0
   };

These names represent a portion of a pathname located in the /dev directory. Each of these names is appended to /dev, so the devices are accessed by client applications via their full names:

When a client application wants to open the second virtual device on the BeGadget card for writing, it would call the Posix open() function like this:

   int fd = open("/dev/begadget/gadget2", O_WRONLY);


find_device()

When a request is made for access to a particular device, the kernel calls the driver's find_device() function to obtain pointers to the hook functions which handle specific I/O operations. The kernel passes into this function the name of the device it is trying to locate. The find_device() function returns a pointer to a structure that lists the driver's hook functions.

These hook functions are called by the kernel to handle specific operations, such as opening, closing, reading, and writing the device. If the device is not handled by the driver, the find_device() function should return NULL.

Before we look at the BeGadget driver's find_device() function, let's take a quick look at an internal utility function implemented by BeGadget for determining the virtual device number of a device based on its name.

   static int
   lookup_device_name (const char *name)
   {
      int i;
      
      for (i = 0; name_list[i]; i++)
         if (!strcmp (name_list[i], name))
            return i;
      return -1;               /* Invalid device name */
   }

The lookup_device_name() function accepts as input a virtual device name (such as begadget/gadget3) and returns the virtual device number of that device, or -1 if the device isn't handled by the BeGadget driver.

All this code does is scan the static list of device names supported by the BeGadget driver and return the index number into the array of the matching device name. The result is therefore an integer between zero and three. If no matching name is found, -1 is returned.

This function is used in several places throughout the BeGadget driver, including the find_device() hook function, to which we now turn our attention.

   device_hooks *
   find_device (const char *name)
   {
      if (lookup_device_name (name) >= 0)
         return &my_device_hooks;   /* Return hooks list */
               
      return NULL;               /* Device not found */
   }

Because of the useful lookup_device_name() function we've already implemented, all find_device() needs to do is call lookup_device_name() to determine whether or not the device exists. If it returns a non-negative number, we return a pointer to my_device_hooks, which is a structure that lists the hook functions required by the kernel. Otherwise we return NULL, indicating to the kernel that the BeGadget driver does not support the device requested.

Since all four devices handled by the BeGadget driver can be handled by the same hook functions, we always return a pointer to the same list of hooks.

The device hooks are specified as follows.

   static device_hooks my_device_hooks = {
      &pci_open,
      &pci_close,
      &pci_free,
      &pci_control,
      &pci_read,
      &pci_write
   };

These functions provide support for the C open(), close(), read(), and write() functions, all of which are Posix-compliant, plus the ioctl() function. The free hook function (called pci_free() in the BeGadget driver) is a special hook which operates in conjunction with the close hook. This will be discussed in greater detail when we look into the pci_close() and pci_free() functions later in this section.


Hook Functions

When the BeOS kernel needs to perform a specific operation on a device, it does so by calling a hook function provided by the driver for that device. The hook functions your driver must provide are:

Hook Description
open Handles the Posix open() function. This function should prepare a device for reading or writing.
close Handles the Posix close() function. This function is called by a client program when it has finished using the device.
free The free hook releases a device after all I/O transactions have been completed. It does not directly correspond to any Posix or other C function.
read Handles the Posix read() function. This function reads data from the device into system memory.
write Handles the Posix write() function. This function writes data from system memory to the device.
control Handles the ioctl() function. This function provides the mechanism by which the system, and client programs, can control how the device functions.

The difference between the close and free hooks is a critical one. In a multithreaded environment such as the BeOS, it is entirely possible for a client program to close a driver before all I/O transactions have completed. Because of this, it is important that the close hook not perform any actions that might interrupt the flow of data.

This is why there is also a free hook function. The free hook is called by the kernel after all I/O operations on a closed device have been concluded. The free handler should perform any necessary deallocation of resources or other activities related to closing the device after a transaction.

To better understand how these hook functions operate, let's take a look at the BeGadget driver's implementations of each them.


pci_open()

This function handles the open hook for the BeGadget driver, and is called by the kernel to handle the Posix open() call. The kernel passes the pci_open() function the name of the virtual device to open, such as begadget/gadget2, as well as the open flags. Also provided is a pointer to a 32-bit space in which we can store a pointer to a "cookie."

A cookie is a device-specific piece of information which can be used to track device status information. The open hook function creates the cookie for the device when it is opened; from then on, all other hook functions receive a pointer to this cookie for identification and data storage purposes.

The cookies used by the BeGadget driver have the PCI information record stored in them, as well as a copy of the device's ID number and a pointer to the registers for the device.

Let's have a look at the open hook function for the BeGadget driver, then examine how it works.

   static status_t   
   pci_open(const char *name, uint32 flags, void **cookie)
   {
      dev_info   *d;
      int         id;
      int32      mask;
      status_t   err;
      char      sem_name [B_OS_NAME_LENGTH];
      
      /* Get the device ID by looking at the name. */
      id = lookup_device_name (name);
      if (id < 0)
         return EINVAL;         /* Invalid device name */
         
      /* Check to be sure if the device is in use; if not,
         set the flag that says it is */
   
      mask = 1 << id;            /* Construct a mask for the bit */
      if (atomic_or (&is_open, mask) & mask)
         return B_BUSY;         /* The device is already in use */
      
      /* Allocate a cookie for the device */   
   
      err = B_NO_MEMORY;
      d = (dev_info *) malloc (sizeof (dev_info));
      if (!d)
         goto err0;         /* Unable to allocate the cookie */
      
      /* Locate the device and fill out the rest of the cookie */
   
      err = ENODEV;            /* If we fail, we'll report this error */
      if (!lookup_pci_device (GADGET_DEV_VENDOR_ID,
                              GADGET_DEV_DEVICE_ID, &d->pci))
         goto err1;            /* We couldn't find the device */
      
      /* Set up other fields in the cookie */
      d->id = id;
      d->regs = (dev_regs *) reg_base + id;
      
      /* Allocate the hardware locking semaphore and make the
         system team its owner */   
   
      sprintf (sem_name, "gadget dev %d hw_lock", id);   /* create the semaphore's name */
      d->hw_lock = err = create_sem (1, sem_name);      /* create the semaphore */
      if (err < 0)            /* If an error occurred, bail out */
         goto err1;
      set_sem_owner(err, B_SYSTEM_TEAM);   
      
      /* Allocate the semaphore for waiting until I/O is done
         and transfer it to the system team */
   
      sprintf (sem_name, "gadget dev %d io_done", id);
      d->io_done = err = create_sem (0, sem_name);
      if (err < 0)            /* Bail out if an error occurred */
         goto err2;
      set_sem_owner(err, B_SYSTEM_TEAM);
      
      *cookie = d;            /* Let the kernel know where the
                              cookie is */
      
      /* Install and enable the interrupt handler */
   
      set_io_interrupt_handler (
         d->pci.u.h0.interrupt_line,      /* interrupt number */
         gadget_dev_inth,            /* Pointer to interrupt handler */
         d                        /* pass this to handler */
      );
      enable_io_interrupt (d->pci.u.h0.interrupt_line);
      
      return B_OK;                  /* We're open for business */
   
   err2:                           /* Error handler when HW lock
                                    semaphore is installed */
      delete_sem (d->hw_lock);
   err1:
      free (d);                     /* Error handler when no
                                    semaphores are installed yet */
   err0:
      atomic_and(&is_open, ~mask);
      return err;
   }

The very first thing the pci_open() function does is determine the ID number of the virtual device which is to be opened. As described earlier, the BeGadget card has four virtual devices. These four devices have ID numbers of zero through three. We make this determination by calling the lookup_device_name() function we defined earlier. If it returns -1, indicating that the device does not exist, we return the EINVAL return code to let the kernel know that an error occurred.

BeGadget devices can only be accessed by one client at a time, so we maintain flags to keep track of which of the four devices are already open. These flags are stored in a variable called is_open. This variable's lowest four bits are used for this purpose; for example, if the value of bit 0 is set to 1, the first BeGadget device (/dev/begadget/gadget1) is currently open. The bits of the is_open variable are diagrammed below.

Before we can proceed to open the requested virtual device, we need to ensure that the device is not already open by examining the state of the is_open variable. If the device is not already open, we need to set the appropriate flag to indicate that it is now open, so that future open() calls on this virtual device will fail.

To do this safely, without risking a dangerous race condition, we make use of the atomic_or() function exported by the kernel. This allows us to examine and alter the value of the is_open variable in a single atomic operation. We construct a mask in which all bits are zero except the one that represents the device to be checked, then use atomic_or() to set that bit to 1. The atomic_or() function returns the previous value of the variable. If the previous value of the variable already specified that the device is open, we return the B_BUSY result code. Otherwise, we have safely set the flag indicating that the device is open.

Now we need to create the "cookie" to represent this virtual device. We simply call malloc() to do this. If the malloc() call fails, we report the B_NO_MEMORY result code and return immediately. The cookie's memory is allocated in the kernel's heap, which is always locked, so we know it will never be paged out by the virtual memory system.

Once we have allocated the memory to contain the cookie, we need to fill out the various fields therein. We begin by locating the BeGadget device, using the lookup_pci_device() function we have already written. If lookup_pci_device() returns false, we immediately return the ENODEV result code, since the device was not found on the PCI bus.

The PCI info record returned by lookup_pci_device() is stored in the cookie. The device ID and a pointer to the command register for this device are also stored in the cookie. Recall that each of the four virtual devices controlled by the BeGadget card has a single 32-bit command register. This pointer, stored within the cookie, provides easy access to this register when we need it.

This device requires two semaphores. The first is used to enforce exclusive access to the device. We call create_sem() to establish this semaphore, called hw_lock, which is a component of the device's cookie. We then call set_sem_owner() to transfer ownership of the semaphore to the system team. This is critical, since otherwise the semaphore is created in the team of the application that called open(). This team goes away when the application is shut down. If the device is still open at that time--possibly while still in use by another team--its semaphores would go away as well. By transferring the semaphores to the system team, we avoid this problem.

Failure to transfer ownership of the locking semaphores to the system team can cause unpredictable behavior and system crashes.

We set the initial value of the hw_lock semaphore to 1 so that it will not block the first time an attempt is made to acquire it.

A semaphore will also be used to block I/O calls until transactions with the device are completed, so we call create_sem() to create the io_done semaphore, which is also stored in the cookie for this device. We set the initial value of the semaphore to 0 so it will block the first time an aquire_sem() call is issued. The interrupt handler which monitors I/O with the BeGadget card will release the semaphore when the I/O transaction is finished. Again, we call set_sem_owner() to transfer ownership of the semaphore to the system team.

Now that the cookie has been created and filled out, a pointer to it is stored in the location passed to us by the kernel, so that the kernel can pass a reference to this cookie when requesting that operations be performed on this open device.

Finally, the interrupt handler for the BeGadget device is installed by calling set_io_interrupt_handler(). The interrupt number is obtained from the PCI info structure for the device. Also passed into the set_io_interrupt_handler() function are a pointer to the interrupt handler function, called gadget_dev_inth(), which is implemented by the BeGadget driver, as well as a pointer to the cookie for this device, which allows the interrupt handler to get access to this information.

Once the interrupt handler has been installed, the enable_io_interrupt() function is called to activate the interrupt for the device.

At this point, the device has been opened successfully, so the B_OK return code is returned to the kernel.

After this are three error-handling exit points,, which are branched to using goto commands upon errors earlier in the code; these points handle removing the hw_lock semaphore, freeing up the cookie memory, and turning off the device open flag for the device, depending on where in the code the error occurred.


pci_close()

The pci_close() hook function handles the Posix close() function for BeGadget devices, which is called when the client program has finished issuing read, write, and control calls to the device.

In a multithreaded world such as the BeOS, it is entirely possible for the close hook to be called even though there are pending I/O requests for the device. The close hook should therefore not deallocate anything; instead, use the free hook to do this, which is called after all I/O transactions for a closed device have been completed.

   static status_t
   pci_close(void *cookie)
   {
      return B_OK;
   }

The BeGadget driver doesn't need to do anything when the Posix close() function is called. Instead, we wait until the free hook is called after all I/O on the device is complete.


pci_free()

The free hook does not correspond to any Posix call. This hook is called when all I/O transactions on a closed device are completed. Since the BeOS is a multithreaded operating system, it is possible for a situation to arise in which read or write operations are queued up for a device even after close() is called.

The free hook allows us to finish closing down the device after those transactions are finished. This includes releasing system resources used by the device, including interrupts, semaphores, and memory; among other things, the device's cookie should be released by this function.

The driver is not necessarily being unloaded after this hook is called. The free hook's sole responsibility is to clean up when a particular device handled by the driver is closed. The uninit_driver() entry point is responsible for cleanup prior to the entire driver being unloaded from memory.

The BeGadget driver's free hook must perform the following actions, in order to undo those actions performed by the open hook:

   static status_t   
   pci_free(void *cookie)
   {
      dev_info *d = (dev_info *) cookie;
      
      /* Disable and remove the interrupt handler for the device */
   
      disable_io_interrupt (d->pci.u.h0.interrupt_line);
      set_io_interrupt_handler (
         d->pci.u.h0.interrupt_line,   
         NULL,
         NULL
      );
      
      /* Mark the device as not open */   
   
      atomic_and (&is_open, ~(1 << d->id));
   
      /* Dispose of the hw_lock and io_done semaphores */   
   
      delete_sem (d->hw_lock);
      delete_sem (d->io_done);
      free (d);
      
      return B_OK;
   }

In the BeGadget driver, the pci_free() function is nearly the exact opposite of the pci_open() function. It begins by disabling the interrupt for the device by passing the interrupt number stored in the cookie to the disable_io_interrupt() kernel function. Once the interrupt is disabled, set_io_interrupt_handler() is called to remove the interrupt handler entirely by setting the interrupt handler for that interrupt number to NULL.

Having removed the interrupt handler for the device, atomic_and() is used to clear the appropriate bit in the is_open variable to zero. This lets us keep track of the fact that this device is no longer open.

Finally, the hw_lock and io_done semaphores are deleted and the cookie is disposed of. The B_OK return code is returned to the kernel.


pci_control()

When the C ioctl() function is called for an open BeGadget device, the kernel passes the call along to the BeGadget driver's control hook, which is handled by the pci_control() function.

The BeGadget device supports three control calls:

GADGET_DEV_RESET Resets the BeGadget device to its default state.
GADGET_DEV_SET_SPEED Sets the speed of the BeGadget device.
GADGET_DEV_GET_SPEED Returns the current speed of the BeGadget device.

Recall that the BeGadget card provides a reset register which, when written to, resets the entire card to its default state.

The GADGET_DEV_SET_SPEED and GADGET_DEV_GET_SPEED commands are handled by accessing the speed register for the device whose cookie is passed in to the pci_control() function.

   static status_t   
   pci_control(void *cookie, uint32 op, void *data, size_t len)
   {
      dev_info *d = (dev_info *) cookie;
      
      switch (op) {
      
      case GADGET_DEV_RESET:         /* reset the device */
         write_pci_config (d->pci.bus, d->pci.device, 
            d->pci.function, 
            GADGET_CONFIG_RESET,   /* offset in config space */
            4,               /* size of register */
            0               /* initialization value */
         );
         break;
         
      case GADGET_DEV_SET_SPEED:      /* set device speed */
         d->regs->speed = *(int32 *)data;
         break;
         
      case GADGET_DEV_GET_SPEED:      /* return current speed */
         *(int32 *)data = d->regs->speed;
         break;
      
      default:
         return EINVAL;
      }
      
      return B_OK;
   }

If the GADGET_DEV_RESET operation is requested, we use the write_pci_config() kernel function to store a zero into the reset register on the BeGadget card. Since any write to the reset register will perform the desired reset, this is all it takes to do the job.

If the GADGET_DEV_SET_SPEED operation is requested, we store the 32-bit integer pointed to by the data parameter in the speed register.

If the GADGET_DEV_GET_SPEED operation is requested, the value of the speed register is retrieved and stored in the 32-bit memory location pointed to by the data parameter.


pci_read()

The Posix read() function is handled by the BeGadget driver's pci_read() hook function, which looks like this:

   static status_t
   pci_read(void *cookie, off_t pos, void *data, size_t *len)
   {
      return do_device_io ((dev_info *)cookie, GADGET_DEV_READ, 
         data, len);
   }

For convenience, the BeGadget driver uses a single function for both reading and writing. This function, called do_device_io(), is described below, after we look briefly at the pci_write() function.

To perform a read, we simply call do_device_io() and forward the pointer to the cookie for the device to read from, the opcode GADGET_DEV_READ, which specifies to the do_device_io() function that it should perform a read, and a pointer to the buffer into which to read as well as a count for the number of bytes to read.

We ignore the pos parameter, which specifies the position within the input data to begin reading, since the BeGadget device doesn't support random-access operations.


pci_write()

The Posix write() function is handled by the BeGadget driver's pci_write() hook, the code for which follows.

   static status_t   
   pci_write(void *cookie, off_t pos, const void *data, size_t *len)
   {
      return do_device_io ((dev_info *)cookie, GADGET_DEV_WRITE,
         (void *)data, len);
   }

As discussed in the section on pci_read() above, the pci_write() function calls do_device_io() to perform the actual work of writing to the BeGadget device. We pass the opcode GADGET_DEV_WRITE to request the write activity.

The pos parameter is ignored, since random-access writes are not permitted by the BeGadget hardware.

Let's have a look at the do_device_io() function. This is where both read and write operations get their start. It accepts, as input, a pointer to the cookie for the device being accessed, an operation opcode (either GADGET_DEV_READ or GADGET_DEV_WRITE), a pointer to a data buffer into which the data will be read, or from which the data will be written, and the number of bytes to transfer.

   static status_t
   do_device_io (dev_info *d, dev_op op, void *data, size_t *len)
   {
      status_t         err;
      size_t         xfer_count = 0;
      int32         flags;
      
      ddprintf ("do_device_io: data=%.8x *len = %.8x\\n", data, *len);
   
      /* If the requested transfer is zero bytes long,
         just exit immediately */
   
      if (!*len)
         return B_OK;
      
      /* Lock down the client buffer in memory */
   
      flags = B_DMA_IO;               /* It's a DMA operation */
      if (op == GADGET_DEV_READ)
         flags |= B_READ_DEVICE;         /* And we're reading, too */
      
      /* Tell the kernel to lock the memory down and why */
      
      err = lock_memory (data, *len, flags);
      if (err != B_OK)
         goto err0;                  /* Unable to lock the memory */
      
      /* Block until the BeGadget hardware is available */   
   
      err = acquire_sem_etc (d->hw_lock, 1, B_CAN_INTERRUPT, 0);
      if (err != B_OK)
         goto err1;                  /* Couldn't get the lock */
      
      d->xfer_count = 0;               /* No data has been transferred */
   
      /* Get a list of pages comprising the client buffer */
   
      err = get_memory_map (data, *len, d->scatter, MAX_SCATTER);
      if (err != B_OK)
         goto err2;                  /* Couldn't get the map */
   
   #if DEBUG_NO_HW
      {
         int j;
         for (j = 0; j < MAX_SCATTER; j++)
            ddprintf ("do_device_io: scatter[j] = %.8x %.8x\\n", d->scatter[j].address, d->scatter[j].size);
      }
   #endif
   
      /* Start up the I/O on the first scatter/gather entry */
      
      d->scat_index = 0;
      d->op = op;
      start_io (d);
      
   #if DEBUG_NO_HW
      /* start up the gadget interrupt provoker thread */
      d->DEBUG_INT_SEM = create_sem (0, "gadget int sem");
      resume_thread (spawn_kernel_thread (
         gadget_interrupter, 
         "gadget_interrupter",
         B_NORMAL_PRIORITY,
         d
      ));
   #endif
   
      /* Block until the entire transfer is complete */
   
      err = acquire_sem_etc (d->io_done, 1, B_CAN_INTERRUPT, 0);
      if (err != B_OK)
         goto err2;
      
   #if DEBUG_NO_HW
      /* stop the gadget interrupter thread */
      release_sem (d->DEBUG_INT_SEM);
   #endif
      xfer_count = d->xfer_count;         /* Update the transfer count */
      err = B_OK;                     /* And note that no error occurred */
      
      /* Handle errors which occurred at various points */
      
   err2:
      release_sem (d->hw_lock);
   err1:
      unlock_memory (data, *len, flags);
   err0:
      *len = xfer_count;
      
      return err;
   }

The first thing do_device_io() does is check to see if the size of the requested transfer is larger than zero bytes long. If it's zero bytes long, the B_OK result code is immediately returned, since we don't have to do anything to successfully fulfill a zero-byte request.

Since the BeGadget driver uses DMA to transfer information between the client buffer and the BeGadget card on the PCI bus, we need to tell the kernel this is the case because some computers don't maintain cache coherency during DMA operations. On these computers, the kernel marks the client buffer's memory as non-cacheable for the duration of the operation.

Also, if this is a read operation, we need to tell the kernel that the DMA activity may alter the contents of the memory so it can mark the altered memory as dirty for virtual memory purposes.

This is accomplished by calling lock_memory() on the data buffer, passing in the B_DMA_IO flag to tell the kernel why we're locking the buffer down. The B_READ_DEVICE flag is added if the request is going to be reading into the memory, so that the memory will be marked as dirty by the kernel. If an error occurs, we branch to an exit point which handles cleaning up and returning the appropriate error code.

Next, exclusive access to the BeGadget virtual device is obtained by acquiring the hw_lock semaphore for this virtual device. All but the first request will block here. This ensures that only one thread at a time is manipulating the device. This acquire_sem_etc() call blocks until the hardware is not in use (ie, when the hw_lock semaphore has been released). If an error occurs, the code immediately routes to code that cleans up and returns an error code.

The transfer count--the number of bytes of data that have been transferred between the BeGadget device and memory--is cleared to zero.

Once the hardware has been acquired, a list of memory pages which the client buffer spans is requested. Our cookie contains an array called a scatter/gather table (named scatter) into which this list will be stored.

In order to keep the scatter/gather table small, I/O transactions are limited to 32k by the BeGadget driver; larger requests will be broken down into numerous requests no larger than 32k. This maximum, as well as the number of pages in the scatter/gather list, is defined as follows in the source code for the BeGadget driver:

   #define MAX_IO         0x8000      /* biggest single i/o we do */
   #define MAX_SCATTER      ((MAX_IO/B_PAGE_SIZE) + 1)

MAX_IO is the number of bytes to which we limit a single transaction. MAX_SCATTER is the number of entries in our scatter/gather table.

As data is actually transferred to or from system memory, the addresses of the pages into which the data will be stored are obtained from this list.

If the get_memory_map() call fails, an error is immediately returned.

Once this has been done, we note that the scatter table entry currently being addressed is the first one, by setting the cookie's scat_index field to 0. The operation to perform (GADGET_DEV_READ or GADGET_DEV_WRITE) is stored in the op field of the cookie, so that the actual I/O routines know which operation to perform. Finally, the start_io() function (which is internal to the BeGadget driver) is called with a pointer to the cookie. This function actually issues the I/O request to the BeGadget hardware, and we'll have a look at it in a moment.

Once the start_io() call is issued, we block by attempting to acquire the io_done semaphore for this driver. The interrupt handler will handle completing the request by starting the I/O for the remainder of the pages in the scatter/gather table, and will unblock the io_done semaphore when the entire table has been processed.

After the io_done semaphore unblocks, the hw_lock semaphore is released, the data buffer's memory is unlocked, and the transfer count, which is in the variable xfer_count, is stored at the address pointed to by the len parameter to the do_device_io() function. Finally, the return code is returned to the caller.

The actual interfacing with the hardware is done by the start_io() function. This function performs the task of writing the needed commands into the 32-bit command register for a given virtual device on the BeGadget card in order to transfer a single contiguous chunk of data either from the client buffer to the BeGadget device or vice versa.

The start_io() function is called by do_device_io() initially, as we have already seen, to transfer the first page. After that, it is called repeatedly by the driver's interrupt handler to transfer the remaining pages.

Exclusive access to the hardware has already been obtained via the hw_lock semaphore, so we don't have to lock down the hardware in here.

start_io() returns true if a page transfer has been performed or false if there are no more pages to transfer. A pointer to the cookie for the device being accessed is passed into this function.

   static bool
   start_io (dev_info *d)
   {
      int i;
      
      /* Check to see if the access has been completed */   
   
      ddprintf ("start_io: index = %.8x\\n", d->scat_index);
      i = d->scat_index;
      if (i == MAX_SCATTER || d->scatter[i].size == 0)
         return false;
      
      d->scat_index++;         /* Increment the entry # to access */
   
      /* Issue the command by writing the opcode, address,
         and data size to the command register. */
      
         
   #if DEBUG_NO_HW
      return true;
   #endif
      d->regs->cmd = d->op;      /* Send the opcode */
      __eieio();               /* Wait until it's sent */
      d->regs->cmd = (int32) ram_address (d->scatter[i].address);
                           /* Send the address */
      __eieio();               /* Wait until it's sent */
      d->regs->cmd = d->scatter[i].size;
                           /* Send the size */
      __eieio();               /* Wait until it's sent */
      return true;
   }

First, this code checks to make sure that there is actually data left to transfer, by comparing the scat_index value in the cookie to the number of entries in the scatter/gather table for the device, as well as the maximum number of entries possible in the scatter/gather table. If the page to be transferred is outside the range of existing pages, false is returned, which indicates that the entire transfer has been completed. Otherwise, the scat_index field is incremented so that the next time start_io() is called, the next block will be transferred.

Recall that I/O transactions are performed by writing three times to the virtual device's command register. The first write is the opcode (GADGET_DEV_READ or GADGET_DEV_WRITE), the second is the address of the data buffer in main system memory, and the third is the number of bytes of data to transfer. These three values are 32-bit integers, and are written one after the other into the command register.

The __eieio() intrinsic function must be used after each of these three writes because the PowerPC processor, which is capable of reordering operations to improve efficiency, needs to be told not to reorder these operations, since they have to be performed in order. The __eieio() function issues the eieio (Enforce Inorder Execution of I/O) PowerPC instruction, which forces all pending memory accesses to complete before the processor proceeds to handle the next instruction.

When creating a driver that requires I/O to be performed in a particular order, be sure to use the eieio operation to ensure that the order of the I/O is maintained, or your driver may not function reliably.

The ram_address() function, which is exported by the kernel, translates an address in main system memory into the equivalent address as viewed from the PCI bus. When we pass the address of the buffer to the BeGadget card, we need to first convert it into the PCI equivalent address.

Finally, start_io() returns true to indicate that the request has been issued to the BeGadget card.


gadget_dev_inth()

The final component of the BeGadget device driver is the gadget_dev_inth() function, which is the interrupt handler used by the driver to issue repeated calls to start_io() each time a chunk of a data transfer is completed.

gadget_dev_inth() is called each time a chunk transfer from or to the BeGadget card is completed. Let's have a look.

   static bool
   gadget_dev_inth(void *data)
   {
      dev_info   *d = (dev_info *)data;
      
      /* start_io() returns false if the transaction is complete */
      
      if (!start_io (d))
         release_sem_etc(d->io_done, 1, B_DO_NOT_RESCHEDULE);
      
      return true;
   }

When gadget_dev_inth() is called, a pointer to the cookie for the virtual device is passed in the data parameter--this was established when the interrupt handler was installed by the pci_open() hook function. A local pointer, properly typecast to the type dev_info *, is created for convenience.

start_io() is called with a pointer to the cookie. As discussed previously, start_io() attempts to transfer another chunk of data to or from the BeGadget device; all the necessary parameters are located within the cookie.

If start_io() returns true, the entire transaction has been completed; there are no further entries in the scatter table for the transaction. The io_done semaphore is released, which indicates that the transaction has been completed. Recall that the do_device_io() call, once a transaction is initiated, blocks until this semaphore is released. Once the interrupt handler releases the semaphore, do_device_io() is unblocked and knows that the transaction is finished, so it can clean up and return to the caller.

Note that when we release the semaphore, we use the B_DO_NOT_RESCHEDULE flag. This is a must if anything of significance is done after the call to release the semaphore. If this flag is not set, the release call could reschedule the current thread that is processing the interrupt. When it wakes up, possibly much later, other driver operations may have been done by other threads, which could lead to unexpected behavior.

Although careful thought and planning could be used to avoid using B_DO_NOT_RESCHEDULE, it's much easier to do it this way. The release call will always return right away.

The interrupt handler returns true since the interrupt was handled.






The Be Book, in lovely HTML, for BeOS Release 3.

Copyright © 1998 Be, Inc. All rights reserved.

Last modified March 27, 1998.