I’ve already introduced an ``easy'' (and broken) way to deal with such memory in Section 2.5.2 in Chapter 2, where I showed how pointers holding the physical hardware address dereference correctly to the requested I/O memory. Though this technique works on the x86 platforms, it is not portable to all Linux platforms. Pointer dereferencing might be a good choice for small and short-lived projects, but it’s not recommended for a production driver.

The recommended interface to I/O memory was introduced into the kernel during the 1.3 development tree and is missing from older kernels. The sysdep.h header released with the short sample code, however, implements the new semantics for kernel versions back to 1.2.

The new interface consists of a set of macros and function calls that are used in place of pointer dereferencing whenever you need to access I/O memory. Such macros and functions are portable; this means that the same source code will compile and run on different architectures, as long as they host the same peripheral bus.

Most of these macros currently expand to pointer dereferencing when you compile code on your Intel-based PC, but their internal behavior could well change in the future. One such change happened, for example, in the initial switch from 2.0 to 2.1, when Linus decided to change the virtual memory layout. With the new layout, ISA memory can’t be accessed in the old-fashioned way described in Chapter 2.

The new interface to I/O memory consists of the following functions:

The portability of these functions across the supported architectures is currently limited, like that of the port I/O functions. The functions are completely missing from some platforms; on some they are macros that expand to pointer operations, and on others they are real functions.

People like me, accustomed to the flat memory model of our old PCs, might hesitate to bother with a new interface just to access ``a region of physical addresses.'' Actually, getting comfortable with the interface is simply a question of getting some practice using the functions. There’s nothing better for gaining confidence than looking at a silly module that just accesses I/O memory. The module I’m going to show you is actually called silly, short for ``Simple Tool for Unloading and Printing ISA Data.''

The module features four device nodes that perform the same task using different data transfer functions. The silly devices act as a window over I/O memory, in a way similar to /dev/mem. You can read and write data, lseek to an arbitrary I/O memory address, and mmap the region to your process (see Section 13.2 in Chapter 13).

/dev/sillyb, featuring minor number 0, accesses I/O memory with readb and writeb. The following code shows the implementation for read, which remaps the address range 0xA0000-0xFFFFF to file offset 0-0x5FFFF. The read function is structured as a switch statement over the different access modes; here is the sillyb case:

case M_8: 
  while (count) {
      *ptr = readb(add);
      add++; count--; ptr++;
  }
  break;

The next two devices are /dev/sillyw (minor number 1) and /dev/sillyl (minor number 2). They act exactly like /dev/sillyb, except that they use 16-bit and 32-bit functions. Here’s the write implementation of sillyl, again part of a switch:

case M_32: 
  while (count >= 4) {
      writel(*(u32 *)ptr, add);
      add+=4; count-=4; ptr+=4;
  }
  break;

The last device is /dev/sillycp (minor number 3), which uses the memcpy_*io functions to perform the same task. Here’s the core of its read implementation:

case M_memcpy:
  memcpy_fromio(ptr, add, count);
  break;

Some ISA device boards carry on-board memory that is mapped to the physical addresses between 14MB and 16MB. These devices are slowly disappearing, but it’s worth introducing how their memory range can be accessed. This discussion, however, only applies to the x86 architecture; I’ve no information on how the Alpha or other architectures behave with such ISA boards.

In the old days of the 80286 processor, when the physical address space was 20 bits wide (16MB) and all the address lines were present on the ISA bus, almost no computer carried more than 1 or 2 megs of RAM. Why couldn’t the expansion board steal some high memory addresses for its buffer? This idea is not new; it’s the same concept that led to ISA memory below 1M, and it was later recycled to implement high PCI buffers. The chosen address range for ISA device boards was the top two megs, though most boards just use the top meg.

Nowadays, there are still a few motherboards that can host these old-fashioned boards even when there are more than 14 megs of physical RAM. Correctly handling this memory requires you to play games with the address ranges to avoid ending up with overlapping RAM and bus addresses.

If you have an ISA device with high memory, and you’re unlucky enough to have less than 16MB RAM, managing the memory is easy. Your software should behave as though it had a high PCI buffer (see the next section), except that it will be slower, because ISA memory is slow.

If you have an ISA device with high memory and you have 16MB or more, then you’re in deep trouble.

One possibility is that your motherboard doesn’t correctly support the ``ISA hole.'' In that case, there’s nothing you can do to access the on-board memory except change the board or remove some RAM. If, on the other hand, the motherboard handles the ISA hole, you still need to tell the Linux kernel about such memory and do some work to be able to access the rest of your RAM (the range over 16M).

The place where you need to do some hacking to correctly reserve the high ISA memory, while not losing access to the remaining RAM, is in the map of the computer’s physical memory. This map is built in arch/i386/mm/init.c, within the mem_init function. The array mem_map holds the relevant information about each memory page; if the bit PG_reserved is set for a page, the kernel won’t use that page for normal paging activity (i.e., the page is ``reserved'' and can’t be touched). Reserved pages can nonetheless be used by drivers; the range between 640KB and 1MB is marked as ``reserved,'' but it hosts usable device memory.

The following code, inserted in mem_init, correctly reserves the memory space between 15MB and 16MB:

while (start_mem < high_memory) {
    if (start_mem >= 0xf00000 && start_mem < 0x1000000) {
        /* keep it reserved, and prevent counting as data */
        reservedpages++; datapages--;
    }
    else
        clear_bit(PG_reserved, &mem_map[MAP_NR(start_mem)].flags);
    start_mem += PAGE_SIZE;
}

Initially all the memory is marked as ``reserved,'' and the lines shown above take care not to unreserve high I/O memory; the original code only has the else branch within the loop shown. Since every reserved page after the end of the kernel code is counted as kernel data, two counters are modified to prevent a mismatching message at boot time. My box, with 32MB and the previous code to access the ISA hole, reports:

Memory: 30120k/32768k available (512k kernel code, 1408k reserved,
        728k data)

I tested this code with my own Intel box (ISA-hole aware) with kernel 2.0.29. If you are running a different kernel version, you might need to tweak the code--the internal structures related to memory management changed slightly in 2.1 and were different in version 1.2 of the kernel. Hacking with kernel code is unavoidable when you’re supporting old-fashioned (and sometimes badly designed) hardware.

Accessing high PCI memory is much easier than accessing high ISA memory. High memory on PCI boards is really high--higher than any reasonable physical RAM address (at least for the next few years).

As discussed in Section 7.3 in Chapter 7, a single call to vremap (ioremap with kernel 2.1) is all it takes to access this memory. If you want to be portable across platforms, however, you should access the remapped memory range only through readb and similar functions. This restriction applies because not all platforms are able to directly map PCI buffers in the processor’s address space.