Chapter 4
Silicon-Specific Initialization
Since when has the world of computer software design been about what people want? This is a simple question of evolution. The day is quickly coming when every knee will bow down to a silicon fist, and you will all beg your binary gods for mercy.
—Bill Gates
So if developers strive to be binary gods, as Bill Gates puts it, then they really need to know what they are doing when it comes to silicon initialization. Hopefully they have more than a binary to start from. They are going to need code and proper documentation.
When initializing silicon should be easy: “It’s a few rdmsrs, and a wrmsr for the processor. It is a 0xCFC here and a 0xCF8 there with the standard config space for PCI device, a peek and a poke to some special-sauce registers. Then just glob on the expansion ROMs or blobs for the add-in devices and voila you are done, right?” Such would be the case in a perfect world.
While silicon initialization is a process in which the BIOS or system firmware must set a few bits and enable the system in the correct sequence, the nuances of the silicon and the recent changes must be updated every design cycle.
Listen to the Designer, Then Experiment, and Fix It
Regardless of the language used to program the firmware or the BIOS or the bootloader, the outcomes still needs to match up to what the silicon design engineer (Si DE) is expecting. Or what the Si DE thought should happen before the first parts were powered on and debugging started. During a power-on, the BIOS toggles the state machines, sets the bits, and, in many cases, is the first test software to run on the product. When this process happens for the first time, bringing silicon to life, it can find silicon bugs, and the ideal initialization algorithms may need to change, sometimes drastically. It may be a case where the default bit settings are no good and have to be programmed to new values through I/O, PCI, or memory-mapped I/O programming. There may be a toggle of a bit to increase or decrease the frequency or decrease the frequency of a clock or add cycles to the data paths to eliminate a potential race condition for a buffer to fill or clear. It’s a lot of very methodical and deliberate step-by-step work to do that debug! Or it may just be flipping a few bits or changing some strapping values and the entire system just starts right up. More on this in Chapter 6.
Silicon programming for Intel architecture really means needing to understand the individual controller’s nuances. When done properly it can be fantastic, making a system come to life and run faster than anything else on the planet. When done without proper preparation, and if the board designer has not followed the prescribed hardware design guidelines, it can be a ferocious nightmare where the board sputters and grinds and makes magic blue smoke. As the first chapter indicated, this is where it is vital to have the right level of data about the component.
Chipsets
For a chipset, the firmware developer has to consider several features:
–Flash programming (NOR, NAND, or integrated)
–Reset controls
–I/O and MMIO base address locations and ranges
–Standard PCI header details
–Timers and clock routing
–General purpose I/Os and bells and whistles.
–Thermal management
–Power management, APM, ACPI tables and ASL code support
–Interrupts
–Bus support per major interface (DMI, PCIe, audio, Ethernet, SMBus, USB, SATA).
The chipset comprises a series of integrated PCI devices connected to a fabric, backbone, or bus, which connects to an interface (such as DMI) to the processor. Each device/function on the component will get PCI BIOS standard initialization, but also some amount of proprietary bit twiddling. Most of these bits will be detailed in the datasheet, but it is best to get the BIOS specification for the particular chipset, which may require a special NDA with the manufacturer. There are also industry standards to follow, depending on the type of controller (SATA, USB, and so on). There may be exceptions to industry standards per component. In the end, we need to keep it all straight and avoid conflicting memory, I/O, ranges, IRQ and interrupt settings, for all of the components to come alive and start talking to one another before the OS takes over with its set of device drivers, settings, industry standards and exceptions, and additional capabilities.
Processors
For a processor, there is much more than a few rdmsrs and wrmsrs. Today we have no less than:
–CPUID and branding strings
–More than 100 model-specific registers, some specific to package, core, or thread.
–Bus-to-core ratios
–Overclocking controls
–Turbo mode
–Cache controls
–CPU RESETs
–Microcode updates
–Multithreading and multicore initialization
–The xAPIC
–System management mode
–CPU ACPI Power states (P-, C-, S-, and T-states)
–System management BIOS
–Thermal management
–Power management
–Intel EM64T
–Intel® Trusted Execution Technology (Intel TXT)
–Intel® Virtualization Technology (Intel VT)
–Machine check architecture
–Security features
For supplemental information regarding MSRs or other processor features, refer to the Intel® 64 and IA-32 Architecture Software Developer’s Manual, Volume 3B, System Programming Guide, Part 2. However, for chip-specific data needed to program an Intel CPU successfully, there is a processor BIOS writer’s guide, which may require an NDA to get to the detailed bits and algorithms.
Basic Types of Initialization
For BIOS, there are four types of programming:
- Bit setting, per industry specification or per designer
- Standard algorithms, per specification
- Custom routines, per designer
- Expansion ROMs, per vendor
Simple Bits
“Bit banging” is slang for the sequence of reading/modifying/writing to registers in the silicon. This can be to CPU model-specific registers (MSRs) or to PCI or I/O or memory-mapped I/O. The register/bit settings are normally done in a priority order as dictated by logic to speed up the system initialization, or it is done out of simple fear in doing a workaround as early as possible to avoid the errata condition, but the order may not matter. Normally these bits are set once at boot time and never looked at again. It is possible that the bits are also locked when set to avoid tampering by malware during runtime.
Standard Algorithms, Minding the Ps and Qs
Standard routines are programmed to follow industry specifications, such as PCI, ACPI, USB, JEDEC, or others. This is done with the assumption that the silicon being programmed is designed exactly per the specification. If this is the case, then the algorithms should never change once written and that standard easily can go from component to component, device to device, year over year. If this is not the case, then you will have to create the exception and determine the trigger for that exception—a vendor ID and device ID combination in the case of PCI headers perhaps? A revision ID, or is that for a specific CPUID? Or is it form factor or board-specific condition that it will have to be read from a GPIO? There are many standards and when something needs to ship and it is not quite in line with the standard, a quick exception and/or workaround is created and the product ships.
Custom Algorithms: It’s All About Me
Custom routines are just that: custom. It is up to the developer and/or the designer to make the calls as to what needs to happen with initialization of a device beyond the standard PCI base address registers, IRQ routing, and so forth. A custom routine can provide the flexibility to do what is needed apart from the industry standards, such as PCI, USB, SATA, or others. Custom routines mean that they could be one-off implementations for specific applications and will need to be redone for the next component design, such as ASICs. Often, custom routines provide the best efficiency in boot speeds overall, as standard implementations typically mean slowing down to detect and meet any unusual scenarios. The algorithms and bit settings could be entirely memory mapped and have absolutely no standard to program to, such as PCI or ACPI. Most OSes, though, do not match up easily to that lack of standards; custom OS-level drivers would also be needed, and that OS driver may be very different from the initialization drivers in a UEFI system firmware.
An interesting example is USB initialization. Per the specification, one needs to use USB protocols to interact with each controller and identify whether any devices are attached to those ports that need further initialization. One alternative mechanism that can be utilized is the access to this information through memory-mapped I/O (MMIO) instead. Such an enhancement would need silicon changes to create the mappings in memory space, but could achieve potentially a much faster and leaner initialization mechanism. Alternatively, beyond the USB specification requirements, study of the actual timing of the silicon and shortening of the “standard” delays down to what is actually required by the component can yield great benefits. Results may vary by the controller and device manufacturers, but the potential time savings are dramatic. More on this in Chapter 12.
Embedded controllers are custom programmable hardware that can interface with and extend the abilities of the system, as well as provide a back-end solution to some interesting design problems. These controllers come with their own firmware and control interfaces to the system BIOS, besides embedding keyboard controllers and other super IO functionality.
Field Programmable Grid Arrays (FPGAs) are examples that provide fixed functionality until they get reprogrammed. Their sizes can vary and their applicability depends on the market segment in which they are found. Like CMOS, they need battery backup to maintain their NV status … along these lines. The usage can follow standard programming needs, like PCI or USB, ACPI, and so on, or it can be completely custom or need no additional programming at all. FPGAs are normally used to provide feature augmentation to a design or early development phases of new silicon.
Option ROMs
Formerly ISA-expansion ROMs, PCI-option ROMs such as video BIOS and now UEFI drivers such as graphics output protocols provide another mechanism for taking things one step beyond industry standards. These objects do so, though, in a very standard way that can be easily implemented either though binary or source or mixture. Expansion ROM code extends the BIOS capabilities beyond what the standard CPU and chipset programming requirements provide.
They can be used for add-in cards or with integrated components. Option ROMs get loaded and executed by the system firmware during initialization and, if needed, shadowed during runtime. Newer capabilities of UEFI option ROMs offer the developers a driver model where the ROM can be loaded but not executed unless it is needed to be enabled via the boot path. There are some other advantages to UEFI option ROMs:
–Prior to EFI capabilities, all legacy option ROMs are located below 1 MB between 0xC0000 and 0xFFFFF and carry around a good deal of 16-bit code.
–Newer option ROMs can be called by various names, including DXE drivers in the Tiano realm and can be relocated above 1 MB, which eliminates the crunch on size requirements of most option ROMs and alleviates the expansion limitation of larger server systems.
Devices such as LAN, SCSI, SATA, RAID, and video often have option ROMs initialize key low-level requirements of proprietary designs. It is possible to embed or integrate the code into the main BIOS ROMs. Sometimes the intellectual property of the various silicon integrated into the design may not allow access to that code or knowledge of the exact registers. There is a positive aspect to binary code: As a developer, you don’t have to fix what cannot be fixed. And as the black box, a legacy option ROM binary gives an excellent chance to innovate along alternative lines when the opportunity presents itself.
Summary
This chapter contained a high-level overview of some of the programming requirements of basic components. For complete details on a particular chip, go to the silicon provider and obtain the required documentation, such as the data sheet and programmer’s reference manual.