Fibre Channel is the protocol used for data transfer due to its high speed and provisions for high availability. It is essentially the classic SCSI protocol, mixed with an address-based networking system, so that multiple hosts can be online and request and send data to and from storage devices.
A speed of 2 Gbits per second is used with all Xsan elements, which yields a theoretical limit of 200 MB/s for transmission and reception, for a total of 400 MB/s per cable.
There are two kinds of entities in Fibre Channel networking: initiators and targets. Initiators are end user computers and servers that receive data from and transmit data to storage. Targets are storage arrays and tape drives that transmit data to and receive data from initiators.
There are three protocols used with Fibre Channel networking: point-to-point, arbitrated loop and fabric.
Point-to-Point
Point-to-Point protocol is used when connecting storage directly to the host. Only two devices are involved.
Arbitrated Loop
Arbitrated loop protocol works as a “party line” for all of the entities on the network. Regardless of the number of connected devices only two devices can communicate at any one moment. An 8-bit addressing system is used to route data from one entity to another.
Fabric
Fabric switches are similar to Ethernet switches, in that multiple communications can be occurring on the switch–as many as are needed at any given moment. A 24-bit addressing system is used to route data from one entity to another.
When planning your SAN, you might require more than one switch to accommodate all of the entities. The following topologies are used to interconnect fabric switches.
Cascade Topology
A cascade topology interconnects two switches, using two or more cables connected to ports on each switch. More than one interconnecting cable is used to increase the bandwidth of data transferred and to provide redundancy in case any one connection goes down.
Cascade Loop
Cascade loop topology is used for three or more switches, where the first and last switch are further interconnected. This allows data to follow the “first or shortest path first (FSPF) rule, a standard function of fabric protocol.
Mesh
Mesh topology has each switch fully interconnected to every other switch. This allows for greatest efficiency and redundancy in the fabric.
Apple’s Fibre Channel 133 MHz PCI-X HBA is used in Xsan integration. At $499, it is often as much as $2000 less than equivalent products from other manufacturers, and comes with two 2.9 m copper cables. This card has two small form factor pluggable (SFP) ports.
Copper Cabling
Two 2.9 m copper cables come standard with each Fibre Channel HBA. SFP transceivers are already attached to each end. Copper cables of longer lengths should be avoided with Xsan.
Optical Cabling
Use 2 Gbits/s LC cable. These cables usually have bright orange jacketing, although multiple-color cables are emerging on the market. Check the printing on the jacketing of the cable to be sure of its specifications.
Cable lengths depend on substrate size:
“The smaller the diameter, the longer the run”
Cable Size | Run Length | SFP Transceiver Required |
62.5 μ | 300 m | Short wavelength (short haul) |
50 μ | 500 m | Short wavelength (short haul) |
9 μ | 10 km | Long wavelngth (long haul) |
Prefabricated optical cables are the easiest to implement, as cable ends are already finely polished to interface with the SFP transceiver. Lengths vary from 1 m to 500 m.
Optical cable is manufactured from fiberglass substrate. It is therefore extremely fragile. When running fibre cable from the core SAN components, great care should be taken not to bend or kink the cable. As often as possible, it should be laid into conduit, rather than pulled through.
Two SFP transceivers are needed for each optical cable used in your integration. Apple recommends the following manufacturers and part numbers.
Short wavelength (short haul)
Finisar FTRJ-8519-P1BNL
PicoLight PL-XPL-VE-S24-11
JDS Uniphase JSM-21S0AA1
JDS Uniphase JSP-21S0AA1
Long wavelength (long haul)
Finisar FTRJ-1319-P1BTL
Most Xserve RAID controllers have SFP connectors as well. However, first-generation Xserve RAIDs have HSSDC2 connectors instead. Those connectors look quite different from SFP ports.
Fibre to SFP Connection
Usually those RAIDs were sold with copper cables that have HSSDC2 connectors on one end and SFP connectors on the other. Use those cables to connect the controllers to the switch. There are no issues with using HSSDC2-ported controllers.
Fibre End
All switches approved for use with Xsan have auto-sensing SFP ports. Most have the ability to suppress registered state name changes (RSCN) on ports that will be attached to the initiator nodes (clients or controllers) of the SAN. This allows uninterrupted communication between the nodes and the storage if any of the nodes are started up or are restarted during SAN activity.
Currently supported switches for Xsan:
SilkWorm 3200 or 3250 8-port fabric switch
SilkWorm 3800 or 3850 16-port fabric switch
SilkWorm 3900 32-port fabric switch
SilkWorm 12000 64-port dual-domain director
Administration
All Brocade switches can be administered through a built-in Web host called Web Tools. The full documentation for this interface is provided on a CD-ROM that comes with each switch.
Out of the box, the switch can be contacted by pointing a browser to:
IP address: | 10.77.77.77 |
Netmask: | 255.0.0.0 |
Gateway: | 0.0.0.0 |
Username: | admin |
Password: | password |
On first launch, Brocade’s Web Tools requires a license. Have the license ready and follow the onscreen instructions to install the license.You will then be prompted to select a new username and password.
To reboot the switch:
Select the desired switch from the Fabric Tree.
The selected switch appears in the Switch view.
Select the Admin icon from the Switch view.
The login dialog appears.
Enter the admin username and password.
Click the Firmware tab.
Click the Reboot radio button.
Click Apply.
To download a new version of the firmware:
Select the desired switch from the Fabric Tree.
The selected switch appears in the Switch view.
Select the Admin icon from the Switch view.
The login dialog appears.
Enter the admin username and password.
Click the Firmware tab.
Click the Firmware Download radio button.
Enter the username, password, and host IP information.
Enter the fully qualified path to the filename.
Click Apply.
To suppress RSCN on initiator ports:
Brocade does not offer a RSCN suppression feature. To inhibit RSCN interruptions when client computers are booted or restarted, separate “mini” zones for each client and the storage must be created. Refer to the documentation that came with your Brocade switch to set up these zones.
To create inter-switch links:
Brocade refers to inter-switch linking as ISL trunking.
Select the desired switch from the Fabric Tree.
The selected switch appears in the Switch view.
Click the Admin icon from the Switch view.
The login dialog appears.
Enter the admin username and password.
Click the Port Setting tab.
For SilkWorm 12000 and 24000 switches: Select the slot subtab that corresponds to the correct slot for the logical switch.
To enable trunking mode on a port, check the box in the Enable Trunking column that corresponds to the port you wish to trunk.To disable trunking mode on a port, uncheck the box.
Click Apply.
Brocade Contact Information:
Corporate | T: (408) 333-8000 |
Tech support | T: (888) 283-4273 |
T: (408) 333-6061 | |
Europe | T: +800 28 34 27 33 |
Tech support: |
Currently supported switches for Xsan:
InSpeed 355 12-port arbitrated loop switch
InSpeed 375 20-port arbitrated loop switch
Full documentation for these switches, including special setup guides for Apple-based implementations, can be downloaded from:
http://www.emulex.com/ts/indexemu.html
Administration
Emulex switches can be administered through a built-in Web host called Web Manager.
Out of the box, the switch can be contacted by pointing a browser to:
IP address: | 169.254.10.10 |
Netmask: | 255.255.0.0 |
Gateway: | 0.0.0.0 |
Passsword: | password |
To reboot the switch:
To suppress RSCN on initiator ports
On the left side of the interface, choose Port > Smart Settings.
Select the “Initiator with Stealth” Smart Setting from the list box.
From the list of port numbers, select the ports that will use the selected Smart Setting under the Assign heading. Remember to seleect only initiator ports for this setting.
Click Submit to save the settings.
To upgrade firmware:
On the left side of the interface, choose Help > Downloads.
The Emulex Corporation’s Web site appears.
Click the “drivers, software and manuals” link, and select the switch product model under the “drivers, software and manuals by product model number” section.
Under Download New Alternate Firmware Version to Switch, choose Browse to navigate to and select the appropriate file on the Mac.
The file must have a .bin extension.
Click Start to load the new firmware image.
Once the firmware has been installed, the new firmware should appear as the AlternateVersion firmware.
Under Next Boot Firmware Version, ensure that the Use Alternate Version on Next Reboot option is selected.
The alternate firmware version currently displayed will be loaded on the next boot cycle.
Click Reboot to reset the switch using the selected firmware.
To create inter-switch links:
Emulex refers to inter-switch linking as trunking.
Ports designated for trunking must be assigned to a Trunk Group.
On the left side of the interface, choose Advanced Functions > Automatic Trunking.
The Automatic Trunking page appears.
Select a trunk group for each port by clicking the appropriate trunk group option.
When you are finished making changes, click Submit.
Emulex Contact Information:
Corporate | T: (800) 368-5391 |
T: (714) 662-5600 | |
F: (714) 241-0792 | |
Tech support | T: (800) 854-7112 |
T: (714) 885-3402 |
General information: | |
Tech support: | |
Repair: |
Currently supported switches for QLogic:
SANbox 2-8 8-port fabric switch
SANbox 2-16 16-port fabric switch
SANbox 2-64 64-port fabric switch
SANbox 5200 or 5202 8 to 20-port scalable, stackable fabric switch
QLogic uses a java-based application called SANbox Manager to administer its switches via Ethernet networking.
QLogic switches currently need a downgrade in their stock switch firmware in order to operate smoothly with Apple HBAs. This will change starting with version 5.0 of their firmware.
SANbox Manager software for OS X, as well as the special 4.0.6.02 firmware for use with Xsan, can be downloaded from:
http://www.qlogic.com/support/oem_detail_all.asp?oemid=364
Administration
Default settings for use with SANbox Manager:
IP address: | 10.0.0.1 |
Netmask: | 255.0.0.0 |
Gateway: | 10.0.0.254 |
Username: | admin |
Passsword: | password |
To connect to the switch:
Launch SANbox Manager.
An initial screen gives four connection options.
Choose “Open Existing Fabric” and click Proceed.
You are then presented with a login window.
If connecting for the first time, enter 10.0.0.1 in the “IP Address:” field, admin in the Login Name field, and password in the Password field.
After a short pause, the main fabric view appears.
Double-click on the image of your switch to observe current port states and to further administer the switch. This view is called the Faceplate.
To reboot the switch:
In Faceplate view, choose Switch > Reset Switch.
There are three reset options:
“Hot Reset” simply allows a reset during communications, to incoroporate new firmware.
“Reset” forces the switch to run its power on self-test (POST).
“Hard Reset” forces the switch to factory default settings and forces a POST.
To suppress RSCN on initiator ports:
In Faceplate view, select the icons for each initiator port on which you wish to suppress RSCN.
Choose Port > Port Properties 1G/2G.
A window appears, allowing you to change the properties of the port(s) you’ve selected.
In the I/O Stream Guard section near the bottom, click the Enable radio button. Remember to select only initiator ports for this setting.
Click OK.
A confirmation is given once the ports have been set.
Click OK.
To upgrade firmware:
A window appears prompting you to browse for the firmware file.
Click Select, and browse to the firmware file.
Click Choose.
Click the Start button on the original window.
The firmware uploads to the switch. You will then be prompted to activate the firmware.
Click Yes to activate the firmware.
To create inter-switch links:
Simply connect two switches with cables. The ports will auto-negotiate to E (expansion) ports.
If the port status lights on the switch remain blinking, it usually means that the two switches being linked have the same domain number. Simply power down and power up one of the switches. When it restarts, it will auto-negotiate to a new domain number.
QLogic Contact Information:
Corporate | T: (800) 662-4771 |
T: (949) 389-6000 | |
F: (949) 389-6009 | |
Tech support | T: (952) 932-4040 |
General information: | |
Tech support: | |
Technical training: | |