- Copyright
- Praise for Building Clustered Linux Systems
- Hewlett-Packard® Professional Books
- List of Figures
- List of Tables
- Preface
- Acknowledgments
- Introduction
- I. Introduction to Cluster Concepts
- 1. Parallel Power: Defining the Clustered System Approach
- 1.1. Avoiding Difficulties with the Word Cluster
- 1.2. Defining a Cluster
- 1.3. The Evolution of a Clustered Solution
- 1.4. Collapsed Network Computing for Engineering
- 1.5. Scientific Cluster Computing
- 1.6. Revisiting the Definition of Cluster
- 1.7. Commercial Cluster Computing
- 1.8. High Performance, High Throughput, and High Availability
- 1.9. A Formal Definition of Cluster
- 1.10. The Why and Wherefore of Clusters
- 1.11. Summary
- 2. One Step at a Time: A Process for Building Clusters
- 1. Parallel Power: Defining the Clustered System Approach
- II. Cluster Architecture and Hardware Components
- 3. Underneath the Hood: Cluster Hardware Components and Architecture
- 3.1. Hardware Categories in a Cluster
- 3.2. A Survey of Cluster Hardware Configurations
- 3.3. High-Throughput Cluster Configurations
- 3.4. High-Availability Cluster Configurations
- 3.5. High-Performance Cluster Configurations
- 3.6. Common Cluster Hardware Architecture
- 3.7. Cluster Hardware Architecture Summary
- 4. Any Way You Slice It: Work and Master Nodes in a Cluster
- 4.1. Criteria for Selecting Compute Slices
- 4.2. An Example Compute Slice from Hewlett-Packard
- 4.3. Thirty-two Bit and 64-Bit Compute Slices
- 4.4. Memory Bandwidth
- 4.5. Memory and Cache Latency
- 4.6. Number of Processors in a Compute Slice
- 4.7. I/O Interface Capacity and Performance
- 4.8. Compute Slice Operating System Support
- 4.9. Master Node Characteristics
- 4.10. Compute Slice and Master Node Summary
- 5. Packet In: Cluster Networking Basics and Example Devices
- 6. Tying It Together: Cluster Data, Management, and Control Networks
- 7. Life in the Fast LAN: HSIs and Your Cluster
- 3. Underneath the Hood: Cluster Hardware Components and Architecture
- III. Cluster Software Architecture
- 8. The Right Stuff: Linux as the Basis for Clusters
- 9. Round and Round It Goes: Booting, Disks, Partitioning, and Local File Systems
- 9.1. Disk Partitioning, Booting, and the BIOS
- 9.2. Booting the Linux Kernel
- 9.3. The Linux Initial RAM Disk Image
- 9.4. Linux Local Disk Storage
- 9.5. Linux File System Types
- 9.6. The Linux /proc and devfs Pseudo File Systems
- 9.7. The Linux ext2 and ext3 Physical File Systems
- 9.8. Standard Mount Options for All File Systems
- 9.9. The Temporary File System
- 9.10. Other Available File System Types
- 9.11. Advanced Performance Tuning
- 9.12. A Word about SMART Monitoring for Disks
- 9.13. Local Disks and File Systems Summary
- 10. Supporting Role: Infrastructure Services and Administration
- 10.1. The Big Infrastructure Picture
- 10.2. Initializing Your Cluster's Software Infrastructure
- 10.3. Infrastructure Implementation Recommendations
- 10.4. Protecting Active Configuration Information
- 10.5. Preparation for Infrastructure Installation
- 10.6. Networking
- 10.7. Enabling and Starting Linux Services
- 10.8. Time Synchronization
- 10.9. Name Services
- 10.10. Infrastructure Services Summary
- 11. Reach Out and Access Something: Remote Access Services, DHCP, and System Logging
- 11.1. Continuing Infrastructure Installation
- 11.2. “Traditional” User Login and Authentication
- 11.3. Remote Access Services
- 11.4. Using BSD Remote Access Services
- 11.5. Kerberized Versions of BSD/ARPA Remote Services
- 11.6. The Secure Shell
- 11.6.1. SSH and Public Key Encryption
- 11.6.2. Configuring the SSH Client and Server
- 11.6.3. Configuring User Identity for SSH
- 11.6.4. SSH Host Keys, and Known and Authorized Hosts
- 11.6.5. Using the Authorized Keys File
- 11.6.6. Fine-Tuning SSH Access
- 11.6.7. SSH scp and sftp Commands
- 11.6.8. SSH Forwarding
- 11.6.9. SSH Summary
- 11.7. The Parallel Distributed Shell
- 11.8. Configuring DHCP
- 11.9. Logging System Activity
- 11.10. Access and Logging Services Summary
- 12. Installment Plan: Introduction to Compute Slice Configuration and Installation
- 12.1. Compute Slice Configuration Considerations
- 12.2. One Thousand Pieces Flying in Close Formation
- 12.3. The Single-System View
- 12.4. A Generalized Network Boot Facility: pxelinux
- 12.5. Configuring Network kickstart
- 12.6. NFS Diskless Configuration
- 12.7. Introduction to Compute Slice Installation Summary
- 13. Improving Your Images: System Installation with SystemImager
- 13.1. Using the SystemImager Software
- 13.1.1. Downloading and Installing SI
- 13.1.2. Configuring the SI Server
- 13.1.3. The SI Cold Installation Boot Process
- 13.1.4. SI Server Commands
- 13.1.5. Installing and Configuring the SI Client Software
- 13.1.6. Capturing a Client Image
- 13.1.7. Forcing Hardware-to-Driver Mapping with SystemConfigurator
- 13.1.8. Installing a Client Image
- 13.1.9. Updating Client Software without Reinstalling
- 13.1.10. Image Management and Naming
- 13.1.11. Avoiding the Big MAC-Gathering Syndrome
- 13.1.12. Summary
- 13.2. Multicast Installation
- 13.2.1. Multicast Basics
- 13.2.2. An Open-Source Multicast Facility: udpcast
- 13.2.3. A Simple Multicast Example
- 13.2.4. A More Complex Example
- 13.2.5. Command-line Prototyping with Multicast
- 13.2.6. Prototyping a Network Multicast Installation
- 13.2.7. Making More Modifications
- 13.2.8. Generalizing the Multicast Installation Prototype
- 13.2.9. Triggering a Multicast Installation
- 13.3. The SI flamethrower Facility
- 13.4. System Installation with SI Summary
- 13.1. Using the SystemImager Software
- 14. To Protect and Serve: Providing Data to Your Cluster
- 14.1. Introduction to Cluster File Systems
- 14.2. The NFS
- 14.2.1. Enabling NFS on the Server
- 14.2.2. Adjusting NFS Mount Daemon Protocol Behavior
- 14.2.3. Tuning the NFS Server Network Parameters
- 14.2.4. NFS and TCP Wrappers
- 14.2.5. Exporting File Systems on the NFS Server
- 14.2.6. Starting the NFS Server Subsystem
- 14.2.7. NFS Client Mount Parameters
- 14.2.8. Using autofs on NFS Clients
- 14.2.9. NFS Summary
- 14.3. A Survey of Some Open-Source Parallel File Systems
- 14.4. Commercially Available Cluster File Systems
- 14.5. Cluster File System Summary
- 15. Stuck in the Middle: Cluster Middleware
- 15.1. Introduction to Cluster Middleware
- 15.2. The MPICH Library
- 15.3. The Simple Linux Utility for Resource Management
- 15.4. The Maui Scheduler
- 15.5. The Ganglia Distributed Monitoring and Execution System
- 15.5.1. The Ganglia Software Architecture
- 15.5.2. Introducing RRD Software: rrdtool
- 15.5.3. Downloading and Installing Ganglia Software
- 15.5.4. Ganglia's gmond and gmetad Daemons
- 15.5.5. Adding Your Own Ganglia Metrics
- 15.5.6. Parallel Authentication with authd and gexec
- 15.5.7. Starting Parallel Programs with gexec
- 15.5.8. Ganglia Summary
- 15.6. Monitoring with Nagios
- 15.7. Cluster Middleware Summary
- 15.8. An Afterword on Linux High-Availability and Open-Source
- 16. Put Tab A in Slot C: OSCAR, Rocks, OpenMOSIX, and the Globus Toolkit
- 16.1. Introducing Cluster-Building Toolkits
- 16.2. General Cluster Toolkit Installation Process
- 16.3. Installing a Cluster with OSCAR
- 16.3.1. OSCAR Initial Software Installation and Configuration
- 16.3.2. The OSCAR Installation Wizard
- 16.3.3. OSCAR Package Configuration
- 16.3.4. Building an OSCAR Compute Slice Image
- 16.3.5. Defining and Installing OSCAR Clients
- 16.3.6. Completing the OSCAR Installation
- 16.3.7. Adding and Deleting OSCAR Clients
- 16.3.8. OSCAR Summary
- 16.4. Installing a Cluster with NPACI Rocks
- 16.5. The OpenMOSIX Project
- 16.6. Introduction to the Grid Concept
- 16.7. The Globus Toolkit
- 16.8. Cluster-Building Toolkit Summary
- IV. Building and Deploying Your Cluster
- 17. Dollars and Sense: Cluster Economics
- 17.1. Initial Perceptions
- 17.2. Setting the Ground Rules
- 17.3. Cluster Cabling and Complexity
- 17.4. Eight-Compute Slice Cluster Hardware Costs
- 17.5. Sixteen-Compute Slice Cluster Hardware Costs
- 17.6. Thirty-two-Compute Slice Hardware Costs
- 17.7. Sixty-four-Compute Slice Hardware Costs
- 17.8. One Hundred Twenty-eight-Compute Slice Hardware Costs
- 17.9. The Land beyond 128 Compute Slices
- 17.10. Hardware Cost Trends and Analysis
- 17.11. Cluster Economics Summary
- 18. Racking Your Brains: Example Cluster Rack Assembly Steps
- 18.1. Examining the Cluster Assembly Process
- 18.2. Assembly Assumptions
- 18.3. Some “Rules of Thumb” for Physical Cluster Assembly
- 18.4. Detailed Cluster Assembly Steps
- 18.4.1. Physical Rack Assembly
- 18.4.2. Physical Management Rack Assembly
- 18.4.3. Physical Compute Rack Assembly
- 18.4.4. Physical Compute Rack System Installation
- 18.4.5. Physical Rack Final Assembly and Checkout
- 18.4.6. Individual System Checkout
- 18.4.7. Physical Rack Cleanup
- 18.4.8. Physical Rack Positioning
- 18.4.9. Interrack Configuration
- 18.4.10. Interrack Cabling
- 18.4.11. Final Cluster Hardware Assembly and Checkout
- 18.5. Learning from the Example Steps
- 18.6. Physical Assembly Conclusions
- 19. Getting Your Cluster Wired: An Example Cable-Labeling Scheme
- 20. Physical Constraints: Heat, Space, and Power
- A. Acronym List
- B. List of URLs and Software Sources
- B.1. Cluster Construction Tool Kits
- B.2. Cluster Design Tools
- B.3. Conversion Factors
- B.4. File Systems and Volume Management
- B.5. General Linux Software
- B.6. Grid Tool Kits and Software
- B.7. Hardware Vendors
- B.8. High-Availability Software
- B.9. High-Performance Graphics
- B.10. HSI Technologies
- B.11. Java Software for Linux
- B.12. Linux Distributions and Open-Source License Examples
- B.13. Monitoring and Event Generation Software and Dependencies
- B.14. Networking Software, Hardware, and Examples
- B.15. Open-Source Databases
- B.16. Parallel Applications and Development Tools
- B.17. Parallel Application Examples
- B.18. Performance Benchmarks and Lists
- B.19. Protocols and Messaging Libraries
- B.20. Resource Management, Parallel Execution, and Scheduling
- B.21. Security and Encryption
- B.22. System Installation and Management Tools
- 17. Dollars and Sense: Cluster Economics
- Glossary
- Bibliography
Library of Congress Cataloging-in-Publication Data
Lucke, Robert W. Building clustered Linux systems / Robert W. Lucke. p. cm. Includes bibliographical references and index. ISBN 0-13-144853-6 (pbk. : alk. paper) 1. Linux. 2. Operating systems (Computers) 3. Embedded computer systems--Programming. I. Title. QA76.76.O63L838 2004 005.4'32--dc22 2004014016
© 2005 by Hewlett-Packard Corp.Pearson Education, Inc.Publishing as Prentice Hall Professional Technical ReferenceUpper Saddle River, New Jersey 07458
Prentice Hall books are widely used by corporations and government agencies for training, marketing, and resale.
Prentice Hall PTR offers discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact: U.S. Corporate and Government Sales, 1-800-382-3419, [email protected]. For sales outside the U.S., please contact: International Sales, [email protected].
Other company and product names mentioned herein are the trademarks or registered trademarks of their respective owners.
All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher.
Printed in the United States of America
First Printing
Pearson Education Ltd.Pearson Education Australia PTY., LtdPearson Education Singapore, Pte. Ltd.Pearson Education Asia Ltd.Pearson Education Canada, Ltd.Pearson Educación de Mexico, S.A. de C.V.Pearson Education—JapanPearson Education Malaysia, Pte. Ltd
This book is dedicated to family and friends, whom I foolishly took for granted—until November 24, 2002.
Most important of all, this book is dedicated to my wonderful wife and best friend, Teri. Without her love, support, and understanding, I would not be where I am today. Thank you for coming into my life!
It is not his mistakes that define the man, but how he handles them.
— Robert W. Lucke
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.