5.3. Hardening the OS and NOS

Any network is only as strong as its weakest component. Sometimes, the most obvious components are overlooked, and it's your job as a security administrator to make certain that doesn't happen. You must ensure that the operating systems running on the workstations and on the network servers are as secure as they can be.

Hardening an operating system (OS) or network operating system (NOS) refers to the process of making the environment more secure from attacks and intruders. The following sections discuss hardening an OS and the methods of keeping it hardened as new threats emerge. They will also discuss some of the vulnerabilities of the more popular operating systems and what can be done to harden those OSs.

5.3.1. Configuring Network Protocols

Configuring an OS's network protocols properly is a major factor in hardening. PC systems today primarily use TCP/IP, but for the exam, you should pretend that they use three primary network protocols:

• NetBIOS Extended User Interface (NetBEUI)

• Transmission Control Protocol/Internet Protocol (TCP/IP)

• Internetwork Packet Exchange/Sequenced Packet Exchange (IPX/SPX)

Each of these protocols can transport the Microsoft native networking protocol Network Basic Input/Output System (NetBIOS) across networks. NetBIOS protocol–enabled systems periodically announce names, service types, and other information on the networks bound to them. NetBIOS is also used for programming interfaces and other purposes.

For several years, Microsoft has been suggesting that TCP/IP be the primary network protocol used in networks. The company is concentrating more effort in making this protocol secure.

In looking at the large picture, don't overlook the simple things. Applications such as Netscape, Internet Explorer, and Office are susceptible to exploitation. Make sure that all your applications are up to the current release level and that all security patches have been installed.

In the following sections, we'll look at how network protocols are configured, how they're installed, and how they operate in a PC environment.

5.3.1.1. Network Binding

Binding is the process of tying a network protocol to another network protocol or to a network interface card (NIC). In a Microsoft network, NetBIOS can be bound to any of the three protocols mentioned in the previous section.

For example, binding NetBIOS to TCP/IP encapsulates NetBIOS messages into TCP/IP packets. TCP/IP can then be used to send NetBIOS traffic across the network. This binding process is where you'll find the security vulnerability. The problem lies in the fact that NetBIOS information becomes encapsulated in TCP/IP packets, making them vulnerable to sniffing (listening in on network traffic). Figure 5.1 illustrates the process of network binding. If the TCP/IP packet is intercepted, critical systems information, including passwords, can be discovered.

Make sure your network protocols and adapters have the proper binding configurations. Don't bind NetBIOS to a protocol unless necessary. Figure 5.2 shows the network binding of a typical Windows XP system. When two computers, such as a server and a client, attempt to communicate with each other, they must first find a common language. They do so by trying different protocols based on the binding order. (Internet Protocol [TCP/IP] is the only default in the latest operating systems from Microsoft.) For that reason, the protocols most commonly used on the server/client should be at the top of the binding list.

5.3.1.2. NetBEUI

NetBEUI is a proprietary protocol developed by Microsoft for Windows networks. If your entire network is configured for NetBEUI, the network will be almost invulnerable to outside attack. This is the case because NetBEUI isn't routable, so you can't connect it to an outside network using a router.

NetBEUI is not available by default on Windows XP Professional or Microsoft Windows Vista.

Tools such as Network Neighborhood, Explorer, and file sharing use NetBIOS for communications. Virtually all internal networking functions operate properly if NetBEUI is used for internal networking. NetBEUI wasn't designed to provide any security capabilities, and its packets disclose a great deal about system configuration, services running, and other information that can be used to identify weaknesses in a system. NetBEUI, isn't however, intended for large networks and is less efficient than Internetwork Packet Exchange/Sequenced Packet Exchange (IPX/SPX) or TCP/IP in such an environment.

5.3.1.3. TCP/IP

TCP/IP is vulnerable to all the threats discussed in Chapter 2, "Identifying Potential Risks." If your system is connected to the Internet or other large-scale networks, the security of the system is tied to the vulnerability of the TCP/IP network protocol.

The current implementations of TCP/IP are relatively secure. Earlier versions of TCP/IP, as implemented by Microsoft, Novell, Apple, and other vendors, had a huge number of technical problems and security vulnerabilities. The security of the network, regardless of the manufacturer, is only as good as the implementation the manufacturer accomplishes.

5.3.1.4. IPX/SPX

IPX/SPX is an efficient, routable protocol that was originally designed for use with Novell NetWare systems. The routers in use today don't generally route IPX/SPX unless they're specifically configured to do so. NetBIOS can be bound to IPX/SPX, and it won't be vulnerable to external attack unless this protocol is routed.

5.3.2. Hardening Microsoft Windows Vista

Security is such a driving component of computing today that it was one of the catalysts behind the development of the most recent version of Microsoft's workstation product. A new feature in this operating system is the ability to apply parental controls to accounts. To do so, choose the Set Up Parental Controls for Any User applet from the Control Panel, choose the user you want to apply them to, and click the On, Enforce Current Settings radio button.

From this same applet, you can also choose the Windows Vista web filter and set a web restriction level or the Time limits settings and restrict hours that the computer can be used. The former also allows you to block file downloads as well as choose websites to allow/block, and is an example of an Internet content filter.

However, the Security applet, beneath the Control Panel, is the main interface for security features in Vista. From here, you can configure Windows Firewall, automatic scans of your computer, and Windows Defender. One of the newest security features that is available only in the Enterprise and Ultimate versions of Vista is Bitlocker. Bitlocker is a whole disk encryption feature that can encrypt an entire volume with 128-bit encryption. When the entire volume is encrypted, the data is not accessible to someone who might boot another operating system in an attempt to bypass the computer's security.

5.3.3. Hardening Microsoft Windows XP

Windows XP functioned as a replacement for both the Windows 9x family and Windows 2000 Professional. There are multiple versions of Windows XP, including the Home, Media Center, and Professional editions, but Microsoft is currently planning to discontinue supporting it in the near future in favor of Windows Vista.

The Windows XP Home edition was intended specifically to replace Windows 9x clients, while Media Center added entertaining options (such as a remote control for TV) and Windows XP Professional was designed for the corporate environment. Windows XP Professional has the ability to take advantage of the security possible from Windows 200x servers running Active Directory.

The service packs fix minor security openings within the operating system, and as of this writing, three such service packs have been released.

One of the best tools to use when looking for possible illicit activity on a workstation is Systems Monitor (also known as System Monitor). This utility can be used to examine activity on any counter, and excessive processor usage is one worth paying attention to if you suspect the workstation is affected or being illegitimately accessed. In previous versions of Windows-based operating systems, this utility was a standalone menu choice. With Windows XP, it became a subcomponent (a snap-in) in the Performance Console. To access it, choose Start Run and type perfmon.msc. By default, System Monitor comes up showing three counters: Pages/sec, Avg. Disk Queue Length, and % Processor Time. To add more counters, right click in the right pane and choose Add Counters from the popup menu.

5.3.4. Hardening Windows Server 2003

Windows Server 2003 was released in four variants:

• Web edition

• Standard edition

• Enterprise edition

• Datacenter edition

This product introduced the following features to the Microsoft server line:

• Internet connection firewall

• Secure authentication (locally and remotely)

• Secure wireless connections

• Software restriction policies

• Secure Web Server (IIS 6)

• Encryption and cryptography enhancements

• Improved security in VPN connections

• PKI and X.509 certificate support

In short, the goal was to make a product that is both secure and flexible. Since a server is only as secure as the workstations connected to it, one of the best ways to keep the network safe is by utilizing Group Policies and regularly reviewing them. Group Policies replace System Policies, which existed in Microsoft operating systems prior to Windows 2000.

With a Group Policy, you create restrictions that will apply to workstations when users authenticate. Upon each authentication, those restrictions are then applied as Registry settings, providing an efficient way to manage a large number of computers.

The restrictions you set come from choices within template files and can be as simple as not allowing the user to access Solitaire, to removing their ability to access the other networks. Security templates are those template files that hold Registry setting choices that relate to security settings.

Not every configuration setting needs to be downloaded through the domain. Every current Microsoft operating system also includes local policies—settings that apply to the workstation when the user has yet to authenticate with the network. The purpose of these policies are to restrict the user locally, just as you would across the network, when they have yet to log into the domain. To see the local polices, choose Start Run and type secpol.msc.

You should routinely monitor the settings made throughout your network in local and Group Policies and tweak them as needed.

5.3.5. Hardening Microsoft Windows 2000

Windows 2000 entered the market at the millennium. It includes workstation and several server versions. The market has embraced these products, and they offer reasonable security when updated. Windows 2000 provides a Windows Update icon on the Start menu; this icon allows you to connect to the Microsoft website and automatically download and install updates. A large number of security updates are available for Windows 2000—make sure they're applied.

In the Windows environment, the Services manager or applet is one of the primary methods (along with policies) used to disable a service.

The server and workstation products operate in a manner similar to Windows NT 4. These products run into the most security-related problems when they're bundled with services that Microsoft has included with them. Some of the more attack-prone services include Internet Information Server (IIS), File Transfer Protocol (FTP), and other common web technologies. Make sure these services are disabled if they aren't needed, and keep them up-to-date with the most recent security and service packs.

Many security updates have been issued for Windows 2000. The Microsoft TechNet and Security websites provide tools, white papers, and materials to help secure Windows 2000 systems.

Windows 2000 includes extensive system logging, reporting, and monitoring tools. They help make the job of monitoring security fairly easy. In addition, Windows 2000 provides a great deal of flexibility in managing groups of users, security attributes, and access control to the environment.

The Event Viewer is the major tool for reviewing logs in Windows 2000. Figure 5.3 shows an example Event Viewer. Using Event Viewer, an administrator can log a number of different types of events and configure the level of events that are logged.

Another important security tool is Performance Monitor. As an administrator of a Windows 2000 network, you must know how to use Performance Monitor. This tool can be a lifesaver when you're troubleshooting problems and looking for resource-related issues.

Windows 2000 servers can run a technology called Active Directory (AD), which lets you control security configuration options of Windows 2000 systems in a network. Unfortunately, the full power of AD doesn't work unless all the systems in the network are running Windows 2000 or higher.

5.3.6. Hardening Unix/Linux

The Unix environment and its derivatives are some of the most-installed server products in the history of the computer industry. Over a dozen different versions of Unix are available; the most popular is a free version derivative called Linux.

Unix was created in the 1970s. The product designers took an open-systems approach, meaning that the entire source code for the operating system was readily available for most versions. This open-source philosophy has allowed tens of thousands of programmers, computer scientists, and systems developers to tinker with and improve the product.

Linux and Unix, when properly configured, provide a high level of security. The major challenge with the Unix environment is configuring it properly.

Unix includes the capacity to handle and run almost every protocol, service, and capability designed. You should turn off most of the services when they aren't needed by running a script during system startup. The script will configure the protocols, and it will determine which services are started.

All Unix security is handled at the file level. Files and directories need to be established properly to ensure correct access permissions. The file structure is hierarchical by nature, and when a file folder access level is set, all subordinate file folders usually inherit this access. This inheritance of security is established by the systems administrator or by a user who knows how to adjust directory permissions.

Keeping patches and updates current is essential in the Unix environment. You can accomplish this by regularly visiting the developer's website for the version/flavor you're using and downloading the latest fixes.

Linux also provides a great deal of activity logging. These logs are essential in establishing patterns of intrusion.

An additional method of securing Linux systems is accomplished by adding TCP wrappers, which are low-level logging packages designed for Unix systems. Wrappers provide additional detailed logging on activity using a specific protocol. Each protocol or port must have a wrapper installed for it. The wrappers then record activities and deny access to the service or server.

As an administrator of a Unix or Linux network, you're confronted with a large number of configuration files and variables that you must work with in order to keep all hosts communicating properly.

5.3.7. Hardening Novell NetWare

Novell was one of the first companies to introduce a network operating system (NOS) for desktop computers, called NetWare. Early versions of NetWare provided the ability to connect PCs into primitive but effective LANs. The most recent version of NetWare, version 6.5, includes file sharing, print sharing, support for most clients, and fairly tight security.

NetWare functions as a server product. The server has its own NOS. The NetWare software also includes client applications for a number of different types of systems, including Macintoshes and PCs. You can extend the server services by adding NetWare Loadable Modules (NLMs) to the server. These modules allow executable code to be patched or inserted into the OS.

NetWare version 6.x is primarily susceptible to denial of service (DoS) types of attacks, as opposed to exploitation and other attacks. NetWare security is accomplished through a combination of access controls, user rights, security rights, and authentication.

A number of additional capabilities make NetWare a product worth evaluating in implementation. These include e-commerce products, document retrieval, and enhanced network printing.

Prior to version 5, NetWare defaulted to the proprietary IPX/SPX protocol for networking. All newer versions of NetWare default to TCP/IP.

5.3.8. Hardening Apple Macintosh

Macintosh systems seem to be the most vulnerable to physical access attacks targeted through the console. The network implementations are as secure as any of the other systems discussed in this chapter.

Macintosh security breaks down in its access control and authentication systems. Macintosh uses a simple 32-bit password encryption scheme that is relatively easy to crack. The password file is located in the Preference folder; if this file is shared or is part of a network share, it may be vulnerable to decryption.

Macintosh systems also have several proprietary network protocols that aren't intended for routing. Recently, Macintosh systems have implemented TCP/IP networking as an integral part of the operating system.

To secure the system, you should verify that it is not configured to automatically log in a user at startup. Every system must require a username and password in order to gain access to the Mac itself, as well as to the network.

You should also configure a screensaver to automatically deploy after a few moments of inactivity. The screensaver can be any that you want to use as long as it requires a password to resume the session.

5.3.9. Hardening Filesystems

Several filesystems are involved in the operating systems we've discussed, and they have a high level of interoperability between them—from a network perspective, that is. Through the years, the different vendors have implemented their own sets of file standards. Some of the more common filesystems are listed here:

Microsoft FAT

Microsoft's earliest filesystem was referred to as File Allocation Table (FAT). FAT is designed for relatively small disk drives. It was upgraded first to FAT-16 and finally to FAT-32. FAT-32 allows large disk systems to be used on Windows systems. FAT allows only two types of protection: share-level and user-level access privileges. If a user has write or change access to a drive or directory, they have access to any file in that directory. This is very unsecure in an Internet environment.

Microsoft NTFS

The New Technology File System (NTFS) was introduced with Windows NT to address security problems. Before Windows NT was released, it had become apparent to Microsoft that a new filing system was needed to handle growing disk sizes, security concerns, and the need for more stability. NTFS was created to address those issues.

Although FAT was relatively stable if the systems that were controlling it kept running, it didn't do so well when the power went out or the system crashed unexpectedly. One of the benefits of NTFS was a transaction-tracking system, which made it possible for Windows NT to back out of any disk operations that were in progress when Windows NT crashed or lost power.

With NTFS, files, directories, and volumes can each have their own security. NTFS's security is flexible and built in. Not only does NTFS track security in access control lists (ACLs), which can hold permissions for local users and groups, but each entry in the ACL can specify what type of access is given—such as Read-Only, Change, or Full Control. This allows a great deal of flexibility in setting up a network. In addition, special file-encryption programs were developed to encrypt data while it was stored on the hard disk.

Microsoft strongly recommends that all network shares be established using NTFS.

Novell NetWare Storage Services

Novell, like Microsoft, implemented a proprietary file structure. Novell's is called NetWare File System. This system allows complete control of every file resource on a NetWare server. The NetWare File System was upgraded to NetWare Storage Service (NSS) in version 6. NSS provides higher performance and larger file storage capacities than the NetWare File System. NSS, like its predecessor, uses the NDS or eDirectory to provide authentication for all access.

Unix Filesystem

The Unix filesystem is a completely hierarchical filesystem. Each file, filesystem, and subdirectory has complete granularity of access control. The three primary attributes in a Unix file or directory are Read, Write, and Execute. The ability to individually create these capabilities, as well as to establish inheritance to subdirectories, gives Unix the highest level of security available for commercial systems. The major difficulty with Unix is that establishing these access-control hierarchies can be time consuming when the system is initially configured. Figure 5.4 illustrates this hierarchical file structure. Most current operating systems have embraced this method of file organization.

Figure 5.4. Hierarchical file structure used in Unix and other operating systems

Unix Network Filesystems

Network File System (NFS) is a Unix protocol that allows systems to mount filesystems from remote locations. This ability allows a client system to view the server or remote desktop storage as a part of the local client. NFS, while functional, is difficult to secure. The discussion of this protocol is beyond the scope of this book; the major issue lies in Unix's inherent trust of authentication processes. NFS was originally implemented by Sun Microsystems, and it has become a standard protocol in Unix environments.

Apple File Sharing

Apple File Sharing (AFS) was intended to provide simple networking for Apple Macintosh systems. This system used a proprietary network protocol called AppleTalk. An AppleTalk network isn't routed through the Internet and isn't considered secure. AFS allows the file owner to establish password and access privileges. This process is similar to the Unix filesystem. OS X, the newest version of the Macintosh operating system, has more fully implemented a filesystem that is based on the Unix model. In general, Apple networking is considered as secure as the other implementations discussed in the section. The major weakness of the operating system involves physical control of the systems.

Each of these filesystem implementations requires careful consideration when you're implementing it in a network. You must evaluate their individual capabilities, limitations, and vulnerabilities when you're choosing which protocols or systems to implement.

Most OS providers support multiple protocols and methods. Turn off any protocols that aren't needed because each protocol or filesystem running on a workstation or server increases your vulnerability and exposure to attack, data loss, or DoS attacks.

Make sure you periodically review the manufacturers' support websites and other support resources that are available to apply current updates and security patches to your systems. Doing this on a regular basis will lower your exposure to security risks.

5.3.10. Updating Your Operating System

Operating system manufacturers typically provide product updates. For example, Microsoft provides a series of regular updates for Windows (a proprietary system) and other applications. However, in the case of public source systems (such as Linux), the updates may come from a newsgroup, the manufacturer of the version you're using, or a user community.

In both cases, public and private, updates help keep operating systems up to the most current revision level. Researching updates is important; when possible, so is getting feedback from other users before you install an update. In a number of cases, a service pack or update has rendered a system unusable. Make sure your system is backed up before you install updates.

Three different types of updates are discussed here: hotfixes, service packs, and patches.

5.3.10.1. Hotfixes

Hotfixes are used to make repairs to a system during normal operation, even though they might require a reboot. A hotfix may entail moving data from a bad spot on the disk and remapping the data to a new sector. Doing so prevents data loss and loss of service. This type of repair may also involve reallocating a block of memory if, for example, a memory problem occurred. This allows the system to continue normal operations until a permanent repair can be made. Microsoft refers to a bug fix as a hotfix. It involves the replacement of files with an updated version.

5.3.10.2. Service Packs and Support Packs

A service pack or support pack (depending upon the vendor) is a comprehensive set of fixes consolidated into a single product. A service pack may be used to address a large number of bugs or to introduce new capabilities in an OS. When installed, a service pack usually contains a number of file replacements.

Make sure you check related websites to verify that the service pack works properly. Sometimes a manufacturer will release a service pack before it has been thoroughly tested. An untested service pack can cause extreme instability in an operating system or, even worse, render it inoperable.

5.3.10.3. Patches

A patch is a temporary or quick fix to a program. Patches may be used to temporarily bypass a set of instructions that have malfunctioned. Several OS manufacturers issue patches that can either be manually applied or applied using a disk file to fix a program.

When you're working with customer support on a technical problem with an OS or applications product, customer service may have you go into the code and make alterations to the binary files that run on your system. Double-check each change to prevent catastrophic failures due to improperly entered code.

When more data is known about the problem, a service pack or hotfix may be issued to fix the problem on a larger scale. Patching is becoming less common, but it's still very much a way of life for many vendors and administrators.