As you probably know by now, eDirectory is an extremely complex environment. Fortunately, it is largely self-sufficient. Most of the day-to-day tasks of maintaining and protecting directory data are handled automatically and transparently. Not only does eDirectory have many built-in integrity features, but it also employs several background processes that keep the directory environment stable and healthy.
This section provides a look at the main background processes that do all the heavy lifting associated with eDirectory operations:
Database initialization
Flat cleaner
Janitor
Replica sync
Replica purger
Limber
Backlinker
Schema sync
Time sync
When you use the various eDirectory monitoring and repair tools, some of which were discussed in Chapter 5, “Novell eDirectory,” and more are discussed later in this appendix, these background processes and their effects are what you monitor and repair. For this reason, it’s a good idea to explore the possibilities.
The Database Initialization (DB Init) background process is automatically initiated whenever the file system is mounted on the eDirectory server. It also executes any time the eDirectory database is opened or when eDirectory is reloaded. DB Init is responsible for
Verifying the usability of the eDirectory database files on this server
Scheduling the running of other eDirectory background processes
Initializing the various global variables and data structures used by eDirectory
Opening the eDirectory database files for use by the version of eDirectory running on this server
DSTrace provides the capability to monitor the DB Init process directly.
The Flat Cleaner background process is used to eliminate eDirectory variables and attributes that are no longer needed by the database. Flat Cleaner is responsible for
Eliminating unused bindery and external reference (X-ref) objects and/or attributes.
Making sure that each of the objects in a partition replica maintained on this server has a valid public key attribute.
Eliminating X-ref obituaries that have been set as “purgeable.”
Making sure that the server objects in partition replicas hosted on this server have maintained accurate Status and Version attributes. The server object maintains an attribute that specifies server status—up, down, initializing, and so on. It also keeps a record of the version of eDirectory running on that server.
Flat Cleaner can be indirectly monitored through the use of Check External References in DSRepair. DSTrace also provides the capability to monitor the Janitor process directly.
As its name implies, the Janitor process is responsible for routine cleanup of eDirectory environment. Janitor is responsible for
Monitoring the value of the NCP status attribute maintained in the eDirectory Server object for this server.
Keeping track of the [Root]-most partition replica on the server and the overall replica depth of the server. The [Root]-most partition is the partition root object highest in the tree (closest to [Root]). Replica depth describes how many levels down from [Root] the highest partition replica hosted by that server is located.
Executing the Flat Cleaner process at regular intervals.
Optimizing the eDirectory database at regular intervals.
Reporting synthetic time use by a partition replica on the server. Synthetic time occurs when a server clock set to a future time is reset to the correct time. Any eDirectory changes made while the clock was set at the future time will bear incorrect timestamps. This problem will self-correct as long as the gap between current and synthetic time is not too large.
Making sure the inherited rights for each partition root object on this server are properly maintained.
Like Flat Cleaner, Janitor can be monitored indirectly by examining the Replica Ring repair options, Time Synchronization status, and Replica Synchronization status operations with DSRepair. DSTrace also provides the ability to monitor the Janitor process directly.
The Replica Synchronization background process is responsible for two primary tasks:
Distributing modifications to eDirectory objects contained within partition replicas maintained by the eDirectory server
Receiving and processing partition operations involving partition replicas hosted by the eDirectory server
DSRepair can report the status of the replica synchronization process from a number of different perspectives:
Report synchronization status of all servers
Report synchronization status on the selected server
DSTrace also provides the ability to monitor the Replica Synchronization process directly.
Replica Sync schedules the execution of the Replica Purger background process. It is responsible for
Purging any unused objects and/or attributes that exist in eDirectory partition replicas hosted on this server
Processing obituaries for objects maintained within partition replicas hosted on this server
DSTrace also provides the ability to monitor the Replica Purger process directly, commonly referred to as Skulker.
After questioning several sources, it is still unclear why this process is named Limber, so that will remain a mystery for now. However, naming issues aside, Limber is responsible for
Making sure that the eDirectory referral information for this server is properly maintained in each partition hosted on this server.
Making sure that the server hosting the Master replica of the partition in which the server object for this server resides has the correct Relative Distinguished Name (RDN) for this server. The RDN identifies a target eDirectory object’s context in relation to the context of the source eDirectory object. For example, the admin object in O = Quills would receive the following RDN for CN = jharris.OU = Education.OU = Provo.O = Quills:
jharris.Education.Provo
The O = Quills is assumed from the location of the admin object itself.
Making sure the server object in eDirectory correctly reflects the operating system version and network address in use on this server.
Making sure the name of the eDirectory tree in which this server resides is correctly reported.
Monitoring the external reference/DRL links between this server and the partition replica that holds this server’s eDirectory server object. This is done to make sure that the eDirectory server can be properly accessed via its eDirectory object.
Making sure this server’s identification information is correct.
Limber can be monitored indirectly through Check External Reference, Report Synchronization Status, and Replica Ring repair options in DSRepair. DSTrace also provides the ability to monitor Limber directly.
The Backlinker background process helps maintain referential integrity within the eDirectory environment. Backlinker is responsible for
Making sure that all external references (X-refs) maintained by this server are still required.
Making sure that each X-ref is properly backlinked to a server that hosts a partition replica holding the eDirectory object specified in the X-ref.
Eliminating X-refs that are no longer necessary. As part of doing this, the server hosting the partition replica that holds the referenced eDirectory object is notified of the elimination of the X-ref.
Backlinker can be monitored indirectly through Check External References in DSRepair. DSTrace also provides the ability to monitor Backlinker directly.
The schema sync background process is responsible for synchronizing the schema updates received by this server with other eDirectory servers. DSTrace also provides the ability to monitor Schema Synchronization directly.
Although time sync is not an eDirectory process, it is necessary to perform some partition operations such as moves and merges. The underlying time sync mechanism is not important as long as the eDirectory servers are, in fact, synchronized. Time sync can be monitored directly through the Time Synchronization option in DSRepair.
Now that you have been introduced to the most common eDirectory processes, it’s important that you know how to keep track of the health and general operation of those processes. To do this you can use iMonitor. iMonitor is presented in Chapter 4, “OES Management Tools,” as one of the principal management tools for OES NetWare. However, this section focuses on the iMonitor options for monitoring eDirectory processes and activities. Refer to Chapter 4 for information on iMonitor installation, general interface, and additional capabilities. For detailed feature information, see the OESOES NetWare on-line documentation.
iMonitor is a web-based replacement for several of the console-based management utilities used with previous versions of NetWare, including DSBrowse, DSTrace, and DSDiag. Because the eDirectory processes discussed previously run on each eDirectory server, iMonitor provides a server-level view of eDirectory health as opposed to a tree-level view. You can view the health of processes running only on the server from which you are running iMonitor. To view another server, launch iMonitor from that server.
Prior to using DSTrace from iMonitor, you must configure the utility and specify the activity that you want to monitor. This is accomplished from the Trace Configuration page, shown in Figure B.1.
When you go into Trace Configuration, you will see four new links in the left navigation frame:
Trace Configuration—This is the default view you see when entering the Trace Configuration page. From this page, you can define the server-based eDirectory events and processes that you want to trace. The following configuration options are available from this page:
Trace On/Off—Enables/disables DSTrace monitoring. When DSTrace is enabled, you see a Trace button (big lightning bolt) in iMonitor’s header frame that you can use to view the active trace (see Figure B.2).
Note
DSTrace can increase CPU utilization significantly and reduce performance, so you should trace only when you are actively looking for something, and not as a standard practice.
Update—Applies new configuration options to an existing trace.
Trace Line Prefixes—These options enable you to specify what type of descriptive information to include with each trace line. These prefixes enable you to identify event sequence, group related messages, and determine how long ago a problem occurred. This can be critical when analyzing DSTrace data, particularly historical trace data.
DSTrace Options—Specifies the eDirectory activities that you want to trace for this particular eDirectory server. To control the amount of data that you have to sift through in DSTrace, it’s best to restrict tracing to only those specific events that are of interest instead of tracing everything. Table B.1 provides a brief description of many of the common trace options.
Table B.1. Common DSTrace Options
|
OPTION NAME |
DESCRIPTION |
|---|---|
|
Allocated Memory |
Trace messages related to allocation of memory for eDirectory processes. |
|
Audit |
Trace messages related to the eDirectory audit process. |
|
Audit NCP |
Trace Audit NCP (NetWare Core Protocol) events. |
|
Audit Skulk |
Trace audit messages related to the replica sync process. |
|
Authentication |
Trace messages related to eDirectory authentication events. |
|
Backlinker |
Trace messages related to the Backlink process. |
|
Buffers |
Trace messages related to allocation of inbound and outbound packet buffers related to eDirectory requests. |
|
Trace messages related to the changing of the eDirectory memory cache. |
|
|
Client Buffers |
Events related to memory buffers maintained for client connections. |
|
Collisions |
Trace messages related to the receipt of duplicate update packets. These duplicate packets usually occur on very busy networks. |
|
Distributed References |
Trace messages related to Distributed Reference Link operations. |
|
DNS |
Trace messages related to Domain Name Service requests. |
|
DS Agent |
Trace messages related to general eDirectory agent activities on this server. |
|
Emulated Bindery |
Trace messages related to Bindery Emulation. |
|
Fragmented Requests |
Trace messages related to the packet fragmenter that breaks up eDirectory messages for transmission in multiple packets. |
|
Inbound Synchronization |
Trace messages related to incoming eDirectory synchronization requests. |
|
Initialization |
Trace messages related to the opening of the local eDirectory database. |
|
Inspector |
Messages related to the Inspector process. Inspector is part of the Janitor that verifies the structural integrity of the eDirectory database. |
|
Janitor |
Trace Janitor messages. The Janitor cleans up eDirectory by removing objects that are no longer needed. |
|
LDAP |
Trace messages related to LDAP communications. |
|
LDAP Stack |
Trace messages related to the memory stack associated with LDAP operations. |
|
Limber |
Trace Limber messages. The Limber monitors connectivity between all replicas. |
|
Locking |
Trace messages related to manipulation of the local eDirectory database locks. |
|
Lost Entry |
Trace messages related to obituaries, eDirectory attributes, and stream files. |
|
Move Object |
Trace messages related to eDirectory object move operations. |
|
NCP Engine |
Trace messages related to the NCP engine. |
|
Trace messages related to background replica synchronization. |
|
|
Partition |
Trace partition operations and messages. |
|
Purge |
Trace replica purger messages. |
|
Resolve Name |
Trace messages related to eDirectory name resolution when traversing the eDirectory tree. |
|
SAP |
Trace messages related to the SAP protocol. |
|
Schema |
Trace schema modification and synchronization messages. |
|
Server Packets |
Trace messages related to server packets. |
|
Streams |
Trace messages related to stream attributes in eDirectory. |
|
Thread Scheduling |
Trace messages related to the management of processor threads used with eDirectory. |
|
Time Vectors |
Trace messages related to transitive vectors, which describe how caught up the replica is in the synchronization process. |
|
Wanman |
Trace messages related to WAN Traffic Manager. |
Event Configuration—This link provides a view similar to Trace Configuration, but it enables you to select eDirectory events that you want to trace. eDirectory events include such things as adding/deleting objects, modifying attributes, and changing a password. The same configuration options described for Trace Configuration are available for Event Configuration, except that instead of listing DSTrace options, DS Events are listed.
Trace History—From this page you can view a list of previous traces. A timestamp indicating the period of time during which the trace was gathered identifies each trace.
Trace Triggers—This page lists some common DS Agent activities and identifies the DSTrace options that must be selected to trace that type of DS Agent activity. Selecting trigger options and clicking Submit will add those DSTrace options to the list of active DSTrace options, if they are not already active.
Every database needs a tool for repairing inconsistencies when they occur. DSRepair has been serving in this capacity as long as eDirectory has existed. Even though Novell is shifting its focus toward web-based tools, DSRepair is still essential for working on your eDirectory on a day-to-day basis. DSRepair offers three main groups of features:
Unattended full repair
eDirectory monitor operations
eDirectory repair operations
To load DSRepair on your OES NetWare server, load DSREPAIR.NLM at the server console. Throughout this section, when describing steps for performing a DSRepair operation, it is assumed that you have already loaded the utility. Figure B.3 shows the main menu of DSRepair.
All DSRepair operations and results can be logged to file for review. The default log file is SYS:SYSTEMDSREPAIR.LOG. DSRepair on NetWare has a menu for configuring the log file. To access this menu, select the Advanced Options Menu and then select Log File and Login Configuration.
The following three sections describe the features available in each of the three main categories of DSRepair operations—Unattended Full Repair, eDirectory Monitor Operations, and eDirectory Repair Operations.
Unattended Full Repair (UFR) is probably the most-used feature in DSRepair—although the huge database sizes now being supported by eDirectory might change that. UFR checks for and repairs most noncritical eDirectory errors in the eDirectory database files of a given server. UFR is activated by selecting Unattended Full Repair from the main DSREPAIR menu.
UFR performs eight primary operations each time it is run, none of which requires any intervention by the administrator. These operations are described in Table B.2. During some of these operations, the local database is locked. UFR builds a temporary set of local database files and runs the repair operations against those files. That way, if a serious problem develops, the original files are still intact. When complete, a UFR log file will be generated that outlines all the activities that have gone on, errors that were encountered, and other useful information in reviewing the state of your eDirectory environment.
Table B.2. Operations Performed by Unattended Full Repair
Tip
Rebuilding the operational indexes used by eDirectory is possible only when the local database is locked. Given this, it is a good idea to schedule a locked database repair on a regular basis, even in large eDirectory environments.
After validating the volume information, the list of trustee IDs is validated. Each object in eDirectory has a unique trustee ID. This ID is used to grant rights to other objects, including NetWare volumes, in the eDirectory tree. This task makes sure that each trustee ID in the volume list is a valid eDirectory object. If not, the trustee ID is removed from the volume list.
Warning
When the local database is locked, no changes are permitted while the operations execute. Some of these operations, when performed on very large eDirectory databases, will take an extended period of time to complete. When working with a large eDirectory database, it is best to schedule these types of operations carefully so as not to disrupt network operations.
DSRepair offers several partition, replica, and server operations that are available to monitor the health of the eDirectory environment. These operations can be performed individually or as groups to help keep eDirectory stable and healthy.
Some of the DSRepair operations described in this section are available only when DSRepair is loaded in advanced mode. Type the following at the server console:
DSREPAIR -a
The first category of operations can be loosely grouped into monitor operations that are designed to report eDirectory status and health. You will likely perform most of these tasks from iMonitor with OES NetWare, but they are still available from DSRepair. Table B.3 describes the monitor operations available with DSRepair.
Table B.3. DSRepair Monitor Operations on NetWare
Although monitoring the condition of the eDirectory database is important, it does little good if there are no tools for repairing inconsistencies when they occur. DSRepair offers several eDirectory repair operations. These repair operations can be organized into three categories:
Database repair operations
Partition and replica repair operations
Other repair operations
All database repair options are accessible from the same menu in DSRepair, as shown in Figure B.4. Access these operations by selecting Advanced Options Menu and then choosing Repair Local DS Database.
Table B.4 describes the repair operations available from this menu.
Table B.4. DSRepair Database Operations on NetWare
In addition to these database repair options, DSRepair offers a menu of partition and replica operations designed to keep the distributed eDirectory environment functioning properly. This changes the focus from the local database to the partition—and all the replicas of that partition stored on servers across the network. To access these operations, select Advanced Options, Replica and Partition Options, and then select the partition with which you want to work, as shown in Figure B.5.
Table B.5 describes the various partition and replica operations available.
Table B.5. DSRepair Partition and Replica Operations
|
OPERATION |
DESCRIPTION |
|---|---|
|
Sync replica on all servers |
Each server holding a replica of the selected partition is contacted and then a synchronization cycle is initiated. |
|
Repair all replicas |
This operation resolves eDirectory database inconsistencies by checking each object and attribute against schema definitions. It also checks the format of all internal data structures. |
|
Repair selected replica |
Performs a replica repair on the selected replica only. |
|
Repair ring—selected replica |
Performs a replica repair operation on each server that hosts a replica of the selected partition. |
|
Repair ring—all replicas |
Performs the replica ring repair operation for each replica ring in which this server participates. |
|
Schedule immediate sync |
Initiates a replica synchronization cycle for each partition with a replica hosted on this server. This is useful for forcing the recognition of recent database changes. |
|
Designate this server as new Master replica |
If the Master replica of a given partition is lost due to hardware failure, this operation can be used to designate a new Master for partition operations to function normally. |
Three other replica operations are available by doing the following:
1. Select Advanced Options Menu and then select Replica And Partition Operations.
2. Select a replica and then choose View Replica Ring.
3. Select a server from the list.
These three operations are described in Table B.6.
Table B.6. DSRepair Replica Ring Operations on NetWare
|
OPERATION |
DESCRIPTION |
|---|---|
|
Synchronize the replica on the selected server |
Reports the synchronization status of the selected partition’s replica that is hosted on this server. |
|
Send all objects to every replica in the ring |
Rebuilds every replica in the ring according to the objects found in this server’s replica. Warning—Any changes made to other replicas that have not yet updated to this server will be lost. |
|
Receive all objects for this replica |
Rebuilds the local replica from object information received from the Master replica. Warning—Any changes made to this replica that have not yet updated to the Master replica will be lost. |
Finally, there are four miscellaneous repair operations that are accessible from other areas of the DSRepair utility. Table B.7 describes these operations.
Table B.7. Other DSRepair Operations
By using the operations described in this section, you will be able to manage most noncatastrophic problems in your eDirectory environment.
Novell recommends that some DSRepair operations be performed on a regular basis to keep the eDirectory tree healthy. To facilitate this, Novell has also made DSRepair functionality available through command-line switches. These switches make it possible to use batch schedulers to perform regular eDirectory tree maintenance automatically without any input from the administrator. Table B.8 describes the various command-line switches that are provided for automating basic DSRepair tasks.
Table B.8. Basic Command-Line DSRepair Switches
|
SWITCH |
PARAMETER |
DESCRIPTION |
|---|---|---|
|
|
Filename(with path) |
Specifies location for |
|
|
None |
Performs Unattended Full Repair and automatically unloads when complete. |
|
|
None |
Creates an eDirectory dump file. This file is a snapshot of the local eDirectory database that can be used for troubleshooting. The dump file is stored as |
|
None |
Repairs Local Database. Executes using default database repair options, which include Structure and Index Check, Rebuild Database, Tree Structure Check, Repair All Local Replicas, Validate Mail/Stream Files, and Check Local References. |
|
|
|
None |
Repairs Network Addresses. |
|
|
None |
Performs Volume Object Repair. |
|
|
Volume Name |
Performs Volume Object Repair and Trustee Check. |
In addition to the numerous functions described in the previous sections, DSRepair also has some advanced features that are hidden from normal use. These advanced features are enabled through switches when loading the DSRepair utility. Table B.9 provides an overview of the advanced functionality available on each platform.
Table B.9. Advanced DSRepair Features
|
SWITCH |
DESCRIPTION |
|---|---|
|
|
Removes all “move inhibit” obituaries. |
|
|
Limits the number of days a user object can be connected to a given server to the number specified as a command-line argument. When this number is reached, the user connection is terminated. The default value is 60 days. |
|
|
Removes the server identified by the Partition Root ID provided as a command-line argument from the replica ring for that partition. This might be necessary if a Read/Write replica becomes corrupt and needs to be eliminated. |
|
Loads
This switch also allows the Designate This Server as the New Master Replica option to assign a Subordinate Reference as the new Master replica. Be careful!! If you select the View Replica Ring option and select a replica, you will see an additional option on that menu as well:
|
|
|
|
Kills all eDirectory objects in this server’s eDirectory database. This operation is used only to destroy a corrupt replica that cannot be removed in any other way. |
|
|
Kills all external references in this server’s eDirectory database. This operation is used to destroy all external references in a nonfunctioning replica. If the references are the source of the problem, eDirectory can then be reinstalled. |
Warning
The features described in this section can—and will—cause irreversible damage to your eDirectory tree if used improperly. These features should be used only under the guidance of eDirectory professionals, such as the Novell Technical Services team, to resolve serious database issues. Always make a full backup of the eDirectory tree before using any of these features on a production tree. If you are going to use these features be sure you understand all the consequences before proceeding.
These advanced options are seldom used because they are needed for only the most serious of cases. However, it is nice to know they exist when you get into a jam.
There are a wide variety of error codes and conditions that can be reported in your Novell eDirectory environment. Specific information on each error is available in the Novell online documentation. You can also link to error code information from iMonitor by clicking an error from the Trace screen. eDirectory error codes are usually displayed in decimal numbers.
Note
Because the eDirectory is designed as a loosely consistent database, temporary errors are normal. Don’t be alarmed if temporary error conditions come and go as part of normal eDirectory operation. However, if errors persist for a significant period of time, you might need to take some action to resolve the problem.
eDirectory error codes can be categorized as shown in the following subsections.
These are the error codes with which you will typically work when tackling some eDirectory problem. They come in two ranges:
–601 to –799
–6001 or higher
The 6001 range is new to recent versions of eDirectory. These error codes identify errors originating in the eDirectory Agent running on your OES NetWareOES NetWare server.
Certain eDirectory background processes or operations, such as network communications or time synchronization, require the use or functionality provided by the operating system on which eDirectory is running. These functions can return operating system–specific error codes to eDirectory. These error codes are passed on to the eDirectory process or operation that initiated a request.
Generally, negative numbers identify all eDirectory-generated operating system errors, whereas positive numbers identify all other operating system errors:
–1 to –256—eDirectory-generated operating system errors
1 to 255—Operating system–generated errors
This is an esoteric distinction for your information only. During troubleshooting, you should treat occurrences of operating system errors with the same number, whether negative or positive, as relating to the same event.
In some cases, an eDirectory server will function as a directory client to perform certain background processes or operations. This can result in client-specific error codes being returned to eDirectory background processes and operations. The eDirectory client that is built into DS.NLM generates these error codes. Client error codes fall in the range of –301 through –399.
Some eDirectory background processes and operations require interaction with other NLMs running on the OES NetWare server. Examples of this include TIMESYNC.NLM and UNICODE.NLM. If any of these external NLMs encounter an error, it can be passed on to DS.NLM. Errors in this category use codes ranging between –400 and –599.