Chapter 10 Programming for eDirectory

In previous chapters, you’ve learned how to use a number of Novell-supplied and third-party directory services (DS) data recovery applications. Because every network environment is unique in its own way, however, you might not be able to find one off-the-shelf utility that totally fulfills your requirements. This chapter offers you an overview on how you can use a basic programming background to develop some complex data recovery tools for your own network. Please note, however, that this chapter does not discuss programming in detail and expects you to already have that knowledge.

Programming for NDS disaster recovery need not always involve the Novell Developer Kit (NDK) or any form of NDS or the NetWare application programming interface (API) set. Some of the most effective programming techniques available are nothing more than text file manipulations. This type of manipulation is good for data conversions such as converting NList output to a format suitable for UImport or ICE to use for input.

Several programming languages include text (or string) manipulation. BASIC, C, Pascal, Perl, and even assembly language include interfaces and libraries for performing string manipulation. One of the best languages for this sort of work, however, is a programming language called awk.

The awk language was developed in 1977 by Alfred Aho, Brian Kernighan, and Peter Weinberger at AT&T Bell Labs; the name of the language comes from the last-name initials of the three authors. The original development of awk was done in a Unix environment, and because of this, many of the concepts are familiar if you have had exposure to utilities such as grep or sed. grep is very similar to the Windows find utility because it searches one or more input files for lines containing a match to a specified pattern. By default, grep prints the matching lines. sed (stream editor) is a non-interactive editor that works from a command line; it’s not a GUI Windows application. sed changes blocks of text on-the-fly, without requiring you to open a screen, push a cursor around, and press Delete, Insert, Enter, or function keys.

awk is an interpreted scripting language; this means that there is no compiler or other means to turn an awk program into a self-sufficient executable program as you would a C or Pascal program.

An awk program takes input a line at a time, parses and processes the input, and produces an output file. This is usually done through command-line pipes or redirection. Normally, this process involves the use of three files: an awk script, an input file, and an output file.

The awk script itself is a set of rules for processing lines of text from the input file. These rules are written using a pattern-matching technique that is common in the Unix world, called a regular expression (regex). Regex pattern matching enables you to specify the format of a line of text in the input file; if a line matches the regex, the text is processed in a manner described by that portion of the script.

Regex pattern matching uses this basic format:

where pattern is replaced by a string that represents an input format. Table 10.1 shows special sequences of characters that can be used in the pattern.

TABLE 10.1 Sample Regular Expressions

For example, if you execute this command:

and use the file OUTPUT.TXT as the input file to the script:

awk searches the input file for the case-sensitive string User:, and if this text is found, it prints the string “Found a user ID.”

awk supports two special commands: BEGIN and END. These are not really patterns but are used to include special instructions—such as variable initialization and final output options—in the script. Here’s an example:

This script searches for instances of the case-sensitive string User: (as in the previous example), but rather than print a string, it increments the variable count, and when it finishes that, it prints a total count of the user objects listed in the input file.

An awk script parses the input line based on a field separator. By default, the field separator is whitespace. Whitespace includes any number of spaces or tabs between the data. The line is then split out into internal variables based on the field separator found.

For example, this line consists of a total of four fields:

These fields are referred to by the names $1, $2, $3, and $4, with these values:

The entire line of text is referred to by the variable $0. This variable always represents the entire line up to the record separator, which is typically a carriage return.

You can change the defaults for the various separators, such as field separator and record separator. For example, if you have a tab-delimited file, you would want the field separator (FS) to be the tab character rather than a space. You typically make this change in the beginning of the script, in the BEGIN segment:

The output field separator (OFS) denotes the character used to separate the fields when awk prints them. The default is a blank or space character. To output a comma-separated variable (CSV) formatted file, you can change the OFS to a comma as follows:

You can also change the record separator (RS) by using the RS variable:

Typically, you do not want to change this, but at times it might make sense to do so.

awk also has a number of string manipulation functions. Table 10.2 shows some of the commonly used functions and what they can be used for.

TABLE 10.2 awk String Manipulation Functions

In addition to doing string manipulation, you can also use awk for numeric manipulation. This type of manipulation of data is done in the same manner as C/C++ numeric manipulation. If you need to manipulate a number with an initial value, you can initialize it in the BEGIN section of the script.

When you’re scanning a line and breaking it into the initial subcomponents ($1 through $NF), or when you’re breaking it down using the split() function, if a numeric value is found, it is automatically treated as a number.

However, if you need to perform string manipulations on it, you are also able to do this. In this respect, awk provides greater flexibility than most other programming languages.

awk also supports the use of several other statements and structures. Table 10.3 lists the most common of these.

TABLE 10.3 awk Language Actions

TIP

awk uses the pound sign (#) to indicate the start of a comment. Anything after a # will be ignored. Therefore, you can include inline comments anywhere within your script.

NOTE

awk resources and documentation are available on the Internet, and the following are two examples of good Web sites for this type of information: www.ling.ed.ac.uk/facilities/help/gawk/gawk_13.html#SEC97 and www.sunsite.ualberta.ca/Documentation/Gnu/gawk-3.1.0/html_chapter/gawk_10.html.

An additional feature of the awk language that is occasionally useful is the capability to create specialized functions for repeated operations. If you are coding an involved script, you can package the code so as to minimize your coding time. However, in disaster recovery situations, you will generally find that the scripts you write will perform very specific manipulations, and as a general rule, you do not need to reuse code within such scripts.

Creating functions within awk is a simple matter. For example, the following would be a function to return the minimum value of two passed-in parameters:

function min(a, b){
  if (a < b)
    return a
  else
    return b
}

Because awk supports the abbreviated if structure that C provides, the preceding can also be coded as follows:

function min(a, b){
  return a < b ? a : b
}

This code sample provides the same functionality as the previous sample.

After you have written awk script, you need to give it an input file. When you’re using the gawk interpreter, you do this by using a command in the following format:

gawk awkfile.awk < inputfile [> outputfile]

This tells gawk to use awkfile.awk as the script and to pipe the contents of inputfile into the script. The input file can be left out, but if it is, you must type the input file by hand. The output file contains the results of the script; if an output file is not specified, the output is written to the screen.

TIP

Writing the output to the screen can be very useful during the development process. By exiting the script with the exit command after processing the first line of text, you can get an idea as to whether the program is working the way you want it to work without processing the entire input file.

In a disaster-recovery situation, it is not absolutely necessary that the script work completely correctly when you finish it. Using awk with a combination of other tools may get you to the end you need more quickly. Consider this example. Say you have an awk script that outputs a UImport file with group membership information in it. It is missing a few of the members because of an extra embedded space for those entries. Instead of trying to perfect the script to handle that small number of exceptions, you can use your favorite text editor to either remove those entries from the file or to make the correction using a search-and-replace function. Remember that during disaster recovery, it doesn’t matter if your script is elegant or pretty; it just matters that you get the job done quickly and correctly.

The C programming language has been, and still is, the programming language of choice for systems programming. It is, therefore, not a surprise that Novell’s initial Software Developer Kit (SDK; now known as the NDK) efforts were placed in C libraries. Even today, the NetWare Loadable Module (NLM) libraries are still C based, and workstation application developers now have more options in their choice of programming language (see the “Other Programming and Scripting Languages” section, later in this chapter).

When you’re programming for NDS, there are a number of operations that your application must perform locally when accessing DS information. The operations include the following:

As mentioned previously, a DS object may be referenced in multiple ways: using its relative distinguished name (RDN) or its distinguished name (DN) in typeful or typeless naming syntax. DS operates on canonical names only—that is, names that include full naming paths, with a type specification for each naming component (typeful DNs). Because it’s not always convenient or practical to store or use canonical names, APIs enable you to use partial names and enable the underlying library routines to handle the conversion.

Most DS API calls that involve character data or interact with the NDS/eDirectory tree require additional information to be supplied. This is to indicate the default context and whether the character data should be translated to Unicode. This information is collectively held in a structure called the directory context and is used to pass the following details:

When a directory context (which is essentially a memory pointer) is created, the default context is the same as the workstation’s current context, and the four operations listed previously are performed. The directory context should be freed when it is no longer needed (normally when the application terminates).

You can read and modify the information in a directory context. You need to be careful when changing the default context. You can set it to an arbitrary string, unlike with the CX command-line utility, which does not set the default context to an illegal value. If the default context is set to an illegal value, any API calls that rely on the directory context will fail.

The same object-naming conventions apply to the NDS API calls as to the NetWare command-line utilities. That is, leading and trailing periods can be used to modify the default context, and the type qualifiers, such as C=, can be inserted into the name to override the default typing.

The Unicode conversion tables have to be loaded before a workstation application can access NDS (this is not necessary for NLMs), and they should be released prior to an application’s terminating. These tables are used to convert character data between the general Unicode format used by the NDS and the local format used by the application. You load the Unicode tables by executing the Unicode API NWInitUnicodeTables() and specifying the local code page. You can obtain the local code page by executing the Internationalization Services APIs NWLsetlocale() and NWLlocaleconv().

DS API functions use buffers for sending and receiving data between the application and NDS/eDirectory. For example, a typical sequence in reading an object’s attribute values would be to initialize an input buffer, load the buffer with the list of desired attributes, execute the read object command, and unload the attribute values from an output buffer. Given that the output buffer is finite, it is sometimes necessary to call the read function repeatedly to get subsequent values. This protocol of using input and output buffers provides a general interface for transferring data.

Memory for directory buffers is allocated using an DS API call. The suggested amount of memory to allocate is given by the constant DEFAULT_MESSAGE_LEN that is set to 4096 bytes. This value is normally more than sufficient for a typical DS application. The only exception under normal circumstances would be when there is a single attribute value (such as a stream file) that will not fit in the buffer (thus resulting in a -119 error [buffer too small]). If you are unsure, you can consider using MAX_MESSAGE_LEN, which has a 63KB value, or using your own setting. Directory buffers should be freed when they are no longer needed (normally when the application terminates); otherwise, memory leaks result.

The directory buffer management tasks make DS-aware applications more complex than bindery-based programs; don’t let that discourage you from developing DS utilities, however, because it only looks harder than it is.

The following is an example of C source code that enables you to change a user’s telephone attribute from a command line. Error checking has been removed to simplify the example:

If you have an external database that contains up-to-date user information, you can use a modified version of this example to read the telephone number (or other attribute values) from this database and repopulate the user objects with the information when needed.

You might be interested in knowing that when you modify eDirectory data via LDAP, you do not need to deal with allocating buffers, directory context, and so on (as is the case when using DS calls). You do, however, have to set up the necessary code to connect and bind to the LDAP server (where a workstation-based DS utility would not require but an NLM would require additional code to authenticate to NDS prior to changes being made). The following is a sample C code snippet (based on Novell’s modattrs.c example) that changes the telephone number:

As you can see, although the LDAP code is longer than the DS-based code presented earlier in this chapter, about half of it actually has to deal with connecting and binding to the LDAP server. The actual code necessary for updating the attribute value is less than 10 lines, and most of it is just setup code.

Although the amount of work involved in using DS calls instead of LDAP calls seem about the same, the big advantage of using LDAP is that your application will also work with directories other than NDS. If you have to update a number of different directories with the same data and they all support LDAP, using LDAP calls would be the better way to go.

Other Programming and Scripting Languages

To help you quickly develop applications for NDS and NetWare without having to fully understand the underlying complexity, Novell Developer Support has several tools available. For example, you can build successful network-ready applications and utilities by using your favorite rapid application development (RAD) Windows programming tool, such as Visual Basic.

Visual Basic Libraries for Novell Services gives programmers full access to all the low-level NetWare APIs that have traditionally been available only to C programmers. The libraries are provided in the form of a single text file and a set of helper functions in a .BAS file. The text file (NETWARE.BAS) contains declarations for most of the NetWare APIs. (The libraries are for the 32-bit Novell Client DLLs and do not support the old 16-bit library.) All the constants, types, and function declarations are in NETWARE.BAS to make it easier to include them in a project. The following is a Visual Basic code snippet that illustrates how to remove an attribute value whose syntax type is SYN_CI_STRING:

Private objName As String
Private attrName As String
Private objType As Integer

Private Sub RemoveValueFromAttribute(objName As String,
                                        attrName As String,
                                        attrValue As String)
Dim inBuf As Long
Dim byteName(127) As Byte
    retCode = NWDSAllocBuf(DEFAULT_MESSAGE_LEN, inBuf)
    If retCode Then
        MsgBox "NWDSAllocBug returned E=" + Str(retCode),
                vbCritical
        Exit Sub
    End If
    retCode = NWDSInitBuf(contextHandle, DSV_MODIFY_ENTRY,
                            inBuf)
    If retCode Then
        MsgBox "NWDSInitBuf returned E=" + Str(retCode),
                vbCritical
        GoTo exit1
    End If

   ’defined in NETWARE.BAS
    Call StringToByteArray(byteName, attrValue + Chr(0))
    retCode = NWDSPutChangeAndVal(contextHandle, inBuf,
                                    DS_REMOVE_VALUE,
                                    attrName + Chr(0),
                                    SYN_CI_STRING,
                                    VarPtr(byteName(0)))
    retCode = NWDSModifyObject(contextHandle,
                                 objName + Chr(0),
                                 -1, 0, inBuf)
exit1:
    retCode = NWDSFreeBuf(inBuf)
End Sub

Alternately, you can use ActiveX Controls for Novell Services, which supports full access to NDS, as well as administration capabilities for NetWare servers, print queues, and volumes. All this functionality is packaged so it can be used quickly and easily in a Windows visual builder and other development tools, such as Visual Basic, Delphi (see Figure 10.1), PowerBuilder, Active Server Pages for Internet Information Server (IIS), Windows Scripting Host, and the Internet Explorer Web browser. ActiveX Controls for Novell Services contains controls that are divided into the following three categories:

   Novell Directory Access Protocol (NDAP; that is, NCP-based) controls:

   Application Administration (NWAppA)

   Browser (NWBrowse)

   Directory (NWDir)

   Directory Administration (NWDirA)

   Directory Authenticator (NWDirAuth)

   Directory Query (NWDirQ)

   NDPS Printer Administration (NWDPPrtA)

   NDS Corporate Edition Domain (NDSDomain)

   Network Selector (NWSelect)

   SecretStore (NWSecStr)

   Server Administration (NWSrvA)

   Session Management (NWSess)

   User Group (NWUsrGrp)

   Volume Administration (NWVolA)

   LDAP controls (to access NDS/eDirectory or any LDAP-complaint directory via LDAP):

   Internet Directory (NWIDir)

   Internet Directory Authenticator (NWIDirAuth)

   Internet Directory Entries (NWIDirE)

   Internet Directory Query (NWIDirQ)

   Internet Directory Schema (NWIDirS)

   Socket controls (to develop applications that communicate by using either TCP or SPX protocols):

   Client and Server Socket (NWCliSkt and NWSvrSkt)

   Peer Socket (NWPrSkt)

FIGURE 10.1 A Novell sample Delphi application.

TIP

After installing the core components for ActiveX Controls for Novell Services, you need to also install the necessary LDAP/NDAP/Socket component to get the necessary controls. These controls use the .DLL extension and not the .OCX extension that you expect for ActiveX controls.

NOTE

Although ActiveX Controls for Novell Services provides broad coverage of the most popular features of the NetWare C APIs, the controls do not wrap 100% of the available API set.

If you are a Delphi developer, you can use Delphi Libraries for Novell Services, which includes the DCU and PAS files needed to call the NetWare APIs from Delphi, instead of using ActiveX Controls for Novell Services.

Java fans will be glad to know that Novell has classes for Java and Java Naming and Directory Interface (JNDI) providers, as well as JavaBeans, available to use. Java applications can access NDS through the JNDI APIs, with or without Novell client software installed. If you’re more comfortable with scripting languages, such as VBScript, you’ll be happy to hear that Novell has support for them, too. For example, using the Novell Script for NetWare (NSN), which is a VBScript-compatible language for script automation and Web development on the Netscape FastTrack Server for NetWare platform, you can use Novell Script’s prebuilt components to access NetWare and integrate eDirectory, Oracle, Microsoft Access, MySQL, and Pervasive (previously Btrieve) databases into your Web applications.

NOTE

You install and configure Novell Script for NetWare when you install a NetWare 5.1 or later server. You can find the latest at http://developer.novell.com/ndk/nscript.htm.

More traditional Web page designers can use Perl 5 for NetWare with the Apache Web Server for NetWare or the NetWare Enterprise Web Server. Perl 5 support enables you to enhance and continue your investment in Perl scripts and Perl applications. At the time of this writing, Perl 5 for NetWare is at Perl 5.8 and provides system administration and common gateway interface (CGI) scripting functionality. You can use the Perl_LDAP modules to access NDS/eDirectory. However, the standard Perl_LDAP modules available from Comprehensive Perl Archive Network (CPAN; www.cpan.org) do not support eDirectory-specific extensions such as those related to partition operations. However, the version from Novell, which can be found at http://developer.novell.com/ndk/perl5.htm, includes a set of functions for performing extended operations (such as ChangeReplicaType) on eDirectory.

With the release of NetWare 6.5, Novell introduced PHP scripting support (used in conjunction with Apache Web Server) to NetWare 6.x servers. (PHP is a recursive acronym for PHP: Hypertext Preprocesser.) PHP for NetWare is based on open-source PHP (www.php.net). On NetWare, PHP supports standard extensions and extensions for LDAP, XML, and MySQL. It also has a UCS extension (PHP2UCS) that can be used to access various Novell services, such as eDirectory, file I/O, server management, and volume administration. Through UCS, PHP can use UCX components, JavaBeans and classes, and remote ActiveX controls.

NOTE

For information on how to set up a NetWare 6 server to take advantage of Apache, MySQL, and PHP/Perl scripting—generally referred to as AMP—see the white paper about NetWare AMP at http://developer.novell.com/ndk/whitepapers/namp.htm.