The real issue with trusting input is this: many applications today distribute functionality between client and server machines or between peers, and many developers rely on the client portion of the application to provide specific behavior. However, the client software, once deployed, is no longer under the control of the developer, nor the server administrators, so there is no guarantee that requests made by the client came from a valid client. Instead, those requests may have been forged. Hence, the server can never trust the client request. The critical issue is trust and, more accurately, attributing too much trust to data provided by an untrusted entity. The same concept applies to the client. Does the client code really trust the data from the server, or is the server a rogue server? A good example of client-side attacks is cross-site scripting, discussed in detail in Chapter 13.

When you’re analyzing designs and code, it’s often easy to find areas of vulnerability by asking two simple questions. Do I trust the data at this point? And what are the assumptions about the validity of the data? Let’s take a buffer overrun example. Buffer overruns occur for the following reasons:

Take a look at this code. What’s wrong with it?

void CopyData(char *szData) {
   char cDest[32];
   strcpy(cDest,szData);
    
   // use cDest
   ...
} 

Surprisingly, there may be nothing wrong with this code! It all depends on how CopyData is called and whether szData comes from a trusted source. For example, the following code is safe:

char *szNames[] = {"Michael","Cheryl","Blake"};
CopyData(szNames[1]);

The code is safe because the names are hard-coded and therefore each string does not exceed 32 characters in length; hence, the call to strcpy is always safe. However, if the sole argument to CopyData, szData, comes from an untrusted source—such as a socket or a file with a weak access control list (ACL)—then strcpy will copy the data until it hits a null character. And if the data is greater than 32 characters in length, the cDest buffer is overrun and any data above the buffer in memory is clobbered. Figure 10-1 shows the relationship between the call to strcpy and the three points I made earlier.

Figure 10-1. The conditions for calling strcpy in an unsafe manner.

Scrutinize this example and you’ll notice that if you remove any of the conditions, the chance of a buffer overrun is zero. Remove the memory-copying nature of strcpy, and you cannot overflow a buffer, but that’s not realistic because a non-memory-copying version is worthless! If the data always come from trusted source—for example, from a highly trusted user or a well-ACL’d file—you can assume the data is well-formed. Finally, if the code makes no assumptions about the data and validates it prior to copying it, then once again, the code is safe from buffer overruns. If you check the data validity prior to copying it, it doesn’t matter whether the data came from a trusted source. Which leads to just one acceptable solution to make this code secure: first check that the data is valid, and do not trust it until the legality is verified.

The following code is less trusting and is therefore more secure:

void CopyData(char *szData, DWORD cbData) {
        const DWORD cbDest = 32;
    char cDest[cbDest];

        if (szData != NULL && cbDest > cbData)
           strncpy(cDest,szData,min(cbDest,cbData));
    
    //use cDest
    ...
}

The code still copies the data (strncpy), but because the szData and cbData arguments are untrusted, the code limits the amount of data copied to cDest. You might think this is a little too much code to check the data validity, but it’s not—a little extra code can protect the application from serious attack. Besides, if the insecure code is attacked you’d need to make the fixes in the earlier example anyway, so get it right first time.

Earlier I mentioned that weak ACLs lead to untrusted data. Imagine a registry key that determines which file to update with log information and that has an ACL of Everyone (Full Control). How much trust can you assign to the data in that key? None! Because anyone can update the filename. For example, an attacker could change the filename to c:oot.ini. The data in this key can be trusted more if the ACL is Administrator (Full Control) and Everyone (Read); in that case only an administrator can change the data, and administrators are trusted entities in the system. With proper ACLs, the concept of trust is transitive: because you trust the administrators and because only administrators can change the data, you trust the data.

The simplest and by far the most effective way to defend your application from input attacks is to validate the data before performing any further processing. To achieve these goals, you should adhere to the following strategies:

First, all applications have a point in the design where the data is believed to be well-formed and safe because it has been checked. Once the data is inside that trusted boundary, there should be no reason to check it again for validity—that is, assuming the code did a good job! That said, the principle of defense in depth dictates that you should employ multiple layers of defense in case a layer is compromised, and that is quite true. But I’ll leave it to you to find that balance between security and performance for your application. The balance will depend on the sensitivity of the data and the environment in which the application operates.

Next, you should perform the check at the boundary of the trusted code base. You must define that point in the design; it must be in one place, and no input should be allowed into the trusted code base without going through that chokepoint. Note that you can have multiple chokepoints, one for each data source (Web, registry, file system, configuration files, and so on), but data from one source should not enter the trusted code base through any other chokepoint but the one designated for that source.

As you can see, the concept of the trusted boundary and chokepoint are tightly related. Figure 10-2 graphically shows the concepts of a trust boundary and chokepoints.

Figure 10-2. The concept of a trust boundary and chokepoints.

Note that the service and the service’s data store have no chokepoint between them. That’s because they’re both inside the trust boundary and data never enters the boundary from any data source without being validated first at a chokepoint. Therefore, only valid data can flow from the service to the data store and vice versa.

A common vulnerability on the Web today is cross-site scripting errors. These errors involve malicious input that is echoed in the unsuspecting user’s browser from an insecure Web site. The malicious input comprises HTML and script. I’m not going to give the game away; it’s all explained in detail in Chapter 13. Many Web sites have them, and the sites’ administrators don’t know it. I spent time in early 2001 providing security training for some developers of a very large Web site that has never had such an issue. They have never had these issues because the Web application has two chokepoints, one for data coming from the user (or attacker) and another for data flowing back to the user. All input and output goes through these two chokepoints. Any developer that violates this policy by reading or writing Web-based traffic by using alternate means is "spoken to!" The chokepoints enforce very strict validity checking. Now that I’ve brought up the subject, let’s discuss validity checking.

When checking input for validity, you should follow this rule: look for valid data and reject everything else. The principle of failing securely, outlined in Chapter 3, means you should deny all access until you determine the request is valid. You should look for valid data and not look for invalid data for two reasons:

The first point is explained in more detail in Chapter 11. It’s common to escape characters in a way that is valid, but it’s also possible to hide invalid data by escaping it. Because the data is escaped, your code might not detect that it’s invalid.

The second point is very common indeed. Let me explain by way of a simple example. Imagine your code takes requests from users to upload files, and the request includes the filename. Your application will not allow a user to upload executable code because it could compromise the system. Therefore, you have code that looks a little like this:

bool IsBadExtension(char *szFilename) {
    bool fIsBad = false;

    if (szFilename) {
        size_t cFilename = strlen(szFilename);
        if (cFilename >= 3) {
            char *szBadExt[] 
                = {".exe", ".com", ".bat", ".cmd"};
            char *szLCase
                = _strlwr(_strdup(szFilename));

            for (int i=0; 
                i < sizeof(szBadExt) / sizeof(szBadExt[0]); 
                i++)
                if (szLCase[cFilename-1] == szBadExt[i][3] && 
                    szLCase[cFilename-2] == szBadExt[i][2] && 
                    szLCase[cFilename-3] == szBadExt[i][1] && 
                    szLCase[cFilename-4] == szBadExt[i][0])
                    fIsBad = true;                    }
    }

    return fIsBad;
}

bool CheckFileExtension(char *szFilename) {
    if (!IsBadExtension(szFilename))
        if (UploadUserFile(szFilename)) 
            NotifyUserUploadOK(szFilename);
}

What’s wrong with the code? IsBadExtension performs a great deal of error checking, and it’s reasonably efficient. The problem is the list of "invalid" file extensions. It’s nowhere near complete; in fact, it’s hopelessly lacking. A user could upload many more executable file types, such as Perl scripts (.pl) or perhaps Windows Scripting Host files (.wsh, .js and .vbs), so you decide to update the code to reflect the other file types. However, a week later you realize that Microsoft Office documents can contain macros (.doc, .xls, and so on), which technically makes them executable code. Yet again, you update the list of bad extensions, only to find that there are yet more executable file types. It’s a never-ending battle. The only correct way to achieve the goal is to look for valid, safe extensions and to reject everything else. For example, in the Web file upload scenario, you might decide that users can upload only certain text document types and graphics, so the secure code looks like this:

bool IsOKExtension(char *szFilename) {
    bool fIsOK = false;

    if (szFilename) {
        size_t cFilename = strlen(szFilename);
        if (cFilename >= 3) {
            char *szOKExt[] = 
                {".txt", ".rtf", ".gif", ".jpg", ".bmp"};

            char *szLCase = 
                _strlwr(_strdup(szFilename));

            for (int i=0; 
                i < sizeof(szOKExt) / sizeof(szOKExt[0]); 
                i++)
                if (szLCase[cFilename-1] == szOKExt[i][3] && 
                    szLCase[cFilename-2] == szOKExt[i][2] && 
                    szLCase[cFilename-3] == szOKExt[i][1] && 
                    szLCase[cFilename-4] == szOKExt[i][0])
                    fIsOK = true;
        }
    }

    return fIsOK;
}

As you can see, this code will not allow any code to upload unless it’s a safe data type, and that includes text files (.txt), Rich Text Format files (.rtf), and some graphic formats. It’s much better to do it this way. In the worst case, you have an annoyed user who thinks you should support another file format, which is better than having your servers compromised.

use strict;
my $filename = <STDIN>;
open (FILENAME, ">> " . $filename) or die $!;
print FILENAME "Hello!";
close FILENAME;

use strict;
my $filename = <STDIN>;
$filename =~ /(w{1,8}.log)/;
open (FILENAME, ">> " . $1) or die $!;
print FILENAME "Hello!";
close FILENAME;

For simple data validation, you can use code like the code I showed earlier, which used simple string compares. However, for complex data you need to use higher-level constructs, such as regular expressions. The following C# code shows how to use regular expressions to replace the C++ extension-checking code. This code uses the RegularExpressions namespace in the .NET Framework:

using System.Text.RegularExpressions;
...
static bool IsOKExtension(string Filename) {
        Regex r = 
    new Regex(@"txt|rtf|gif|jpg|bmp$",
    RegexOptions.IgnoreCase);
        return r.Match(Filename).Success;
}

The same code in Perl looks like this:

sub isOkExtension($) {
        $_ = shift;
        return /txt|rtf|gif|jpg|bmp$/i ? -1 : 0;
}

I’ll go into language specifics later in this chapter. For now, let me explain how this works. The core of the expression is the string "txt|rtf|gif|jpg|bmp$". The components are described in Table 10-1.

If the search string matches one of the file extensions and then the end of the filename, the expression returns true. Also note that the C# code sets the RegexOptions.IgnoreCase option, because filenames in Microsoft Windows are case-insensitive.

Table 10-2 offers a more complete regular expression elements list. Note that some of these elements are implemented in some programming languages and not in others.

Let’s look at some examples in Table 10-3 to make this a little more concrete.

RegExp r = [a-z]{1,8}.[a-z]{1,3};
if (r.Match(strFilename).Success) {
        //Cool! Allow access to strFilename; it’s valid.
} else {
        //Tut! tut! Trying to access an invalid file.
}

^[a-z]{1,8}.[a-z]{1,3}$

Historically, regular expressions dealt with only 8-bit characters, which is fine for single-byte alphabets but it’s not so great for everyone else! So how should your input-restricting code handle Unicode characters? If you must restrict your application to accept only what is valid, how do you do it if your application has Japanese or German users? The answer is not straightforward, and support is inconsistent across regular expression engines at best.

Three aspects to Unicode make it complex to build good Unicode regular expressions:

Now here’s the good news: more engines are adding support for Unicode expressions as vendors realize the world is a very small place. A good example of this change is the introduction of Perl 5.8.0, which had just been released at the time of this writing. Another example is Microsoft’s .NET Framework, which has both excellent regular expression support and exemplary globalization support. In addition, all strings in managed code are natively Unicode.

At first, you might think you can use hexadecimal ranges for languages, and you can, but doing so is crude and not recommended because

The following regular expression will find all Japanese Katakana letters from small letter a to letter vo, but not the conjunction and length marks and some other special characters above u30FB:

Regex r = new Regex(@"^[u30A1-u30FA]+$");

The secret to making Unicode regular expressions manageable lies in the p{category} construct, which matches any character in the named Unicode character category. The .NET Framework and Perl 5.8.0 support Unicode categories, and this makes dealing with international characters easier. The high-level Unicode categories are Letters (L), Marks (M), Numbers (N), Punctuation (P), Symbols (S), Separators (Z), and Others (O and C) as follows:

Let’s put the character classes to good use. Imagine a field in your Web application must include only a currency symbol, such as that for a dollar or a euro. You can verify that the field contains such a character and nothing else with this code:

Regex r = new Regex(@"^p{Sc}{1}$");
if (r.Match(strInput).Success) {
    // cool!
} else {
    // try again
}

The good news is that this works for all currency symbols defined in Unicode, including dollar ($), pound sterling (£), yen (¥), franc (), euro (), new sheqel (), and others!

The following regular expression will match all letters, nonspacing marks, and spaces:

Regex r = new Regex(@"^[p{L}p{Mn}p{Zs}]+$");

The reason for p{Mn} is many languages use diacritics and vowel marks; these are often called nonspacing marks.

The .NET Framework also provides language specifies, such as p{IsHebrew}, p{IsArabic} and p{IsKatakana}. I have included some sample code that demonstrates this named Ch10Lang.

When you’re experimenting with other languages, I recommend you use Windows 2000, Windows XP, or Microsoft Windows .NET Server 2003 with a Unicode font installed (such as Arial Unicode MS) and use the Character Map application, as shown in Figure 10-3, to determine which characters are valid. Note, however, that a font that claims to support Unicode is not required to have glyphs for every valid Unicode code point. You can look at the Unicode code charts at http://www.unicode.org/charts.

Figure 10-3. Using the Character Map application to view non-ASCII fonts.

Regular expressions are incredibly powerful, and their usefulness extends beyond just restricting input. They constitute a technology worth understanding for solving many complex data manipulation problems. I write many applications, mostly in Perl and C#, that use regular expressions to analyze log files for attack signatures and to analyze source code for security defects. Because subtle variations exist in regular expression syntax between programming languages and execution environments, the rest of this chapter outlines some of these variations. (Note that my intention is only to give you a number of regular expression quick references.)

$_ = "We leave at 12:15pm for Mount Doom. ";
if (/.*(d{2}:d{2}[ap]m)/i) {
        print $1;
}

var =~ /expression/;

// C# Example
String s = @"We leave at 12:15pm for Mount Doom.";
Regex r = new Regex(@".*(d{2}:d{2}[ap]m)",RegexOptions.IgnoreCase);
if (r.Match(s).Success)
    Console.Write(r.Match(s).Result("$1"));

‘ Visual Basic .NET example
Imports System.Text.RegularExpressions

…

Dim s As String
Dim r As Regex
s = "We leave at 12:15pm for Mount Doom."
r = New Regex(".*(d{2}:d{2}[ap]m)", RegexOptions.IgnoreCase)
If r.Match(s).Success Then
    Console.Write(r.Match(s).Result("$1"))
End If

// Managed C++ version
#using <mscorlib.dll>
#include <tchar.h>
#using <system.dll>

using namespace System;
using namespace System::Text;
using namespace System::Text::RegularExpressions;
…
String *s = S"We leave at 12:15pm for Mount Doom.";
Regex *r = new Regex(".*(\d{2}:\d{2}[ap]m)",IgnoreCase);
if (r->Match(s)->Success) 
        Console::WriteLine(r->Match(s)->Result(S"$1"));

var r = /.*(d{2}:d{2}[ap]m)/;
var s = "We leave at 12:15pm for Mount Doom.";
if (s.match(r))
        alert(RegExp.$1);

Set r = new RegExp
r.Pattern = ".*(d{2}:d{2}[ap]m)"
r.IgnoreCase = True

Set m = r.Execute("We leave at 12:15pm for Mount Doom.")
MsgBox m(0).SubMatches(0)

#include <AtlRX.h>
…
CAtlRegExp<> re;
re.Parse(".*{\d\d:\d\d[ap]m}",FALSE);
CAtlREMatchContext<> mc;
if (re.Match("We leave at 12:15pm for Mount Doom.", &mc)) {
        const CAtlREMatchContext<>::RECHAR* szStart = 0;
    const CAtlREMatchContext<>::RECHAR* szEnd = 0;
        mc.GetMatch(0,&szStart, &szEnd);

        ptrdiff_t nLength = szEnd - szStart;
        printf("%.*s",nLength, szStart);
}

One way to enforce that input is always validated prior to being accessed is by using languages that support classes, such as C++, C# and Visual Basic .NET. Here’s an example of a UserInput class written in C++:

#include <string>
using namespace std;

class UserInput {
public:
    UserInput(){};
    ~UserInput(){};
    bool Init(const char* str) {
        //add more checking here if you like
        if(!Validate(str)){
            return false;
        } else {
            input = str;
            return true;
        }
    }

    const char* GetInput(){return input.c_str();}
    DWORD Length(){return input.length();}

private:
    bool Validate(const char* str);
    string input;
};

Using a class like this has a number of advantages. First, if you see a method or function that takes a pointer or reference to a UserInput class, it’s obvious that you’re dealing with user input. The second is that there’s no way to get an instance of this class where the input has not passed through the Validate method. If the Init method is never called or fails, the class contains an empty string. If you wanted to, you could create such a class with a Canonicalize method. This approach might save you time and bug-fixing because you can ensure that input validation always takes place and is done consistently.

I’ve spent a great deal of time outlining how to use regular expressions, but do not lose sight of the most important message of this chapter: trust input at your peril. In fact, do not trust any input until it is validated. Remember, just about any security vulnerability can be traced back to an application placing too much trust in the data,

When analyzing input, have a small number of entry points into the trusted code; all input must come through one of these chokepoints. Do not look for "bad" data in the request. You should look for good, well-formed data and reject the request if the data does not meet your acceptance criteria. Remember: you wrote the code for accessing and manipulating your resources; you know what constitutes a correct request. You cannot know all possible invalid requests, and that’s one of the reasons you must look only for valid data. The list of correct requests is finite, and the list of invalid requests is potentially infinite or, at least, very very large.