Appendix 2
SAS Tokens
SAS Tokens
When SAS processes a program, a component called the word scanner reads the program, character by character, and groups the characters into words. These words are referred to as tokens.
List of Tokens
SAS recognizes four general types of tokens:
Literal
One or more characters enclosed in single or double quotation marks. Examples of literals include the following:
Table A2.1 Literals
’CARY’ |
"2008" |
’Dr. Kemple-Long’ |
’<entry align="center">’ |
Name
One or more characters beginning with a letter or an underscore. Other characters can be letters, underscores, and digits.
Table A2.2 Names
data |
_test |
linesleft |
f25 |
univariate |
otherwise |
year_2008 |
descending |
|
Number
A numeric value. Number tokens include the following:
• integers. Integers are numbers that do not contain a decimal point or an exponent. Examples of integers include 1, 72, and 5000. SAS date, time, and datetime constants such as ’24AUG2008’D are integers, as are hexadecimal constants such as 0C4X.
• real (floating-point) numbers. Floating-point numbers contain a decimal point or an exponent. Examples include numbers such as 2.35, 5., 2.3E1, and 5.4E− 1.
Special character
Any character that is not a letter, number, or underscore. The following characters are some special characters:
= + − % & ; ( ) #
The maximum length of any type of token is 32,767 characters. A token ends when the tokenizer encounters one of the following situations:
• the beginning of a new token.
• a blank after a name or number token.
• in a literal token, a quotation mark of the same type that started the token. There is an exception. A quotation mark followed by a quotation mark of the same type is interpreted as a single quotation mark that becomes part of the literal token. For example, in ’Mary’’s’, the fourth quotation mark terminates the literal token. The second and third quotation marks are interpreted as a single character, which is included in the literal token.