LIBNAME statement, see SAS Statements: Reference. For more information about the
COMPRESS= system option, see SAS System Options: Reference.
Disabling a Compression Request
Compressing a file adds a fixed-length block of data to each observation. Because of the
additional block of data (12 bytes for a 32-bit host and 24 bytes for a 64-bit host per
observation), some files could result in a larger file size. For example, files with
extremely short record lengths could result in a larger file size if compressed.
When a request is made to compress a data set, SAS attempts to determine whether
compression will increase the size of the file. SAS examines the lengths of the variables.
If, due to the number and lengths of the variables, it is not possible for the compressed
file to be at least 12 bytes (for a 32-bit host) or 24 bytes (for a 64-bit host) per
observation smaller than an uncompressed version, compression is disabled and a
message is written to the SAS log.
For example, here is a simple data set for which SAS determines that it is not possible
for the compressed file to be smaller than an uncompressed one:
data one (compress=char);
length x y $2;
input x y;
datalines;
ab cd
;
The following output is written to the SAS log:
Log 26.1 SAS Log Output When Compression Request Is Disabled
NOTE: Compression was disabled for data set WORK.ONE because compression
overhead would increase the size of the data set.
NOTE: The data set WORK.ONE has 1 observations and 2 variables.
Extending the Observation Count for a 32-Bit SAS
Data File
Definition of Extended Observation Count
An extended observation count is an attribute of a SAS data file that counts observations
beyond the 32-bit long integer maximum. In SAS 9.4, when you create an output 32-bit
SAS data file, the resulting enhanced file format behaves like a 64-bit file regarding
counters. A SAS data file with an extended observation count, however, is incompatible
with releases prior to SAS 9.3.
For more information about the maximum observation count, see “Understanding the
Observation Count in a SAS Data File” on page 608.
Understanding the Extended Observation Count
Historically, the BASE engine had a limitation in which up to approximately two billion
observations could be counted and fully supported for operating environments with a 32-
662 Chapter 26 • SAS Data Files
bit long integer. In SAS 9.3, functionality was added to allow the limit to be extended to
match that of operating environments with a 64-bit long integer. To extend the
observation count in an output SAS data file, the EXTENDOBSCOUNTER= data set
option and LIBNAME statement options were provided, with the default set to NO. To
create a SAS data file with an extended observation count in SAS 9.3, you specify
EXTENDOBSCOUNTER=YES as either a SAS data set option or a LIBNAME
statement option.
SAS 9.4 enhances the extended observation count functionality by automatically
creating a 32-bit SAS data file with an extended observation count and by providing the
EXTENDOBSCOUNTER= system option. In SAS 9.4, the EXTENDOBSCOUNTER=
data set option, LIBNAME statement option, and system option are, by default, set to
YES. Therefore, in SAS 9.4, you must specify EXTENDOBSCOUNTER=NO if you do
not want a SAS data file created with an extended observation count.
Using the EXTENDOBSCOUNTER= Option
To control whether to extend the observation count in an output SAS data file, use the
EXTENDOBSCOUNTER= option. By default, the option requests an enhanced file
format that counts observations beyond the 32-bit long maximum.
Note: The EXTENDOBSCOUNTER= option has the alias EOC=.
The EXTENDOBSCOUNTER= option is supported in the following situations:
• As a SAS data set option, EXTENDOBSCOUNTER= applies to the newly created
output file with which the option appears. You can use the data set option to override
the setting of the LIBNAME statement and system option. For details, see
“EXTENDOBSCOUNTER= Data Set Option” in SAS Data Set Options: Reference.
• As a LIBNAME statement option, EXTENDOBSCOUNTER= applies to the newly
created output files in the SAS library. For details, see “LIBNAME Statement” in
SAS Statements: Reference.
• As a system option, EXTENDOBSCOUNTER= affects newly created output files in
the SAS session. For details, see “EXTENDOBSCOUNTER= System Option” in
SAS System Options: Reference.
To request that an output SAS data file not extend the observation count, you must
specify EXTENDOBSCOUNTER=NO. For example, the following code creates an
output SAS data file that does not extend the observation count beyond the 32-bit long
maximum when processed in an operating environment with a 32-bit long integer. The
EXTENDOBSCOUNTER= option is specified as a data set option in the DATA
statement. Requesting that an output SAS data file not extend the observation count
might be necessary if you need to process the file in a SAS version prior to SAS 9.3.
libname myfiles 'C:MyFiles';
data myfiles.bigfile (extendobscounter=no);
.
.
.
run;
Specifying EXTENDOBSCOUNTER= as a LIBNAME statement option controls the
enhanced file format for all files created for the SAS library. The following code uses the
COPY procedure to re-create all of the files in the SAS library without the extended
observation count:
Extending the Observation Count for a 32-Bit SAS Data File 663
libname new 'C:NewFiles' extendobscounter=no;
libname old 'C:OldFiles';
proc copy in=old out=new;
run;
Recovering from an Exceeded Maximum Observation Count
To recover a SAS data file that exceeds the maximum number of observations that can
be counted, simply re-create the SAS data file in SAS 9.4. One method to convert a file
that exceeds the maximum observation count is to use the DATA step SET statement.
For example, the following code creates a new file with an extended observation count
by making a copy of the existing file. By default, EXTENDOBSCOUNTER=YES.
libname lib 'C:Myfiles';
data lib.b;
set lib.a;
run;
Extended Observation Count File Attribute
When a SAS data file is created with an extended observation count, the file contains an
attribute to indicate that the file format is enhanced to extend the observation count. For
example, the following CONTENTS procedure output lists ExtendObsCounter
information under Engine/Host Dependent Information.
Note: If a SAS data file does not contain the extended observation count file attribute,
the ExtendObsCounter field is not listed.
Output 26.7 CONTENTS Procedure Output Showing ExtendObsCounter Attribute
664 Chapter 26 • SAS Data Files
Extended Observation Count Behavior Considerations
A SAS data file that contains the extended observation count attribute is supported
starting in SAS 9.3. The enhanced file format is incompatible with releases prior to SAS
9.3. If you attempt to open a SAS data file that contains the extended observation count
attribute in a SAS release prior to SAS 9.3, an error message occurs. For example:
ERROR: File MYFILES.EXTEND.Data not compatible with this SAS version.
The extended observation count attribute is not inherited by a new file. However,
because EXTENDOBSCOUNTER= is set to YES by default, you do not need to
explicitly request to extend the observation count when you create a new file from one
that contains the extended observation count attribute. For example, the following code
creates a new file named Extend2 from an existing file named Extend1. Extend1
contains the extended observation count attribute. The extended observation count
attribute is not inherited by the new file, but the DATA statement for the new file does
not need to specify the EXTENDOBSCOUNTER=YES data set option.
EXTENDOBSCOUNTER= is set to YES by default:
data extend2;
set extend1;
run;
Conversely, to create a file without an extended observation count from a file that does
not contain the extended observation count attribute, you must specify the
EXTENDOBSCOUNTER=NO option for the new file. For example, the following code
creates a new file named NoExtend2 from an existing file named NoExtend1, which
does not contain the extended observation count attribute. The DATA statement for the
new file must specify the EXTENDOBSCOUNTER=NO data set option:
data noextend2 (eoc=no);
set noextend1;
run;
Operations that are affected by the extended observation count attribute include the
following:
• SAS functionality that copies files (such as the APPEND procedure, COPY
procedure, MIGRATE procedure, and SET statement) does not copy the extended
observation count attribute.
• In a SAS/SHARE client session, the EXTENDOBSCOUNTER= option in the
LIBNAME statement is ignored if it is specified in combination with the SERVER=
option.
• When you specify the FIRSTOBS= or OBS= option for a SAS data file that is
created with an extended observation count, the performance is improved for a file
with 2
31
-1 observations or more in a 32-bit environment.
Extending the Observation Count in a 64-Bit Operating Environment
In an operating environment that stores the observation count as a 64-bit long integer,
extending the observation count is ignored. For example, when processed in an operating
Extending the Observation Count for a 32-Bit SAS Data File 665
environment with a 64-bit long integer such as HP-UX on 64-bit Itanium, the following
code results in a message to the SAS log:
libname myfiles '/u/myid/myfiles';
data myfiles.bigfile (extendobscounter=yes);
.
.
.
run;
NOTE: EXTENDOBSCOUNTER=YES is ignored on a SAS data set with a 64-bit long
observation counter.
However, if you are using the OUTREP= option to create an output file with a 32-bit
data representation, the observation count is stored as a 32-bit long. In this case,
extending the observation count in order to create a 32-bit file that behaves like a 64-bit
file with respect to counters is supported. For example, the following code is submitted
in the 64-bit operating environment HP-UX on 64-bit Itanium and requests a Microsoft
Windows 64-Bit Edition data representation, which uses the 32-bit model long integer
data type.
libname myfiles '/u/MyFiles';
data myfiles.bigfile (outrep=windows_64);
.
.
.
run;
T I P Even though Microsoft Windows 64-Bit Edition is a 64-bit operating
environment, the observation count is stored as a 32-bit long integer. Therefore,
extending the observation count for SAS data files in the Microsoft Windows 64-Bit
Edition operating environment provides the ability to count observations beyond the
32-bit long integer maximum.
When Extending the Observation Count Is Supported
Extending the observation count affects only an output SAS data file whose internal data
representation stores the observation count as a 32-bit long integer.
In an operating environment that stores the observation count as a 64-bit long integer,
EXTENDOBSCOUNTER=YES is ignored, unless you are creating an output file with a
32-bit data representation by specifying the OUTREP= option.
In an operating environment that stores the observation count as a 32-bit long integer, if
you are creating an output file with a 64-bit representation by specifying the OUTREP=
option, EXTENDOBSCOUNTER=YES is ignored.
The following table lists the SAS 9.4 operating environments, the data representation
value for each operating environment, the default observation count size, and whether
extending the observation count is supported. For example,
EXTENDOBSCOUNTER=YES is ignored if your operating environment is AIX on 64-
bit platform. However, in the AIX on 64-bit platform, if you use the OUTREP= option to
specify the data representation value WINDOWS_64, then
EXTENDOBSCOUNTER=YES is supported.
666 Chapter 26 • SAS Data Files
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.