In Chapter 3 we will delve into the details of developing, deploying, and invoking Java applications in the database. Before that, this chapter takes you inside Java in the Oracle database. You may ask, “What for? Why dive into the internals of the OracleJVM?” Java developers do not usually care about the internals of the Java Virtual Machine they are using. First, understanding the implementation of the OracleJVM and contrasting its architecture, its memory management, its thread management, the class-sharing techniques, the native Java compiler, and the security management with the JDK VM is, in my opinion, a prerequisite for using it efficiently, at least for those who build infrastructures and frameworks. Second, when you want to better know your car, you look under the hood; similarly, database developers and DBAs will be in a better position to take full advantage of OracleJVM by reading this chapter.
The initial design goals of the OracleJVM were tight integration with the RDBMS (à la PL/SQL), J2SE compatibility, scalability (memory footprint), portability across the platforms where the RDBMS ships, robustness, and performance of Java with SQL. The attempt to support J2EE in the database (pre-9.2 releases) was an afterthought, which did not work very well. Scalability, the ability to support a large number of simultaneous users while keeping memory requirements under control, was another key requirement that has been addressed by sharing system classes, class state, and strings as much as possible. Support for large Java programs has been addressed by designing efficient memory management. Robustness, or resilience, was another requirement that was addressed through the integration with the RDBMS session model.
The tight integration with the RDBMS consists of allowing Java to execute in the same address space as SQL and PL/SQL, while making no compromise on security and data integrity. This goal was accomplished by espousing the session model of the Oracle database. Each database process hosting a Java-enabled session literally has its own private Java VM along with SQL and PL/SQL engines; there is, therefore, no need for a separate process for running Java.
J2SE compatibility is a key requirement from a Java developer’s perspective. The OracleJVM keeps up with J2SE specifications through database releases. In other words, you cannot upgrade the Java VM in the database independently of the database release. As examples, the Oracle 9i Release 2 is J2SE 1.3 compatible, while the Oracle Database 10g Release 1 and 2 are J2SE 1.4 compatible. OracleJVM support for J2SE allows the extension of the database capabilities by the use of standard Java libraries. As an example, the Java Secure Socket Extension (JSSE) allows Java programs to communicate with other systems, using SSL. This feature is optional in J2SE 1.3 and therefore not offered by default in the Oracle 9i Release 2, but you can extend the database by loading the standard JSSE libraries in OracleJVM. What is the usefulness of JSSE for Java in the database? It can communicate securely with external systems, using HTTPS callouts. See, in Chapter 4, step-by-step instructions for secure communication with external systems, using HTTPS callout.
OracleJVM Is J2SE 1.3 Compatible in Oracle9i Release 2
There are few implementation differences with the JDK VM, because of the server-side (or embedded) nature of the OracleJVM runtime:
It does not materialize GUI objects. (This is now standardized in J2SE 1.4 by headless AWT.)
Another implementation difference is that OracleJVM does not support delete-on-close mode for zip and jar.
An additional feature in Oracle 9i Release 2, beyond the standard J2SE features, includes the support for Java Debug Wire Protocol (JDWP), a standard protocol (part of the Java Platform Debugger Architecture) that allows debuggers to communicate with Java virtual machines.
OracleJVM Is J2SE 1.4.x in Oracle Database 10g (Release 1 and 2).
This includes mostly support for standard JDK 1.4 features such as:
Support for Logging (JSR-041)
Complete Security (JAAS, JCE)
Support for Preferences API (JSR-010), a replacement for Properties
Assertions (JSR-041)
Exception chaining
Support for Headless AWT allows the manipulation of graphic objects without materializing the UI, and therefore without throwing an exception
Regular expression java.util.regex
JDBC 3.0 support in the server-side JDBC
The JDK VM stores, loads, and runs Java sources/classes from the file system; the OracleJVM stores each Java class (and derived classes), its source, the potential/optional Java resources (used for properties and miscellaneous needs), and the metadata in a managed, stand-alone database object called “libunit” (short for library unit). Let’s see the storage structures involved.
Unlike PL/SQL sources, which are stored in human-readable format as VARCHAR columns of system tables, Java sources are stored in nonhuman-readable format as Opaque types,[1] using UTF8 character set, in internal system tables. Upon the reception of the instruction to create a new managed Java source—from SQL*Plus interactive session (“CREATE JAVA SOURCE...”), from client-side loadjava utility, or from server-side dbms_java.loadjava()[2] or IDEs[3]—the SQL DDL creates a library unit (“libunit”) in the internal system table. It is also possible to create managed Java sources from existing Java sources in external system files using the BFILE mechanism (described later) or regular database structures such as VARCHAR, BLOB, CLOB, or any other mechanism that can transmit large sequences of characters separated by newlines (i.e., source code).
Java resources allow storing, of properties files but also .xml, .jar, .zip files in the database, as well as any arbitrary file type used by Java classes and applications running in the database. They are used mainly during the resolution of Java classes. They are not compiled by the OracleJVM. They are loaded as is in the database using the loadjava utility or the SQL command CREATE JAVA RESOURCE NAMED "<name> ... . The resources objects are looked up by the Java VM (within the scope of the resolver spec of the calling class), using ClassLoader.getResource() or ClassLoader.findResource() methods, which return a URL of the actual resource. The Native Java compiler (described later) stores platform-specific executables (i.e., DLL) in the database as Java resource. Here is a basic example of a config.properties file:
Example 2.1. Propty.sql
create or replace and resolve java source named propty as
import java.sql.*;
import java.util.*;
import java.io.*;
public class propty
{
public static void getppty(String fname)
throws java.io.IOException
{
InputStream is = null;
// Get input stream from the properties file
is = ClassLoader.getSystemResourceAsStream(fname);
Properties p = new Properties();
p.load(is);
is.close();
// Print poperties values
System.out.println ("foo in config file: " +
p.getProperty("foo"));
System.out.println ("bar in config file: " +
p.getProperty("bar"));
System.out.println ("url in config file: " +
p.getProperty("url"));
System.out.println ("sid in config file: " +
p.getProperty("sid"));
}
}
/
show errors;
create or replace procedure getproperties
as language java name
'propty.getppty(java.lang.String)';
/
show errors;
rem At this stage, you need to grant scott the permission to read the
the properties file for loading
rem
rem SQL> connect sys as sysdba
rem Enter password: *****
rem Connected.
rem SQL> call dbms_java.grant_permission
rem ( 'SCOTT','java.io.FilePermission',
rem 2 'C: empconfig.properties', 'read' );
rem
rem Call completed.
rem
rem SQL>connect scott/tiger
rem
call dbms_java.loadjava('-v -f C: empconfig.properties'),Notice the literal name of the resource object that is created (with the “ROOT/” prefix):
SQL>select object_name from user_objects where object_type = 'JAVA
RESOURCE';
OBJECT_NAME
----------------------------------------------------
ROOT/tmp/config.properties
set serveroutput on
Call dbms_java.set_output(50000);
call getproperties('ROOT/tmp/config.properties'),Alternatively, you may archive the resource file in a JAR file as follows:
C:Book>jar -cf resource.jar config.properties C:Book>jar -tf resource.jar META-INF/ META-INF/MANIFEST.MF config.properties
Notice that dbms_java.loadjava will explode the JAR file; the config.properties file can now be referenced without the “ROOT/” prefix:
SQL*Plus scott/tiger
SQL>call dbms_java.loadjava('-v -f /tmp/resources.jar'),
SQL>select object_name from user_objects where object_type = 'JAVA
RESOURCE';
OBJECT_NAME
--------------------------------------
META-INF/MANIFEST.MF
config.properties
SQL>call getproperties('config.properties'),In the current implementation, a Java class must be physically stored in the database before it can be executed. You can compile the Java source on the client-side and then upload the resulting class, but you can load the source file and compile directly within the database. During the in-database compilation phase (i.e., using “ALTER JAVA COMPILE..." or "create or replace and resolve java source.."), the Java source is “loaded” into the compiler’s memory by reading the previously stored text out of the internal system table. This is to ensure that the original Java source has not been altered out of the control of the DDL since it has been loaded—such alteration could break the dependencies maintained within the OracleJVM. The compiler produces a set of bytecodes. During the compilation phase, the references to other objects are “resolved” by a “resolver,” using the “resolver specification,” which is similar to the notion of CLASSPATH in JDK VMs (see section 2.1.6). However, a resolver specification is used only once to resolve referenced names to other classes. Once the resolution has been done, the name bindings persist indefinitely. On the other hand, the CLASSPATH may be different at each invocation, which may bind different classes to the referenced names.
Optionally, but recommended, the bytecodes can be compiled into machine code (static native compilation), to accelerate the execution speed; in this case, the native compiler library information (or metadata)—including the name of the DLL that contains the machine code, the name of the function stored in the DLL that loads the class, and flags—is associated with the class.
In summary, a “stand-alone Java object” or “managed Java object” contains the source text (if loaded in the database), the bytecode of the class, the “resolver specification,”[4] OracleJVM-specific metadata, and, optionally, the native compiler (NCOMP) metadata. It is stored as a “library unit.” Similar to PL/SQL objects, the metadata of “Java objects” does not currently reference external objects. During runtime, the bytecode of the Java classes is loaded into shared memory (see section 2.2).
Java/J2EE applications are usually packaged as JARs (Java ARchives), WARs (Web ARchives), and EARs (Enterprise ARchives). The OracleJVM does not currently handle JARs as a unit of deployment; as a result, signed JARs[5]are not currently supported. However, it allows uploading JAR and ZIP files into the database. Once loaded, the JAR and ZIP files are unbundled into individual classes before compilation and resolution. By default, JAR files are treated as resource files (only looked up during runtime) and are not processed recursively. However, the “–recursivejars” option of loadjava forces the recursive processing of JAR files.
In section 2.1.3, I have described how Java is stored in the database. In order to support thousands of simultaneous sessions and users, the OracleJVM must reduce the average memory footprint per session. This goal is achieved through class sharing, both for system and user classes.
The following classes and metadata, from largest sized to smallest sized, are shared between all clients in the same database instance: the bytecodes of interpreted methods, the in-memory representation of the constant pool,[6] the virtual method tables (i.e., a list of pointers to actual methods that support polymorphic[7] methods), the list of implemented methods and fields, the “hot-loaded” states, and various implementation-specific metadata. Figure 2.1 pictures the sharing of system classes.
Each active Java class comprises a shared part and a nonshared part, described here as follows:
The read-only and hence shareable part includes the bytecodes and the method tables. Sharing the read-only part of Java classes results in smaller average memory footprint and scalability.
The unshared part contains the static class variables; this private part is carved out of the Sessionspace (a session memory area described later).
The invocation of Java methods causes the read-only part to be brought into the Shared Global Area (SGA) by the class loader—if not yet present, since there is only one copy per instance—and the creation of the private part. Keeping private copies of Java classes in every schema would be a waste of resources and would quickly become a maintenance nightmare. Figure 2.1 depicts the sharing of user classes. The best practice to derive from this is that applications libraries should be designed to be shareable across multiple database schemas.
Applications class sharing occurs automatically as soon as two or more schemas share the same classes and jars. At any given time, there is only one in-memory (SGA) representation of each Java class per database instance.
To further reduce the per-session footprint, OracleJVM also uses a memory space in SGA to hold string literals that are common to several Java classes and arrange these in a way that allows quick search and retrieval.
Another memory optimization technique, known as “class hot loading,” consists of sharing constant states across all sessions, based on the observation that many system classes have static variables that are initialized to the same value across all sessions and then never changed (i.e., constant values). This technique is implemented by saving in system tables the static states of selected system classes, during the installation of Java (part of or post database installation), and then restoring the saved states into shared memory when loading the classes in question.
By design (e.g., scalability, data integrity), the Oracle database sessions are isolated and watertight, whereas the Oracle JVM manages to share selected system objects. This sharing is achieved by persisting selected read-only objects into the database and then loading these into shared memory. Sharing read-only system objects significantly reduces the per-session memory. This technique is used internally only by the OracleJVM and Oracle products that use Java in the database but is not productized and therefore not exposed or available for user objects.[8]
As explained earlier, the session architecture of the Oracle database does not allow sharing user data across sessions. This lack of data sharing across sessions is often pointed out as a limitation of Oracle’s implementation of Java in the database. In the JDK VM world, sharing across threads works fine within a single Java VM boundary; however, this perceived advantage goes away in a multiple Java VM environment, which is now the norm for large-scale deployment. There is a growing need (i.e., scalability, security) for interapplication isolation. The JSR 121,[9] which specifies the Application Isolation API, is a materialization of such a requirement. OracleJVM already offers interapplication isolation by design without the need for a specific API (a by-product of session isolation) but will support JSR 121 when it is finalized.
Although OracleJVM maintains J2SE compatibility, it currently differs from the JDK VM in terms of architecture, scalability, robustness behavior, threading, invocation mechanisms, isolation, and security.
Session isolation is the foundation for scalability and data integrity in the Oracle database; an application should not be able to see/read uncommitted data or update another application’s data. The OracleJVM is integrated into this architecture while keeping system resources (needed by each session) minimal to maximize the number of concurrently executing Java sessions on a given database instance. Conceptually, as illustrated in Figure 2.1, there is a dedicated, private Java VM per session. In reality, all sessions share the same JVM system classes—only statics and private states are kept in individual session space, resulting in unbound scalability as far as the capacity of the system permits. Row-level locking and multilevel read consistency ensure that one application would not be able to prevent another application from performing its database operations. Session isolation, row-level locking, and multilevel read consistency result in high concurrency or scalability.
As a poster on Slashdot.org stated: “We were doing some benchmarking and found that our app ‘died’ at about 3,000 sessions (as defined by us) in the tweaked/tuned native JDK JVM, whereas the OracleJVM handled almost 150,000 before croaking.”
Because each session has its own “virtual Java VM,” a process failure impacts only the virtual JVM of the session being serviced by the failed process; as a result, it does not crash as a whole (unless the entire system crashes), and it has the same robustness as the Oracle database.
JDK-based Java VMs are multithreaded (i.e., use threads for scalability to serve multiple users or requests concurrently). OracleJVM, on the other hand, is based on the database session model, which is a single-client nonpreemptive model. Although Java in the Oracle database allows running threaded programs, such as SQL and PL/SQL, it is single threaded at the execution level. To be clearer, multiple clients or threads cannot be multiplexed to run simultaneously within the same process. Therefore, unless the currently running Java thread completes or blocks on socket operation or voluntarily pauses (i.e., calls java.lang.Thread.yield()), another thread will not be capable of running. This model is independent of the server process type (i.e., dedicated or shared server); even in shared server mode, clients are scheduled and run serially. This model increases thread safety and robustness by reducing the need for object locking (i.e., synchronization) and therefore reduces the likelihood of deadlocks; it also simplifies memory management and Garbage Collection.
Currently, and unlike JDK VMs, the outputs of OracleJVM go to the trace files (.trc) associated with the session (process). However, you can redirect the standard output to the console (i.e., the SQL output) using the DBMS_JAVA.SET_OUTPUT() method. But, the output is only printed when the stored procedure exits, and this setting works only for one call (i.e., the SQL call that immediately follows the invocation of DBMS_JAVA.SET_OUTPUT()). The minumum and default value is 2,000 characters and the maximum is 1,000,000 (1 million) characters. Notice the “SET SERVEROUTPUT ON” command, which enables displaying the outputs of stored procedures (Java or PL/SQL blocks) in SQL*Plus.
Usage: SQL> SET SERVEROUTPUT ON SQL> call dbms_java.set_output (5000);
See examples in Listing 2.1.
The top-level invocation of Java is primarly done via a PL/SQL wrapper called Call Spec. As of Oracle Database 10g, you can also use a JPublisher-generated client-stub, which shields you from defining a Call Spec at the expense of the overhead of serializing/deserializing parameters and return values (see “Developing and Deploying Java in the Database” in Chapter 4 for more details). The notion of CLASSPATH is replaced by the corresponding notion of “Resolver Spec,” which is short for “Resolver Specification” (see section 2.3 later in this chapter for more details). To be executed, the Java classes must reside in database schemas, and all references to other classes and objects must be “resolved” within the database (i.e., cannot reference external classes).
Starting from JDK1.4, the preferences API (in package “java.util.prefs”) allows Java applications to persist small amounts of states. Designed as a replacement for java.util.Properties, the Preferences API provides persistent storage and retrieval of key-value pairs, mostly to store user and system configuration values as well as GUI/Window positions. The system preferences are available to all users, whereas user-specific preferences are private to that user. In Oracle Database 10g, OracleJVM is J2SE 1.4 compatible and therefore implements java.util.prefs.Preferences. The JDK implementation uses platform-dependent mechanisms to persistently store the name/value pairs, such as registries on Windows and xml file on UNIX. The OracleJVM implementation uses a platform-independent storage (i.e., an internal database table); the name is restricted to VARCHAR2(2000); and the value is stored as a BLOB. In addition to supporting the standard API for setting and retrieving the stored values, the OracleJVM implementation also offers a PL/SQL interface dbms_java.set_preference() for setting preferences. However, there currently is no symmetric function to retrieve the value (i.e., dbms_java.get_preferences()) .
The PL/SQL interface, which is implemented by the Java method, is as follows:
procedure set_preference(user varchar2, type varchar2, abspath varchar2, key varchar2, value varchar2); user: <login schema> or 'SYS' type: User preference('U') or System preference ('S') abspath: absolute path key: for looking up the value value: the value associated with the key Usage: Setting preferences using dbms_java.set_preferences() SQL> call dbms_java.set_preference('SCOTT','U', '/my/package/class/method','CONSTANT','20'),
The Java method:
java.util.prefs.OraclePreferences.DbmsSetPreference(
java.lang.String, java.lang.String, java.lang.String,
java.lang.String, java.lang.String);See java.util.prefs.Preferences[10] for more details on the API itself.
As part of the Oracle database code, the OracleJVM is built once and then ported to the various platforms where the database code runs; as a result, its behavior is consistently the same across all of the platforms where the Oracle database is available; most of you are familiar with the headaches incurred by the difference in behavior of JDK VM across platforms. As already mentioned, there is no need to install or upload an external JDK or JRE to run Java in the database; however, you need to make sure that the Java option is selected during the database installation.
OracleJVM also differs from the JDK VM in terms of Java 2 Security implementation and permission administration, which are covered in detail in section 2.3 in this chapter.
The Oracle database limits system resource consumption through DBA-defined profiles (i.e., CREATE PROFILE p, ALTER USER u PROFILE p). In addition, the Database Resource Manager allows you to group users and applications into resource consumer groups and allocate a percentage of CPU resources to each consumer group through a resource plan. The Database Resource Manager is not addressed in this book, but the OracleJVM enforces a database resource profile. Defining and enforcing resource limitations on a schema involves the following steps:
Enable resource limitation at the database system level using either of the following two ways:
Statically, in the database initialization file, using the
RESOURCE_LIMIT parameter RESOURCE_LIMIT = true | false
Dynamically, using the “
ALTER SYSTEM” command (requiresALTER SYSTEMprivilege)ALTER SYSTEM RESOURCE_LIMIT = true | false
Define a resource profile: a set of limits on database and system resources (requires
CREATE PROFILEprivilege).Example CREATE PROFILE greedy LIMIT SESSIONS_PER_USER UNLIMITED CPU_PER_SESSION UNLIMITED CPU_PER_CALL 4000 CONNECT_TIME 50 LOGICAL_READS_PER_SESSION DEFAULT LOGICAL_READS_PER_CALL 2000 PRIVATE_SGA 30K COMPOSITE_LIMIT 5000000;
Assign the defined profile to a user (schema).
ALTER USER Java_nerd1 PROFILE greedy;
See the Oracle Database 10g SQL Reference for more details on defining resource limitations.
One of the motivations for embedding Java in the database is to take advantage of the close proximity to the SQL engine for efficient data access and manipulation. OracleJVM furnishes a type-2 server-side JDBC driver, as well as a SQLJ runtime, which runs in the same address space as the SQL engine. The Oracle JDBC drivers are covered in detail in Part II; in this section, I will briefly address the server-side JDBC driver, which is used when calling SQL directly from Java code running in the database. This JDBC driver is instantiated and preregistered at Java initialization in the session. Java initialization happens during the first invocation of Java since the creation of the session. Because you are already connected to the database by the time you invoke Java, the connection object you get by invoking getConnection() is a mere handler to the default connection. The following three methods look syntactically different but all return the same kind of connection (i.e., a lightweight server-side default connection handler):
Connection conn =DriverManager.getConnection("jdbc:default:connection:");Connection conn =DriverManager.getConnection ("jdbc:oracle:kprb:");OracleDriver t2server = new OracleDriver();Connection conn = t2server.defaultConnection();These syntaxes are required because the OracleJVM does not yet allow associating a JDBC data-source name with the actual connect string through the JNDI namespace. The consequence of this limitation is that you need to code the connection section in such a way that you can determine where your code is running and then use the right connection string.
/* * Where is your code running: inside or outside the database? */ Connection conn; if (System.getProperty("oracle.jserver.version") != null) { /* * You are in the database, use the default connection */ conn = DriverManager.getConnection("jdbc:default:connection:"); } else { /* * You are not in the database, use standard JDBC connection */ DriverManager.registerDriver(new oracle.jdbc.OracleDriver()); conn = DriverManager.getConnection("jdbc:oracle:thin:", "scott","tiger"); }
Alternatively, you may use properties files to adapt and pass-in the JDBC URL.
All SQL operations in OracleJVM are part of the current transaction; however, because auto-commit is disabled (not supported is the right statement), you must explictly COMMIT or ROLLBACK all of your changes. JDBC best practices recommend closing all statements and result sets when they are no longer needed. However, the close() method closes the connection object instance but not the physical database connection; you cannot close this one as long as the session lives.
For accessing remote Oracle databases from within Java in the database, a type-4 “Server-side JDBC-thin” is provided. Existing JDBC applications that access a remote Oracle database should work as is, but the cancel() and setQueryTimeout() methods are not supported. Also, in order to open a socket from within the server-side JDBC-thin, the database session must be granted java.net.SocketPermission as follows:
SQL> create role CALLOUT;
SQL> call dbms_java.grant_permission('CALLOUT',
'java.net.SocketPermission',
'*', 'connect' );
SQL> grant callout to scott;See section 2.3, for more details on permissions, and “JDBC Call-out to Non-Oracle Databases” in Chapter 4 for an example of accessing a remote non-Oracle database from within the Oracle database.
Both server-side drivers (type-2 and type-4) and their classes are automatically installed as part of the database install (when the Java option is selected) or during manual OracleJVM reinstall. In Part II, all Oracle JDBC driver types are compared, including their features’ differences.
DMBS_JAVA is a comprehensive package of PL/SQL wrappers (see the “PL/SQL Packaged Call Spec” section in Chapter 4) that exposes a set of Java methods for administering the OracleJVM from SQL (and, therefore, JDBC). You will see a lot of references to DBMS_JAVA throughout the book, so that is why I am introducing this topic at this early stage. DBMS_JAVA contains documented interfaces for DBAs and Java developers but also undocumented interfaces that are used internally. Each interface is in the form of a DBMS_JAVA.<Java_method()> procedure of function and each has a corresponding Java method that implements the procedure or function in question. However, the implementation classes and the Java methods may change from one release to another. Therefore, it is highly recommended to use only the PL/SQL interface (or wrapper); your applications will not brake throughout successive database releases. The corresponding DBMS_JAVA interfaces will be highlighted under the appropiriate sections (e.g., security, compiler setting).
Let’s look at one example of DBMS_JAVA usage through the longname/ shortname interfaces.
The maximum length of a SQL identifier or a database schema object name is 31 characters, and all characters must be legal and convertible to the target database character set. The full name of Java classes and methods usually exceeds this limit and may contain illegal characters such as dots. To work around this issue, the Oracle database uses abbreviated names (i.e., shortname) internally, but maintains a correspondence between this shortname and the full name (i.e., longname), as well as the methods used to transform shortname into longname and vice versa. A longname is only generated when the actual name is regarded as invalid (i.e., it has passed the maximum size or contains illegal characters). The longname and shortname functions have been designed to help you query shortname and longname of your Java classes, sources, and resources.
longname: Returns the longname of a class when given the shortname, if it exists.
FUNCTION longname (shortname VARCHAR2) RETURN VARCHAR2 as
language java name
'oracle.aurora.rdbms.DbmsJava.longNameForSQL(java.lang.String
) return java.lang.String';shortname: Returns the shortname of a class when given the longname.
FUNCTION shortname (longname VARCHAR2) RETURN VARCHAR2 as
language java name
'oracle.aurora.rdbms.DbmsJava.shortName(java.lang.String)
return java.lang.String';
.Example: Let’s create a Java source and class where the package name contains illegal characters (e.g., dots are illegal, from a database object name perspective) and see its conversion.
SQL>CREATE OR REPLACE AND RESOLVE JAVA SOURCE NAMED LongNameSample AS
package longnamepkg.subpackage.foo;
class LongNameSample {
public static void main (String argv[]) {
System.out.println ("Hello LngNameSample");
}
}
/
Java created.
SQL> show errors;
No errors.
SQL>
SQL> col DBMS_JAVA.LONGNAME(OBJECT_NAME) format a45
SQL> col DBMS_JAVA.LONGNAME(OBJECT_NAME) format a45
SQL> select dbms_java.longname(object_name), object_type, status from
user_objects
2 where object_type not in ('TABLE', 'VIEW', 'INDEX') and
3 dbms_java.longname(object_name) like '%LongNameSample'
4 order by dbms_java.longname(object_name)
5 /
DBMS_JAVA.LONGNAME(OBJECT_NAME) OBJECT_TYPE
STATUS
--------------------------------------------- -----------------------
longnamepkg/subpackage/foo/LongNameSample JAVA CLASS
VALID
SQL> col DBMS_JAVA.SHORTNAME(OBJECT_NAME) format a40
SQL> l
1 select dbms_java.shortname(object_name), object_type, status
2 from user_objects
3 where object_type not in ('TABLE', 'VIEW', 'INDEX')
4 and dbms_java.shortname(object_name) like '%LongNameSample'
5* order by dbms_java.shortname(object_name)
SQL> /
DBMS_JAVA.SHORTNAME(OBJECT_NAME) OBJECT_TYPE STATUS
---------------------------------------- ------------------- -------
/3277131c_LongNameSample JAVA CLASS VALIDThe goal of the OracleJVM memory manager is to map Java memory management semantics (automatic memory management) to the Oracle database memory management framework. I will first introduce the memory structures of the Oracle database—including SGA, CGA, PGA, and UGA (see section 2.2.1), and then describe the OracleJVM memory structures, memory allocation, and Garbage Collection techniques. Once again, this information might appear unnecessary for Java developers, but remember this is Java in the database, and you have to have the big picture by factoring in the overall RDBMS memory management. You will learn how to retrieve and alter the default values of memory areas. Another reason why you want to know more about memory management for Java in the database is when the DBA sets a maximum size target for the instance’s shared memory, a.k.a SGA (i.e., setting PGA_AGGREGATE_TARGET parameter), it influences the behavior of the Garbage Collector in Java-enabled sessions. In this case, when the overall RDBMS memory allocation approaches PGA_AGGREGATE_TARGET, the OracleJVM will make sure to use less memory at the expense of the execution speed.
In order to understand the memory management of OracleJVM, you need a basic knowledge of the memory structures of the Oracle database. Those familiar with the Oracle database structures can skip this section and go to section 2.2.2. The installation, configuration, and administration of the database is beyond the scope of this book; however, for those interested in these topics, I recommend the Oracle Database 2 Day DBA documentation[11]—a concise, task-oriented book that will get you up to speed with the Oracle database in two days!
A Database Instance: An Oracle database instance comprises one (and only one) SGA and a set of database processes. A database is usually accessed and managed by a single instance. However, an Oracle database can also be concurrently accessed and managed by multiple instances—Oracle’s Real Application Clusters (RAC) technology[12]—for scalability and high-availability requirements. RAC is also the cornerstone of Oracle database deployment in Enterprise Grid environments (which will be addressed in Part II of this book). Figure 2.2 depicts the key memory structures of an Oracle database instance.
The Shared Global Area (SGA): This is a shared memory area that contains data and control information, which is visible and accessible to the server processes that are part of the database instance. The data buffer cache, the redo buffer, and the various pools are part of the SGA and therefore visible and accessible to all Oracle server processes. The SGA has the same lifetime as the database instance.
The Process Global Area (PGA): This is a memory area that contains data and control information, which is private, visible and accessible only to a server process (dedicated or shared). The PGA has the same lifetime as the server process.
The User Global Area (UGA): This is a memory region associated with a user session. With dedicated server processes, the session is tied to the process; therefore, the UGA is allocated out of the PGA. With shared servers, the session is not attached to any process; therefore, its UGA is allocated out of the SGA.
CGA (Call Global Area): This is a memory area allocated out of the PGA (dedicated or shared servers) for the duration of a SQL call; its contents are visible and accessible only to this call. It is freed at the end of the call.
In the OracleJVM memory manager lexicon, “object” refers to a program state, and “object memory” or “objmem” refers to a memory space used to hold live objects as well as garbage objects. Live objects are objects that a program will likely access at some point in the future, as opposed to garbage objects, which are regarded as dead objects. Before examining the various object memories, their allocation, and clean-up policies, let’s first define a baseline in terms of memory allocation and clean-up vocabulary and techniques used by the OracleJVM.
OracleJVM uses mainly the “frontier consing” and “buddy allocation” memory allocation techniques for the various object memories.
Frontier consing (or “cons operation”) is a very fast allocation strategy, which consists of incrementing a pointer that maintains a frontier boundary between allocated and unused memory space in the heap.
Buddy allocation is a strategy that rounds up memory allocation requests to predetermined sizes (e.g., next multiple of 16, next power of 2). For example, an allocation request for 10-Kb will result in a 16-Kb space allocation, a request for 28-Kb will result in a 32-Kb allocation and so on.
Garbage Collection designates a set of techniques used by Java VM for automatic memory reclamation; those familiar with these techniques can skip this section. Garbage Collection generally consists of three steps: (1) mark, (2) collect, and (3) clean-up. The mark step finds the objects that need to be retained. The collect step removes unused objects. The clean-up step returns the reclaimed memory to the pool of free memory (i.e., return to a usable state). The OracleJVM memory manager uses a set of GC techniques for its various memory structures (listed in Table 2.1), including the generational GC, mark-sweep GC, and copy GC.
Table 2.1. Summary of OracleJVM Memory Structure
Name | Contents | Location | Property | Allocation | GC Type |
|---|---|---|---|---|---|
New-space | Small & New Objects | PGC (CGA[*]) | Contiguous, fixed-size | Frontier Consing | Copy GC |
Old-space | Large & Old Objects | PGC (CGA[*]) | Contiguous, fixed-size | Buddy | Mark-Sweep |
Stackspace | Java Stack Stratch Objects | PGC (CGA[*]) | Segmented, may grow | Frontier Consing | No GC |
Sessionspace | Java Objects (Session Lifetime) | PGC (UGA[*] Javapool) | Unpaged, may grow (Paged[*]) | Buddy/(Frontier Consing[*]) | Mark-Sweep (Copy GC[*]) |
Run-space | System Objects | PGC (CGA[*]) | Contiguous, fixed-size | Buddy | Mark-Sweep |
[*] Shared Server mode. | |||||
Generational GC is a technique that segregates memory areas by their age (i.e., generations). Allocations happen in the most recent area. When an area is full, a collector is used to “move” the live objects to the next older area (hence, the term generational). Different collection algorithms may be used across generations. As a result, most objects are reclaimed upon the first GC, resulting in many inexpensive GCs instead of fewer large ones.
A mark-sweep GC consists of two phases: (1) the mark phase (garbage detection) and the (2) sweep phase (Garbage Collection). The sweep phase places the “garbaged” memory on the freelists. The allocation is serviced out of the freelists. Lazy sweeping is a variation technique that marks objects normally, but, in the sweep phase, leaves it up to the allocation phase to determine whether the marked objects are live (no need to reclaim) or not (can be reclaimed), rather than sweeping the object memory at that time.
Copy GC consists of recursively copying objects, referenced by root objects, into an empty region of the object memory. It combines marking and compacting into one operation. This technique is very efficient when most of the objects need to be reclaimed—which is the case most of the time.
In reference-counting GC, each object stores a count of the number of references to it. This count is incremented every time a reference to that object is stored and decremented every time a reference to that object is overwritten. The object is reclaimed when the count reaches zero.
Starting with Oracle 9i Release 2, the Garbage Collector automatically and dynamically adjusts the threshold at which the GC is triggered; for most applications, you don’t have to play with GC thresholds, using OracleRuntime.getSessionGCThreshold() and OracleRuntime.setSessionGCThreshold(). In Oracle Database 10g Release 2, the GC supports PGA_AGGREGATE_TARGET and a smoother memory growth.
Supporting PGA_AGGREGATE_TARGET: The DBA or database developers with DBA privileges can set the PGA_AGGREGATE_TARGET[13] parameter, which will instruct the GC to trade memory for speed and vice versa. The values differ as follows:
Low values optimize memory at the expense of execution speed. In other words, if the only active Java thread at the end of the top-level call is the main thread (i.e., there are no user threads),[14] then perform a full GC of oldspace to free up memory, irrespective of the type of server (dedicated or shared). This results in optimizing the PGA space usage, much like the end-of-call migration, which happens systematically in pre-10g and only in shared server mode in 10g.
High values optimize the execution speed at the expense of memory. In other words, state migration is not performed at the end of the call, when in dedicated mode.
Smoother Memory Growth: In Oracle database 10g Release 2, the clean-up and growth of the memory space used for holding long-lived or large objects (i.e., Oldspace) is smoother. In previous releases, the growth of Oldspace was too aggressive, resulting in that a user who allocates a lot of objects and then drops these ends up with a large unused memory space. In 10g Release 2, Oldspace grows only if the growth is inversely proportional to the amount of space freed. In other words, Oldspace cannot grow more than 100 percent of its current size.
In OracleJVM, the states of Java objects are preserved in special data structures called “object memories.” There are several object memory types, including Newspace, Oldspace, Runspace, Stackspace, and Unscanned Stackspace. Each object memory type is used for a specific need. Why would you care? Well, although the default sizes of the various memory areas, described as follows, suffice for most applications, some Java applications or frameworks running in the database might have extreme memory requirements; in this case, you want to know the default allocations and behavior of the Garbage Collector. Table 2.1 summarizes what each object memory type is used for and how these are allocated and cleaned (i.e., garbage collected).
Sessionspace, also called “session memory,” is an object memory residing in the User Global Area (UGA), which is associated with each session (each session has its own Sessionspace) and lives as long as the session lives. Sessionspace is used to preserve state of system objects, user objects, and metadata used by the Java runtime. In shared server mode, the UGA resides in the SGA (hence, the Sessionspace), and the states of the objects that are live at the end of the previous call are recursively copied from the Call Global Area (see Newspace and Oldspace in the following text) to the Sessionspace; this process is called “end-of-call migration” or “session migration.” In dedicated server mode, the Sessionspace resides in the PGA (as opposed to the SGA); starting with release 10g, the end-of-call migration is no longer performed when in dedicated mode. You can programmatically get the size of the Sessionspace using OracleRuntime.getSessionSize() (see memtest.sql code sample).
Oldspace is an object memory used for holding long-lived or large objects (i.e., larger than 1-K Bytes) for the duration of a call. It is cleaned up using Marksweep GC (described earlier) for large objects; it uses a variation of “lazy sweeping” for small objects.
Runspace is an object memory residing in CGA or PGA and is used to hold system objects (e.g., classloader objects) allocated for the duration of a call. It is managed (allocation and clean-up) in the same way as Oldspace. You can use the OracleRuntime.getMaxRunspaceSize() method to see the current value in your system (see the following memtest code sample in Listing 2.2).
Stackspace is an object memory residing in CGA or PGA and is used to allocate system objects and objects used by the Java interpreter (no user objects). Memory allocation in stackspace is achieved using the “Frontier Consing” technique. You can use OracleRuntime.getJavaStackSize(), OracleRuntime.setJavaStackSize(), OracleRuntime.getThread-StackSize(), and OracleRuntime.setThreadStackSize() to get and set the size of the Stackspace (see the following memtest code sample in Listing 2.2).
Newspace is the default memory for allocating almost all Java objects, except large objects, which are allocated in Oldspace. Memory allocation in newspace is achieved using “Frontier Consing,” and Garbage Collection is done using the “Copy GC” technique. You can use OracleRuntime.getNewspaceSize() and OracleRuntime.setNewspaceSize() to programmatically get and set the size of Newspace (see Listing 2.2).
Example 2.2. Memtest.sql
create or replace and resolve java source named memtest as
import oracle.aurora.vm.OracleRuntime;
public class memtest
{
/*
* The following code sample is provided for for illustration purposes
only.
* The default values should work for most applications.
* Before altering these values for your production system, please
* go under tuning and testing exercises beforehand
*/
public static void Tests ()
{
System.out.println("getSessionSize(): "
+ OracleRuntime.getSessionSize());
System.out.println("Old NewspaceSize(): "
+ OracleRuntime.getNewspaceSize());
OracleRuntime.setNewspaceSize(2 *
OracleRuntime.getNewspaceSize());
System.out.println("New NewspaceSize(): "
+ OracleRuntime.getNewspaceSize());
System.out.println("Old MaxRunspaceSize(): "
+ OracleRuntime.getMaxRunspaceSize());
OracleRuntime.setMaxRunspaceSize(2 *
OracleRuntime.getMaxRunspaceSize());
System.out.println("New MaxRunspaceSize(): "
+ OracleRuntime.getMaxRunspaceSize());
System.out.println("getJavaPoolSize(): "
+ OracleRuntime.getJavaPoolSize());
System.out.println("getSessionSoftLimit(): "
+ OracleRuntime.getSessionSoftLimit());
System.out.println("Old SessionGCThreshold(): "
+ OracleRuntime.getSessionGCThreshold());
OracleRuntime.setSessionGCThreshold(2 *
OracleRuntime.getSessionGCThreshold());
System.out.println("New SessionGCThreshold(): "
+ OracleRuntime.getSessionGCThreshold());
System.out.println("Old NewspaceSize: " +
OracleRuntime.getNewspaceSize());
OracleRuntime.setNewspaceSize(2 *
OracleRuntime.getNewspaceSize());
System.out.println("New NewspaceSize: " +
OracleRuntime.getNewspaceSize());
System.out.println("Old MaxMemsize: " +
OracleRuntime.getMaxMemorySize());
OracleRuntime.setMaxMemorySize(2 *
OracleRuntime.getMaxMemorySize());
System.out.println("New MaxMemsize: " +
OracleRuntime.getMaxMemorySize());
System.out.println("Old JavaStackSize(): "
+ OracleRuntime.getJavaStackSize());
OracleRuntime.setJavaStackSize(2 *
OracleRuntime.getJavaStackSize());
System.out.println("New JavaStackSize(): "
+ OracleRuntime.getJavaStackSize());
System.out.println("Old ThreadStackSize(): "
+ OracleRuntime.getThread-StackSize());
OracleRuntime.setThreadStackSize(2 *
OracleRuntime.getThreadStackSize());
System.out.println("New ThreadStackSize(): "
+ OracleRuntime.getThreadStackSize());
}
}
/
show errors;
create or replace procedure memtests
as language java name
'memtest.Tests()';
/
show errors;
set serveroutput on
Call dbms_java.set_output(50000);
call memtests();The Java VM runtime was initially designed for the shared server mode and reused as is in dedicated mode. In mode, because there is no guarantee that subsequent calls (within the session) will be executing in the same server process, an end-of-call migration is performed in order to preserve program/objects state across calls in the Sessionspace. Starting with Oracle database 10g Release 1, the memory management has been optimized for dedicated servers; in this case, the session always runs in the same process, and an end-of-call migration is not necessary for preserving states across calls. As a result, threads remain quiescent, and file handles persist (remain open) across calls. Applications that needed to keep file handles open across calls will now work properly. Use the (SERVER = SHARED) or (SERVER = DEDICATED) connection descriptor in the tnsnames.ora file to specify the server type. See the Oracle Database Administration Guide for more details on configuring shared server or dedicated servers.
Figure 2.3 summarizes the location of the main object memories in a dedicated server environment. The user memory (UGA) is allocated out of the dedicated process memory (PGA) and not out of the Javapool in SGA.
Figure 2.4 summarizes the location of object memories in a shared server environment. The user memory (UGA) is allocated out of the Javapool in global database instance memory (SGA). End-of-call migration is enforced in this case.
The size of the Javapool in SGA is defined by the JAVA_POOL_SIZE parameter. JAVA_POOL_SIZE specifies (in bytes) the size of the instance-wide Javapool, from which the Java class loader allocates space for bytecodes and other read-only class metadata when a class, not actually in SGA, needs to be loaded. This memory also contains the Java states that are migrated from the Session space at the end of call. You can see its value using the SHOW PARAMETER command under SQL*Plus using SYSTEM or SYS schemas, or programmatically from any session, using OracleRuntime.getJavaPoolSize() (see Listing 2.2). Starting with Oracle database 10g, JAVA_POOL_SIZE can be dynamically adjusted through the ALTER SYSTEM command.
JAVA_SOFT_SESSIONSPACE_LIMIT specifies (in bytes) a soft limit on Java memory usage in a session. When a user’s session exceeds this size, Oracle generates a warning that goes into the trace files. You can also use OracleRuntime.getSessionSoftLimit() to programmatically get the value of the soft session space limit.
JAVA_MAX_SESSIONSPACE_SIZE specifies (in bytes) the maximum amount of session space made available to a Java program executing in the server. When a user session attempts to exceed this limit, the session is killed, with an out-of-memory message. You can programmatically get its value, using OracleRuntime.getSessionSize(). See the previous memtest code sample.
Also, two new views, V$JAVA_POOL_ADVICE and V$JAVA_LIBRARY_CACHE_MEMORY, have been added to help enforce the Self-Tuning SGA.
V$JAVA_POOL_ADVICE displays information about estimated parse time in the Javapool for different pool sizes.
Column | Datatype | Description |
|---|---|---|
|
| Javapool size for the estimate (in megabytes) |
|
| Size factor with respect to the current Javapool size |
|
| Estimated memory in use by the library cache (in megabytes) |
|
| Estimated number of library cache memory objects in the Javapool of the specified size |
| Estimated elapsed parse time saved (in seconds), owing to library cache memory objects being found in a Javapool of the specified size. This is the time that would have been spent in reloading the required objects in the Javapool if they had aged out because of insufficient amount of available free memory. | |
|
| Estimated parse time saved factor with respect to the current Javapool size |
|
| Estimated elapsed time (in seconds) for parsing in a Javapool of the specified size |
|
| Estimated load time factor with respect to the current Javapool size |
|
| Estimated number of times a library cache memory object was found in a Javapool of the specified size |
V$JAVA_LIBRARY_CACHE_MEMORY displays statistics about memory allocated to library cache memory objects in different namespaces for Java objects. A library cache object consists of one or more memory objects.
Column | Datatype | Description |
|---|---|---|
|
| Library cache namespace |
|
| Number of library cache memory objects currently in use in the Javapool |
|
| Total size of library cache in-use memory objects (in megabytes) |
|
| Number of freeable library cache memory objects in the Javapool |
|
| Size of library cache freeable memory objects (in megabytes) |
The top-level call to Java in the database is generally through a SQL call; in other words, entering the OracleJVM is achieved by invoking public static methods through PL/SQL wrappers. Starting with Oracle Database 10g Release 1, a client-side stub generated by JPublisher and referred to in the Oracle Database Java Developers documentation[15] as “Native Java Interface” offers another alternative for calling into Java in the database, with an additional overhead associated with serializing and deserializing the parameters and return values. A detailed example of the various invocation mechanisms is given in Chapter 3.
Once you enter Java in the database, you can recursively invoke other Java classes within the same top-level call, provided the classes in questions are visible to you (see section 2.3) and you have been granted execute rights on these. When Java is recursively invoked,[16] the same session and the same JVM are used. Therefore, the inner Java code (i.e., the recursively called java class) has access to the same Java states as the originating call; in other words, the Java state(s) manipulated in the recursive procedure is visible to the outer calls.
Java in the database operates repeatedly on the same object instances, but the OracleJVM was initially designed for shared database server environments (i.e., the database sessions are not attached to a particular server process). At the end of a call, and in order to preserve Java states that originate (i.e., are reachable) from the static variables of the active classes in the session, all of the significant information necessary for subsequent calls is saved in sessionspace (described earlier); this state migration is known as “end-of-call migration.” End-of-call migration happens systematically in pre-10g releases of the Oracle database irrespective of the server type (i.e., dedicated or shared). Starting with Release 10g of the Oracle database, in dedicated server mode, end-of-call migration is no longer performed, because the session is bound to the same process during its entire lifetime. By saving the time spent after each call performing state migration and restoration, the absence of end-of-call migration results in better overall execution performance, at the expense of an overall increase in memory consumption (a result of keeping all states). Listing 2.3 illustrates state preservation across calls.
Example 2.3. StateXCall.sql (contains both Java source and SQL scripts)
create or replace java source named StateXCall as
import oracle.*;
import oracle.aurora.vm.OracleRuntime;
public class StateXCall
{
public static String state1 = "initial state";
public static int state2;
// state2 is initialized each time the class is loaded
static {
state2 = 0;
}
public static void pass1() {
state1 = "got this state in pass1";
state2 = 111;
System.out.println("Pass 1" + "
state1: " + state1 + "
state2: " +
String.valueOf(state2));
OracleRuntime.exitCall(0); // graceful end-of-call, preserves states
}
public static void pass2() {
System.out.println("Pass 2" + "
state1: " + state1 + "
state2: " +
String.valueOf(state2));
if (state2 != 0 || !state1.equals("initial state"))
System.out.println("SESSION STATE PRESERVED!");
else
System.out.println("SESSION STATE LOST!");
state1 = "got this state in pass2";
state2 = 222;
OracleRuntime.exitSession(0); //terminates the VM
}
public static void pass3() {
System.out.println("Pass 3" + "
state1: " + state1 + "
state2: " +
String.valueOf(state2));
if (state2 != 0 || !state1.equals("initial state"))
System.out.println("SESSION STATE PRESERVED!");
else
System.out.println("SESSION STATE LOST!");
OracleRuntime.exitSession(1);
}
}
/
show errors;
alter java source StateXCall compile;
show errors;
create or replace procedure pass1
is language java
name 'StateXCall.pass1 ()';
/
show errors
create or replace procedure pass2
is language java
name 'StateXCall.pass2 ()';
/
show errors
create or replace procedure pass3
is language java
name 'StateXCall.pass3 ()';
/
show errors
exit;And here is the output of the invocation of pass1, pass2, and pass3:
set serveroutput on call dbms_java.set_output(50000); call pass1(); Pass 1 state1: got this state in pass1 state2: 111 Call completed. SQL> call dbms_java.set_output(50000); Call completed. SQL> SQL> call pass2(); Pass 2 state1: got this state in pass1 state2: 111 SESSION STATE PRESERVED! call pass2() * ERROR at line 1: ORA-29515: exit called from Java code with status 0 SQL> call dbms_java.set_output(50000); Call completed. SQL> SQL> call pass3(); Pass 3 state1: initial state state2: 0 SESSION STATE LOST! call pass3() * ERROR at line 1: ORA-29515: exit called from Java code with status 1 SQL>
Be aware that when a Java class being used is recompiled, all sessions using this class in question will receive an "ORA-29549 Java Session State Cleared" message the next time they invoke methods belonging to the class in question. Although this is normal during the development process (develop/test/debug cycles), this is annoying in production environments. DBAs just need to pay attention before redeploying new versions of Java classes on a live system.
Most Java database applications work well with the default/automatic end-of-call migration or avoid it using dedicated servers. However, you may want to programmatically control the end-of-call migration and trade the memory overhead—incurred by keeping statics in Sessionspace across calls—for faster execution by releasing objects at the end of the call and recreating these on demand (lazy initialization), using the oracle.aurora.memoryManager.EndOfCallRegistry and oracle.aurora.memoryManager.Callback interfaces, documented in Oracle’s Java Developer’s Guide.
As you have probably noticed in the state preservation code samples, we used both OracleRuntime.exitCall() and OracleRuntime.exitSession(); what are their differences and how about the standard System.exit()? Let’s look into this in greater detail:
System.exit()terminates the Java VM (i.e., all threads) in the RDBMS session with no state preservation. As illustrated by the state across calls sample, it does not and should not terminate the RDBMS session nor cause the client to disconnect (i.e., the SQL*Plus session continues).OracleRuntime.exitSession()terminates the Java VM as well as the RDBMS session, causing the client to disconnect.The behavior of
OracleRuntime.exitCall()requires a little more explanation, because it varies depending on theOracleRuntime.threadTerminationPolicy(). This method returns a Boolean, which, if true, means that any active threads should be terminated (rather than left quiescent) at the end of a database call.In 9i R2, all threads (i.e., daemon and nondaemon) are terminated abruptly, irrespective of the server type (i.e., shared and dedicated).
In 10g, shared server mode,
threadTerminationPolicy()is always true, and all active threads are terminated abruptly.In 10g, dedicated server, the value may be changed by the user by calling
OracleRuntime.setThreadTerminationPolicy().If the value is false (the default), all threads are left quiescent but receive a ThreadDeath exception for graceful termination.
If the value is true, all threads are terminated abruptly.
In addition, the
OracleRuntime.callExitPolicy()controls when a call exits, if none of theOracleRuntime.exitSession(), OracleRuntime.exitCall()orSystem.exit()methods were ever called.OracleRuntime.callExitPolicy() may be set (by usingOracleRuntime.setCallExitPolicy()) to one of the following:OracleRuntime.EXIT_CALL_WHEN_ALL_NON_DAEMON_THREADS_TERMINATEThis is the default value; the call ends when only daemon threads are left running.[17]
If
threadTerminationPolicy() true, always in shared server mode, the daemon threads are killed. In 9i R2, the Java VM behaved as if thecallExitPolicy()were set toOracleRuntime.EXIT_CALL_WHEN_ALL_NON_DAEMON_THREADS_TERMINATEandthreadTerminationPolicy()= true.If
threadTerminationPolicy() false, the daemon threads are left quiescent until the next call (default for dedicated servers).
OracleRuntime.EXIT_CALL_WHEN_ALL_THREADS_TERMINATEThe call ends only when all threads have returned (or ended upon uncaught exception), irrespective of the value of
threadTerminationPolicy().OracleRuntime.EXIT_CALL_WHEN_MAIN_THREAD_TERMINATESWhen the main thread returns (or an uncaught exception occurs):
In summary:
In all database releases—namely, 9i R2, 10g R1, and 10g R2—
System.exit()andOracleRuntime.exitSesssion()end the Java VM abruptly; all threads are terminated without running the finally blocks.[18]In all releases,
OracleRuntime.exitCall()attempts to end each thread “gracefully” by throwing aThreadDeath exceptionon each thread, which results in the execution of finally blocks.Otherwise, if none of these applies, in other words, no invocation of
System.exit(),OracleRuntime.exitSesssion(), orOracleRuntime.exitCall()—which is the case for most Java programs—thenOracleRuntime.callExitPolicy()controls how the call is terminated.
Security requirements are even more stringent when it comes to the database. Java code running in an Oracle database benefits from a rich database security environment, including user authentication, database-schema security, login-user security, and effective-user security. The OracleJVM itself provides additional features such as class-resolution security and Java 2 security.
An Oracle database user—which is typically a client application or a JDBC connection—needs to be identified, authenticated, and authorized before it can create a session. Oracle Net and Oracle JDBC drivers support several user-authentication mechanisms. These methods include traditional username/password authentication, as well as strong authentication mechanisms such as Kerberos, CyberSafe, RADIUS, token cards, smart cards, biometrics, public-key infrastructure (PKI), certificate-based authentication, proxy authentication and authorization, and Single Sign-On (SSO). Once the database client is authenticated, a new database session is associated with it. For more details on Oracle database authentication, see the Oracle Net documentation.
JDK JVM lets you execute Java classes downloaded from the Web. With OracleJVM, Java classes reside in the database and are governed by the same security features that protect other database objects. Specifically, Java classes, sources, and resources are organized, stored, searched, and executed within database schemas. Before you can execute a Java class with OracleJVM, you need to load it into a specific database schema and then validate it (i.e., resolve it). To load a Java class, you can use the loadjava command-line utility or an integrated development environment (IDE) such as Oracle JDeveloper. You must have CREATE PROCEDURE privilege in order to load classes into your own schema and CREATE ANY PROCEDURE privilege to load classes into a schema owned by another user. These protections provide a reliable Java execution environment, which is less likely to be attacked by malicious code.
Java classes are owned by the schema, which loads them (i.e., the defining schema). You can provide different levels of security for Java classes by grouping them into separate schemas. For example, suppose you define class A and class B in different schemas. In order for class A to refer to class B, the defining schema of class A must be granted execute rights on class B. To accomplish this, you can use the grant option in the loadjava utility. By creating new schemas for particular classes and controlling execute rights to those classes, you can design fine-grained security for your Java applications. In the following example, Betty and Bob are granted the right to execute class alpha in the schema TEST:
loadjava -thin -schema test –u SCOTT/
TIGER@localhost:5521:orcl
-grant BETTY,BOB alpha.classThe OracleJVM class-resolution architecture provides flexible and fine-grained control over class visibility and security. It uses a resolver specification (also known as resolver spec) to search for and locate Java classes within database schemas. A resolver spec usually contains multiple schemas. It is similar in concept to the CLASSPATH in the JDK world. In contrast with JDK, resolver specs are defined on a class-by-class basis. There is no general CLASSPATH, however; there’s a default resolver spec per schema, instead.
The following examples show different ways of defining the scope of a resolver spec.
The default resolver spec contains the defining schema (i.e.,
SCOTT) and thePUBLICschema (i.e., all classes exposed through thePUBLICsynonym).The default resolver spec for the
SCOTTschema is'{(* SCOTT) (* PUBLIC)}'.The following command loads a Java class using the default resolver spec; as a result, loadjava will search the definer’s schema and
PUBLIC:loadjava –resolve Foo.java
In the following example, a Java class is loaded using a resolver spec that will search the
SCOTT,OTHER, andPUBLICschemas:loadjava -resolve -resolver”((* SCOTT)(* OTHER)(* PUBLIC))” ...
Before resolving a class, it must be visible and the requester must have execution permission on it. All references to other classes must have been resolved before a class can be executed. Furthermore, OracleJVM requires that the resolution be done at deployment time, unless it is told to defer such resolution until runtime. For example, the following resolver spec skips the references not found within SCOTT or PUBLIC schemas:
loadjava -resolve -resolver"((* SCOTT)(* PUBLIC))" - genmissing
genmissing is an option of loadjava that lets you deal with nonexistent classes. It instructs loadjava to create and load placeholders for classes that are not defined but referenced. By skipping the missing classes, genmissing allows the Java code to run, but the methods in the missing classes cannot be invoked.
Note
A Java resource can have a “resolver spec” but is only looked up and used during the resolution of the class, which refers to it; in other words, loadjava does not resolve Java resources. The following PL/SQL interface and the Java method return the resolver spec of a specific object.
function resolver (name VARCHAR2, owner VARCHAR2, type VARCHAR2)
RETURN VARCHAR2;
oracle.aurora.rdbms.DbmsJava.resolver(java.lang.String,
oracle.sql.CHAR,
java.lang.String)
return oracle.sql.CHAR;Example: returning the resolver spec for a given object:
SQL> select dbms_java.resolver('TrimLob', 'SCOTT', 'CLASS') from dual; DBMS_JAVA.RESOLVER('TRIMLOB','SCOTT','CLASS') ----------------------------------------- ((* SCOTT)(* PUBLIC))
This output indicates that all the references to external objects in Trimlob class should be resolved by looking up the SCOTT and PUBLIC schemas (PUBLIC schema is a pseudoschema that refers to all objects that have a public synonym).
When a user logs in to the database, a new session is created with a login-user identity or schema identity. All database operations—including SQL statements, PL/SQL packages, and PL/SQL wrappers for Java stored procedures—are executed under the login-user identity. A session can execute Java classes defined in other schemas, provided the login user’s schema has been granted execute permission on the other classes. However, you can dynamically override the effective identity under which a Java class runs. By default, the effective identity is the login-user identity. You can alter the identity at the Java class level or the PL/SQL stored-procedure level by using the database’s effective-user mechanism, similar to the UNIX setuid facility.
You can set the effective identity by assigning invoker’s rights or definer’s rights to a class; these are defined as follows:
Invoker’s rights. By default, a Java class is associated with invoker’s rights; the class is not bound to its defining schema. The current user of any session executing the class is the same as the session’s login user. The privileges are checked at runtime, and external references are resolved in the schema of the current user. With invoker’s rights, you can centralize and share code across multiple schemas and applications while keeping data stored in each schema private and isolated.
Definer’s rights. By assigning definer’s rights to a class, you bind the class to its defining schema. The effective user of any session executing the class is changed temporarily to the identity of the class’s defining schema, and all unqualified references are looked for in the defining schema. Java classes loaded with definer’s rights can be executed without requiring the invoking session to be connected as the schema to which the code belongs.
You can specify effective-user rights on the PL/SQL wrapper (by using Call Spec, which exposes public static Java methods to the SQL world) or on the Java class itself (by using the loadjava utility). These rights are in effect only when a Java class or PL/SQL package is invoked through server-side JDBC. In the following examples, the effective-user rights are defined on the PL/SQL wrapper and on the Java class:
Specifying invoker’s rights (current user) through the PL/SQL interface:
CREATE [OR REPLACE] PROCEDURE [schema_name.]procedure_name [(parameter_list)] [AUTHID CURRENT_USER] AS ...Specifying invoker’s rights (current user) on the Java class with loadjava:
loadjava {-user | -u} <user>/<password>[@<database>] [options] <file>.java | <file>.class | <file>.jar | <file>.zip |<file>.sqlj | <resourcefile> ...[- nodefiner]Specifying definer’s rights through the PL/SQL interface:
CREATE [OR REPLACE] PROCEDURE [schema_name.]procedure_name [(parameter_list)] AUTHID DEFINER ...Specifying definer’s rights on the Java class with loadjava:
loadjava {-user | -u} <user>/<password>
[@<database>] [options]
<file>.java | <file>.class | <file>.jar |
<file>.zip |<file>.sqlj | <resourcefile> ...-definerFor more details on effective-user rights, see Oracle’s PL/SQL User’s Guide and Reference.
The goal of Java 2 security is to provide an easily configurable security policy, fine-grained access control (i.e., permissions), and an easily extensible access-control structure. In this section, I will briefly describe the key Java 2 security concepts, including permissions, security policy, and the security manager, and then describe how they are implemented and supported in the OracleJVM.
Permissions are represented by (encapsulated in) Java objects, which represent the authorization or access to system resources. Each permission object controls one or several targets and zero, one, or several actions it authorizes on those targets. As an example, files and directories are the targets of java.io.FilePermission, which controls their access and authorizes read, write, delete, and execute actions (or access) on those files and directories.
Here is a sample of built-in fine-grained security permissions, with their targets and actions. (See the Java Security Architecture documentation for the complete list of Java security permissions.)[19]
java.util.PropertyPermission. Controls read/write access to JVM properties such as
java.home,os.name.java.lang.RuntimePermission. Controls use of some system/runtime functions like
exitVM(),getClassLoader(),setSecurityManager(),setIO(), and so on.java.io.FilePermission. Controls access to files and directories; the actions on files and directories are read, write, delete, and execute.
java.net.SocketPermission. Controls use of network sockets; the actions on sockets are accept, connect, listen, and resolve.
java.net.NetPermission. Controls use of multicast network connections, no action.
java.lang.reflect.ReflectPermission. Controls use of reflection for class introspection; it has one target and no action.
java.security.SecurityPermission. Controls access to security methods such as
getPolicy(),setPolicy(), ... and so on but has no action.java.security.AllPermission. Allows access to all permissions, just as if you were running Java without a SecurityManager
The SecurityManager (java.lang.SecurityManager) is a runtime object that controls permissions. Once activated, Java classes/methods request permission to allow or disallow certain operations proactively (before performing the operation). Each operation type has its specific checkXXX() method. For example, the Connect() method invokes the security manager’s checkConnect() method to approve the socket connection operation:
SecurityManager security = System.getSecurityManager();
if (security != null) {
security.checkConnect(status);
}
. . .
// catch SecurityExceptionIf the security manager approves the connect operation, the checkConnect() returns normally; otherwise, the checkConnect() method throws a SecurityException.
Figure 2.5 illustrates how it works.
The following operations or activities are regulated by the Java security manager (i.e., have “checkXXX” methods):
Network operations (i.e., opening, waiting, or accepting a socket connection from a host and a port number) require SocketPermission and RuntimePermission .
Thread operations (i.e., modify a thread) require RuntimePermission.
File System operations (i.e., read, write, delete a file) require FilePermission and RuntimePermission.
Operating System operations (i.e., create a new process) require FilePermission, RuntimePermission, and AWTPermission.
JVM operations (i.e., create a new class loader, exit the current application) require FilePermission, PropertyPermission, and AWTPermission.
Packages and Classes operations (i.e., add a new class to a package, unload a class from a package) require FilePermission and RuntimePermission.
Security operations (i.e., access or modify system properties) require SecurityPermission.
See the
java.lang.SecurityManagermethod permissions checks for more details.[20]
OracleJVM fully complies with Java 2 security; it supports all default Java 2 permissions as well as Oracle-defined permissions but differs from JDK VM implementation on the following points: permission determination, security policy management, and security manager.
A security policy is a set of permissions that represents what a code source[21] is and is not allowed to do.
In the JDK VM:
The security policies are represented by entries in one or several policy file(s), which can be updated with proper permissions, using a text editor or specific tool. The system policy file is by default located at:
$JAVA_HOME/lib/security/java.policy (Unix platforms)
$JAVA_HOMElibsecurityjava.policy (Windows platforms)
Although there is a
java.securityfile under$ORACLE_HOME/javavm/lib/security, the security policy is not read from the file system. Security policy objects are defined as rows in the PolicyTable, which is a secure/internal database table. This table can be updated (i.e., update policies, insert user-defined policies) through theDBMS_JAVApackage, but you must be grantedPolicyTablePermissionon the specific permission type (i.e.,PolicyTablePermissionon FilePermission) before doing so. By default, only theJAVA_ADMINrole is grantedPolicyTablePermissionto update the PolicyTable.
The policy table is an internal system table that is not queryable as a traditional table; however, two views have been provided for viewing its contents: USER_JAVA_POLICY gives each schema access to its permissions rows, and DBA_JAVA_POLICY for viewing all permissions requires DBA privilege.
Example:
SQL*Plus scott/tiger SQL> describe user_java_policy; Name Null? Type ----------------------------------------- -------- ------------------ KIND VARCHAR2(8) GRANTEE_NAME NOT NULL VARCHAR2(30) TYPE_SCHEMA NOT NULL VARCHAR2(30) TYPE_NAME VARCHAR2(4000) NAME VARCHAR2(4000) ACTION VARCHAR2(4000) ENABLED VARCHAR2(8) SEQ NUMBER
Notice the SEQ column, which serves as the row key!
SQL> column grantee_name format a15
SQL> col grantee_name format a10
SQL> col type_schema format a10
SQL> col name format a35
SQL> col type_name format a35
SQL> col action format a10
SQL> select kind, grantee_name, name, action, enabled from
user_java_policy
SQL> /
KIND GRANTEE_NA NAME ACTION ENABLED
-------- ---------- ------------------------------ ---------- --------
GRANT PUBLIC createSecurityManager ENABLED
GRANT PUBLIC exitVM ENABLED
RESTRICT PUBLIC loadLibrary.* ENABLED
GRANT PUBLIC modifyThread ENABLED
GRANT PUBLIC modifyThreadGroup ENABLED
GRANT PUBLIC preferences ENABLED
GRANT PUBLIC * read ENABLED
GRANT PUBLIC user.language write ENABLED
RESTRICT PUBLIC 0:java.lang.RuntimePermission# ENABLED
loadLibrary.*
KIND GRANTEE_NA NAME ACTION ENABLED
-------- ---------- ------------------------------ ---------- --------
GRANT PUBLIC DUMMY DISABLED
GRANT PUBLIC LoadClassInPackage.* ENABLED
RESTRICT PUBLIC LoadClassInPackage.java.* ENABLED
RESTRICT PUBLIC LoadClassInPackage.oracle.auro ENABLED
ra.*
RESTRICT PUBLIC LoadClassInPackage.oracle.jdbc ENABLED
.*
14 rows selected.
SQL> select name, type_name from user_java_policy;
NAME TYPE_NAME
------------------------------ -----------------------------------
createSecurityManager java.lang.RuntimePermission
exitVM java.lang.RuntimePermission
loadLibrary.* java.lang.RuntimePermission
modifyThread java.lang.RuntimePermission
modifyThreadGroup java.lang.RuntimePermission
preferences java.lang.RuntimePermission
* java.util.PropertyPermission
user.language java.util.PropertyPermission
0:java.lang.RuntimePermission# oracle.aurora.rdbms.security.Policy
loadLibrary.* TablePermission
NAME TYPE_NAME
------------------------------ -----------------------------------
DUMMY oracle.aurora.security.JServerPermi
ssion
LoadClassInPackage.* oracle.aurora.security.JServerPermi
ssion
LoadClassInPackage.java.* oracle.aurora.security.JServerPermi
ssion
LoadClassInPackage.oracle.auro oracle.aurora.security.JServerPermi
ra.* ssion
NAME TYPE_NAME
------------------------------ -----------------------------------
LoadClassInPackage.oracle.jdbc oracle.aurora.security.JServerPermi
.* ssion
14 rows selected.Security permissions are assigned to the defining schema; therefore, all classes within the same schema are regarded as belonging to the same “protection domain.”[22]
You can grant, restrict, and revoke permissions (granting, restricting, and revoking permissions is explained later).
End users with proper privileges can define and store their own permissions. However, a custom permission (i.e., not a default/built-in permission) can only be granted by the schema that created such permission in the PolicyTable. The granting schema must have been granted the PolicyTablePermission or the
JAVA_ADMINrole before being in position to grant defined permission to other schemas.
In the JDK environment:
The Security Manager is not initialized or enforced by default. You can use the default Security Manager or write and launch your own security manager.
In OracleJVM:
Security Manager is always initialized/enforced at database startup. It is recommended to use the default database security manager, but you may extend it (see the Oracle Database Java Developer’s Guide for more details).
In JDK VM:
The permissions are determined by code location:
CLASSPATH,URL, and keycode (a set of private keys and associated certificates). All classes within theCLASSPATHare trusted. TheCodeSourceis used to identify the code/class:grant { permission java.io.FilePermission”/tmo/ myfile”,”read”;};Permissions can also be granted programmatically; the following instruction in a Java program creates a permission object for reading the file named “myfile” in the /tmp directory:
p = new java.io.FilePermission(”/tmp/myfile”,”read”);
In the standard Java 2 security environment, permissions are only granted, by contrast in the OracleJVM implementation, as described later; permissions can also be revoked or restricted.
In OracleJVM:
Each Java 2 permission is represented by a permission object, stored in the PolicyTable and identifiable by a row key. The permissions are determined by the defining schema (i.e., the schema in which they are loaded). In OracleJVM, the notion of
CLASSPATHis replaced by the notion of Resolver Spec. Classes in the database are trusted on a class-by-class basis (even classes within the reach of the Resolver Spec are not blindly trusted). The permissions are granted using either thedbms_java.grant_permission()method or the underlying Java methodPolicyTableManager.grant()programmatically.
dbms_java.grant_permission(grantee, permission_type, permission_name, permission_action, key); grantee: is the name of a schema. permission_type: is the fully qualified name of a class that extends java.lang.security.Permission. If the class does not have a public synonymn then the name should be prefixed by <schema>:. For example 'myschema:scott.MyPermission'. permission_name: is the name of the permission permission_action: is the action of the permission key: is set to the key of the created row or to -1 if an error occurs.
This command attempts to create an active row in the policy table that will grant the permission specified to the grantee. If a row granting the exact permission does not already exist, then a new row is inserted. If a row granting the exact permission already exists, then the new row is not inserted and the table remains unchanged. If a row granting the exact permission exists but is disabled, then the permission is reenabled.
Alternatively, you can directly invoke the PolicyTableManager grant method from a Java program as follows:
oracle.aurora.rdbms.security.PolicyTableManager.grant(grantee , permission_type, permission_name, permission_action, key); Signature: oracle.aurora.rdbms.security.PolicyTableManager.grant( java.lang.String, java.lang.String, java.lang.String, java.lang.String, long[]);
Examples:
The following command creates a permission entry in the PolicyTable for reading the file named “myfile” in the
/tmpdirectory:SQL>call DBMS_JAVA.GRANT_PERMISSION('scott','java.io.FilePermiss ion', '/tmp/myfile','read'),The SCOTT schema may execute this command, provided it has been granted the PolicyTablePermission.
In order to open a socket from within the server-side JDBC Thin, the database session must be granted
java.net.SocketPermission; in the following example, we first grant the permission to a role and then grant the role to the schema; this technique is useful when you want to predefine a set of permissions as a role and then assign such a role as a coarse-grained permission to one or several schemas.SQL> create role CALLOUT; SQL> call dbms_java.grant_permission('CALLOUT', 'java.net.SocketPermission', '*', 'connect' ); SQL> grant callout to scott;
Let’s grant the SCOTT schema, the permission to read an existing file, C: empmyfile:
SQL>connect system/xxxx as sysdba SQL> call dbms_java.grant_permission('SCOTT','java.io.FilePermission', 2 'C: empmyfile', 'read, write'), Call completed. SQL>connect scott/tiger Connected. SQL> select kind, grantee_name, name, action, enabled from user_java_policy; KIND GRANTEE_NA NAME ACTION ENABLED -------- ---------- ------------------------------ ---------- -------- GRANT PUBLIC createSecurityManager ENABLED GRANT PUBLIC exitVM ENABLED RESTRICT PUBLIC loadLibrary.* ENABLED GRANT PUBLIC modifyThread ENABLED GRANT PUBLIC modifyThreadGroup ENABLED GRANT PUBLIC preferences ENABLED GRANT PUBLIC * read ENABLED GRANT PUBLIC user.language write ENABLED RESTRICT PUBLIC 0:java.lang.RuntimePermission# ENABLED loadLibrary.* KIND GRANTEE_NA NAME ACTION ENABLED -------- ---------- ------------------------------ ---------- -------- GRANT PUBLIC DUMMY DISABLED GRANT PUBLIC LoadClassInPackage.* ENABLED RESTRICT PUBLIC LoadClassInPackage.java.* ENABLED RESTRICT PUBLIC LoadClassInPackage.oracle.auro ENABLED ra.* RESTRICT PUBLIC LoadClassInPackage.oracle.jdbc ENABLED .* GRANT SCOTT C: empmyfile read,writ ENABLED 15 rows selected.
See more examples of permissions granting in Chapter 4, “Pragmatic Applications Using Java in the Database,” “Advanced Applications Using Java in the Database,” as well as “Running Non-Java Languages in OracleJVM” in Chapter 5.
Permission restriction is usually applied to an existing generic/ broad permission grant (such as allowing all users the permission to load Java classes in their own schema), by placing a restriction to it such as: (1) restricting the permission to a specific schema, (2) restricting a specific permission on a specific target, or (3) restricting the permission on a specific action.
The PL/SQL interface(s) is:
procedure restrict_permission ( grantee varchar2, permission_type varchar2, permission_name varchar2, permission_action varchar2); procedure restrict_permission ( grantee varchar2, permission_type varchar2, permission_name varchar2, permission_action varchar2, key OUT number);
The corresponding Java method(s) is:
oracle.aurora.rdbms.security.PolicyTableManager.restrict(
java.lang.String, java.lang.String, java.lang.String,
java.lang.String);
oracle.aurora.rdbms.security.PolicyTableManager.restrict(
java.lang.String, java.lang.String, java.lang.String,
java.lang.String, long[]);As an example, in the previous section (grant_permission) we granted both read and write on C: empmyfile to SCOTT. We may change our mind and place a restriction on write action; therefore, SCOTT can only read the file in question.
SQL> connect system/manager as sysdba;
Connected.
SQL> call dbms_java.restrict_permission('SCOTT','java.io.FilePermission',
2 'C: empmyfile', 'write'),
Call completed.
SQL> connect scott/tiger;
Connected.
SQL> select kind, grantee_name, name, action, enabled from user_java_policy;
KIND GRANTEE_NA NAME ACTION ENABLED
-------- ---------- ----------------------------------- ---------- --------
GRANT PUBLIC createSecurityManager ENABLED
GRANT PUBLIC exitVM ENABLED
RESTRICT PUBLIC loadLibrary.* ENABLED
GRANT PUBLIC modifyThread ENABLED
GRANT PUBLIC modifyThreadGroup ENABLED
GRANT PUBLIC preferences ENABLED
GRANT PUBLIC * read ENABLED
GRANT PUBLIC user.language write ENABLED
RESTRICT PUBLIC 0:java.lang.RuntimePermission#loadL ENABLED
ibrary.*
KIND GRANTEE_NA NAME ACTION ENABLED
-------- ---------- ----------------------------------- ---------- --------
GRANT PUBLIC DUMMY DISABLED
GRANT PUBLIC LoadClassInPackage.* ENABLED
RESTRICT PUBLIC LoadClassInPackage.java.* ENABLED
RESTRICT PUBLIC LoadClassInPackage.oracle.aurora.* ENABLED
RESTRICT PUBLIC LoadClassInPackage.oracle.jdbc.* ENABLED
GRANT SCOTT C: empmyfile read, writ ENABLED
e
RESTRICT SCOTT C: empmyfile write ENABLED
16 rows selected.
SQL>Having initially granted all users (i.e., PUBLIC) the right to load any class in their own schemas, the following commands place restrictions on the JServerPermission permission, preventing all users from loading system classes in their own schema:
call dbms_java.restrict_permission('PUBLIC',
'SYS:oracle.aurora.security.JServerPermission', 'LoadClassInPackage.java.*',
null);
call dbms_java.restrict_permission('PUBLIC',
'SYS:oracle.aurora.security.JServerPermission',
'LoadClassInPackage.oracle.aurora.*', null);
call dbms_java.restrict_permission('PUBLIC',
'SYS:oracle.aurora.security.JServerPermission',
'LoadClassInPackage.oracle.aurora.*', null);As a result, the ability for any schema to load classes/packages in their own schemas won’t apply for system classes.
Unlike RESTRICT_PERMISSION, which adds restrictions to a broader permission grant, REVOKE_PERMISSION revokes entirely the permission in question. In the previous example, instead of placing a write restriction on SCOTT on C: empmyfile, we could have revoked the initial permission from SCOTT and granted it a new one.
PL/SQL interface(s):
procedure revoke_permission( grantee varchar2, permission_type
varchar2,
permission_name varchar2, permission_action varchar2)The corresponding Java method is:
oracle.aurora.rdbms.security.PolicyTableManager.revoke(
java.lang.String, java.lang.String, java.lang.String,
java.lang.String)';The command dbms_java.disable_permission(key number) disables the existing permission row, identified by the specified key (SEQ#). The corresponding Java method(s) is:
oracle.aurora.rdbms.security.PolicyTableManager.disable(long);
Deletes the existing permission row identified by the specified key (SEQ#). The permission must have been previously disabled (see previous section), in order for the delete to work; otherwise, if the permission is still active, then this procedure does nothing.
PL/SQL interface(s):
dbms_java.delete_permission(key number);
oracle.aurora.rdbms.security.PolicyTableManager.delete(long);
User Java class needs to share the J2SE system classes. The system classes are loaded once in the SYS schema and are made accessible from any other schema through PUBLIC synonyms (a database mechanism that provides public visibility to common objects or resources). As described in rectrict_permission, OracleJVM prevents all users from replacing system classes (e.g., java.*, oracle.aurora.*, and oracle.jdbc.*), loading new classes into system packages, or loading system classes into their own schema. However, in order to load system classes into your own schema, a user schema may be explictly granted JserverPermission on the classes or package in question. As an example, the following command grants SCOTT the permission to load its own version of the oracle.aurora.tools.* package into its schema call:
dbms_java.grant_permission('SCOTT','SYS:oracle.aurora.sec
urity.JServerPermission',
'LoadClassInPackage.oracle.aurora.tools.*, null);Do not grant unnecessary permissions. Use the principle of “Least Privileges”; in other words, only grant explicit or specific permission to a particular code for accessing a particular resource. Such attitude gives you a fine-grained security control, in the spirit of Java 2 security.
An example of a bad practice would be to grant the SCOTT schema “read” access to all files, like this:
call dbms_java.grant_permission('SCOTT',
'SYS:java.io.FilePermission',
'<<ALL FILES>>','read'),On the other hand, a good practice would be to grant permission only to the explicit document root file path, as we did earlier.
call dbms_java.grant_permission('SCOTT',
'SYS:java.io.FilePermission',
'(<actual directory path>)','read'),Discourage the use of granting a coarse-grained group of permissions such as JAVASYSPRIV and JAVAUSERPRIV roles that are still provided for backward compatibility with Oracle 8i so as not to break programs that used to work in previous RDBMS releases.
In general, be conservative about granting permissions related to Classloader and java.lang. For example, granting RuntimePermission to create-ClassLoader has the same effect as granting all permissions.
Java Native Interface (JNI) is a standardized interface to call procedures written in other languages to access platform facilities such as AWT fonts, file locks, and registries from Java. There are safety considerations when calling native methods; JavaSoft/SUN recommends that, only experienced programmers should attempt to write native methods or use the Invocation API! In order to reduce the need for JNI, the latest J2SE specifications have added most platform-specific facilities such as java.util.prefs to deal with registries. Because OracleJVM does not have control over the behavior of foreign code, and in order to guarantee data integrity, it does not allow end users the use of JNI calls. See the “Calling-out SAP from within a Stored Procedure” case study in Chapter 4 for a discussion of working around the JNI restriction. However, OracleJVM uses JNI calls internally for its own needs (by expert programmers only!), such as the C-based byte-code verifier, the headless AWT fonts, and miscellaneous interactions with the RDBMS kernel.
One frequent question people ask is, “Can I upgrade the Java VM in the database?” As you have seen throughout this chapter, the OracleJVM is intimately integrated with the RDBMS release and cannot be upgraded independently; however, similar to most Oracle database components, its packaging allows independent install or reinstall. OracleJVM is automatically installed as part of the Oracle database software installation, but you can choose to disable this option, knowing that some database components such as interMedia, UltraSearch, Data Mining, Oracle Spatial, the XML Developers Kit (XDK), the XML SQL Utility (XSU), the XML Query, and so on require a Java-enabled database in order to function properly. In case you need to uninstall or manually reinstall the OracleJVM, see Metalink note 209870.1, “How to Reload the JVM in 9.2.0.X,” and note 276554.1 “How to Reload the JVM in 10.1.0.X.” In Oracle Database 10g. The native compiler libraries (NCOMP) and the compiled system classes reside on the companion CD and have to be installed in a second step.
You may have noticed that sometimes the first invocation of Java in the database session is slower than subsequent calls. It is likely that the first call is paying either the cost of Java VM initialization in the session and/or the cost of the initialization of the application. But this is not a high price remember that only the private/unshared part of the runtime and the applications reside in Sessionspace; everything else is in Javapool in the SGA so as to be shared by all sessions.
See the “End-of-Call, VM Termination, and Session Termination” discussion in section 2.2.9.
In general, there are many ways to execute Java bytecode, including strict interpretation, threaded interpretation, JIT compilation, native compilation, and so on. Discussing Java bytecode execution techniques is beyond the scope of this book.
The OracleJVM currently furnishes a threaded interpretation and static native compilation (NCOMP) of Java; both techniques are described in the following sections. The portability of the interpreter and the compiler is a key requirement for the Oracle database.
There are three modes for running Java in the database: (1) interpreted system classes and interpreted user classes, (2) compiled system classes and interpreted user classes, and (3) compiled system classes and compiled user classes. For (1), you only need to install Java during the database installation. For (2) and (3), you need to install the NCOMP libraries (covered later) from the companion CD, in a second step.
Note
To check whether NCOMP is installed (from the companion CD), use the following query, which gives ORA-29558 when it has not been installed.
select dbms_java.full_ncomp_enabled from dual;
I will briefly describe what a threaded interpreter is, and then, more important for application developers, indicate best practices for optimizing the performance of interpreted Java code.
A traditional Java interpreter can be regarded as a large switch-and-loop statement over an array of bytecodes:
char *current_vm_instruction; while(1) { char vm_instruction = *current_vm_instruction; switch (vm_instruction) { case OPCODE_0: /* C code for VM instruction */ ... ; current_vm_instruction += OPCODE_0_LENGTH; break; ... ... case OPCODE_255: /* C code for VM instruction */ ... ; current_vm_instruction += OPCODE_255_LENGTH; break; } }
It fetches an array of bytecodes at the current_vm_instruction, and for each bytecode entry, switches to corresponding vm_instruction"opcode " for its execution and then starts the loop over for the next byte-code. The cost of dispatching interpreted bytecode is expensive and may often surpass the execution cost. For simple bytecodes, this overhead can be extremely costly. Direct-threaded interpreters reduce this overhead by replacing the loop-and-switch code—which a traditional interpreter will place after every case to execute the next bytecode—with a direct jump (goto) to an address offset corresponding to the bytecode implementation (see following example).
char **current_vm_instruction; OPCODE_0: /* C code for VM instruction */ ... ; current_vm_instruction += OPCODE_0_LENGTH; goto *current_vm_instruction; ... ... OPCODE_255: /* C code for VM instruction */ ... ; current_vm_instruction += OPCODE_255_LENGTH; goto *current_vm_instruction;
Performance optimization starts in general with profiling to reveal the hot spots or bottlenecks; then you tune the specific codes by implementing the appropriate best practices, and validate your changes with another profiling and iterate the process until you achieve satisfaction. The following standard programming best practices—not specific to Java—will help you get a decent performance out of the Java interpreter:
Avoid excessive interface and virtual method calls. Virtual method dispatch incurs a runtime lookup cost. Interface method dispatch incurs an even greater runtime lookup cost. As a result, calling interface methods or virtual methods are slower than calling methods on a regular Java class. Use private, static, or final methods whenever possible, because these method calls are directly bound to the Java class, incurring less overhead.
Use the following optimization techniques, described in detail in Practical Java Programming Language Guide:[23]
Empty method removal
Dead code removal
Strength reduction: replacing expensive operations with more effective ones such as compound assignment operators
Constant folding: precalculating constant expressions
Common subexpression elimination: replace duplicate expressions by a temporary variable
Loop unrolling: replace the loop construct by the sequence of the operations to be performed
Algebraic simplification: use algebra rules to simplify expressions
Loop invariant code motion: avoid redundant object accessors in tight loops (see following example).
Instead of:
int[] a = ...; for(int i = 0; i < a.length; i++) { a[i] = o._x; }Use this one:
int[] a = ...; int x = o._x; int max = a.length; for(int i = 0; i < max; i++) { a[i] = x; }
Browse online Java performance-tuning Web sites such as JavaPerformanceTuning.com.[24]
However, the best Java performance is achieved by the means of natively compiled code (i.e., dynamic compilation [JIT] or static compilation); OracleJVM Native Compiler is described in the following section.
Java developers are more familiar with Just-In-Time (JIT) compilation in JDK VM environments. JIT compilation consists of dynamically compiling the Java bytecode into platform-specific executables on the fly. It is a runtime process that happens transparently under the covers, which is great from a usability perspective but has a recurrent recompilation cost (this is minor, though). NCOMP, which is short for Native Compilation of Java (also known as JAccelerator), aims at the same goal—that is, speed up Java execution by compiling the Java bytecode into platform-specific binary executables. However, NCOMP uses a static compilation approach, which must be performed ahead of the deployment time. It is an explicit, additional operation (you have to invoke it), which, from a usability perspective, is not great but which, on the plus side, avoids the repeating cost of on-the-fly compilation. The JIT versus NCOMP discussion is beyond the scope of this book, because the Oracle database does not currently offer a JIT compiler.
The ultimate goal of NCOMP is to convert Java bytecode into platform-dependent binary, in the most efficient and portable manner. To reach this goal, it had to meet the following requirements: portability, scalability, security, and interoperability between interpreted bytecode and native code. The first design choice was to take advantage of the portability of C code across all platforms where a C compiler is available. The second design choice was to take advantage of the optimization techniques offered by C compilers.
The main piece of the native compiler is a Java-to-C translator. Unlike traditional “platform-specific assembly-based” native compilers, NCOMP must be portable across multiple platforms and support the different hardware platforms with which the Oracle database ships. To meet this requirement, the NCOMP team leveraged the Oracle standards C Pre-Processor (CPP) macros, a highly portable subset of ANSI C, used for writing the Oracle database software. This design choice allows performance optimizations such as caching and simplified maintenance.
The native code must offer the same scalability, throughput, and support for large loads as the database. To meet these requirements, the designers chose to store native code as shared libraries (called DLLs)—using Java resources—that can be dynamically loaded and shared concurrently by a large number of user sessions.
The native code must be safe for execution directly into the server’s address space, avoiding context-switching overhead while preserving the security aspects of the Java runtime. Security requirements were met by preserving all necessary runtime checks, thus making native code as safe as interpreted Java bytecodes.
Java bytecodes residing in the database have been resolved and verified during the loading phase (loadjava) and can therefore be trusted and expected to execute correctly. The translator takes the trusted bytecodes as inputs and converts them into a subset of ANSI C, applying data flow analysis and optimization techniques, as described by the following internal steps (transparent to the end user), which you might want to skip, unless you share my curiosity.
Each operation on Java stack is replaced with an equivalent but much faster operation using local temporary C variables.
Wherever possible, Java VM calling conventions are replaced by faster C calling conventions. The natively compiled Java code executes directly on the C stack and does not use the Java VM stack.
Performance-critical properties of Java classes such as instance layouts and virtual method table layouts are computed at compile time, and the corresponding C code is generated.
The translator performs a range of interprocedural optimizations by analyzing large groups of classes at compile time, thereby reducing the overhead of method dispatch in Java code.
The translator emits data structures that enable caching of the results of runtime method dispatch, further improving the speed of runtime execution.
In summary, the NCOMP-generated code can be regarded as a regular Java class, which preserves the semantics of Java such as class invalidation and reloading. The OracleJVM class loader can invalidate both interpreted or NCOMPed classes and their dependents by marking every affected NCOMPed class as non-ncomped and a candidate for reload.
A Java application may result in a mix of bytecodes (interpreted) and natively compiled binaries. Therefore, the natively compiled code must be fully interoperable with the Java VM. The native compiler has been designed to generate C code that allows natively compiled routines to call (and be called from) interpreted Java bytecode. The interaction between the OracleJVM runtime, the interpreted bytecode, and the native code is pictured in Figure 2.6.
The interaction can be summarized as follows (see Figure 2.7):
At the OracleJVM runtime level: The NCOMP translator generates CPP macros that behave just like the system native code written by hand by OracleJVM developers. This ensures tight integration at the runtime level. In addition, the generated code expands inline into platform-specific raw C code, resulting in greater configurability and flexibility.
At the OracleJVM services level: The OracleJVM preserves dynamic aspects of code execution through a special execution context passed as an argument to all routines that require dynamic services. The native compiler produces C code that follows the same calling convention. In addition, the generated C code contains calls to high-level VM APIs, such as exception handling primitives, predefined and cached standard objects, and so on.
At the OracleJVM interpreter level: The OracleJVM supports a mixed execution model, which allows mixing interpreted byte-code and natively compiled code.
NCOMP comprises three commands/steps: 'NCOMP', 'STATUSNC', and 'DEPLOYNC'. Figure 2.8 depicts the compilation and linkage phase.
the normal/traditional deployment phase of Java in the database (loadjava, interpreted mode execution)
The translation, compiling, and linking phase
Translation of the Java bytecode (application JAR) into C source files and headers
Compilation of the generated C files resulting in platform-specific object files
Linkage of the object files, resulting in platform-specific binaries library (DLL)
The deployment phase
Loadjava uploads every resource by default. When loadjava loads an NCOMP deployment JAR, it treats it as a Java resource. These are “special resources”; a hook in loadjava dumps these DLLs in the
$ORACLE_HOME/javavm/deploydirectory on the database server for the duration of the deployment.Loadjava invokes the installers serially. When a Java package is successfully validated, the corresponding DLL file is enabled by the corresponding installer and copied from
$ORACLE_HOME/javavm/deployto$ORACLE_HOME/javavm/admin.
Figure 2.8 depicts the NCOMP process.
From the end-user’s perspective, the NCOMP command does three things:
Gathers class info, generates a script that will drive the NCOMPing process, and produces the list of NEED_NCOMPNG or ALREADY_NCOMPED methods
Pauses for the Java-to-C translation: C compilation and linkage of each package involved into platform-specific DLLs
Deployment/installation of the resulting DLL (unless -noDeploy) is specified
ncomp [ options ] <class_designation_file> -user | -u <username>/<password>[@<database_url>] [-load] [-projectDir | -d <project_directory>] [-force] [-lightweightDeployment] [-noDeploy] [-outputJarFile | -o <jar_filename>] [-thin] [-oci | -oci8] [-update] [-verbose]
Argument | Description and Values |
|---|---|
| The full path name and file name of a JAR file that contains the classes that are to be natively compiled. If you are executing in the directory where the JAR file exists and you do not specify the |
| The full path name and file name of a ZIP file that contains the classes that are to be natively compiled. If you are executing in the directory where the ZIP file exists and you do not specify the - |
| The full path name and file name of a CLASSES file, which contains the list of CLASSES to be natively compiled. If you are executing in the directory where the classes file exists and you do not specify the - |
| Specifies a user, password, and database connect string; the files will be loaded into this database instance. The argument has the form <username>/<password>[@<database>]. If you specify the database URL on this option, you must specify it with OCI syntax. To provide a JDBC Thin database URL, use the -thin option. |
| The native compilation is performed on all classes, excluding previously compiled classes. |
| Provides an option for deploying shared libraries and native compilation information separately. This is useful if you need to preserve resources when deploying. |
| Executes |
All natively compiled classes output into a deployment JAR file. This option specifies the name of the deployment JAR file and its destination directory. If omitted, the | |
| Specifies that the native compilation results only in the output deployment JAR file, which is not deployed to the server. The resulting deployment JAR can be deployed to any server using the deploync tool. |
| The database URL that is provided on the - |
| The database URL that is provided on the -user option uses an OCI URL address for the database URL syntax. However, if neither - |
| Specifies the full path for the project directory. If not specified, Accelerator uses the directory from which |
| If you add more classes to a <class_designation_file> that has already been natively compiled, this flag informs Accelerator to update the deployment JAR file with the new classes. Thus, Accelerator compiles the new classes and adds them to the appropriate shared libraries. The deployment JAR file is updated. |
The normal usage is to natively compile already loaded (then trusted) Java classes or packages. NCOMP requires that each library (JAR, ZIP) contains only a single package. It recompiles only packages that contain classes that have changed since the last time these were loaded in the database. However, you can “force” the recompilation of any package using the –force option.
ncomp -user SCOTT/TIGER Project1.jar, Project2.zip, Class1.class ncomp -user SCOTT/TIGER –force Project1.jar, Project2.zip, Class1.class
NCOMP only uses loaded and trusted classes; however, you may ask for the compilation of a class or library that has not yet been loaded using the “load” option.
ncomp -user SCOTT/TIGER@<dbhost>:<port>:<sid> -<driver_type> -load <library> ncomp -user SCOTT/TIGER@host1:5521:orcl -thin –load Project1.jar
The OracleJVM offers the STATUSNC command as a standard method for checking whether your JAR files, ZIP files, or CLASSES files are NCOMPed or not.
statusnc [ options ] <file>.jar | <file>.zip | <file>.classes. -user <user>/<password>[@database] [-output | -o <filename>] [-projectDir | -d <directory>] [-thin] [-oci | -oci8]
The possible values of the status are:
ALREADY_NCOMPED. The class is natively compiled.
NEED_NCOMPING. A class within the shared library probably reloaded after the native compilation process. Needs to natively recompile the entire shared library.
INVALID. The native compiler tried to validate an invalid class but failed. The class will be excluded from the natively compiled shared library.
The results of
STATUSNCare stored in theJACCELERATOR$STATUStable.
Sometimes, not all methods within a class are NCOMPed. To find out whether a specific method within a class has been natively compiled or not, you can leverage new points in the OracleJVM runtime (as of 10g Release 2) to programmatically determine the status of a method with reference to native compilation.
From SYSTEM or SYS account grant SCOTT the following permission:
call dbms_java.grant_permission('SCOTT','SYS.java.lang.Runti mePermission','accessDeclaredMembers', ''),Once granted
SYS.java.lang.RuntimePermissionyou can query the status of loaded methods, using the program shown in Listing 2.4:
Example 2.4. Method-Level Check
CREATE OR REPLACE AND RESOLVE JAVA SOURCE NAMED Method_nc AS
import java.lang.reflect.*;
import oracle.aurora.vm.OracleRuntime;
public class Method_nc {
public static void foo(String s) throws Exception {
Class c = Class.forName(s);
printMethods(c);
}
public static String methodStatusString(int status) {
switch(status) {
case OracleRuntime.INTERPRETED_METHOD:
return "interpreted";
case OracleRuntime.SYSTEM_NATIVE_METHOD:
return "system-native";
case OracleRuntime.JNI_METHOD:
return "JNI-native";
case OracleRuntime.NCOMP_METHOD:
return "NCOMP'd";
default: return "Unknown";
}
}
public static void printMethods(Class c) throws Exception {
System.out.println("Methods report for class "+c);
Constructor[] ctors = c.getConstructors();
for(int i = 0; i < ctors.length; i++) {
int status = OracleRuntime.methodRuntimeType(ctors[i]);
System.out.println(" "+ctors[i]+" is a
"+methodStatusString(status)+" method");
}
Method[] meths = c.getDeclaredMethods();
for(int i = 0; i < meths.length; i++) {
int status = OracleRuntime.methodRuntimeType(meths[i]);
System.out.println(" "+meths[i]+" is a
"+methodStatusString(status)+" method");
}
}
}
/
show errors;
/
CREATE OR REPLACE PROCEDURE MethodNC (C VARCHAR2) AS LANGUAGE JAVA
NAME
'Method_nc.foo(java.lang.String)';
/
show errors;
/Generates the following output:
SQL> call dbms_java.set_output(50000);
Call completed.
SQL> call MethodNC('Method_nc'),
Methods report for class class Method_nc
public Method_nc() is a interpreted method
public static void Method_nc.foo(java.lang.String) throws
java.lang.Exception
is a interpreted method
public static java.lang.String
Method_nc.methodStatusString(int) is a
interpreted method
public static void Method_nc.printMethods(java.lang.Class)
throws
java.lang.Exception is a interpreted method
Call completed.
SQL>Create the list of classes to be natively compiled into the file: my.classes:
my.classes ========== import Method_nc;
Invoke the native compiler with my.classes as input:
C:> ncomp -u scott/tiger my.classes # # this list is produced by query # select status, class_name from jaccelerator$status; # NEED_NCOMPING Method_nc # Deployment History, produced by query: # select timestamp, status, dll_name from jaccelerator$dlls order by dll_name 2005-06-09 09:41:18 installed / libjox10_104648c91ac_scott_UnnamedPackage.so
Let’s run the method-level status check again to find out if the class has been NCOMPed:
SQL> set serveroutput on
SQL> call dbms_java.set_output(50000);
Call completed.
SQL> call MethodNC('Method_nc'),
Methods report for class class Method_nc
public Method_nc() is a NCOMP'd method
public static void Method_nc.foo(java.lang.String) throws
java.lang.Exception is a NCOMP'd method
public static java.lang.String
Method_nc.methodStatusString(int) is a NCOMP'd method
public static void Method_nc.printMethods(java.lang.Class)
throws
java.lang.Exception is a NCOMP'd method
Call completed.
SQL>NCOMP takes as input already compiled and resolved classes. The following command will dump the Java class FooBar:
echo "include FooBar; " > my.classes ncomp -u sys/install -noDeploy -force my.classes ls -l classes/foo/Bar.class
C compiler optimizations are controlled through platform-specific options. These settings are specified in the $ORACLE_HOME/javavm/jahome/Settings_os.properties file.
On Linux (on Solaris these are slightly different but quite similar to Linux):
CC = gcc c.compile.opt.level = -O3 LINK_COMMAND = ld -shared -h $(@:f) -o $@ $<
CC = $(visual.c.home)/bin/cl c.compile.flags.platform = -Ox -Oy- LINK_COMMAND = $(visual.c.home)/bin/LINK /DLL /OUT:$@ /LIBPATH:"$(visual.c.home)/lib" /DEF:$(*:b).def $< $(oracle.home)/lib/orajox10.lib /NODEFAULTLIB msvcrt.lib kernel32.lib
The flag c.compile.opt.level controls the level of optimization. In general, a higher level of optimization produces faster code at the expense of a longer NCOMPing process.
Platform dependencies information comes from three sources:
The javavm/jahome/jtc.h file, which contains platform-specific CPP definitions
The JDK Java system properties
The OracleJVM system properties and other platform-specific variables and system methods
NCOMP gathers most platform-specific information from a file produced by the Oracle database porting teams (i.e., the javavm/jahome/Settings_os.properties file), which contains the C compiler optimization specifics, file extensions, flags, and so on. This file can be modified to reflect a specific path to the C compiler or the location of C temporary storage or specify a new level of C compiler optimization (e.g., O3 performs level 3 optimization).
On Linux:
file.extension.obj = o library.prefix = lib file.extension.dll = so LINK_COMMAND = ld -shared -h $(@:f) -o $@ $< CC = gcc c.compile.opt.level = -O3 c.compile.flags.platform = $(c.compile.opt.level) -fPIC -DLINUX -D_GNU_SOURCE -DSLTS_ENABLE -DSLMXMX_ENABLE -D_REENTRANT -DNS_THREADS -D_SVID_GETTOD makefile.maker = $(one.c.unit.per.dll.makefile.maker)
file.extension.exe = exe file.extension.obj = obj library.prefix = "ora" file.extension.dll = dll visual.c.home = c:/devstudio/vc oracle.home = $(ORACLE_HOME) LINK_COMMAND = $(visual.c.home)/bin/LINK /DLL /OUT:$@ /LIBPATH:"$(visual.c.home)/lib" /DEF:$(*:b).def $< $(oracle.home)/lib/orajox10.lib /NODEFAULTLIB msvcrt.lib kernel32.lib CFLAGS = $< -Fo$@ $(c.compile.include) $(c.compile.flags.platform) $(c.compile.flags.aurora) CC = $(visual.c.home)/bin/cl COMPILE.c = $(CC) $(CFLAGS) $(CPPFLAGS) -c c.compile.flags.platform = -Ox -Oy- makefile.maker = $(one.c.unit.per.dll.makefile.maker)
The system classes are provided fully NCOMPed; their deployement in the database happens during database creation. For the end users or application classes, the NCOMPing process consists of, as described previously, three steps/phases: (1) the C-to-Java translation, (2) the compilation and linkage of the generated C code, and (3) the deployment/uploading of the platform-specific binary libraries (also called DLLs) in the database. Here is an estimate of resource requirements for each phase based on the proof of concept “Run Groovy Scripts in the Database” (see Chapter 5):
The translation of 500 Java classes (i.e., 80 MB of uncompressed bytecodes) generates approximately 24 million lines of C code and takes from one to three hours depending on the CPU power of the platform.
The compilation and linkage phases depend on the speed of the C compiler and its optimizer; it may take from three to five times the duration of the translation phase; in other words, from 3 hours up to 15 hours!
The duration of the deployment phase depends on the speed of JDBC LOBs access. It takes, on average, a few seconds to deploy a class. Based on the same application, the deployment of NCOMPed 500 classes—a 10-MB deployment JAR—takes less than three minutes on modern Intel boxes under Linux. The deployment process requires the pinning in memory of the classes and access to the database storage. Tuning the speed of deploying large NCOMPed JARS involves the typical database tuning process (i.e., table storage, rollback segment, and Javapool).[25]
How much disk space does NCOMP need? To answer that question, let’s look at everything NCOMP produces and their disk space considerations:
Using the same Groovy example as above: unloading 500 Java classes requires 8 MB zipped or 17MB uncompressed.
NCOMP generates temporary C source code: for 500 Java classes, 1400 C files (and headers) generated, approximately 1.3 million lines, occupying 39 MB disk space; the object file occupies 19 MB.
An NCOMP deployment Jar contains:
The installer: Java code that contains tables used to validate and enable the DLLs.
The DLLs: platform-specific binaries that are managed as a Java resource file. The typical bytecode/DLL ratio ranges from 1:2 to 1:5 (on average), up to 1:10, depending on the size of the individual classes and their “density.”[26] With JDK 1.4, classes are less dense, resulting in a ratio of bytecode size over DLL size closer to 1:1. However, for heavyweight C-style applications with very large methods and few classes, it will be closer to 1:10. As an example, the Groovy bytecode/DLL ratio is 8 MB/10 MB.
The ultimate question you were dying to ask is: “What is the performance improvement of NCOMP?” As usual, there is no absolute figure; it all depends on what exactly you are NCOMPing. NCOMP speeds up only Java code, not the embedded SQL, NCOMP may give you up to an order of magnitude speed up (i.e., 10 times) compared with interpreted execution. But remember, in Chapter 1 we saw that the combination of Java and SQL runs faster in the database, even interpreted. This section gives you some tips to get the best performance out of NCOMP.
General considerations:
Private, final, static, and constructor methods turn into direct function calls within the DLL, so there is almost no overhead. When static methods are called within the same package (and hence the same DLL), the overall performance is even better, because you save the overhead of indirectly calling into another DLL.
Interface methods[27] are slower, so avoid them if possible.
Not all methods in an NCOMPed class are natively compiled:
Some are not translated from Java to C because they are too complicated or because of other mitigating circumstances.
Methods that are not NCOMPed are skipped based on some rules. Among them are:
Methods whose compiled bytecode is larger than 16K are not NCOMPed.
Static initializer blocks[28] are not NCOMPed either.
Try-catch-finally blocks:
The Java compiler generates complex exception-handling code out of try-catch-finally; avoid using this in performance-critical code.
If you absolutely need try-catch-finally semantics, put the critical code in another method (ideally a private, static, or final method so it will be a fast C function call) and call that method from the try-catch-finally block.
Do not use try-catch as a control primitive. For portability reasons, the NCOMP translator converts try-catch blocks into code relying on the nonlocal
GOTOof ANSI C:setjmp/longjmp. These facilities are indeed portable but may incur an overhead compared with machine codeGOTOs. Hence, following these recommendations helps avoid the performance penalty ofsetjmp/longjmp:Don’t use the try block as conditionals and “catch” as a branching construct as follows:
try { ((CCC)obj).fooBar(); // the main case } catch (NullPointerException npe) { // obj was null, do something specific to that case } catch (ClassCastException cce) { //it was a different type, do something else yet.. }Instead, use explicit checks:
if (obj == null) { // obj is null, do something to that case } else if (obj instanceof CCC) { ((CCC)obj).fooBAr(); // the main case } else { // obj is of a different type, do something else yet.. }Do not use exceptions to terminate loops!
Instead of this:
try { myArray = .... for (int i;;i++) dosomething(myArray[i]); } catch (ArrayIndexOutOfBoundsException e) { // exit loop }myArray = .... int myArrayLen = myArray.length; for (int i;i < myArrayLen ;i++) dosomething(myArray[i]);
Avoid redundant array accessors and object accessors in tight loops.
Example:
Instead of:
int[] a = ...; for(int i = 0; i < a.length; i++) { // redundant array accessors a[i] = o._x; // redundant object accessor }Use:
int[] a = ...; int x = o._x; int max = a.length; for(int i = 0; i < max; i++) { a[i] = x; }This code snippet saves
(max-1)*timesthe array check andlengthattribute retrieval.Array access is more expensive than primitive operators because array must be validated (i.e., not null and array bounds) before an access happens. In some cases, the NCOMP translator eliminates null checks, but not always, and the bounds checks are not eliminated. Similarly, instance accessors need null pointer checks and also are expensive if repeatedly performed in loops. Hence, the programmer can maximize performance by making sure object and array accessors are not needlessly repeated in performance-critical loops.
The same is true for other expensive operations such as ‘
instanceof’ and casts. Example:DON'T: for (int i = 0; obj instanceof CCC && ((CCC)obj).getSomeArrray != null && i < ((CCC)obj).getSomeArrray.length; i++) ... DO: if (obj instanceof CCC) { CCC myobj = (CCC)obj; SomeArray myarray = myobj.getSomeArrray; if (myarray != null) { int max = myarray.length; for (int i = 0; i < max; i++) ... } }
The performance gain will be significant, and code maintenance will also be improved.
Crossing the NCOMP/non-NCOMP boundary is relatively expensive (especially calling interpreted methods from the NCOMPed code).
Avoid using finalizers. Finalizers are inadvisable from a memory point of view: objects with finalizers (especially ones that release large amounts of memory) typically need to be held onto across session call boundaries (end-of-call migration), which could be expensive in shared servers mode.
If you use the permissions framework (e.g.,
doPriviledged), there is a significant one-time startup cost.
Having explored the internals of the OracleJVM through this chapter, you are now in a position to understand its specifics and take the best of it. In the next chapter, I will walk you through the process of developing, compiling, and executing Java in the database.
[1] A data type whose details are not visible (i.e., hidden).
[2] See Oracle documentation for more details on DBMS_JAVA supplied package.
[3] IDEs such as JDeveloper use loadjava under the covers.
[4] Once resolved, the referenced names are bound to (and depend on) particular database objects (i.e., other classes).
[5] Signed JARs tracks the origin/signature (digest attribute, magic number) of each member in the Manifest file, along with other security files.
[6] The constant pool is an area in the Java class that keeps a reference of strings and other metadata, similar to the symbol table in conventional languages.
[7] Object-oriented concept where a virtual method has multiple implementations (behaviors).
[8] This is discussed here in the context of memory footprint reduction techniques.
[11] http://www.oracle.com/pls/db10g/db10g.to_pdf?pathname=server. 101%2Fb10742.pdf&remark=portal+%28Getting+Started%29.
[12] See the Oracle Database RAC documentation for more details.
[13] See the Oracle Database Performance Tuning Guide for a more detailed discussion.
[14] Pre-10g Database releases, OracleJVM kills any remaining live threads at the end of the call.
[15] Look also for Publishing Server-side Java in the Oracle JPublisher documentation.
[16] Consider procedures A (Java), B (Pl/SQL), and C (Java); recursive call occurs when A calls B, which in turn calls C.
[17] Normal Java VM behavior, when all nondaemon threads terminate, the VM exists (i.e., the call ends).
[18] Block of statements prefixed by finally { } for cleaning up the state of the method before passing control.
[21] A code source is an object that captures the location (URL) from which a code originates (loaded from) and also the certificate(s), if any, that was used to sign the code in question.
[22] A protection domain is a dynamic object (created by class loaders) that associates a set of permissions to given code source; in other words, it defines which permissions are granted to each running Java class.
[23] Practical Java Programming Language Guide, by Peter Haggar (New York: Addison-Wesley, 2000).
[25] See Oracle High-Performance Tuning for 9i and 10g, by Gavin Powell (Boston: Elsevier Science, Digital Press, 2003).
[26] The ratio between the size of the bytecodes and the list of strings, methods, and fields names (also known as constant pool).
[27] Method signature for all classes implementing the interface.
[28] Static initializer blocks are used to initialize static variables the first time the class is loaded.