24. Advanced Multimedia

THE JSR 234 Advanced Multimedia Supplements (AMMS) specification defines standard extensions to MMAPI. Most of the API is composed of additional Controls, but some additional infrastructure is also provided. The AMMS specification defines six capabilities that serve as groups of functionality.

• imagepostprocessing supplies a structure for applying image effects like blurring, sharpening, or posterization.

• imageencoding allows applications to encode images using standard file formats.

• music provides volume and equalization controls for audio playback.

• audio3d enables your applications to place sound sources in a virtual 3D space surrounding the listener.

• camera provides access to camera features like flash and exposure control.

• tuner allows applications to control a device’s FM or AM radio.

A device that supports a capability provides some minimum functionality, which is described in the specification. At runtime, you can find out which capabilities are supported by retrieving the system property supports.mediacapabilities. It contains a list of capability names separated by spaces.

The MSA specification requires devices to support imagepostprocessing and imageencoding. If the device has a camera, the camera capability must also be supported. If the device has a radio tuner, the tuner capability must also be supported.

24.1. Image Processing

If your platform supports it, applications can use AMMS to perform common image operations like blurring, sharpening, posterizing, resizing, and rotating. Image processing is performed by a MediaProcessor, which knows how to manipulate a certain type of input. You can get a MediaProcessor from GlobalManager by specifying the input type. Both MediaProcessor and GlobalManager are in javax.microedition.amms.

For example, to process JPEG images, you would do this:

You can find out what input formats are supported by calling GlobalManager’s static getSupportedMediaProcessorInputTypes() method. The Sun Java Wireless Toolkit returns image/png, image/jpeg, and image/raw, although only image/jpeg and image/raw are required. Raw means that the MediaProcessor will manipulate Image objects rather than manipulating byte streams.

Tell the MediaProcessor about your input (source) image with setInput(). One form of the method accepts an InputStream and a byte length, which can be MediaProcess.UNKNOWN. For example, you can use a JPEG image as input like this:

The MediaProcessor writes the processed image to an output stream. Use setOutput() to tell MediaProcessor where your output should go.

The next step is picking the transformations you want to apply to the image. These are accessible as Controls on the MediaProcessor. You can get useful controls from MediaProcessor by specifying their class names, just like you can with Players. Common photographic effects are encapsulated by ImageEffectControl, which includes useful presets. Choose a preset by passing its name to setPreset(). monochrome and negative must be supported, but other possibilities include emboss, sepia, solarize, and redeyereduction. Call getPresetNames() for a list of available presets. Take a look at the API documentation for ImageEffectControl for a description of each preset.

In this example, the emboss preset is used. Note that you must enable the control for it to affect the output image.

Other controls might be available. The following example shows how to scale an image to 80% of its original size using an ImageTransformControl:

Useful controls for image processing live in the javax.microedition.amms. control.imageeffect package.

You can control the output of the MediaProcessor with an ImageFormatControl.

When you are finished working with the Controls of the MediaProcessor, the start() method kicks off the processing. If you’ve registered a MediaProcessorListener, you’ll be notified when the processing is finished. If you’d prefer to wait instead, you can call complete().

If you prefer, you can use a MediaProcessor that works with Image objects directly instead of a MediaProcessor that works on byte streams. In this case, use an image/raw MediaProcessor. The other form of setInput() accepts an Image object directly:

In the example code for this chapter, HanzUndFranzMIDlet contains two methods that illustrate aspects of image processing:

The first uses a JPEG file as InputStream input. The other shows how to pass an Image directly to an image/raw MediaProcessor. In both cases, the source image is shrunk and embossed. The source and processed images are both displayed. See Figure 24.1.

Figure 24.1. Processed fish

24.2. Controlling Image Format

Unless you say otherwise, MediaProcessor spits out the same format you feed it. For example, if you create an image/png MediaProcessor, the output will be a PNG image.

You can control the output format with an ImageFormatControl. Here is an example that sets the output format to PNG:

This capability can be used without any processing to encode images in any available formats. The available formats are returned by ImageFormatControl’s method getSupportedFormats().

24.3. Music

The music capability of AMMS is simple. All it means is that a VolumeControl and an EqualizerControl must be available from GlobalManager. These controls affect all Players.

24.4. 3D Audio

The AMMS audio3d capability allows applications to make sounds seem as though they are coming from different locations. A 3D sound engine processes sounds to make the listener perceive that they are in a particular location. The sound is physically delivered using stereo headphones. A sound that emanates from only the left side of the headphones is perceived to be located on the listener’s left side. More sophisticated frequency and temporal processing can make a sound appear to be in front, behind, over, or under the listener.

For an excellent introduction to the fascinating topics of sound and audio perception, read the Interactive Audio Special Interest Group (IASIG) guidelines for 3D audio rendering:

http://www.iasig.org/pubs/3dl1v1.pdf

http://www.iasig.org/pubs/3dl2v1a.pdf

Many 3D APIs follow the IASIG specifications. Developers who are familiar with such APIs will find the AMMS 3D Audio API easy to understand.

If a device has hardware or software support for 3D audio, the MSA specification requires support for the AMMS 3D Audio API.

The simplest way to use the 3D Audio API is to create a Player, then add it to a SoundSource3D. You can use controls on the SoundSource3D to affect the perceived position of the sound.

For example, you could create a Player as usual, like this:

Get a SoundSource3D from GlobalManager and add the Player to it:

SoundSource3D can have a variety of Controls. LocationControl must be available, and other control types might also be available. The 3D audio controls all live in javax.microedition.amms.control.audio3d.

Modifying the perceived position of the audio is as simple as obtaining a LocationControl and calling one of its methods:

Then you can start the Player as usual:

One obvious use of the 3D Audio API is to provides sounds for 3D graphics objects that are shown using the Mobile 3D Graphics API (JSR 184). Just move 3D objects and their corresponding audio sources simultaneously. Remember, mobile devices have slower processors and less memory than desktop computers, so you might need to trade quality for speed in order to produce an application that won’t overwhelm a small device.

Together3DCanvas, included in the example code for this chapter, animates a rotating musical tetrahedron (see Figure 24.2).

Figure 24.2. A musical tetrahedron

As with most MIDP applications, device testing is crucial to success. You should run your application on a representative sampling of target devices to observe how it performs and then adjust accordingly.

Recognizing the exact or even approximate location of the sound is very subjective and depends on the listener. Each listener’s ears and frequency response are different, and it is impossible for any 3D audio implementation to work perfectly for all listeners. Distinguishing between left, right, front, and back for an audio source is usually possible. It’s quite difficult to distinguish between above and below. Providing visual clues to the sound’s location greatly increases the chance of convincing the listener of the location of the sound.

24.5. Audio Special Effects

Special effects like reverberation, chorus, and equalization might be available through AMMS, although they are not explicitly grouped as a capability. These types of audio effects are often available from the same audio engine that handles 3D effects.

The audio3d capability requires that a ReverbControl be available from GlobalManager, but other types of audio effects might or might not be available. As usual, you must be careful when you attempt to use these controls.

The audio effect controls, unsurprisingly, are in the javax.microedition.amms. controls.audioeffect package.

24.6. More Camera Control

AMMS does not change the basic technique for using a camera, but it allows your applications to control the camera in much more detail.

You’ll still get a Player representing the camera with capture://video. The javax.microedition.amms.control.camera package contains a bevy of useful controls.

• CameraControl provides general methods that will be familiar to anyone with a digital camera. You can learn about the supported resolutions of the camera and supported exposure modes, which have understandable names like auto, sunset, portrait, and so forth. You can set the exposure mode and examine the camera’s current rotation state.

• ExposureControl allows your application to manipulate the F-Stop, ISO, exposure time, and other photographic settings of the camera.

• FlashControl has methods for controlling the flash mode. Constants represent different modes: AUTO, AUTO_WITH_REDEYEREDUCE, FORCE, and OFF are a few of the possibilities.

• FocusControl enables applications to manually focus the camera or request automatic focus.

• SnapshotControl supports burst shooting by which the camera takes pictures as fast as it can and saves them automatically as files.

• Finally, ZoomControl provides methods for querying and setting both the optical and digital zooms of the camera.

Remember that these controls are not necessarily available on all devices. Code defensively, testing the results of getControl() for null and only exposing features to your users that are actually available on the device.

24.7. Plain Old Radio

Mobile phones always have a radio in them, the one that communicates with the cell tower that connects the phone to the rest of the network. Some phones also feature an old-fashioned FM or AM receiver.

You can access a radio tuner as a Player by requesting capture://radio. Once you’ve got the Player, TunerControl and RDSControl give you methods for using the radio. Both control types live in javax.microedition.amms.control.tuner.

TunerControl allows you to select the radio band and find a specific station, while RDSControl is useful for receiving information about stations and songs.

The system property tuner.modulations contains a list of supported radio types, separated by whitespace. The Sun Java Wireless Toolkit emulator returns the following:

The emulator doesn’t have a real radio, of course, but it simulates one with a few different stations.

This code sets up the radio and seeks an FM station, starting at 99.9 MHz and moving downward:

Using RDSControl is a little more complicated. First register a PlayerListener. When an event comes to the listener that is RDSControl.RDS_NEW_DATA, you can use the RDSControl to examine the incoming information.

For a comprehensive example, consult TunerDemo in AdvancedMultimediaSupplements, part of the Sun Java Wireless Toolkit demonstrations.

24.8. Summary

AMMS expands on the promise of MMAPI with all sorts of goodies. 3D audio, image processing and encoding, camera and radio control are a few of the highlights. MSA requires support for the image encoding and image processing capabilities of AMMS. In addition, if the device has 3D audio infrastructure, a camera, or a radio tuner, MSA requires support for the corresponding AMMS APIs.