Melting the ICE

Here is a summary of those operations and actions which are accelerated by ICE (info supplied by Max Waterman of SGI):

• Real-time PAL or NTSC sized video capture from screen, camera or video source to disk when encoded as MJPEG or H.261.

• Selected image processing functions in imgview, imgworks, Photoshop, etc.

• Real-time video playback in mediaplayer for MJPEG, MPEG, H.261, QuickTime, Cinepak or AVI format video files.

• Real-time or accelerated video-editing functions in moviemaker; operations such as cut & paste are instantaneous with MJPEG, while frame-processing operations such as image-negation, find-edges, etc. are considerably faster compared to older systems which do not have an imaging accelerator.

• certain aspects of dmconvert use ICE, though at present dmconvert does not use ICE to the fullest extent possible (future changes are probable in this area).

This naturally leads onto the question of which operations are currently not accelerated by ICE:

• Generic any-size capture from screen/video/camera (only PAL and NTSC frame sizes use ICE).

• Various image processing functions in imgworks, etc. (most of the new features available for imaging work in 6.5 have been incorporated into imgview).

• None of the Graphics Library Image Tools or Graphics Library Tools.

• One can capture full-size full-frame-rate PAL or NTSC video from screen, camera or video source straight to disk as hardware-compressed MJPEG with compression ratios as low as 4:1 (at 100% quality setting, I typically saw 5:1). Once recorded, the post-recording processing time is practically zero and the movie can be played back in real-time by mediaplayer.

  Basic editing functions are also at real-time rates in moviemaker, such as crop, cut, copy & paste functions, saving loading files and various special effects. A videoout window can be used to send the contents of a mediaplayer window (or anything else on screen) to the video outputs, and in 6.5 one can send movie data straight to video-out.

• Exporting data in moviemaker does not use dmconvert and so can sometimes be slower, though improvements are likely here in the near future that will probably make exporting of data from moviemaker faster than the equivalent operation in the current release of dmconvert.

• If one wanted to capture a PAL or NTSC sized frame from a 3D Inventor or Performer application, ICE will accelerate the capture to real-time levels. But if one only wanted to capture a very small area (eg. because the movie had to be split into frames for post-processing) then ICE currently is not available to accelerate the operation in dmrecord or mediarecorder. One must use uncompressed capture or a software codec.

• With 6.5, image manipulation functions such as brightness-adjust, contrast, saturation, etc. are accelerated to almost real-time levels in imgview, and the introduction of texture tiling (as used in the 'roam' demo) offers real-time pan/roam/zoom operations. imgworks has not been changed since older OS versions, so use imgview instead for imaging work in 6.5 (or something like PhotoShop if you prefer).

• ICE can be used to process a live video stream, with operations such as edge detection, image subtraction/addition, image inversion, psychedelic ghosting; combined with the graphics engine's zoom/roam/pan abilties (using texture tiling and hardware video mipmapping), O2 is capable of a wide range of effects. Since one can use 32bit colour, the results are suitable for cable broadcast and indeed many US cable stations use O2 for just this purpose.

• 6.5 imgview offers a much more intuitive facility for pan, roam and zoom operations. On systems with hardware acceleration, these operations are real-time and operate in much the same manner as the 'roam' demo permits, although the roam demo appears to be somewhat more optimised at the moment.

On systems without hardware acceleration, the same interface controls are used, but one is not forced to wait for all of the image to be updated before making a further change in position, zoom, rotation, etc.; thus gives a good level of interactivity on systems that cannot hardware accelerate the texture mapping, eg. Indy, Indigo, etc. And if one does not like using the slider bars, then alternative controls are also available (rotational pointer, numeric zoom factor, etc.)

For enhanced image clarity, imgview offers a selection of filtering functions to prevent pixelisation. Though some methods are slower than others, simple bilinear filtering does not slow down the pan/roam/zoom functions at all, obviously because the texturing process can use the mipmapping aspects of the CRM graphics hardware. Of course, PCs could theoretically use this tecnique too, but they would not be able to offer the advantages O2 has due to its UMA architecture, ie. being able to manipulate large images.

Bicubic filtering gives better results, and one can choose from several bicubic algorithms (different types of image benefit from different filtering techniques). Of course, on very fast systems such as Onyx2, even the most complex functions will be performed in real-time.

In addition to the above, 6.5 imgview has a series of new control panels for manipulating image features such as brightness, contrast, sharpness, etc. ICE is used to accelerate these functions, with slider buttons as the means of manipulation - excellent! And again, for systems without hardware acceleration, one does not have to wait for all of the image to be updated before making further changes, giving a good level of interactive feedback.

If you're using a system which does have hardware acceleration, try turning it off (type CTRL+C in imgview and use the Edit menu to toggle hardware acceleration) and then use the various image manipulation features - you will see the way the image is altered using a tiling technique, which I suspect is ImageVision at work.

As I said on this page before 6.5 was released:

• Improved interactivity reduces the amount of time needed to come to a decision about a suitable degree of image/video alteration.

Finally, here is a summary of the differing connections between the GUI-based tools and the low-level command line programs available to the user:

• mediaconvert uses dmconvert,

• mediarecorder doesn't use dmrecord,

• mediaplayer doesn't use dmplay,

• moviemaker's 'Export As' does not use dmconvert.

• Support for a RAM disk to carry out image/video operations in memory, only writing to disk when absolutely necessary or asked for by the user. At present, disk speed can prevent some operations from going at full speed. The ability to preload the data into a 'RAM disk' would improve some operations enormously. RAM disks are a concept popular with old 8bit computers (eg. Atari ST) but for image & video work I think they'd still be useful today. People may argue about the 'safety' of a RAM disk concept, but the benefits outweigh the risks IMO - one should at least have the option. Disk-caching techniques ensure that, most of the time, a RAM disk would not give much improvement, but I have observed many situations where such a facility would definitely help, eg. script-based batch processing of multiple movie frames.

• Any-size video capture from screen or video at any relevant location in a manner that is repeatable. mediarecorder should also retain all settings when exited (just as capture used to do). This is possible, but difficult to do.

• dmconvert should use ICE to the highest possible extent (ie. whatever can be done, should be done). Some actions should become real-time if this was done properly. Again, this is possible, but difficult to do.

• All Graphics Library Image Tools and Graphics Library Tools should be rewritten to use ICE if possible. It is highly unlikely that this will ever be done.