Bristol Wearable Computing

[ Home | Plans | Meetings | Members | Search | Papers | Links | CyberWear | LocoSoft]

The Render Manager

The job of the render manager is to act as a server for client processes wishing to render notes. A standalone server is required as multiple requests to render notes may occur on the jacket. Consider the following scenario:

A person wearing a jacket is approaching a supermarket. Approximately 20 meters from the jacket a GPS fix and match leads to the rendering of the following note: "The Cyber Market. You can get really cheap chips hear! Other bargains that can be found hear are...". This note is approximately 2 minutes long and continues to be rendered as audio over the ear piece. Meanwhile, the user has entered the Cyber Market and has connected to an InfoPost. The user then picks up a visual display unit found at the kiosk. The TGPS driver then picks up and successfully recognises a pinger signal triggering a welcome note. While the audio note triggered outside is still playing, the graphical and textual part of the welcome note is rendered to the user on the visual display. The audio over the ear piece now ends. However another pinger triggers a note about the offers on the meat counter. The audio part of the note then gets rendered over the ear piece.

This example clearly illustrates the requirement for a full-blown render manager. Multiple notes triggered from the trigger manager must be able to be successfully rendered in real time. The render manager must be able to find the most suitable, or rather nicer method for rendering the note.
The render manager makes heavy usage of the device manager. It is the device manager's responsibility to keep an online record of which services are available. The render manager is particularly interested in the devices that can render notes.

The program

The render manager initially creates a Unix socket for other processes to use. This socket is then properly registered with the device manager with its correct service name being advertised. The output name is given the name "output/note". This socket is continually checked for messages using the Unix polling method. If a message is detected, a child process is forked in order to handle the message. The program then keeps on polling for more messages. When the polling function detects an incoming message, the program reads the data and copies it into a buffer. The message is an ASCII string as formally specified by the device manager. It is important to mention that once a process is forked to serve a message, the program will continually expect messages from this point until a "/done" message is received. This is also specified by the device manager.

The program then checks to see if the message is requesting the correct kind of service from this render manager. The program therefore checks that the message is requesting service from "output/note". If the type is correct the program then reads in the path of the note to be rendered. The note is therefore assumed to exist somewhere on the cyber jacket ready to be rendered. The render manager only requires information about where the note can be found.

The program must then identify what data types the note contains. The note may contain postscript, text, images, audio etc. and may be rendered in a number of ways. The program reads the data types specified in the mime code. The program detects for data declarations by simply looking out for " content type: " strings.

mime.gif (9283 bytes)

Once a picture of what the content of the note is, the program must find suitable rendering devices. It does this by a call to the device manager. Calling the function GetAvailableDevices(char ***devices) causes an array of device names to be malloc'd and pointed to by the passed parameter. The function then returns the number of available devices. The render manager then has a complete list of note data types and an up to date list of available devices. An important issue then arises:

How do we issue the data types to the various render devices?

It is known what the render devices are capable of rendering. However, certain forms of LocoMedia may be rendered in different ways. If for example all render devices are available and we get a note with text and audio. Does the program:

This gets complicated as different ways of rendering notes appear as different services become registered and unregistered with the device manager. From the available render devices, the best way to render the note elements must be chosen. In the future, this issue will become even more of a problem.
It has been decided that the rules that govern how LocoMedia is rendered is down to user policy. At this stage, policy has been hard wired into the render manager code. In the future, policy such as this may be defined elsewhere and may even be dynamic.

It has been decided to categorise the note data types as follows:

1) PCM / (PCM/INT) - this is raw audio data to be rendered directly over the ear piece.  The optional /INT indicates whether the rendering of the audio may be interrupted by a newly triggered note. This attribute may be associated with very important notes. For example an audio note containing information about a potential hazard would not want to be over ridden by a note regarding the prices of an ice cream van.   Separate renderable parts of a note may contain this /INT attribute. In the future, policy could dictate whether certain notes a interruptible. By default, all created notes are /INT

2) TTS / (TTS/INT) - this is text that may be rendered as text to speech

3) NINOTYPE - This is basically postscript with text and images.

For now, the render manager will sequentially deal with note contents as follows:

1. If a note contains NINOTYPE, this will firstly be issued to the Nino
2. If a note contains images, they will be issued to the Nino. If unavailable, no Nino will be present or NINOTYPE will be rendering on Nino.
3. If a note contains formatted text, this will be issued to the Nino. Again NINOTYPE may already be using text.
4. If a note contains raw ASCII text, this will be issued to the Nino and the audio device for text-to-speech. If the audio device is unavailable, it will not get rendered via audio. If the Nino is unavailable, it will not get rendered via this device. However a previous NINOTYPE render may be displaying the text.
5. If a note contains audio, this will get issued to the audio player. If this is unavailable, it will not get rendered in any way

The render manager is clever enough to strip out the various data types from the mime encoded note and send them on there way to the various devices. It achieves this by reading the note using the mime boundaries specified in the note. The NINOTYPE data types are nested within boundaries in the mime encoded note. When the render manager renders a NINOTYPE note, the entire NINOTYPE data block is stripped out using the nested boundaries. The Nino device then gets sent the entire NINOTYPE block which itself contains boundaries for the Nino device to handle.

Managing the rendering of notes

The render manager must have be aware that certain notes may be triggered from the same context.  When this happens the notes are put on a queue.  The render manager sequentially goes through this queue rendering the notes as it does so.  Every note triggered from the trigger manager is also put on this queue.  However if the context of the note differs from the contexts that triggered all the notes previously put onto the queue, the note is immediately rendered.  The previously queued notes are also discarded. 


If note triggered from other process:

             If note differs from current context:  discard Q, interrupt current note rendition put note on Q

             else put note on Q


             If note on the Q,  render note at start of Q

goto START

Rendering queries

During the use of the cyber jacket, many situations arise where the system requires a Yes/No answer from the user. For example, the user may be asked whether they want a certain block of notes to be down loaded over the footbridge link.  Yes/No responses are also frequently required to resolve situations in policy where the policies involved do not dictate an unambiguous action.

Queries may be rendered in many ways. They may be rendered as a text-to-speech question requiring the user to reply with a spoken Yes/No response. A query could also be rendered over the hand held Nino requiring the user to click on a Yes/No button as a response.  This functionality to achieve this kind of service is also built into the render manager.   A query is in the same form as a note. It may contain multiple parts and data types which may be rendered on separate devices.  The only difference between a note and a query is that only one data type from the query may be rendered to the user.  It would be pointless to ask the user a question over the ear piece as well as on the Nino.  

Again an issue of policy arises.  In the future, queries may contain independent policies allowing them to be rendered in specific ways.  For now, a query containing a NINOTYPE query will be rendered first (if Nino present), blocking any other data type renderings.  If no Nino types are present, then any text-to-speech data types are rendered.

unicrest.gif (4191 bytes)

The material displayed is provided 'as is' and is subject to use restrictions.
For problems or questions regarding this web contact Cliff Randell.
Last updated: January 14, 2000.
logoep.gif (1404 bytes)
ęCopyright Hewlett-Packard 1997-2000.