What is LocoMedia?
LocoMedia is a wearable computer system that allows users to 'place' media objects such
as voice notes at specific locations. These objects are subsequently 'found' by LocoMedia
wearables that enter the vacinity of the object. For example a user may record a voice
note and associate it with his current location just outside a baker's shop: "This
shop sells great doughnuts". The voice note then becomes accessible to other
LocoMedia users. When these users pass the same location, the note is played.
In contrast to its desktop counterpart, the ideal general-purpose wearable computer has
four distinctive characteristics: it is unconsciously wearable, environmentally aware,
dynamically connected and personalized. Through LocoMedia we plan to demonstrate the value
of such a wearable, thus our objectives are:
- To demonstrate the principles of an unconsciously wearable computer.
- To show how an environmentally aware wearable computer can interact in radical
new ways with its users.
- To show how a dynamically connected wearable computer can allow users to work
effortlessly with whatever peripherals are to hand.
- To show how a personalized wearable computer can be both an open component in a
public system and at the same time a personal device.
Lets be more specific about these objectives:
We plan to show that a useful wearable computer does not have to offer a
visually-biased, desktop style interaction. LocoMedia will initially emphasise speech and
audio yet still allowing opportunistic use of visual displays and handheld input as
appropriate. An audio-centric wearable is more appropriate in situations where the user
must be able to see where they are going and to use their hands for other tasks.
Nonetheless in other circumstances a visual channel may be useful (for example, for rapid
browsing) and thus we want to experiment with what will essentially be an audio-biased but
otherwise flexible device (see dynamic connectivity).
In general, there are many factors that raise people's consciousness of using a
wearable, some relate to the interaction media, but others are broader. Click here to see a list of the factors.
Situated computing concerns the ability of computers to detect, interpret and respond
to the local physical environment. In cases where the user's physical environment reflects
their mental context it now becomes possible for the computer to automatically calculate
the mental context of the user. With shared context automatically established human
computer interaction becomes radically better. This is 'Situated HCI'. The effectiveness
of Situated HCI depends on how accurately mental context can be deduced from the user's
physical environment. This will vary across different scenarios and thus in order to
evaluate Situated HCI we would like LocoMedia to support as many different scenarios as
possible. For a description of different sorts of Situated Computing and a discussion of
how to support them see: LocoMedia and Situated Computing.
Following this discussion, here are some examples we would like to demonstrate with
- A user standing in a square records the sound of a brass band that plays there
regularly. Subsequently as other users pass the spot at different times they are still
able to hear the band playing. (ta MH)
- A user and a friend leave a note in the foyer of a museum discussing which parts of the
museum they liked and disliked. Later another user is able to plan thier visit after
hearing this note.
- A user assigns a personal note to the logical location: "postboxes". The note
says: "Don't forget to post those letters". (ta DM)
- Further out work. A user walks onto Park St and hears a note about the history and
facilities on the street. The difficult thing here is that Park St is not a single
physical point or even a fixed radius from a point, it is an irregular shaped object. How
do we define such regions and then assign notes to them?
- A user walks up to a bus stop. The wearable says how long before the next bus will
arrive and where it is going.
- Apative interfaces. The wearable detects conversation and suppresses the automatic
playing of notes. Instead it sounds a quiet beep and waits for the user to command:
"play". Or the device uses ambient sound and travel speed to decide whether the
user is travelling on foot, on a bike or in a car and adjusts the range at which notes are
- The user asks for directions to a location. If the user is holding a display, this
is used, else audio directions are relayed through the earpiece.
- Attach pingers to people and thence attach notes to people. Now we nearly have ContactLense.
We plan to show how a wearable computer can work effortlessly with whatever peripherals
are to hand. In contrast to the way that desktop computers work with their peripherals,
there must be no need for wires, for software drivers or for any formal human-computer
interaction to reconfigure the computer. Local peripherals are automatically and dynamically
connected to the wearable computer in response to natural cues such as proximity or
touching. Specifically we would like to build a wireless display, wireless earpiece,
wireless microphone, wireless data-exchange bracelet, walk up LAN access point and walk up
display and keyboard terminal. Here are the scenarios we would like to support:
- The user wants to stop using their wearable earpiece. They pull it out of their ear and
put it in their pocket. There are no wires or buttons involved. Indeed it is not even
necessary to switch off the earpiece.
- A user takes a handheld display out of their pocket. It automatically switches on and
displays a map showing the current position (and orientation?) of the user. After use the
user puts the display back in his pocket there are no wires or buttons involved.
- A user can touch an "information post" outside the local Tourist Information
Centre and download from the centre a set of "official guide" audio notes stored
on the internet.
- Two users shake hands. Their wearables automatically negotiate to exchange LocoMedia
notes. Should the wearable look for all new notes on only ones originally created by the
- Further out work. To record background noises a wireless microphone can be passed
between users. Whoever is holding the mike is able to record the notes using the mike.
- Support for media other than just audio. A digital camera stores images on the
LocoMedia device worn by the person holding the camera. For example, a user photographs
balloons taking off at the Bristol Balloon Festival. (ta MH)
- A user walks up to a public display device and is able to use it with their
wearable. In this case the display may be used as an electronic whiteboard, with users
developing a schedule for the remainder of the day. Ok, so they will later want their
small personal viewers to review what the note that they agreed!
To show how the wearable can be both an open component of a public system and at the
same time a personal device. We are not particularly concerned with the sort of
personalization that is internal to individual applications or operating systems. Our
concern is more about allowing users to perform what is essentially 'programming in the
large' (object-level composition and management) without feeling like programmers or
system administrators. Our intention is to design and develop easy ways for users to
express personal policies based around the idea of publishing and subscribing
to a set of domains that reflect the theme of associated LocoMedia objects.
- As you leave a coffee shop you pass an information post from which the company offers to
add a cappucinno 'earcon' to be sounded whenever you are within range of any of their
other outlets. The user is able to accept or decline this offer.
- There is an easy way (TBD) of establishing what LocoMedia notes are exchanged when two
LocoMedia users shake hands. We need to be more specific here! The simplest approach would
be for all LocoNotes to be exchanged unless marked "private". Thereafter the
question is whether to enable receipients to impose further restrictions on what they
receive. For example, do they want to receive third party authored notes available on the
other system and how about a speech command to control the precise domain of notes to
accept ("Accept Bristol Tour Guide"). The same transfer controls could apply to
information posts as well as to two users shaking hands. Features like these will be
necessary in order to avoid overflowing recipient devices.
- There is an easy way for LocoMedia users to manage the notes on their system. Note exist
within a hierarchy of named domains that allows them to managed in groups. Users can
delete existing notes and can chose to hear only notes within a specific set of domains.
It may be that most aspects of note management are only supported using a display and pen
and not using speech commands. Of all the management commands only domain labelling at the
time of note creation really must be possible using speech, for example: "Record
'bristol tourist guide' note...". Further out it: should be possible for notes to
exist in multiple domains. Notes are automatically registered in a domain corresponding to
Making LocoMedia Usable and Useful
There are a few other user-level features that we would like to provide that perhaps
don't prove anything about wearable computing, but nevertheless make the system more
usable and useful.
- Users can visit a location "virtually", by using a map displayed on the
wireless display, or by speaking the location name. This causes the trigger of any
information as if they actually visited the site.
- Notes can be partitioned by the author. Normally, authors supply a short "title
note" before the contents of a long note. Users can ask for "more" if they
wish to continue listening to the main note having heard the title. (LocoMedia itself
makes no distinction between such note types.)
- Users can also add notes to follow existing notes left by others.
- A 'back' command would allow users to back up through a history list and then 'more'
would allow them to listen to notes they had previously skipped.
- Location and domain names can have aliases. Eg "Clifton Suspension Bridge" or
"The Brunel Bridge".
- Further out work. Location names can have position dependent interpretation. Eg
"visit newsagent" will make a virtual visit to the nearest newsagent.
- Triggering conditions can be customized by the user. Eg trigger only if user pauses
at a location that has a note. Also trigger only if note has not already been heard within
the last 'n' days. Or allow author or user to set a larger trigger range than average (eg,
by saying 'yes' at some point as a list of ranges is offered)
- We can extend the system to do comparative ranking of sights rather like the Media
Lab's film database. Users rank the sights as they tour the locality. A new user is
provided with access to these rankings and can chose where to go. As the user enters their
own opinions, these are aligned with the registered opinions and subsequent suggestions
are biased towards the suggestions of users with similar preferences.
- Automatic keyword generation based on the content of a voice note.
- A display will mark all the LocoMedia objects available in the locality.
- It is possible to find out how many more minutes of audio recording could be added
to the remaining memory.
Here are a few more LocoMedia scenarios that bring out some other value points using
system features already described:
- User says "Visit the Bristol Museum" and is then able to hear details about
the museum before visiting it.
- Two users arrange a liason point and time before separating. One user subsequently needs
extra time and leaves a message at the appointed place, saying that they will be a little
- Further out work. With a mobile web viewer, web pages of institutions could be
brought up autmotically as you pass their premises: Railway stations, Museums, Shops,
Restaraunts even home pages (eg house for sale)!
- Wireless Microphone (FootnMike). For voice commmands we need a directional mike
placed near to the mouth, yet many audio notes might actually be ambient sounds or
conversations, so a separate audionote recording mike might be handy.
How will it work?
Storing, transferring and triggering LocoMedia Objects
LocoMedia objects (aka 'notes') are stored on the wearable and triggered
according to location. In the first instance we want to support transfer of notes by user
handshake using footbridge technology (near-field radio bracelets). We also want to be
able to support downloading of voicenotes from "information posts" connected to
the internet. We considered offloading LocoMedia objects to servers and then either using
some intelligent caching strategy or even downloading them in real time, but this leads to
additional hardware and architectural challenges that are better tackled separately from
the main programme if at all.
The shape of locations
Eventually locations will correspond to user-defined rather than sensor-dictated
regions, but this is seen as further work. Initially we will accept locations defined by
the characteristics of GPS or of Pingers. So, a (differential) GPS location will probably
correspond to a longditude and latitude defined down to minutes and seconds. Theoretically
this will be a square shaped region with sides of a few metres. Pinger locations will
correspond to a fixed radius about a point, though it may also be affected by interference
and signal obstruction. In both cases we will plan for simple error correcting code to
eliminate obviously implausible sensor data caused by noise or signal reflection.
Controlling LocoMedia with Speech
Speech control will be achieved using a speaker dependent recogniser running on the
wearable computer. The recogniser will use three vocabulary banks. These will contain the
fixed command set, place names and domain names. We accept that with a speaker dependent
system every recognisable phrase will have to be trained by the registered user. This will
mean that it will be hard for outsiders to get a feel for the system without training.
Nonetheless they will be able to witness trained users demonstrating the system and this
will allow us to assess how a powerful speaker independent system might feel. By chosing a
speaker dependent system we can expect higher recognition rates, especially in noisy
environments. Accuracy might also be improved by using a discrete opposed to continuous
speech recogniser and a predefined limited grammar. The cost of this is a less
A series of heirarchical domains will be defined in order to represent an endless set
of different themes. each domain represents a different 'special interest group'. Authors
may publish their LocoMedia objects in one or more domains reflecting the content of the
note. Consumers may subscribe to any number of domains. A 'root' domain includes every
other domain, thereby providing consumers with a simple way to gain access to all notes
available at a location.
The point about domains is that they allow users to stay in control of what will
eventually be a highly open device connected to both the environment and the internet. Of
course, domains are not the end of the story, we also need notions of acess policy that
are expressed in terms of domains. One area where policies are needed is when the user
comes into contact with an 'information post'. An information post is a high bandwidth
connection to the internet (usually via a PC). It provides an opportunity to both load and
save LocoMedia objects (and Java-style applications) onto your wearable. So the kind of
policies that you might envisage include:
- Only play notes from the 'Tourist Board' domain.
- Only play notes from the 'my friends' domain and the 'commercial' domain.
- If at an information post accept all LocoMedia objects from the 'Tourist Board' domain.
- Always reject LocoMedia objects from the 'UK companies' domain.
- When shaking hands only accept notes from the other user's domain - no third party
If different users are to be able to refer to the same domains, then we need a common
naming scheme. This is not to be confused with the fact that each user will need to train
a speech recogniser to recognise their way of pronouncing their local name for the domain.
The real issue is how do we map one wearable's domain identifier onto the equivalent
identifier on another LocoMedia wearable? To solve this issue we might use CORBA object
identifiers (but what happens if the wearable is disconnected from the web for a long
time?) or an RMI registry object or in general use the URL of some kind of web-based
domain-id name server as the first part of any domain id. This is a big issue to pursue,
but initially we just need some kind of place holder.
The Speech interface
Here is the complete speech interface for the initial LocoMedia wearable:
- stop = stop | cancel | replace
One of stop, cancel or replace
is issued at the end of a piece of free audio, domain name or location name. These are
standard delimiters used with different commands as described below. stop
is the normal delimiter. cancel causes the preceeding command to abort. replace
is used to scrap the preceeding free audio or name and replace it with what
follows. replace can be used repeatedly until the user is satisfied with
their entry or until they give up with cancel.
- publish <domain name> [stop]
This causes all subsequently created LocoMedia objects to be associated with the domain
identified by <domain name>. Note that objects can be associated with as many
domains as the user likes. The association continues to apply to newly created objects
until a corresponding unpublish command is issued. Note that speech
interface provides no way for objects to change their associated domains after the time of
creation. This can only be done from the LocoMedia manager application (see below). Note
that the stop sequence is optional; after a timeout period of silence the system will say done
and assume that command is completed as if the user had finished with stop.
If the system does not recognise <domain name> it will fail with Sorry, I
dont know the domain <domain name>. If the system is already publishing to
that domain (or a parent domain of that domain) then it will silently ignore the request.
Note that the system always publishes to a domain corresponding to the user. In this way
users can never deny authorship of their own notes. By default this is the only domain
that the system will associate with newly created objects.
- unpublish <domain name> [stop]
This withdraws the association of subsequently created objects from the domain identified
by <domain name>. If the domain is not currently being associated with created
objects then the request is quietly ignored. If the <domain name> is not recognised
then it fails with the same error message as for the publish command.
Domains can be related in a hierarchy. all is pre-defined as the
"root" domain. Thus unpublish all cancels all preceeding publish
commands. Note however that the system will always still publish to a domain
corresponding to the user (see above).
- [at <location name> [publish <domain
name>]] note <free audio> stop
This is the principle command for creating new LocoMedia objects. In its simplest form the
user says note followed by the <free audio> contents and finally stop.
The optional <location name> field is used to place the note other than at the
user's current location. The optional <domain name> field is used to associate just
this particular note with the specified domain as well as with any domains declared
using the stand alone publish command described in 2. above. Note that
the at <location name> and publish <domain
name> subsequences can be repeated to register notes with multiple locations and
multiple domains. In addition, cancel can be used at any stage to abort
the whole command. Also, replace can be used to re-enter any preceeding
name or piece of free audio in the command. Finally note that in contrast with most other
commands the <free audio> must be terminated, silence will not be detected.
This is so that quiet ambient sounds can be used as valid <free audio> without being
mistaken for a "null" entry!
Aside from the uses given above, cancel can also be used to undo
the most recent publish, unpublish, note,
at or add command, or attempt to copy notes with another
user or an informatin post. If successful the command is always confirmed with done,
otherwise the system replies Sorry, I cant cancel the last command!
This command causes the system to stop the play back of the current note and any
"queued" notes that have been triggered but not yet played. Actually any
utterance from the user will have the same effect, but this is the official way to do it!
- at <location name> [stop]
This command allows the user to perform a "virtual visit" to the named location.
If the location is not recognised the system responds with Sorry, I dont know the
location <location name>! If there are no LocoMedia objects at the location
the system responds with There are no notes at this location.
- play | more | continue
Either resumes or starts playing the current note. The three terms are actually
synonymous, but users might well use different terms depending on their context.
- repeat | back
Repeats the last thing uttered by the computer. This maybe a note or a computer
reponse to a command such as space.
Stops any current playback and retriggers all notes associated with the most
recently visited location.
Suspends playback of a note, pending play, more or
continue. If a different command is issued the pause is treated
as a stop.
This skips playback of the current note but plays any following notes associated with the
same location. If there are no unplayed notes associated with the same location this
functions like stop.
- [publish <domain name>] addition <free
Stops any current playback and adds a new note associated with the current or most
recently visited location. Works in basically the same way as the note command.
If the most recently visited location is named, then at...note..
would achieve the same thing. For simplicity, when a user visits a location notes are
replayed in order of creation, oldest first. Issuing an addition command
at a particular point in the replay of notes associated with a location does not effect
the position of the note that you add in the note sequence. We may revise this in the
- subscribe <domain name> [stop]
This command together with unsubscribe is used to control the triggering
of notes. For a note to be triggered, the user must visit a location at which it is
registered and they must subscribe to at least one of the domains with which it is
associated. The subscribe command adds the specified domain to a list of
subscribed domains. If the domain or a parent domain is already subscribed the command
quietly does nothing. If the domain is not recognised, the system replies Sorry, I
dont know domain <domain name>! By default the system
starts off subscribed to all domains.
- unsubscribe <domain name> [stop]
The unsubscribe command removes the specified domain from a list of
subscribed domains. If there is no reference to the domain or any of its parents on the
list of subscribed domains, then the command is quietly ignored. If the domain is not
recognised, the system replies Sorry, I dont know domain <domain
name>! unsubscribe all can be used clear the entire list of subscribed
domains. Pending a subscribe command you would hear no further notes,
even ones belonging to the user's own domain (in contrast to unpublish all)
- new location <location name> [stop <location name> [stop] ]
This command allows the user to add a name for the current location. The speech system
requires that new vocabulary is entered twice for training purposes, hence the format of
this command. [In the interesting case where the location is indicated by more than one
sensor (eg both a pinger and a GPS fix) we might prefer to associate the location with the
GPS fix since the pinger may actually be pinned to a mobile object such as a person, but
this isnt very elegant. There are two deeper issues here; firstly if we label a location L
where Sensor1=x and Sensor2=y, then if in future we detect Sensor1=x but Sensor2=? or even
worse, Sensor2=z then what do we deduce? Secondly, by using the term location
opposed to say person in the command syntax we are adding unfortunate semantic
overtones. Specifically had we used the term person then we would prefer to use
the pinger data since pinger may be attached to people whereas GPS data is fixed at one
location. How do we solve these two issues?]
This command allows users to find out how much storage space remains on their
wearable computer. The computer replies: There is space for another <n>
minutes of notes or when less than a minute remains: There is
only space for another <n> seconds of notes
Responses / queries
In reponse to a successful cancel command or following a Copying
notes message if all notes were successfully copied.
- Sorry, I didnt understand that!
In response to an utterance that it could not understand or that didnt comply
with the grammar described above.
- Would you like to accept notes to do with <domain name>?
If a user contacts another user or an information post, the system can be
configured to check with the user before accepting notes from specific domains. If the
user responds with an unrelated command, then the notes are declined (why not!) If the
user does not respond within a preset period (5 seconds) the system again declines and
- I assume not
- There are no notes at this location
In response to an at command
- There is space for another <n> minutes of notes
or ...seconds... when less than a minute
In response to a space command
- Sorry, there is no more space for new notes!
When a note (or addition) has
to be cut short or rejected entirely through lack of space.
- Sorry, I dont know the domain <domain name>!
In response to a note, addition, (un)publish or (un)subscribe
command. Ideally the system will interject before the user has attempted to add an entire
piece of free audio. Or even better it will accept the note and then allow the user to
re-enter the domain name.
- Sorry, I dont know the location <location nme>!
In reponse to a note, addition, at command. Again we must protect the
user against the possible loss of a just recorded note.
- Sorry, I cant cancel the last command!
In response to a cancel command
- Copying notes
To indicate that notes are being transferred between the user and another user or
an information post
- Copy interrupted
To indicate when communication was abandonned with notes not completely
The LocoMedia Manager Application
There are other commands to do with more detailed management of LocoMedia objects that
are not supported through the speech interface, but through a graphical application called
the LocoMedia Manager. This application can be used to:
- run new users through command, domain name and location name training
- delete notes
- change the location and domains associated with a note.
- change the playing order of notes associated with the same location
- define new location names
- relate "physical" locations to positions on a map
- relate "logical" locations to other "logical" locations and/or
- remove, rename or alias location names
- define new domain names
- relate domains to other parent or child domains
- remove, rename or alias location names
- Further out work. associate additional media types and URLs with locations
- Allow definition of shaped regions as logical locations.
The exact form of this interface is TBD.
The method of invoking this application is also TBD.
The basic components of the LocoMedia system working with a handheld display are:
(click for clearer view)
For plans click here.
George Fitzmaurice at the University of Toronto has done work on "situated
information spaces" . His work focused on a small hand held display with a 6D
local position sensor. The thrust is on the labelling of 'objects' rather than places.
Though he also labelled places on a wall map. The main differences are in that we are
considering doing this for a larger physical space (Bristol) and so this exposes a
different set of application values. And that we are contemplating a system where the
information consumers can also be information creators, adding their own annotations to
places they ahve visited. In this sense what we have is a tool that allows users to create
new application values and to use the device for messaging and reminders.
The CyberGuide at
Georgie Tech  provides audio information for visitors to Georgia Tech. Again there was
no facility for visitors to add information at will.
The Lancaster University has also done a tourist system , and goes one step further
in that the information is stored on the web rather than locally on the wearable, but even
so, there is still no facility for tourists to add their own notes into the system.
The Esprit HIPS
programme goes all the way in proposing an extensive system that will allow users to add
their own media dynamically. This system is not yet complete - the programme is in the
first of three years. When this system is complete it will subsume the functionality of
the LocoMedia system we are proposing here!
At HP Labs Bristol we have built a prototype situated reminder system based on both
audio and text notes and using the Pinger active badge system . Setting aside the
incidental application domain of reminders, this system is very similar to the LocoMedia
system. The main extension is that notes in LocoMedia can be shared amongst different
users. This brings in notions of domains and keywords to organise the notes. The LocoMedia
system is also more evolved in that it will incorporate sensor fusion (active badge and
differential GPS data is combined) and dynamic connection to communication and IO
Also at HP Labs Bristol the (now deceased) Crescent project supported the notion of
location-oriented web servers, where web pages were presented as hyperlinks from a
geographical map of a particular region (and some other organisation actually built a
generic server system for such sites). This system was not necessarily to be linked to
users with wearable computers. It could as readily be used from a desktop PC by a would-be
visitor to the particular region.
Brad Rhodes has built a Remembrance
Agent (RA) and has suggested its extension to wearable computing . Our system can
be viewed as an audio remembrance agent, triggered by location. Compared to the work of
Fitzmaurice, the RA perspective emphasises that the creator and consumer of notes may be
the same person but at different times. The initial RA implementation was entirely
tetxually based with no location sensing. Barry Crabtree at BT is building a wearable
version with location sensing.
 Fitzmaurice G., Situated Information Spaces and Spatially Aware Palmtop Computers,
Communications of the ACM, July 1993. Vol 36, no 7.
 Long, S. et al, Cyberguide: Prototyping Context-Aware Mobile Applications, CHI 96
Vancouver, proceedings ACM.
 Davies, N., Developing a Context Sensitive Tourist Guide, 1st Workshop on HCI for
Mobile Devices, Glasgow, 1998.
 Hull, R., Neaves, P., Bedford-Roberts, J., Towards
Situated Computing, ISWC 97 Boston, IEEE Comp.Soc. Press, 1997.
 Rhodes, B.J., The Wearable Remembrance Agent: A system for augmented Memory, ISWC 97,
Boston, IEEE Comp.Soc. Press, 1997.