Documentation : Hardware | Software | Tutorials    Wardrobe : Jackets | Applications
Papers : Notes | Proposals | Research    Administration : Stock | Minutes | Progress

Equator CITY Project: Mack Room

Dynamic Positioning Tests

Abstract

This report describes research into dynamically determining the accuracy of indoor positioning systems. It outlines tests carried out in the Mack Room of the Lighthouse in Glasgow, and in the atrium of the Hewlett-Packard Labs in Bristol. The results show a 50% CEP accuracy of 54cm in the Mack Room including error components introduced by the test procedure. The equivalent figure for H-P Labs was 31cm indicating that the use of reflected ultrasonics contributes an error in the order of 20cm.

Recommendations for improving the system are made, in particular that beacon technology should be used to determine when the user is in occluded locations e.g. the cubicles. Minor hardware and software improvements are also identified.   

Introduction

The need to determine Vee’s position in the Lighthouse Mack Room was identified as a requirement for the Vee Scenario in the Equator CITY Project. Positioning systems have previously been studied and built as part of the Wearable Computing Project at Bristol University, though the maximum area covered by their indoor system had been 4.2m x 6.5m.

An initial survey of the Mack Room recommended that we " identify ~6 zones within the room. The sensing system, using a single ultrasonic transmitter for each zone, would determine which zone the user is in - but NOT their precise location. The use of heading data would assist in determining the user's field of interest." Nevertheless, after reconsideration and some tentative testing in Bristol, it was decided that it could be worthwhile to investigate a positioning system based on ultrasound signals reflected off the ceiling of the Mack Room. This proposal would meet aesthetic considerations, though it would rely on using the signals that bedevil other ultrasound positioning systems.

An initial installation in July 2001 demonstrated that the theory worked in practice, however there were serious problems with range, deadspots, accuracy and jitter. These were addressed by hardware changes (e.g. repositioning of sensors, increased power, new receiver designs) and the introduction of a range of software algorithms. Nevertheless the errors in the system remained problematic and hence in need of objective appraisal. This document outlines the test procedures, briefly analyses the results, and presents options for the future.

Test Procedures.

The tests were mainly based on following a predetermined trail around the Mack Room over a period of two minutes. Though this route might be typical of a visitor, the speed at which it was undertaken was considerably faster than usual. This had the advantage of making testing more efficient, however it also made the test more challenging. The tester stopped for five seconds at each of eight reference spots around the room. Synchronisation of the tester and the desired route was attempted by the use of a metronome set to 1Hz. The trail is shown as the pink curve in Fig. 1. The blue diamonds show the actual positions as measured by the ultrasonic system; they are connected sequentially by the dotted lines.

In addition static measurements were taken, and finally the timings between physically entering a trigger zone, the trigger being generated, and the corresponding web-page refresh signals were measured. The triggering test result is shown in Fig 4 with 'R' symbols representing the positions where the '$R;' web-page refresh signals were received.

A total of ten satisfactory tests were carried out comparing different receivers, correction algorithms and the effectiveness of triggering. 

To further understand the results from these tests, a 'control' experiment was carried out at H-P Labs in Bristol. The installation here also uses eight transducers; the area of coverage - 12m x 20m - and height of the transducers is similar to the Mack Room. Apart from eight large columnar trees, the configuration here is close to ideal. A similar trail to that used in the Mack Room tests was laid out with eight reference points where the tester stopped for five seconds at each one. The total test time was again 120secs. Ten sets of readings were recorded using Vee's Jacket. The trail and a sample result can be seen in Fig 5.

Error measurement.

As noted above system errors were caused by dead spots e.g. in cubicles; range/shadowing limitations;  geometric inaccuracy; and jitter. However other errors were introduced by the testing procedure. These were primarily:

The determination, and reduction, of the system errors are our main objective. To achieve this, and to produce results that may be comparable with other installations, we need to understand and to take account of these other components.

If the tester is one second late in reaching a reference spot when he is travelling at 1m/s, this will result in a one metre error in a synchronised analysis.  The time taken to walk the test route ranged between 116s and 122s. The average speed required over the route is  0.48m/s. It is likely thus that mistiming generated errors in the order of 2m.

The position of the test receiver in the hand – is around 40cm from the centre of the body. For the jacket with it’s two receivers – one on each shoulder – the potential mean error is 20cm.

Whilst every effort was made to position the sensors and reference spots accurately, the limitations or our measuring techniques resulted in further errors of around 10cm (in the order of 1% of the dimensions of the Mack Room ).   

Device comparison.

Three different receiving devices were used for the tests. The original prototype handheld, a new compact ("Lite") handheld, and Vee's Jacket with a receiving transducer on each shoulder.   

Testing with the prototype receiver had to be abandonded due to it's poor response. This receiver had recently been successfully used at H-P Labs and in Nottingham. It is thought that the recessed design of it's transducer mounting is likely to have been problematic for the indirect signals in the Mack Room.

The new Lite Rx gave good coverage in comparison to the prototype and a typical response curve for this can be seen in Fig 3.

Vee's Jacket was the most successful of the three receivers with the second transducer aiding considerably in reducing errors due to signal blocking by the tester's head and body. Fig 1 illustrates it's performance.

Correction algorithms.

A variety of correction algorithms have been devised over the past year. Of the remaining four still in recent use two have been discarded following hardware improvements. These were threshold filtering which blocked very small movements, and ultrasonic delay filtering which prevented large movements in the delay measurements.

Two algorithms remained - x/y speed limiting and the averaging of the current and previous readings to give the final output. The speed limit was tested at 1m between readings, 2m between readings and 100m between readings (i.e. off).

The algorithm which had the most effect was the speed limiting. Fig 2 clearly shows a considerable error where the tester appeared to vault across the Mack Room into the tower and back again. This exaggerated movement would have been caught by speed limiting. The 1m limit was found to cause lag, particularly when the tester was required to move at close to 1m/s. This would result in sections of the test route being missed altogether as the algorithm produced short cuts in order to catch up. A setting of 2m eliminated this effect and was used for the majority of the tests. This choice was confirmed as appropriate during the control experiments where the 1m limit gave the poorest results (by 5cm 50%  and 45cm 95%, average over three tests).

The averaging of the current and previous readings introduces a small delay of around 1/2 sec. In the Mack Room there was a small improvement in accuracy. In the control experiment, the averaging improved the 95% error by 13cm with no significant effect on the 50% measure.

Overall results.

The variety of errors (see above) made identification of the individual error components particularly difficult.

Three different analysis techniques were tried. The first, or synchronised, measured the difference between the current reading and the point at which the tester should have been at on the test route. This would include all the error components noted above, but in particular it would be subject to errors where the tester was in the right place at the wrong time.

The second approach was to measure the deviation from the specified path. This eliminates timing errors, but does not give an error reading as long as the reading was on the path. This gives more favourable results.

The third approach was to analyse the data by hand using a scaled print of Fig 1 - and a ruler. Though laborious and lacking in resolution, it gives a result which clearly indicates the size of the errors that our automatic analysis should reveal.

The path deviation technique averaged over three identical tests was used for the control result.

The results are shown in the table below:

  50% error<distance (CEP) 95% error<distance
Synchronised 0.98m 2.89m
Path deviation 0.54m 2.29m
Manual 0.5m 2.5m
Control 0.31m 2.02m

A curve showing the error distribution for the Mack Room is also shown in Fig 6. It is based on the path deviation results - apologies for the imperfect 'y' axis.

Given the potential static errors, and the system limitations, the 50% 0.5m results appear to realisatically reflect the overall performance of the installation. This is consistent with an ultrasound system error of 20-30cm; an offset of ~20cm of the receiver from the test route; and a ~10cm error due to the precision of the transducer and reference spot placement. The control experiment confirmed this distribution of error components, indicating that the use of the ultrasonic reflections was introducing an error in the order of 10-20cm.

The 95% results are much harder to analyse because readings are included from a cubicle where there is no effective signal present, and from the top left hand corner of the room (-1,+11) where the signal is intermittent due to range/shadowing problems. The loss of signal in these areas gave rise to large errors and should thus be our main focus for concern. In the control experiment, the 2.02m 95% error figure was disappointing, however this appears to have been caused by shadowing from the trees.

It is worth also considering the effect of reflections off the side walls of the Mack Room. These are not a significant factor for Vee's Jacket - except in the bottom left hand corner (-1,-8), however the handheld receiver regularly has it's 'direct' signal blocked by the user. In particular it is getting spurious readings along the right hand wall (+10), as well as in the bottom left hand corner. This effect is taken into account along the left hand wall (-2), where the ultrasonic signal is assumed to arrive after bouncing off the wall. A similar technique may be suitable for improving the handheld's performance along the right hand wall.   

Conclusions and recommendations.

The tests have shown that it is possible to track a user around the Mack Room with around 0.5m accuracy 50% of the time. We have also generated reliable triggers within one second of entering each zone, and a web page refresh is received approx two seconds later.

We have, however, failed to find an algorithm which will track users while they are inside the cubicles. We are also operating at the limit of the current system when the user is in the top left hand corner (-1,11) of the room. Under these conditions spurious readings are obtained. There are also minor problems with geometry which need to be followed up.

It would be interesting to have some user feedback on the current (big)zone sizing. It appears generous and should work reasonably well with the current system. The H-P installation uses 36 zones in a similar area and tests with around 100 users have proved favourable. It would also be worth experimenting with small zones away from the cubicles until the cubicle problem is resolved. 

We recommend:

It is possible to extend the current design with two further transducers (it would require a new unit to be built) and these could assist with improving the performance. The most suitable place for these would be in the cubicles; and/or one located at +4, +10 could solve the top left corner problem if a suitable object could be found to place it on top of.

A further option is to apply higher level techniques to determine the likely position of the user. For instance, by using the existing model of the room it should be possible to eliminate errors where it appears that the user has passed through a solid object. These techniques are potentially of more use when using GPS in a city environment and we hope to do some ground work on this during the March workshop. 

Should these measures prove insufficient for our purposes, then consideration should be given to more radical changes such as:

Acknowledgements.

Special thanks to Ian MacColl for his time, unfailing assistance, suggestions and hospitality. Thanks also to the Lighthouse Staff for their welcomes and tolerance; and also to the bemused visitors. Also thanks to Richard Hull, Jo Reid and all those at Hewlett-Packard's Bristol Labs who assisted with the ultrasonic installation.

 

Cliff Randell

1st Feb 2002


This page last updated April 26, 2002
The material displayed is provided 'as is'. Contents may not be reused without prior permission.
For problems or questions regarding this web contact Cliff Randell.