iSonar: an obstacle warning device for the totally blind

Abstract

Existing electronic travel aids have not been widely used by the visually impaired communities worldwide due to their low performance, unattractive appearance, impracticality, and high cost. This paper proposes iSonar – a miniaturized, high performance, and low cost obstacle warning device for those who are visually impaired. It was designed in accordance with the specific requirements of Thailand's visually impaired users. To avoid collision, an ultrasonic transducer is used to detect obstacles by providing tactile feedback in different vibration frequencies at head and upper body levels; the coverage area is 130 cm in length and 80 cm in width. Our experiment compared performance between visually impaired volunteers’ standard white cane with and without our device. Fifteen visually impaired volunteers (nine males and six females) with total blindness, aged between 20 and 50 years (average age, 35.5 years), were enrolled from the Thailand Association of the Blind. Information and instructions on how to use iSonar, including device limitations, coverage area, and tactile feedback, were provided to users prior to commencing testing. Results found iSonar reduced the average collision rates by 26.66% (from 33.33% to 6.67%). iSonar also received a score of 4.13 rated as good–excellent on a Likert-scale from the visually impaired subjects.

• visually impaired
• obstacle detection
• travel aid for visual impairments
• ultrasonic transducer

There are 285 million visually impaired people worldwide, of which 39 million are totally blind and 246 million have low vision (1). Long canes and guide dogs are famous mobility tools for them. Unfortunately, these cannot protect their head and upper body from collisions, such as with street signs, bus poles, open windows, tree branches, and fences, especially in urban environments. Various detecting technologies including the Global Positioning System (GPS), lasers, cameras, and ultrasonic waves were used in research and commercial products to solve the aforementioned problems (2).

There have been some approaches that make use of GPS to localize and navigate the visually impaired in urban environments. The GPS-based Navigation System (3) is a commercial product that uses satellite signals to locate a user in an approximate range of 20 m. This device is not suitable for indoor environments as high signal loss rates and low accuracy may present dangerous situations for the visually impaired. The map-based priors for localization approach (4) generate signals from fixed location GPS receivers, and a mobile receiver is used to correct the device position. However, this GPS service is not available in all areas and cannot provide information about the obstacles in front of the visually impaired.

A tool for range sensing and environment discovery (5), laser technique improves safety (6), and LaserCane (7) uses laser distance measurement techniques. This detection has a limited coverage, which is hazardous due to blind spots. Obstacle Detection and Warning System by Acoustic Feedback approach (8) presents a stereo camera vision system to detect potential obstacles in 3D indoor and outdoor scenarios. In particular, it uses a stereo audio feedback with different frequencies to represent the presence of obstacles. Despite the reported performance, this device needs to be improved upon so that the main components fit into a smaller device.

Ultrasonic detection devices can be classified into two types: an audio feedback and a tactile feedback. Examples of ultrasonic audio feedback devices are Echolocation (9), Navbelt (10), and the project from Florida International University (FIU) (11). The disadvantage of these audio feedback devices is that the visually impaired users cannot access auditory sense in urban scenarios. Regarding the tactile feedback approach, Guidecane (12) has a limited detection on the ground and sideways obstacles only. As for the disadvantages of CyARM (13), the user always needs to hold it to scan the environment, and the device is too heavy. As for the Tactile Handle (14, 15), this device uses four ultrasonic sensors to detect obstacles toward the front, left, right, and bottom of the user's hand. The 4×4 tactile arrays represent different obstacle positions. To ensure its effectiveness, the results show that excessive user training is necessary.

In addition to these research-led devices, some commercial products have been made available, but with high costs. Miniguide (16) is a small device that represents the feedback via a small vibrating motor or earphone socket; the device is priced at approximately $330. K-Sonar Cane (17) is a device that is attached to the white cane; its price is approximately $700. The LaserCane (18) uses three laser sensors to detect obstacles for head-height, straight-ahead, and drop-off levels. This device is priced at $3,000.

Having encountered limitations of previous developments, this article presents iSonar – an obstacle warning device for avoiding head and upper body collision. Its working is based on an ultrasonic pulse transmitter. By using the tactile feedback, it represents the collision distance but does not block the user's auditory sense. Key features of iSonar include high performance, low cost, miniaturized hardware, rechargeable, and easy to wear.

Method

Specification requirements

In this research, user requirements are gathered directly from the Thailand Association of the Blind. Key requirement factors include delay, portability, user friendliness, and low cost. Specifically, the device must be responsive to obstacles. A typical walking speed of the visually impaired is approximately 1 m per second and they require a reaction time of 1.5 s. The maximum distance for detection should not be more than 1.5 m for avoiding a collision. Portability is another essential factor. It is unpleasant for the visually impaired to always hold a device. The device should be designed to be worn or attach to a part of user's body as well as the ability to use the device along with their canes or mobility aids. However, it should be a small, light device with an ergonomic shape so that the visually impaired user can carry it for a long time. The third key requirement is that the user would prefer a friendly device. The device should be easy to learn and to use, and its battery should retain charge for at least a week. The last key requirement is that the product must be reasonably priced.

Description of the approach

Our approach is divided into three main parts. The first part is the obstacle detector that uses an ultrasonic transducer to detect objects in front of a user. And for the second part, which is the main control unit with embedded software design, a microcontroller is used to calculate obstacle distance, control motor vibration, and check battery status. The last part is the device implementation that presents our hardware design, the main components, and device specifications.

Obstacle detector

The reflection of sound waves with an ultrasonic transducer is used in our obstacle detection technique. The ultrasonic transducer transmitter releases a burst of ultrasonic sound waves toward the air and receives a corresponding echo (19). The echo amplitude also depends on the reflecting material surface and the sound absorption. The distance between the transducer and the object can be calculated by the reflection echo time and the speed of sound in the air, as shown in Equations (1) and (2):1 $V_{sound}\left(m/s\right)=331.3+(0.606\times T(°c))$1

Where V _sound is the speed of sound in meters per second, T(°c) is the temperature in the air in degree Celsius. Therefore, every 1°C change of temperature is equal to approximately 60 cm/s change in the speed of sound.2 $D_m=\frac{V_{sound}(m/s)\times t(s)}{2}$2

Provided that D(m) is the distance between ultrasonic transducer surface and the obstacle in meters, t(s) is the time taken for the ultrasonic burst to travel the distance from the transmitter to the object and back to the receiver in the second unit. Assuming that the speed of sound in air is 345 m/s at room temperature, the time taken for the reflection sound wave is 8.7 ms, since the distance between the device surface and the obstacle is 150 cm. The distance measurement diagram is shown in Figure 1.

Figure 1.  The obstacle detection with ultrasonic pulse.

Our device uses the 40T/R-12B ceramic ultrasonic transducers to transmit and receive specific ultrasonic sound pulses. The 12-cycle burst in 40 kHz square wave signal is generated by the microcontroller's pulse-width modulation every 300 ms. The echo burst received by the 40 kHz ultrasonic transducer is amplified by the operational amplifier in 1000x-gain; the received signal from the oscilloscope is shown in Figure 2 (left). The first burst signal on the receiver trace represents the direct signal received from the transmitter. The next burst represents the echo reflected by the object or obstacles. The time interval between the first and second burst is measured by the microcontroller. This width represents the time taken for the reflection sound wave. It is used to calculate the distance from the device surface with the obstacles. The coverage area of iSonar is 130 cm in length and 80 cm in width, as shown in Figure 2 (right).

Figure 2.  The ultrasonic echo pulse (left), the iSonar coverage area (right).

Embedded software design

iSonar uses a PIC16F684 Microchip© Flash-based, 8-bit CMOS microcontroller to control the main function. Figure 3 presents a software flowchart of the overall function. In the first step, the 40 kHz Pulse-Width Modulated (PWM) signal is generated to an ultrasonic transmitter. A comparator module is used to store a reflection time from the ultrasonic receiver. Then, the obstacle's distance is calculated from the time. If the object distance is less than 130 cm, the motor is started and it starts to vibrate. The vibration frequency will increase as the obstacle distance decreases. The device's battery voltage is monitored by the microcontroller's Analog-to-Digital Converter (ADC) Module. If the voltage reaches a lower level or less than 3.4 volts, an alerting sound from the device's speaker will warn the user to recharge the battery. For security reasons, the device will shut itself down when the voltage reaches a critical level or less than 3.0 volts.

Figure 3.  The embedded software overview flowchart.

Hardware implementation

The main components of iSonar are presented in Figure 4 (left), including seven parts. A microcontroller is used for processing and controlling the collision warning. The 40 kHz ultrasonic ceramic transducer consists of a transmitter and a receiver, which are used to detect obstacles in front of the user. A Tactile feedback with a 3-volt 80 mA 12,500 RPM vibration motor is used to generate the warning signal. An electronic magnetic speaker informs the device status and battery information to the visually impaired with a beep. An In Circuit Serial Programming (ICSP) port is a serial interface used by the microcontroller to download a program into the microcontroller's program memory. Users can charge the device using a 3.7-volt 1,150 mAh rechargeable Li-PO battery with a 5-volt DC switching adaptor via a DC-Plug charger at the bottom of the device. The power consumption by the device is about 75 mW when used with a Li-PO battery 1,150 mAh at 3.7 volts. Average battery life is over 24 h of continuous use, and 48 h of standby. The dimensions of the device are 50×40×25 mm with a weight of 65 g. Our device can be mounted at the user's chest level with a neck strap, as shown in Figure 4 (right).

Figure 4.  The device's main component (left), iSonar's outside with a neck strap (right).

Results

Testing and evaluation

Our experiments aim to compare performance of a standard white cane with and without our device. Fifteen visually impaired volunteers (nine males and six females) with total blindness, aged between 20 and 50 years (mean age, 35.5; standard deviation, 9.6) were enrolled from the Thailand Association of the Blind. Information and instructions on how to use iSonar were provided to users prior to commencing testing. This information included device limitations, coverage area, and tactile feedback. Time was also given to volunteers to become familiar with the iSonar device.

A test field with obstacles was created that simulated the type of urban environment that the visually impaired people typically face. The straight walking test field, which was 12 m long and 2 m wide, contained five obstacles which were not harmful to our subjects. These obstacles were posted at five different levels: head height, chest height, above the waist, below the waist, and whole body level. Their positions were changed randomly in each round of the test, as shown in Figure 5 (right).

Figure 5.  A totally blind person with the device attached to his body (left), the volunteer in the simulated test field (right).

Two experiments were carried out for each volunteer. In the first experiment, the users used only the standard white cane to reach their destination. In the second experiment, an iSonar was attached to the users along with the use of the cane in Figure 5 (left). Collision rates and total travel times were recorded.

After the experiments were completed, the volunteers were interviewed with regard to satisfaction level of using iSonar. The data were collected using a standard 5-point Likert-scale (1=poor, 2=fair, 3=average, 4=good, 5=excellent). The evaluation factors included distance detection ability, tactile feedback, user friendliness, accuracy, shape and size, and cost.

Results and discussion

The results of the comparison of performance between a standard white cane with and without our device from the simulated testing field are shown in Table 1. The collision percentage was calculated from the number of collisions that the volunteers had with the simulated obstacles, divided by the number of volunteers.

Table 1. The collision percentage and total travel time

	Collision percentage (%) (n=15)
Obstacle level	Only standard white cane	Standard white cane with iSonar	Difference
Overhead	73.33	13.33	+60.00
Chest	26.67	0.00	+26.67
Above waist	33.33	6.67	+20.00
Below waist	13.33	13.33	0.00
Whole body	20.00	0.00	+20.00
Total	33.33	6.67	+26.66
The average of total travel time	1:21.57	0:57.27	+0:24.30

Results of the experiments showed that when the volunteers used only their standard canes, collisions occurred mostly at the head level and only a few times at waist level or below. This was due to the fact that the standard white cane cannot detect obstacles above waist level. When iSonar was equipped, the collision rate above waist level reduced dramatically. Specifically, the average total collision rate was reduced from 33.33% to 6.67%. Experimental results also showed that the average travel time of volunteers with and without the device was similar. This is because when the volunteers were warned by iSonar, they had to pause for a moment before resuming walking to avoid the obstacles.

The satisfaction of iSonar performance was evaluated by interviewing the volunteers. The 5-point Likert-scale score was 4.13, which can be interpreted as good–excellent, as shown in Table 2. Among the evaluation items, the volunteers were most satisfied with iSonar's shape and size. There were two items with average Liker-scale scores of less than 4 – tactile feedback and distance detection. Volunteers preferred a greater density of response and longer distance of obstacle detection. For further development, this device should be put to test in different temperatures and humidity levels to assure accuracy.

Table 2. The iSonar performance satisfaction score

Evaluation items	Average Likert scale (Total=5)
Distance detection	3.87
Tactile feedback	3.53
Convenience	4.13
Accuracy	4.33
Shape and size	4.67
Product cost	4.27
Total	4.13

In addition, improvement on the coverage limit of the device should be done through some design modifications aimed at reducing the collision rate, especially with waist-level obstacles. However, when testing for safety and usefulness of the device, long-term testing needs to be conducted to examine possible side effects when the device is in use for a long period of time. Also, several experiments must be designed and carried out in order to measure and compare the efficiency of iSonar with other existing electronic assistive devices for the visually impaired.

Conclusions

iSonar is an obstacle warning device for the visually impaired and acts as an electronic travel aid. It was designed keeping in mind the specific requirements of visually impaired users. The device helps to bring down the obstacle collision rates (above waist to head level) in visually impaired people from 33.33% to 6.67%. In addition, the device has received an average Likert score of 4.13, which is satisfactory. iSonar is a new assistive technology for serving the visually impaired while travelling, and it should be improvised further.

Conflicts of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.