16. ADAS - Advance Driver Assistance Systems - Chat with Dr Jing Xu research with visually impaired driver use

Main photo Dr Jing Xu, small photo on bottom right Belinda O’Connor

I came across Dr Xu’s work through The Eye, The Brain, The Auto “10th World Research Congress on Vision and Driving and the Impact of Autonomous Vehicle Technology on Healthcare”https://www.henryford.com/hcp/research/clinical-research/vision/research-congress/eye-brain-auto . And her paper “Clinical Report: Experiences of a Driver with Vision Impairment when Using a Tesla Car” in which the driver was also a bioptic driver: https://www.researchgate.net/publication/358559671_Experiences_of_a_Driver_with_Vision_Impairment_when_Using_a_Tesla_Car “Conclusions: This clinical report demonstrates how assisted and semiautonomous driving systems in a Tesla car were used to support daily driving by a driver with vision impairment. Codriving with these systems allows him to confidently drive more often and to avoid less situations than he used to.”

 

About Dr Jing Xu

 

Dr. Jing Xu holds the position of Research Associate at the Envision Research Institute and has recently transitioned to Harvard University to continue the same area of research. Prior to joining Envision, she worked as a postdoctoral research fellow at Dr. Alex Bowers’ lab at Schepens Eye Research Institute of Mass Eye and Ear, Department of Ophthalmology, Harvard Medical School. Dr. Xu received her PhD in Industrial Engineering with a concentration in human-machine interaction and driving safety research at Northeastern University. Her current research is focusing on investigating the impact of vision impairments on driving, developing new rehabilitation training device to train drivers with impaired vision, and investigating driver assistance technologies and self-driving car accessibility features to promote safe mobility and life independence for road users who are blind and visually impaired. Dr. Xu’s fellowship was funded by Bosma Enterprises, an Indianapolis-based manufacturer and provider of job training, employment services, rehabilitation and outreach for people who are blind or visually impaired. Bosma Enterprises is a sister agency to Envision through National Industries for the Blind based in Alexandria, Virginia.

 

 

Discussion

 

I commenced discussions with my interest in this area noting my former role within the Australian Government’s Office of Future Transport Technology section of Connected and Automated Vehicles (CAV) working on secretariat and policy matters for CAV for first supply and in-service, particularly my interest in how both ADAS (Advanced Driver Assistance Systems, SAE level 1-2) and ADS (Automated Driving Systems, SAE level 3-5) could support disabled drivers. I also expressed how impressed I was at the operation I visited whilst in Indiana for Bosma Enterprises, referring to the information in the blog for Indiana. Noting Dr Xu’s research was funded by Bosma Enterprises. Dr Xu noted her admiration of the extensive operation and that it hires 80% visually impaired people, including in manufacturing and front of office of blind people. I mentioned that Australia has transitioned away from government funding for disability organisations to a National Disability Insurance Scheme that funds the participant instead, but that transition has resulted in excluding people with low vision from access to disability supports. Australia does have ‘sheltered workshops’ where the government subsidises the wage and not something that resembles the ethos nor scale of Bosma or Envision.

 

Dr Xu said at the beginning of her research she was looking at vision impairment and factor for driving because in the United States has a lot of drivers who have moderate visual impairment who are still permitted to drive, and states have different laws. She is also interested in the bioptic and human factors for driving. Jing stated she comes from an engineering background with interest in the interaction of human factor and advance driver systems. This includes blind spot warning where the car will warn when a car is coming up from behind you, forward collision warnings to give a beep or other warning including may slow your car and then adaptive cruise control. In exploring this area there was no research on vision impairment use of this technology, only ‘normal’ sighted drivers.

 

Dr Xu started her research during the COVID pandemic with social distancing meaning she started with interviews instead of in-car research. Her team and collaborators interviewed in-lab, about 300 drivers, many with Age Related Macular Degeneration (AMD) or central vision loss. She wanted to understand whether the ADAS systems were helpful and why they were or not. They found many people with vision impairment use this technology with the user self-exploring and the users found the technology helpful. Her research group were people aged 60-80 years of age. They found the driving patterns were more frequent when using this technology. These drivers were more likely than age matched normally sighted drivers to have newer cars with ADAS to support their vision impairment. Further, one of her team’s studies found that the bioptic drivers reported GPS are very helpful and can compensate for their vison loss. Bioptic users will use their bioptic to track the traffic lights and verify road signs, if possible, by the GPS as they have difficulty reading road signs and GPS and prefer to use the audio to hear the GPS. Many of them reported the GPS has replaced the need for using the bioptic to read road signs. Conversely, users also had the most difficulty with GPS because they had difficulty reading the interface such as the little arrows.

 

We talked about the use of Mobile Eye as a portable device to be put on the dash to give audible warnings. I noted one state give their bioptic drivers a Mobile Eye when they get their licence.

 

Dr Xu stated the AI technology can detect pedestrians or obstacles and give warnings as an aftermarket product. She noted their research is focusing on systems that come with the car as integrated at time of purchase.

 

After interviewing 100 drivers with central vision loss who use these systems on their perceptions and views, they focused on one driver who uses a Tesla. He bought his car in 2020, ordered during the pandemic with it being delivered to his door and no trainings so he had to self-explore. Dr Xu followed him for two years. It took the driver about three months to get use to the car’s functions. The driver is unable to read the 15” display without a magnified support but can read the speedo. He used his iPhone App to make adjustments and voice control e.g., turn off windscreen wipers. Whilst the voice activation function was really useful for this and other vision impaired drivers the drivers stated they wanted a system that also gave audio interaction to check and verify function and ask for options.

 

The Tesla ADAS/ADS assisted this driver in tandem with his bioptic telescope glasses to help check the roadside, check traffic lights or front road situation, spot hazards. Overtime the driver developed a new/different driving strategy as the Tesla being a part of his extended body. This raised questions about law reform to allow drivers to use ADAS features whilst learning to drive, the test and onroad if that is going to be their way of driving.

 

In compensating for his vision loss, he developed at random strategies. For example, city driving one system, shift to another for highway driving that moved from active driving to monitoring and giving permission to Telsa prompts such as a manoeuvre to change lanes. So the driver still need to monitor traffic to ensure the change lane suggested by the Tesla is safe. In two years, he drove 30,000 miles, had multiple trips to different states which he was not able to do with his decreasing vision now being 20/180. Noting he had no accidents or citations. He was able to visit friends and family during the pandemic that contributed to supporting his wellbeing.

 

Despite having this technology, as usually before he leaves the house, he carefully plans his driving route taking into account he glare sensitivity when to avoid driving or routes such as unprotected left turns. However, the car allowed him to reengage with night driving. He reported being less stressed and more confident. Before he would avoid certain intersections, city areas. Dr Xu noted the driver is a very safe driver as being very careful and does not do anything beyond his capability in that is aware of his visual limitations/ struggles. He is also aware of the car’s limitations in that it only driver assist, and not complete automation and he is always monitoring/ supervising the driving to take control if and when needed.

 

We had a chat about on road research of autonomous vehicles with some high-profile accidents and deaths and people expecting more from the car then available and getting into trouble from that.

 

Dr Xu said she sees the potential future of this is allowing driver rehabilitation with ADAS built in addition to allowing standard driving to learn to drive, driver on road testing and renewing a licence – but there is still a long way to go. Ensuring the technology is known by the driver and they can take advantage of all its features, to try to avoid misuse to help the drive be safer and drive longer. I mentioned from my interaction with a Certified Driver Rehabilitation Specialist in Kentucky I was advised that state does allow a person to do the onroad test using the reversing camera if the car has one, unlike all other places that do not allow use of any of the ADAS features. The use if demonstrated must be to popper use meaning the driver must stop when looking at the camera and not use it whilst in motion.

 

Dr Xu noted this driver’s use of his bioptic telescope decreased about 30%. The Tesla has traffic light detection and can stop before the intersection, the driver just needs to monitor and give permission, so he just checks using the bioptic whether it is red or green. So the bioptic’s purpose has shifted a lot for this driver, it is now often used to confirm the car’s actions.

 

Dr Xu and I discussed that more models of cars are coming onto the market with varying ADAS features and its up to the consumer to review what is available and decide what features they will use and find helpful. We discussed that however the concept of ‘self-driving’ is still a very long time away.

 

In discussing human factors for drivers with vision impairment, Dr Xu stated another part of their research is In-lab driving simulators. Research questions she is trying to understand include: how different types of vision impairments affect the driving safety, how different ADAS technologies can provide varying benefits for drivers with different types of vision loss, and how different designs of the ADAS user interface would best benefit visually impaired drivers such as different warning modalities (auditory, visual, tactile), different warning timing thresholds. Which types of vehicle interface designs should be provided to best benefit visually impaired drives. Different vision impairments experience different challenges from E.g., Peripheral visual field loss, central vision loss. Working out what works for various groups. We discussed we know vision impaired users are too small a group for vehicle manufactures to consider changes based on this research. But at the same time, we know innovation comes from users and disability as an example it was a blind man who the first version of cruise control so who knows where this can lead. Dr Xu noted this research can lead to further refinement of current ADAS and future ADS technology to assist universal design such as making the user interface for all cars more accessible and thus available to wider population.

 

 

 

I wish to thank Dr Jing Xu for agreeing to chat with about this exciting field of research and I look forward to seeing further.

 

To learn about the reasons for this blog, go to my Churchill Trust Australia webpage to read about the fellowship I was awarded being "The NRMA - ACT Road Safety Trust Churchill Fellowship to identify success factors and barriers for low vision and telescopic glasses driving" The page is here:  https://www.churchilltrust.com.au/fellow/belinda-oconnor-act-2022/