Machine learning systems have for years now been besting their human counterparts at everything from Go and Jeopardy! to drug discovery and cancer detection. With all the advances that the field has made, it’s not unheard of for people to be wary of robots replacing them in tomorrow’s workforce. These concerns are misplaced, argues Gerd Gigerenzer argues in his new book How to Stay Smart in a Smart World, if for no other reason than uncertainty itself. AIs are phenomenally capable machines, but only if given sufficient data to act on. Introduce the acutely fickle precariousness of human nature into their algorithms and watch their predictive accuracy plummet — otherwise, we’d never have need to swipe left. In the excerpt below, Gigerenzer discusses the hidden privacy costs of sharing your vehicle’s telematics with the insurance company.
Excerpted from How to Stay Smart in a Smart Worldby Gerd Gigerenzer. Published by MIT Press. Copyright © 2021 by Gerd Gigerenzer. All rights reserved.
When Your Car Reports You to the Police
If self-driving cars are not going to happen, one alternative appears to betraining humans to use AI as a support system but to stay alert and retain control if it fails — which is called augmented intelligence. It amounts to partial automation, that is, to sophisticated versions of Level 2 or 3. Yet augmented intelligence entails more than just adding useful features to your car and may well lead us into a different future, where AI is used to both support and surveil us. That possible future is driven more by insurance companies and police than by car manufacturers. Its seeds are in telematics.
Young drivers are reckless, overconfident, and an insurance risk, according to the stereotype. Some indeed are, but many are not. Nevertheless, insurers often treat them as one group and charge a high premium. Telematics insurance can change this by offering better rates for safe drivers. The idea is to calculate the premium from a person’s actual driving behavior instead of from that of the average driver. To do so, a black box that connects to the insurer is installed in the car (using a smartphone is possible and cheaper but less reliable). The black box records the driver’s behavior and calculates a safety score. Figure 4.6 shows the scoring system of one of the first telematics insurers. It observes four features and assigns them different weights.
Rapid acceleration or harsh braking is assigned the greatest weight, followed by driving over the speed limit. Each driver starts with a monthly budget of 100 points for each of the four features. An “event” results in points being subtracted, such as 20 points for the first rapid acceleration or for driving over the speed limit. At the end of the month, the remaining points are weighted as shown and summed up to a total safety score. Although telematics is often called black box insurance, the algorithm is not at all a black box like most love algorithms. It is explained in detail on the insurer’s web-site, and everyone can understand and verify the resulting score.
Personalized tariffs are advertised as promoting fairness. They do so by taking individual driving style into account. But they also create new sources of discrimination when driving at night and in cities is punished. Hospital staff, for instance, may have little choice to avoid working at night and in cities. Thus, some of the features are under the driver’s control, but not all. Interestingly, one feature that is under the driver’s control is absent in virtually all personalized tariffs: texting while driving.
And the black box that allows fairness also enables surveillance. Consider a possible future. Why should the black box send a record of speeding only to the insurer? A copy to the police would be extremely handy and save them much effort. It would make all speed traps obsolete. If you speed, the car prints out the ticket on time or, more conveniently, deducts the fine automatically from your online account. Your relationship to your beloved car may change. There is a slippery slope between fairness and total surveillance.
Would you be in favor of a new generation of cars that send traffic violations directly to the police? In a survey I conducted, one-third of the adults said yes, more so among those over sixty and less among those younger than thirty. The technology for this future already exists, as most new cars come with a black box installed. The data it collects do not belong to the car owner and can be used in court against the driver. In Georgia, the police obtained black box data without a warrant after a deadly accident, and the driver was found guilty of reckless driving and speeding.
While the motives for surveillance vary, digital technology supports all of them. One need not even buy telematics insurance. Modern cars have built-in internet connections, and — without it being made transparent inthe owner’s manual — most send their car manufacturer all the data they can collect every couple of minutes, including where the driver currently is, whether harsh braking occurred, how often the position of the driver seat was changed, which gas or battery-charging stations were visited, and how many CDs and DVDs were inserted. Moreover, as soon as you plug in your smartphone, the car may copy your personal information, including contacts’ addresses, emails, text messages, and even photos. Car manufacturers are remarkably silent about this activity, and when asked with whom they share this data, they typically do not reply. That information helps to find out many other things of interest, such as how often drivers visited McDonald’s, how healthily they live, and whom they occasionally visit at night. Connected cars can support justice and improve safety but also spy on you. Telematics insurance embodies the double face of digital technology: surveillance in exchange for convenience.