Drivers who use hands-free commands remain distracted for up to 27 seconds after they have made a call or changed music, according to the AAA Foundation for Traffic Safety.
Working with researchers at the University of Utah, the auto club's safety branch found that this lingering distraction raises new concerns for using phones and vehicle infotainment systems.
"Overwhelming scientific evidence concludes that hands-free is not risk-free," said Marshall Doney, AAA's chief executive.
The distraction continues even when a motorist turns attention back to the road and has both hands on the steering wheel, the AAA report said.
"The results indicate that motorists could miss stop signs, pedestrians and other vehicles while the mind is readjusting to the task of driving," said Peter Kissinger, chief executive of the AAA Foundation for Traffic Safety.
AAA and the university looked at 10 cars from the 2015 model year and three phone systems and discovered that not all systems are equally distracting.
Rating mental distraction on a five-point scale, with a lower score being better, the researchers found the best performing system was on Chevrolet's Equinox crossover. It had a rating of 2.4.
The worst was the Mazda 6 sedan, with a cognitive distraction rating of 4.6.
Mazda said it already has improved the system for it 2016 model year lineup.
The other cars tested and their ratings were the Buick LaCrosse, 2.4; Toyota 4Runner, 2.9; Ford Taurus, 3.1; Chevrolet Malibu, 3.4; Volkswagen Passat, 3.5; Nissan Altima, 3.7; Chrysler 200c, 3.8; and Hyundai Sonata, 3.8.
AAA judged a mental distraction rating of 2 or higher to be potentially dangerous.