The poll for driving safety charity Brake found 49% of 17-24 year olds had been in work calls while on the road, compared with 17% for all age groups.
About 35% of young people said they had spoken to family, and 21% with friends – compared with 23% and 15% overall.
Brake asked about hand-held and hands-free calls. They are both a distraction “risking devastating crashes”, it said.
“It is a sobering thought that a significant number of these life-threatening distractions come from drivers’ own friends and family,” said Julie Townsend from the charity.
The survey questioned 1,000 drivers from across the UK on whether they had spoken on the phone while driving, hand-held or hands-free, in the last 12 months. If so, they were asked who they were talking to.
Using a hand-held phone while driving was made illegal in 2003. Using a hands-free kit is allowed but a driver can still be stopped if police believe they are being distracted.
Brake says that making calls hands-free is still a cause of distraction and are campaigning for it be banned.
The charity is advising drivers to put their phones on silent while driving, with their phone away from them. The safest way to make a call is to stop, they say.
According to an observational study by the Department for Transport last October, motorists who use phones are more likely to be texting or using social media than making calls.
Brake has also warned about the dangers of being tired while driving, saying that it is wise to break for at least 15 minutes every two hours.
When asked how often they driven for more than two hours without a break in the last 12 months, 8% said weekly or more, 16% once a month and 29% once a year.