Many voters would have forgiven David Cameron if he had failed to deliver on his campaign promise to hold an EU referendum, a study suggests.