It could work really well if someone had a voice that fit their character and was a good actor. But most people aren't good actors. Lots of people think they are, but most people aren't. So like you said in the OP, you'd have to do "auditions".