In my opinion a few things make a villain scary, I'm not saying these are things which make a villain because that's a different set of values, this is just me distinguishing the traits I find truly terrifying in a villain. (remember these are my opinions.)
1. The villain has to have some measure of power, this is a must. I'm not necessarily talking about the ability to shoot lazers from his eyes, although that is certainly part of it. A combination of being physically more capable than the hero (mentioned in being a threat to him) and also mentally is important. However, a villain isn't particularly effective as a single man, he needs some kind of support base especially if he's only human. The trick is making it seem as if the villain has support everywhere, the hero can't have anyone they can truly trust, that's fear.
2. The villain must show a disregard for life, honorable villains aren't scary because they're easier to relate to and even like in some respects.
3. The villain must have a clear and terrifying reason for what he does, being pure evil for the sake of it isn't good enough to cause fear in my opinion. (Demons etc are an exception to this because it's in their nature. However in the case of humans and others their reasons have to be carefully balanced so you don't feel too sorry for them)
4. The villain has to win in almost everything he does. Always one step ahead, beating down on the hero and making it seem like his defeat is impossible. Hell, if necessary have the villain win out in the end, that's a scary villain in itself. (A keen example of this, in my opinion, is John Doe in Se7en.)
5. The villain can't be traditional, there has to be a unique characteristic which makes them stand out in my opinion. If I see the same thing twice I relate two villains and find them immensely less scary.
That's about all I can think of right now.