So what should you do? Well, here it is - my #1, works every time, no fail technique for letting a guy down easy. Be honest! Yeah, I know how incredibly difficult and unrealistic that advice is, so let's get to useful stuff.
After more first dates than I bothered to count and almost as many let-downs, I developed a simple and efficient "breakup text". After a bad date, I'd wait to see if the guy called or texted. If he didn't, great! If he did, I'd send him this text:
"Hey, I really enjoyed my time the other night, but I'm not interested in pursuing any kind of relationship with you. Sorry and I hope you find the right gal for you."
It seems harsh. It's supposed to be. Whether he was a shmuck or you just weren't that into him, this is your opportunity to make your disinterest clear and give him the opportunity to move on and not look back. If you're overly sweet in this message, if you compliment him, and drag it out he'll wonder why you aren't interested. If you think he's such a great guy, why wouldn't you just go out with him again? If you absolutely must add some sugar to his medicine I recommend something along the lines of, "You're very [insert compliment - cute, smart, funny], but you're just not the right guy for me."
You may be thinking to yourself, "but this is super bitchy! Why would I want to say this?" Because once he thinks of you as a bitch he can move on and not look back. He'll be upset if he loses his opportunity with a great, sweet gal. It may even hurt his ego and damage his confidence. He wont, however mourn the loss of a bitch. If you play the bitch for him, you'll never hear from him again and you'll be doing him a favor by letting him move on.
I don't recommend doing this in a phone call or in person. It opens the doors for discussion, which you don't want. If he continues trying to argue with you about your point Do Not argue back. This is not a debate and should not be treated as such. You said your peace and now it's time to move on.
Letting a guy down after a date (or even two) is difficult, but it doesn't have to be. The sooner you both put it behind you, the sooner you can find another date.