I'm a waiter, and I'm adamant about the importance of leaving 20% tips, never leaving 10% (because a waiter can't survive on that, which many people don't understand).
I will fight tooth and nail to express to people the importance of good tipping, how even a good waiter can be overwhelmed by more than 5 or 8 tables, how people have to not be so selfish and understand that a tip is part of the price of the meal etc; HOWEVER - this is a big however - if a server provides exceptionally poor service, like 15 minutes for drinks, especially if he/she is managing to handle the other tables efficiently, then...
Then, I don't know. I mean, I want to say leave the penny, but I think you should only do that if you're absolutely furious. And if the waiter was truly miserable. From the wait and the fact that other servers brought your food, I am considering that the server may have been handling more tables than he was supposed to or than is normal. In other words, he was probably overworking himself to make up for the restaurant's faults. That happens to me often enough, and, by default, I cannot serve everybody efficiently (better service than no service at all, but even still) and then I get awful tips.
I can't honestly expect people to be aware of that sort of situation though, and, to be fair, the other side of the coin is that the patron is receiving bad service regardless, which means that a good tip is still a charity. Serving, after all, isn't about hard work. The less hard your server works, the better tips he'll get (unfortunately, that's oddly true).
Bottom line:
There's a saying, "Don't tip an ugly stripper."
She'll quit stripping and do something else with her life. Starving her is your good deed for the day.
The same principal works for servers. If the server himself is actually a very bad server, then a penny or a zero tip can be used to send a message; however, I always suggest that people try and analyze the situation. If it is taking forever for the service, but you see the server running around with food in his hands, taking care of multiple tables and not leaning up against a pillar, chances are he's doing the best he can and you, by not tipping, are taking the bad service out on the server instead of the restaurant.