Slicehost is a hosting provider which specializes in providing VPS service to developers at a reasonable price. They have a number of OS's to choose from and a lot of tutorials to help newbs learn the ropes of setting up and maintaining a VPS.
This VPS services is just for developers and is excellent! The base VPS package (256 RAM) is only $20 a month. The services is reliable and the support has been quick the few times I've needed it. I have had no downtime except for issues over a 48 hour period. Despite not having a SLA I was given a free year of hosting for that incident. What fantastic service and high standards!
As they say on their site, Slicehost is for developers. That means you have to install and manage everything on your own. That includes the OS, LAMP, etc - even security updates and configuration. I was hoping it was something I could easily pick up, and I eventually did get it to work. Unfortunately that's not something I want to spend so much time on.