$300 or less is assuming that you already have a PC, and we're talking about PC based recordings. You can also get really good recordings using stand alone devices, but in most cases you will eventually need a PC even if it is just for making CD's or sending out MP3's. I'll try to remember to make a page on just stand alone recorders. I do have a page showing my Portable Zoom H2. It was under $150, but I still use a computer and software to trim up the clips and process the sound.
Come back soon and I will discuss free software like Audacity and AcidExpress. I'll also tell you about cheap, but good mics like Shure's SM57 and SM58. M-Audio makes a decent large diaphragm mic, the Nova. I'll also introduce you to "break out" boxes, also from M-Audio, like the FastTrack and the FastTrack Pro.