If you're looking for a single self-contained device, I've been quite pleased with my Zoom H4n. When I glanced online, it looks like they're running about $250ish.

Although it's self contained, it also contains two combination inputs that can accept either 1/4" or XLR, which can record simultaneously with the (excellent) internal mics, which means you could have a more expanded micing situation (2 externals and the stereo internals) if you wanted to.

It also has a tripod mount as well as a mic-stand mount that could be useful if you're setting it up down my the pews and want to point it up at the loft easily.

You can also play back you're recordings once easily on the device, so you can show your choir right away.

The sound quality is also excellent. I've recorded radio commercials for broadcast on it's internal mics, and the audio quality fit right in with the rest of the spots being aired.

Overall, I've been quite satisfied with it and would really recommend it.

I'm thrilled with my Edirol R-09. It's pretty idiot-proof and records CD quality audio directly to mp3 format for easy sharing. You can use an external mic suspended from the choir loft if you have to, or better yet have someone in the pews who can press start/stop for you.

The placement, type, and quality of the microphones used is the most important factor. Omnidirectional mics will pick up unwanted sounds and echos. Mics with limited frequency response will not let the music blossom, although they can certainly supress some unwanted noises and limit bass distortion from the organ. In my experience, unidirectional microphones are great in suppressing unwanted noises, aiming them say at a choir, but they may not be suitable for recording the sounds of the liturgy in the whole church. In that case there should be several microphones placed stategically throughout the church fed into a mixer with a soundman controlling the levels.
That said, I have had acceptable results recording from the choirloft using even the humblest recorders such as the Olympus DS 30/40 but equipped with a directional stereo micophone aimed directly at the sanctuary such as the very inexpensive Sony ECM-DS70, and of course even much better results with the more expensive but bulkier ECM-MS957 on that same recorder.
A good self contained fairly inexpensive recorder with adjustible cardiod pickup pattern is the Tascam DR-40 4-track digital recorder, which additionally has 2 XLR/TRS inputs, and sells for about $150 these days.

I finally got my own Zoom H4N ... very very nice sound from the built-in microphones. Simple 4-track capabilities allow overdubbing, mixing, etc. The instruction manual remains picture-english, not entirely easy to follow, but the capabilities seem very good.

The simple attachment to a camera tripod was immediately useful.

If you're going to spend some good money on a recorder, I'd recommend the Zoom Q3HD, which has a built-in video camera. You don't have to use the camera to record, but I promise you that it's very, very useful if you want to use the device to analyse your own performances at some stage.