there are ways...

I believe that only the melody of a tune can be copyrighted. Chord progressions cannot. If you have permission to use the melody, you can put whatever you want underneath it.

The intention of the performance factors in as well. For-profit concertizing would be viewed more sternly.

As I hinted at earlier, I imagine re-harmonization at the organ in the context of a service would be viewed as permitted. Even if it isn't, it's such a custom I would imagine it would be difficult to argue with.

Am I correct that the problems begin when things start getting written down? And they become compounded when you start passing it out to people (i.e. your choir)?

This is not that complicated.

PERFORMANCE IN CHURCH is copyright-restriction free. You can sing or play anything you want, any way you want, as much as you want without asking anyone permission or paying for anything at all.

MAKING PHYSICAL COPIES OF MUSIC NOTATION is copyright protected. The same restrictions apply whether you photocopy them from a hymnal or scribble it in pencil on some manuscript paper.

So if you want to SING a re harmonization there is nothing legally stopping you from doing so.
If you want WRITE DOWN a reharmonization, the fact that you are writing down the under-protection melody (in, for example, the top staff of your score) is the thing that has to be done with permission.

It would probably be too difficult for the singers to manage having the legally-obtained original on the left hand side of their folder, while seeing ATB parts without a melody on another piece of paper on the right hand side. Much too difficult. I wouldn't try that.

My elementary school teachers put something to that effect on my IEP.

PERFORMANCE IN CHURCH is copyright-restriction free. You can sing or play anything you want, any way you want, as much as you want without asking anyone permission or paying for anything at all.

Very close; actually 17 U.S.C. § 110(3) provides an exception to copyright restrictions for use of "a nondramatic literary or musical work or of a dramatico-musical work of a religious nature, or display of a work, in the course of services at a place of worship or other religious assembly . . . ." What is not covered by the exemption, then, is (a) use of dramatic musical works of a non-religious nature, and (b) use of exempted works in church but outside the context of a religious service.

The second issue is further covered under subsection (4), which provides a limited exception for some concerts and other events, but only under the following conditions: "Assuming that there is a public performance of a nondramatic literary or musical work as defined by the statute, first, the use of the work must not be for a direct or indirect commercial advantage to the user. Second, there must not be any payment of any fee to performers, promoters or organizers. Third, those attending the performance of the work must not be subject to a direct or indirect admissions charge. Conversely, if there is an admissions charge, the proceeds from the performance must be used for educational, religious or charitable purposes, rather than for private financial gain. This, however, must be further qualified by the opportunity for the copyright owner to serve a notice of objection to the performance of his or her work." Julien H. Collins III, Note, When in Doubt, Do Without: Licensing Public Performances by Nonprofit Camping or Volunteer Service Organizations Under Federal Law, 75 Wash. U. L.Q. 1277 § IV(B) (1997).

Also, this:

When in Doubt, Do Without

is the perfect title for a chilling effects pamphlet campaign

I did a re-harmonization a couple of years ago to a very infamous mass setting. I was given permission to do both, compose a new harmony and print it for the choir. I have not been to confession yet on this one, and unfortunately, it still exists.

Well you obviously do not need to go to confession for the re-harmonization if you received permission. Now depending on how infamous the ordinary setting was, well, maybe that is why you feel you need to confess it.