Weekly (although we're taking June 30 to the beginning of August off). We used to do Vespers monthly on the third Sunday in order to get people a taste of it, and to keep it alive when our church was hit by a tornado. I found that this start-and-stop routine frustrating, and as @Charles_Weaver points out, it doesn't make sense in Advent or in Lent (for those of us who do some form of preconciliar Vespers, it's even worse because the rest of the year is so stable: Advent has proper antiphons every week, and Lent a proper chapter for each Sunday, in addition to the hymn and then the parts which always change weekly). We always did Gaudete and Laetare Sundays no matter what Sunday it was.

I'm mindful that while it is great to have a substantial attendance, but it is also important that we make it predictable, regular and sustainable for the schola.

Our best attendance, and average, ish, is around twenty-five, and while I'd do it with only five, it's obviously nice to have good numbers, and the benefits for the schola translate to the congregation as well. We get almost no one on Easter, but conversely, we get upwards of 100 on Christmas eve (so long as it's not a Sunday).

I would strongly recommend 5 pm and no later than 6 even in summer, even if you live in a place where the sun sets around 10 in summer. Plus, 7 is too late and only possible because moderns have an obsession with fixed times and things that can be done by electricity. Ours are at 6 because my pastor wants it so and I only acquiesce because I get home at noon at the earliest, but I'd rather be there at 5 at the latest. (We do unfortunately have to share space in the future with an Eastern Catholic community, but frankly, we don't have room, and we are the only parish doing full Vespers every week, so I'm not sure why we are expected to compromise here — plenty of parishes don't even have evening Mass in such a small diocese)

I have not much of a clue about the new antiphonal and the Liturgia Horarum, but I offer this observation: evening Mass is less than ideal, but it will often be the case that you have the possibility or need to offer the evening Mass as the sung Mass on major feasts where singing Vespers would be ideal, in which case II Vespers isn't really possible (I mean, you can do the Vespers + Mass combination, but there's a sense in which it doesn't do justice to either Vespers or Mass). Also, we now think that II Vespers are more solemn, and I often prefer the contents of II Vespers, but historically I Vespers were more important, it's when the bishop or abbot pontificated in cope and was the actual celebrant, as he would sing Mass the following day…

Also, do the people not sing the antiphons? Ours give it a go, which I find really wonderful, and they're slow on the psalms, but it's still edifying in its way (although there are reasons why unfortunately the support from the schola isn't the best on those verses).

I also… don't understand the pointing. Two asterisks in one verse confuses me, and I have never been able to make heads or tails of systems with accent marks (versus the accents and preparatory syllables from the Liber Usualis). I would prefer no pointing to this (what I don't know is what inexperienced people outside of the schola want, but trad congregations don't seem to mind the bolded and italic syllables). And ten minutes of silence? I understand that the rubrics make a point about silence, but that does seem excessive and again, it leads to watch-checking in a way that is rather odd.

From my experience of vespers in English I would find the psalm tones in your example intolerably dull. There is probably good psychology (and certainly long experience) leading to the Latin chant having typically over a dozen notes per verse, also Anglican chant having ten chords per verse, Bevenot 16 notes per stanza, Mundelein ...

I would also suggest replacing the parts from the Liber with the Gregorio version, and I pretty much agree with the rest.

Thank you all for so many helpful comments and for sharing your experiences, and for having a look at our handout. I'm certainly getting the consensus view that focus on Advent, Easter and Lent and other solemnities is perhaps a better way of getting the parish to focus and participate.

You asked a lot of questions, and brought up very good points. I'm extremely wary of fiddling around with the format of a service like this and before we started I consulted with our liturgist at some length before arriving at this format. So if I change it again I want to do it once and stick with the new format and a new schedule and not fiddle around anymore!

As you (@matthewroth and @smvanroode) point out there are 4 points where our format diverges from the bare minimum that would be prescribed in the LoTH:
1) we used an entrance hymn which precedes the vespers. As you rightly point out we used Adoro te which is usually used for adoration and while the hymn itself is perfectly appropriate, I suppose the title is what gives everyone the idea it is only for adoration and nothing else. Nonetheless, another entrance/exit hymn I would certainly consider.
2) We added a 2nd reading snatched from the office of readings for the day. Everyone seems to love it and it seems to be a high point of the service for many. This is a group that are overall astonished to hear those readings - most of them know nothing apart from what they have encountered in NO mass and many have never even heard of vespers itself. So I think at this point, in this parish, it has a very important educational function right now. Our liturgist thought it was a great idea as well so I'm inclined to keep it there.
3) we sing a motet after the 2nd reading. This was to make it interesting to the schola. We love chant but we also wanted to stretch their polyphonic capacity as well and we know the parishoners love hearing it. Certainly never heard any complaints about this and again our liturgist was quite supportive.
3) We extend the slience for close to minutes. Almost the only complaints I have ever heard were when we cut this down to 3 or 4 minutes - people wondered why it wasn't the usual length. The regulars in particular really love it, I think it is the high point of the service for many - sitting in silence staring that the crucifix in a mostly dark church with a 12 candles on the alter and alongside the tabernacle.

There were a couple of questions about the psalm tones and antiphons. The tones are from Liber Usualis and pretty standard, and exactly the way the priests who do ER in our diocese do it so for the latin mass folks at least that is what they're used to so we always have some of them chanting along because they are familiar with the format, which is quite important. I know the bold/italic pointing is easier to read but the guy who does the typesetting likes accent marks better - I pick my battles. I don't find them hard to read, though when I'm cantoring I do mark up my script a lot (as with any music). Regarding the antiphons, a cantor chants them before and after the psalms but I hadn't printed them when I made the handout I shared. We recently changed to chanting the antiphons in Latin and printing the translation in the handout, since translating the antiphons never works, and the neumes fall in all the wrong places in English.

All that said, I take your points and the two things I think might make the most sense to change are the entrance hymn and the motet.

With the entrance hymn, it is optional just as it is in the Mass and not part of the prescribed liturgy, but giving the congregation something that is repeated a lot and that they can easily learn and join in, in Latin, is still I think a worthy thing to include. As I said above, I'm considering that we just process (and recess after the closing) chanting the marian antiphon (Alma Redeptoris Mater, Regina Caelorum, etc) that allows them to join in easily, and we could use a different one for the entrance and exit but it needs to be something that we will stick with.

With the motet, the obvious change, if we were to drop it from after the 2nd reading, would be to simply sing the prescribed hymn in a polyphonic setting if there is one - we've sometimes done settings where the odd verses are chanted and even verses are polyphonic (as we do in the Magnificat every time) which keeps the congregation engaged but gives the schola a chance to show off.

Back to the original question about the calendar - again thank you for sharing your experiences. I'm thinking we really need to get it done on Sundays around the year ultimately - I thought of 5pm in winter and 7pm in summer so it is around sunset. I even thought of starting at sunset every time which would certainly be more mediæval! Doing it on other week nights will only work where there are holidays, and doing it at 5pm during the week for example on 31 October at 5pm will never work since we all have jobs.

I'm thinking right now about proposing to our priest and liturgy committee that we do 3rd Sundays of the month until advent, then do every Sunday in Advent, Lent and Passiontide + solemnities to the extent we can manage them, and keep that up until Trinity of 2025 and see how we are going. At that point we would have done every sunday from the start of lent through Trinity and by that time I'm hoping we will have a momentum to carry it on as we permanent fixture.

Thanks all and please share any more thoughts or suggestions you have!

@MatthewRoth do you have an organ accompaniment with this? We do everything a capella, don't know if that changes much but in my experience it certainly changes how confident people are in singing along. We have one good organist who is in his nineties and we've decided to try to move the congregation toward getting used to a capella chant and a little polyphony going forward since that is what we can sustain.
Basically my experience has been that whatever is repeated every time, the congregation will sing once they are used to it (Magnificat, pater noster etc) but if they see it for the first time, they will never sing along. Sight singing chants are not something they have ever done, so I'm impressed that you give them a new antiphon every week and the sing along every time! Even using a familiar tone with new words they are not comfortable with. To be honest it is a very old and not a very musical congregation overall, but they do really appreciate the services a lot. This is why I'm tempted to keep bookending the service with something familiar and often repeated that they can join in. How do you usually start and end the service, do you process and recess in silence?
Another question, you say the Magnificat changes weekly? Do you mean you sing it with a different tone every week? Maybe I misunderstood your comment. Also, do you do it all in English?

We used to have sung Vespers, in English, on Solemnities, at 6.30 before the evening Mass which was at 7pm.
Recently the time of Mass was arbitrarily changed to 6pm. 5.30 for Vespers was too early for several of our singers to get there- some of them work at a distance , some students have late classes - so we changed to Compline after the Mass, starting around 8pm. It's actually easier for the choir because it's shorter and doesn't change much, and the congregation seem to like it very much. We give them leaflets, so that they can join in with their side, which they do. Often we start with a short appropriate motet or chant (eg Cor Jesu) to set the scene. Compline is particularly effective by candlelight in the dark winter evenings we get here.
We miss Vespers though.