Help with automating book compiling
  • OMagnumMysterium
    Posts: 256
    Hello,

    I am wondering if any of you with more tech experience than I have could help me out.

    In my typesetting of chant books, one of my biggest goals is to put together a book which is the entire 1961 Graduale with the addition of the other missal parts (in Latin), like in the Liber Usualis. So for each Mass, the chant for the Introit, text for the Collect, text for the Epistle, chant for the Gradual, etc.

    However, since the project is such a massive undertaking (at least by my standards), I am trying to find if there is any shortcut. That is, I am looking for a way to get a lot of the gabc files and/or Mass texts in order and in the document automatically, so that I mostly just have to focus on formatting and editing. I am planning on doing this in LaTeX with the scores from Gregobase. Even if I could just get all the scores from the 1961 GR in one folder with all their file names in "\gregorioscore" in order, that would save me days (more like months at the pace I go) of work with just that.

    I have been using Divinum Officium for the texts, and have just been copying and pasting them one by one, but if there was some way to automatically retrieve the right ones and put them in the correct order, that would also save a tremendous amount of time.

    Basically, anything to make this go quicker would be wonderful, and I figure if anyone would know, it would be someone on this forum.

    Thanks.
  • Cantus
    Posts: 13
    Look at the Benjamin Bloomfield's Mass Propers Tool (https://bbloomf.github.io/jgabc/propers.html) — it is a great tool! You can get some ideas there: in the source code, you can find an array with references to the scores on GregoBase (e.g., "Adv1": {"inID": 132, "ofID": 962, ...} means that Introitus for the First Sunday of Advent is available on Gregobase via the URL with ID 132 https://gregobase.selapa.net/chant.php?id=132). With the help of some scripting, the downloading of these files and the preparation of the LaTeX code could be automated.
    Thanked by 2Chrism OMagnumMysterium
  • Chrism
    Posts: 871
    For Divinum Officium you could generate the whole office, either on your local or on the divinumofficium.com website, and then scrape the HTML output using some algorithm of your own design. For example, search for the word "Collect"...

    There would be special cases where there are multiple collects and readings, for example Ember Days and Holy Week...

    It might take you a while to automate but if you develop a good process you can tweak it to your liking and rerun it multiple times until you get everything right.
    Thanked by 1OMagnumMysterium
  • igneusigneus
    Posts: 386
    Even if I could just get all the scores from the 1961 GR in one folder with all their file names in "\gregorioscore" in order, that would save me days (more like months at the pace I go) of work with just that.


    Depending on how in/complete the data of the 1961 gradual on GregoBase currently are, this listing https://gregobase.selapa.net/source.php?id=2 may have all you need. Just write a simple web scraper which downloads gabc of each referenced item and produces the LaTeX document.

    You could also take the database dump here https://github.com/gregorio-project/GregoBase and work with it instead, but unless you are well versed in SQL and database design, web scraping is probably the easier way. Also the dump isn't refreshed too frequently and web scraping provides you the most up-to-date data.
    Thanked by 1OMagnumMysterium
  • OMagnumMysterium
    Posts: 256
    Thank you all for your ideas.

    I am familiar with Benjamin Bloomfield's Mass Propers Tool, and I actually asked him first, but have not heard back from him yet, and I don't want to bother him unnecessarily.

    Basically, all of the ideas sound wonderful, but I have no idea how I would do them. I am half decent at certain types of tech (gabc, LaTeX), but webscraping, scripting, source code, arrrays, and all the other things mentioned are super confusing to me. It looks like webscraping would involve python (which I have never used before either)?

    I suppose it is a question of how much of it is actually worth the trouble of automating, verses the trouble of doing it manually.

    I am familiar with the GregoBase page for the 1961 GR, and from my experience using it, it seems complete (if something was missing, it wouldn't be a big deal to add it in manually).

    If in your charity any of you would be willing to help me with setting up some of the code (or whatever it is), I would be most appreciative. Otherwise, I will try to do some reading up on it over the next couple of weeks.
    Thanked by 1Chrism
  • Chrism
    Posts: 871
    .
  • smt
    Posts: 62
    I think I could help. I have a similar project (and will surely need help in the near future too), but I already downloaded all of the gregobase files in one folder. Please contact me and describe what exactly you want.
    Thanked by 1OMagnumMysterium
  • joerg
    Posts: 137
    In LuaLaTeX there's incorporated a programming language Lua which can be used for tasks like this. Here's such a TeX file with an embedded Lua program. It parses the raw data of Benjamin Bloomfield's Mass Propers Tool and produces two new files GR61_prolog.tex and GR61_out.tex. The latter file can either be compiled as is, resulting in GR61_out.pdf or it can be edited before compiling. Compile time was more than 4 hours for 709 pages.
    Thanked by 2OMagnumMysterium tomjaw
  • OMagnumMysterium
    Posts: 256
    joerg,
    Thanks so much for the TeX file and explanation (which I slightly understand). That is incredible, and my mind has just been blown. I have been playing around with it the last hour, to try to see how it works. One thing I'm wondering about is if there is anyway to edit the GABC files manually, or if they all are just taken from Mr. Bloomfield's website. I was hoping to be able to adjust scores, add line breaks if necessary, etc.
  • joerg
    Posts: 137
    The file GR61_out.tex consists of thousands of lines of the form
    "\includeGB{132}{IN}", i.e. "Download the gabc file with id 132 directly from Gregobase and typeset it as Introitus". You can of course replace each of these lines with a standard call "\gregorioscore{MyEditedFile}" or You can add such lines whereever You want.
    Is that what You meant?
    Thanked by 1OMagnumMysterium
  • OMagnumMysterium
    Posts: 256
    Yes, I guess so. It isn't the way I had imagined it would work, I had imagined the code/script/whatever it is would download all the gabc scores into a folder instead of just taking them directly from the website, but I guess the way you set it up is even easier. Thank you.
Add a Comment