If I had to pick a "best-of" reading list, I'd say the 9th-century "Musica Enchiriadis", Berno's "Prologus in Tonarum", and Boethius's "De Institutione Musica". These all assume a familiarity with Platonic music theory, so at least skim through Plato's "Timaeus" first. There will also be more references to astronomy than will make sense to somebody in the modern age, because music and astronomy were believed to be essentially the same thing. Also more recent research has found that actual musicians of the time seem to have completely ignored the theorists. Some things never change.
In the 18th Century Presbyterian Church, they only sang Biblical psalms set to simple (but distinct) tunes and without any musical accompaniment. This was known as exclusive psalmody.
The psalms weren't the prose-style you'll find in any print Bible though, they were their own translation called the Metrical Psalms. Its 1650 edition proved very long lasting.
Many of these psalms were set to common metre, meaning that if you found such a psalter and tried singing it to some well-known tune it will very likely 'fit' the lyrics. You could try Psalm 23 set to the tune of The British Grenadiers, or Amazing Grace, or even The Star-Spangled Banner: https://thewestminsterstandard.org/psalm-23/
This was actually how the psalms were practised by congregations in my home country. The psalm lyrics were considered too sacred for practice, so they set the same tunes to other (sometimes comical) lyrics and practised those during the week. Then on Sunday the minister would say something like 'we will now sing Psalm xx to the tune of lilies', and the congregation would know the tune even if they hadn't memorised the psalm lyrics.