Maybe they were part of the Internet but not the Web?

No.

AOL was its own independent proprietary network, closer to a very large BBS, with email, chat, and content standards. It bridged to the Internet, as many networks at the time did (and many others did not).

The World Wide Web is a specific set of protocols (mostly: HTML and HTTPS, most would now include CSS and JS a well as other stsndards), which can run across the Internet or other networks, e.g., a disconected LAN.

AOL's joining the Internet had the largest immediate impact on Usenet, another protocol facility which can oerate over the Internet, but also transacted over phone lines (via UUCP) and avian carriers --- 747s carrying tapes between the US and Australia.

John Waterman's The Matrix (1990) is an excellent overview of early networking systems. That edition (there were several revisions) predates even the Web.

Oh, I might be misremembering because I was very small but I could have sworn I used AOL as an ISP and used web browsers (maybe Netscape Navigator?) to browse the web on AOL (maybe version 5 or 6?) in addition to their own weird portal thing which in my very vague memory was a lot like what you are describing.

Is my memory completely false here? It's very possible I'm conflating things from earlier in the 90s with things later in the 90s.

I can't seem to find the book (?) that you're suggesting in a search.

Ah, I think I've finally found it - or at least I've found what appears to be that by a John _Quarterman_.

Nah, that was years before the WWW (as we used to call it). Those were walled garden networks (CompuServe was another one), which eventually offered connectivity to the Internet too (and not long thereafter either ceased to exist or converted to Internet service provider and/or web services).