The button next to them (a grey box with a plus symbol
    on it) doesn't explain to me at all what it does

If so, maybe they should use a 'read later' button instead. Or put a sentence on top 'Click + to read a story later'. The more people bounce the stronger the inclination to try that and see if engagement increases. That's why all 17000 visitors would be valuable testers, no matter how they behave.