I used to feel just like that. Then I learned that academic philosophy studies this phenomenon as "metaethics". There are arguments such as yours that would be considered "moral skepticism". Read up on those (or watch a course like https://youtu.be/g3f-Lfm8KNg); I think you'll find these arguments agreeable.