Couldn't you just ask "would you agree"? Mabye it's confusing because you are making statement like if you were going to ask "would you agree", but then ask a question expecting opposite answer (if you were going to agree, you have to say "no, I don't have a problem"). And I think there is no real difference between saying I agree and I don't have a problem with that.