That is absolutely “doing sketchy shit with people’s data”.

So you need to tell people you are doing that, so they can consent.