Not putting headphones down or just scrolling during conversation is considered rude around me. It is ok to scroll to find info about something you discuss, it is ok to scroll when not conversing during longer trip etc ... but it is not ok to just take phone out mid conversation and scroll a bit.