Of the top of my head: start off with big up/down buttons. Then show a list of floors. Some people won't notice the second press required, but it's a start. I was confused by those things the first time too.