It depends on your driver. I got conned in Athens too - the driver had a little button beside his gear stick that would add 1 Euro everytime he hit it. He pressed that button a lot, and the fare was around 40 Euros. I have him a 50, and he did a trick with his hand to produce a 5 Euro note that he said I have him, trying to get yet more money out of me.
Honestly, as a tourist I absolutely fucking hate taking taxis :-/
This seems very different from my experience in Athens two weeks ago. According to Google maps, getting from the city to the airport by car takes around 40 minutes by car and the alternative of taking the train costs 10 euros per person for the same amount of time. Paying around 40 euros seems reasonable for such a long trip if you have more than one person.