In my experience not the optimization is the hard part. But the eventually necessary scaling down you have to do first. Doing this with Imagemagick, Pillow or whatever will result in possible OOMs or gigabyte sized, not so temporary files filling your /tmp dir for large source images.

The only tool I ever found which does this job reliable even for huge images is http://www.vips.ecs.soton.ac.uk .