Python startup time: milliseconds matter

Re:Python startup time: milliseconds matter by eesmith on Wednesday May 02, 2018 @03:54PM (#16980590)

I worked on one Python application that had a startup time problem because it was on a network filesystem with slow metadata/stat times. It took several seconds to start Python.

We were able to solve most of the problem by zipping up the Python standard library and the our application.

That is, if you look at sys.path you'll see something like:

  >>> sys.path
  ['', '/usr/local/lib/python36.zip', '/usr/local/lib/python3.6', ...]

If you zip up the python3.6 directory into python36.zip then it will use that zip file as the source of the standard library, and use the zip directory structure instead of a bunch of stat calls to find the data.

This should also include getting access to the pre-compiled byte code.

You can also have Python byte-compile all of the .py files in a directory as part of your build/zip process.

  python -m compileall --help

[ Parent | on HN ]

Re:Python startup time: milliseconds matter by tomc1985 on Wednesday May 02, 2018 @05:42PM (#16981402)

Don't forget

  find . -type f -name "*.py" -delete

right afterwords.

Also note calls to imp.load_source need to change to imp.load_compiled, and any .py files references directly in code need to be changed to .pyc (this is with 2.7, not sure about 3.x)

[ Parent | on HN ]

Re:Python startup time: milliseconds matter by std_throwaway on Wednesday May 02, 2018 @06:10PM (#16981597)

Thanks, I will try that!

[ Parent | on HN ]

Re:Python startup time: milliseconds matter by a-dub on Wednesday May 02, 2018 @05:23PM (#16981267)

cool!

[ Parent | on HN ]