Coming from JS, that was a pretty frustrating realization.
It does not though, unless you have altered the default sys.path to always contains `.`.
When running a file / script, Python will add that file's directory as the first lookup path, so these two invocations should have the exact same sys.path, and thus the same module resolution throughout[0].
If you have added `.` to sys.path (which is not the default), then the first invocation will also have the "grand-parent" folder on the sys.path, while the second won't.
This also doesn't seem to have anything to do with relative imports, you need to already be inside a package for relative imports to do anything.
[0] that is not the case of `-c` and `-m`, both add CWD to the path, and they differ in what they do: `-c` adds `''` to sys.path, which is the in-process CWD. `-m` stores the actual value of CWD at invocation, so changes to CWD while the program runs don't influence module resolution
Assuming `foo.bar` is a script inside the package `foo` that you want to both import and run directly (without `python -m ...`), this lets you do so without too much hassle.
This.
IMO that's also one of the main issues of Bash – you can't modularize a script unless you make sure your working directory is the directory containing the script. (And good luck with finding out the latter! [0])
[0]: https://stackoverflow.com/questions/4774054/reliable-way-for...
I want to organize my code logically in directories. As a script grows, I want the ability to spin out parts of that file to separate files.
In order to do that in python, I need separate directories between the script and the spun-out functionality. This ends with a script that says "do function from module" and all code being in the module.
Having code in different directories for no reason except "the import system" sucks. How is this supposed to go?
I used to include a simple stub Python program (~10 lines) as a "script" in my setup.py that would import the right code and call it.
Then I learned that "entry_points" implemented most of the dispatch behavior I wanted.
I no longer have those scripts, just an entry that basically says "from abc.cli.prog1 import main; main()". The prog1.py, prog2.py, etc. look like normal command-line scripts, assuming the usual:
if __name__ == "__main__":
main()
The major downside is I can't suppress SIGPIPE and SIGINT until main() is called, leaving a wider window where something like ^C gives a unwanted Python stack trace.Because unless I'm missing something, the changes to the import system make this more reliable, not less: in Python 2, implicitly relative imports means `import x` might depend on the file's location. In Python 3, it does not, it depends solely on the setup of the sys.path.
https://stackoverflow.com/questions/14132789/relative-import...
from . import foo
from .foo import bar
Implicit relative imports don't have such an indicator: from foo import bar
In py2, that second one could be a relative import or from anywhere in sys.path, while in py3 those implicit relative imports were removed so that means it'll only only look in sys.path and not the local directory.https://stackoverflow.com/questions/48716943/what-is-python-...
Exactly that: in Python 3, if an import doesn’t start with a . it will always be absolute aka looked up from sys.path.
In python 2, it would try performing a relative import (so importing a sibling module) first.
There's also ~/.local/lib/python3/site-packages (or whatever your distribution made of that). Virtualenvs are only necessary if you want to isolate dependencies between environments. That's useful if you have projects with conflicting dependencies, because Python doesn't allow you to install multiple versions of the same package, for better or worse. However, if you've written some simple scripts that don't care much about the exact version of their dependencies, it's perfectly fine to install their dependencies glboally.
https://nixos.org/manual/nix/stable/#use-as-a-interpreter
I wrote a Python script yesterday which calls out to a couple of external commands (`mlr` and `xq`), with a shebang like this:
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p python3 -p miller -p yqSay you have a main package "mylib" with a subpackage "mylib.utils". Typically I like to see "mylib.utils" imports as being in one of (roughly) 4 categories:
- standard imports, that I would put first in the file (e.g. "import logging")
- external imports, that I would put second in the file (e.g. "import requests")
- library local imports, bits and pieces from the "mylib" package that you want to reuse in "mylib.utils", but are external to the current package (e.g. "from mylib.email import client")
- and package local imports, which I see as implementation details of the current subpackage, and should be agnostic from the overall architecture of "mylib" (e.g. "from .helpers import help_function")
The last category are modules that only make sense from within "mylib.utils", should relocate with it even if it is renamed or moved elsewhere, and shouldn't require a change whatever the structure of "mylib" becomes, which is why I would use "mylib.utils" relative imports in there.
This is not supported in python. For reasons beyond my understanding, you are supposed to put the script with python, or with the shebang, in a different directory.
Alternatively, you can always use `python -m` to run your code.
Come again? This is what "import" does, obviously. This is not a clear explanation of your problem.
I say script here because the apparent intention is that scripts and modules are separate. That is why it's not easy to import functions from a file in the same directory as a script.
This restriction is very unexpected. And it is not borne out well by the fact that it is not obvious this distinction exists.
When you run a module, the root package containing it is on the path by definition and imports work.
Does that still work if you call the script from a different directory (so `python bar/foo.py`)?
Imma guess it doesn't work with symlinked scripts. But I don't think it should without doing an install, in which case a module does make sense.
I don't think Python will resolve symlinks here, so if that needs to work, you probably need to do it yourself before touching other code / files.
If the package uses absolute paths then I would need to tweak all the imports for the new absolute path.
Also the way that JS imports are just relative paths is very nice because it means that the imports are statically determinable, your editor can understand them and fix them automatically and you can trust that refactoring. Python has turing complete imports because there's so much dynamic messing about with sys.path that goes on in Python due to inadequacies of the import system.
Yes, just like absolute imports...
> relative paths is very nice because it means that the imports are statically determinable, your editor can understand them and fix them automatically and you can trust that refactoring
1) Just like absolute imports?
2) I absolutely contest that relative imports are easier for an IDE to refactor. I've never had VSCode hang while refactoring a Java package; I've had VSCode hang while refactoring "create-react-app" app...
import a.b.c
sys.path.insert(0, 'some/other/path')
import x.y.z
Or env PYTHONPATH="foo/bar:$PYTHONPATH" python ...
There is no way in general for Python tooling to figure out where an import points. All it can do is guess. It's a very regrettable situation.Yes in Java it's fine because Java has a build stage. So tooling can figure out where imports point to from your static configuration.
However from the tool's perspective there is nothing better than a relative path. Relative paths require no messing about with configuration files at all to resolve. It's just a path to another file or directory on disk.
When a tool sees
import c from "./a/b/c"
It can resolve it immediately.So what is the advantage of absolute imports really? If import lines are mainly written and maintained by tooling then shouldn't we pick the representation that is easiest for the tooling? Then we can have more and better tools and the tools will be more reliable.
And it turns out that, relative paths are easy for humans to understand too. The same configuration-free resolution algorithm also works in your head when you are reading code! At least when the language doesn't overcomplicate them too much (JS is guilty of this to a certain extent, although nowhere as bad as Python)
Well whatever the language or runtime, you need to tell it how to find its dependencies. Be it with a configuration file, environment variables or parameters.
Your statement is not exactly correct either. You don't need a fake installation _and_ bend PYTHONPATH.
Virtualenv leverages the fact that "../lib/python<version>/site-packages/" relative to the interpreter is a default import path (default value of PYTHONHOME). It doesn't use PYTHONPATH.
> Why couldn't the import system resolve versions, too?
Not sure that would be really great. I prefer to have my dependencies all grouped together in a setup.py rather than scattered in various files at import time.
If I understand what you mean by package management in this context, I wonder if editable installs will help you.
https://www.python.org/dev/peps/pep-0660/#abstract
https://discuss.python.org/t/pronouncement-on-peps-660-and-6...
What do you do about conflicts? Or say you have `import foo >= 1.0` in a file, and `import foo == 2.4`, but the latest version is 2.5, so the first import grabbed the latest version, and you later realize you need 2.4?
Imagine running a report generator for 5 hours, only to have the formatting module require a conflicting version of something and erroring out at run time...
I think the thing to remember about using relative imports is that it requires modules. Using relative imports to import something into a script will fail because the scripts don't belong to modules.
> Number two is, in order to do package management, you have to create a fake python installation and bend PYTHONPATH. Virtualenvs are the canonical way to do it, but to me it feels like a hack - the core language seems to wants all packages installed in /usr. So now I have all these virtualenvs lying around and they are detached from the scripts.
This is a pain. You can abstract this away with with pyenv or virtualenvwrapper, though.
Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...).
Number two is, in order to do package management, you have to create a fake python installation and bend PYTHONPATH. Virtualenvs are the canonical way to do it, but to me it feels like a hack - the core language seems to wants all packages installed in /usr. So now I have all these virtualenvs lying around and they are detached from the scripts.
Why couldn't the import system resolve versions, too? You could say `import foo >= 1.0` and it would download it to some global cache, and the import the correct versions from the cache.