News for hackers. Stuff that matters.
Re:CLI tools hidden in the Python standard library by lizard on Thursday June 29, 2023 @02:05AM (#36516921)
Its notable, perhaps, that the

  if __name__ == "__main__":
block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py`

You can also use either of these throughout your code so that you can have

  python -m foo
  python -m foo.bar
  python -m foo.bar.baz
each doing different, but hopefully somewhat related, things.
Re:CLI tools hidden in the Python standard library by BiteCode_dev on Thursday June 29, 2023 @03:11AM (#36517360)
It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path.

This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path.

It is my favorite way of running my projects.

Unfortunalty it means you can't use "-m pdb", and that's a big loss.

Re:CLI tools hidden in the Python standard library by scruple on Thursday June 29, 2023 @03:29AM (#36517495)
I also develop and run projects this way. I really, really enjoy it. It's a very pleasant experience, on both the development side and execution side.

I'm relatively new to Python (used it for ~1 year in 2007/2008, again briefly in 2014 -- which is when I believe I picked this module trick up -- and then didn't touch it again until March of this year). It's made an impression on my team and we're all having a good time developing code this way. I do wonder, though, what other shortcomings might exist with this approach.

Re:CLI tools hidden in the Python standard library by sharikous on Thursday June 29, 2023 @03:42AM (#36517579)
You can use pdb

Just

python -m pdb -m module

Re:CLI tools hidden in the Python standard library by BiteCode_dev on Thursday June 29, 2023 @04:35AM (#36517895)
Damn, 15 years of python and I learn you can use -m twice. I've never even tried, didn't occur to me it would be supported.
Re:CLI tools hidden in the Python standard library by abhishekjha on Thursday June 29, 2023 @04:47AM (#36517977)
Like whats the general usage though?

python -m http.server is the most I have done.

Re:CLI tools hidden in the Python standard library by BiteCode_dev on Thursday June 29, 2023 @05:11AM (#36518123)
-m pdb is post mortem debugging, it drops you in the debugger when it encounters the first unhandled exception. This is much easier than trying to pin point where to put a breakpoint.
Re:CLI tools hidden in the Python standard library by kzrdude on Thursday June 29, 2023 @05:12AM (#36518135)
There's no magic, only layers. `python -m pdb <args>` runs pdb with the rest of the arguments. pdb handles the second `-m`.

If you have a fancy IDE feature, open a new python file, type "import pdb", use go to definition on pdb to jump to that file in the standard library, and read its main function - it handles -m explicitly :)

Re:CLI tools hidden in the Python standard library by sharikous on Thursday June 29, 2023 @06:53AM (#36518783)
Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm?

Multiline statements are not accepted, nor things like if/for

Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL

Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab are very used to a trial and error programming style in a console and bare python does not address this need

Re:CLI tools hidden in the Python standard library by BiteCode_dev on Thursday June 29, 2023 @07:30AM (#36519025)
There are not workaround, pdb is a rustic tool.

Since I still enjoy a cmdline debugger more than a graphical one, I use ipdb, which doesn't suffer from the multiline limitation.

However, scoping issues with lambda and comprehension are actually a Python problem, not a pdb problem.

Re:CLI tools hidden in the Python standard library by johnnymellor on Thursday June 29, 2023 @11:19AM (#36521574)
Not supporting multi-line statements is just because pdb doesn't bother to parse the statement to work out if it is an incomplete multi-line statement. That could be easily fixed (I have a prototype patch for that using `code.compile_command`).

The scope problems are more fundamental:

The pdb REPL calls[1] the exec builtin as `exec(code, frame.f_globals, frame.f_locals)`, which https://docs.python.org/3/library/functions.html#exec documents as:

"If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition."

And https://docs.python.org/3/reference/executionmodel.html#reso... documents that:

"The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods - this includes comprehensions and generator expressions since they are implemented using a function scope."

This is a fundamental limitation of `exec`. You can workaround it by only passing a single namespace dictionary to exec instead of passing separate globals and locals, which is what pdb's interact command does[2], but then it's ambiguous how to map changes to that dictionary back to the separate globals and locals dictionaries (pdb's interact command just discards any changes you make to the namespace). This too could be solved, but requires either brittle ast parsing or probably a PEP to add new functionality to exec. I'll file a bug against Python soon.

[1]: https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae...

[2]: https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae...

Re:CLI tools hidden in the Python standard library by johnnymellor on Thursday June 29, 2023 @02:47PM (#36524937)
Oh, and multi-line statements will be supported from Python 3.13 (https://github.com/python/cpython/issues/103124)
Re:CLI tools hidden in the Python standard library by mejutoco on Friday June 30, 2023 @02:53AM (#36531427)
After triggering pdb by having "breakpoint()" in tour python code and dropping in the debugger you can type "interactive" in the console to enter multiline strings.
Re:CLI tools hidden in the Python standard library by BiteCode_dev on Friday June 30, 2023 @04:22AM (#36532011)
TIL again. Very good thread!

It's "interact" though, not "ive".

You can exit it to carry on with the regular pdb.

Re:CLI tools hidden in the Python standard library by mejutoco on Sunday July 02, 2023 @02:22PM (#36564159)
Thanks. I make that mistake every time I use it:)
Re:CLI tools hidden in the Python standard library by misnome on Thursday June 29, 2023 @05:29AM (#36518247)
to be fair, it was only added in 3.7
Re:CLI tools hidden in the Python standard library by andreareina on Thursday June 29, 2023 @03:35AM (#36517531)
I've taken to adding a --pdb option to my scripts.
Re:CLI tools hidden in the Python standard library by maleldil on Thursday June 29, 2023 @07:24AM (#36518975)
What does it do? Just `import pdb; pdb.set_trace()`?

Or `breakpoint()` after 3.7 (better because the user can override pdb with ipdb or other with `PYTHONBREAKPOINT`)?

Re:CLI tools hidden in the Python standard library by andreareina on Thursday June 29, 2023 @12:02PM (#36522278)
Either breakpoint() or setting sys.excepthook to call the postmortem debugger.
Re:CLI tools hidden in the Python standard library by quietbritishjim on Thursday June 29, 2023 @06:54AM (#36518788)
> which means suddenly relative imports works

Not just relative imports but also (properly formed) absolute imports. For example, if you have a directory my_pkg/ with files mod1.py and mod2.py, then

    # In my_pkg/mod2.py
    import my_pkg.mod1
will work if you run `python -m my_pkg.mod2` but will fail if you run `python my_pkg/mod2.py`

However, the script syntax does work properly with absolute paths if if you set the enviornment variable `PYTHONPATH=.` (I don't know about relative paths - I don't use those). That would presumably allow pdb to work (but, shame on me, I've never tried it).

Re:CLI tools hidden in the Python standard library by jxramos on Thursday June 29, 2023 @06:43PM (#36527870)
I've always been curious how that mechanism works exactly what is it about that invocation technique that satisfies the relative imports? I think it changes the pythonpath somehow right in a way related to the module being ran, something like appending the basedir where the module is saved to the PYTHONPATH?