News for hackers. Stuff that matters.
Re:Guide to Python Debugging by abiogenesis on Monday June 01, 2020 @11:48PM (#23387218)
I second PyCharm, not only from a debugging perspective but as a whole. I don't understand people struggling with generic text editors, development oriented text tools (sublime etc), or even vscode just because they are free when there is a much better tool money can buy (and PyCharm has a free, community version as well).
Re:Guide to Python Debugging by raziel2p on Tuesday June 02, 2020 @05:40AM (#23389312)
Anecdotal, but I have never seen any of my colleagues with PyCharm work more quickly than me in Sublime Text. I think if you "struggle" in an IDE or editor, it's not really the IDE or editor that's the problem. A "jump to definiton" key and plaintext search tools (I use grep or ag in the command line) tend to be enough for me 99% of the time.

I also think there's something to it being forced to think about your code rather than just relying on your IDE. In the past I saw random functions getting extracted to random modules and imported everywhere because PyCharm makes it so easy to do so, with no real thought as to where it belongs, which led to some weird circular import error happening pretty much every month.

Re:Guide to Python Debugging by soVeryTired on Tuesday June 02, 2020 @06:08AM (#23389438)
I work with data scientists and ML people more than hardcore developers, but I've noticed that a lot of people who use sublime text tend to resort to print() debugging.

IMO python's lack of explicit typing makes it difficult to reason about by inspection alone ("does foo() return a dataframe or a numpy array?!"). For me at least, I need to get into the guts of a system and watch it execute to really understand it. The print() debug crowd tend to be much better than me at reasoning about code by inspection alone, but when you work on something complex that you didn't write yourself, that only gets you so far.

Re:Guide to Python Debugging by naniwaduni on Tuesday June 02, 2020 @07:10AM (#23389757)
Part of it is that the print() debug crowd tends to prefer to externalize more of the reasoning into the code itself (rather than into their tooling, or into their brains—though sometimes it is the latter, and that can be a disaster coming back a few months later).

That doesn't work great on other people's code, obviously.

Re:Guide to Python Debugging by themoops36 on Tuesday June 02, 2020 @08:25AM (#23390218)
Can you recommend anything to learn debugging the proper way (or maybe the article here is a good resource)?

I might be one of these print() people- usually when I hit bugs I read through the stack trace and can figure it out, but if it's more of an "unexpected result" I resort to print() so I know exactly what I'm doing. Would love to learn a more efficient way

Re:Guide to Python Debugging by JackC on Tuesday June 02, 2020 @08:43AM (#23390363)
For me the next step after print debugging was adding an `import pdb; pdb.set_trace()` shortcut to my editor (I use "pdb<tab>" as the shortcut, but whatever works.) Do that whenever you would add a print statement and re-run to resolve a confusing bug. It drops you into a REPL at that point in the code, and you can then print values and try running things. It basically just collapses a series of "add print statement and run" into a much tighter loop, which can save a ton of time depending how long it takes your program to throw the error and how many print statements it was going to take you to figure out what was wrong.

Key commands I use while in pdb: ? for help; n to run the next line; s to step into the next function call; w to see where I am in the stack; u and d to go up and down the stack; unt to run until a new line is reached (if you're stuck in a loop); interact to go to a normal python shell (if the pdb shell is confused by what you typed).

This won't replace print statements 100% of the time, because it might be simpler to print a bunch of logs to see where the problem is than to step through the program in the debugger; use it when you know where the problem probably is but not what it is.

Re:Guide to Python Debugging by alexhutcheson on Tuesday June 02, 2020 @10:19AM (#23391281)
Andreas Zeller’s Software Debugging course on Udacity is excellent and also free: https://www.udacity.com/course/software-debugging--cs259
Re:Guide to Python Debugging by soVeryTired on Tuesday June 02, 2020 @03:16PM (#23394988)
This seems to be a decent intro. It's focused on pycharm but the same functionality is available in VS Code and most other IDEs.

https://www.youtube.com/watch?v=QJtWxm12Eo0

Re:Guide to Python Debugging by nicoburns on Tuesday June 02, 2020 @09:59AM (#23391044)
It depends a lot on whoch platform you're working on. I do a lot of frotend stuff, which means I have an excellent debugger available in the browser. Having one in my editor doens't buy me much.

I do tend to reach for a simple log first though. I find it's ofte quicker to place a bunch of log statments than step through every statment. The debugger comes out for more complex issues.

Re:Guide to Python Debugging by dangoljames on Tuesday June 02, 2020 @08:35AM (#23390296)
Don't underestimate the usefulness of basic instrumentation with print statements; it's just a simple (simplest?) form of logging.
Re:Guide to Python Debugging by _coveredInBees on Tuesday June 02, 2020 @12:03AM (#23387305)
My main reason for wishing to be able to switch to Vscode instead of Pycharm is because vscode's remote development UX is really really nice and works a lot better for me compared to the Pycharm way of SFTP-ing files between your local and the remote.

My use-case where Pycharm feels a bit deficient is that I have an Ubuntu deeplearning box at work that I can VNC into and develop on whenever needed. I have all my repos cloned locally and work on various projects locally (and Pycharm is my IDE of choice on the Ubuntu box). But when I work from home, VNC feels a bit too sluggish and remote development would be much better for me. The vscode experience is pretty fantastic. I can setup SSH access to my box and have an open project for the same repos that I have cloned locally on my Ubuntu box and it's as if I am working on it natively. Everything is super snappy and even my bash sessions have the correct remote venvs activated correctly.

In contrast, every time I tried mimicking this in Pycharm (the paid version), it seemed like there was no way to link my already cloned repos on the Ubuntu box to my local version on my laptop so that work I did locally would always also be "live" on my Ubuntu box so that were I to VNC into my Ubuntu box and open the same project in my local Pycharm IDE, it would exactly mirror the state of my Pycharm IDE on my laptop. Is what I want possible? Everything I tried seem to result in my laptop Pycharm setting up a new project somewhere on my Ubuntu box where it would SFTP my project files to, but that resulted in a duplicate of my repo in some other location, which is not what I want. Hopefully this made sense?

Re:Guide to Python Debugging by sooheon on Tuesday June 02, 2020 @12:38AM (#23387499)
> vscode's remote development UX is really really nice

I'm very interested in this, so looked it up.

For your use case, I think it's just a matter of setting the correct Deployment Mappings. There's a setting when you're creating the remote interpreter to specify your own mapped directory (not the default /tmp/pycharm-foo) and a checkbox to not have it automatically upload what you have on your local machine (since in your case the source of truth is the remote). Then you can manually sync the diffs yourself with pycharm or git.

Your workflow's interesting in that the remote machine is the main and your laptop's the satellite, and pycharm's model has it the other way around.

Also, if you've already got the interpreter set up, you can just change the deployment mapping settings.

Re:Guide to Python Debugging by hermitdev on Tuesday June 02, 2020 @12:54PM (#23393160)
My one current gripe about PyCharm is its seemingly lack of support for git submodules. As in I don't see any way to do a pull or push or work with branches on a submodule. I resort to doing all of my git work on the command line (which I don't mind - it's how I learned to work with git from the beginning).
Re:Guide to Python Debugging by _coveredInBees on Tuesday June 02, 2020 @01:12AM (#23387675)
Thanks for taking the time to look into this. I'll give this a shot again when I have a bit of time to tinker with this.
Re:Guide to Python Debugging by ngcc_hk on Tuesday June 02, 2020 @01:52AM (#23387933)
Thanks as well. Help people like me who walks by.
Re:Guide to Python Debugging by albertzeyer on Tuesday June 02, 2020 @03:33AM (#23388592)
Btw, this is exactly this feature request: https://youtrack.jetbrains.com/issue/IDEA-226455

Current statement:

> We're very aware of how important this feature is, and we have several initiatives in place to provide a great experience for fully remote development. Unfortunately, at this time we're unable to share any specifics or timeframes, as getting the best possible experience requires massive-scale architectural changes to our codebase, but we've been working on this for quite some time already and we believe that we will have a solution that will go above and beyond what VS Code delivers today.

Re:Guide to Python Debugging by thoraway1010 on Tuesday June 02, 2020 @12:49PM (#23393096)
Glad to know I'm not the only one. I still have the issue but I do still use pycharm (I do docker local and then just go from there - but my preferred was always basically "remote" develop - I used to use that so if I was on my laptop or desktop etc I was always on the same (linux) machine even if coming in from mac or windows.
Re:Guide to Python Debugging by _coveredInBees on Tuesday June 02, 2020 @12:56PM (#23393187)
Thanks for that link. Glad to see that it is prioritized on their roadmap, and also to know that there is no point in my trying to work around their current implementation because it seems it just won't work the way vscode can work seamlessly.
Re:Guide to Python Debugging by darkteflon on Tuesday June 02, 2020 @08:42AM (#23390356)
I have the same use case as you and went through exactly this. I ended up feeling that Pycharm’s implementation wasn’t really a suitable solution. Some others had mentioned the alternative of mounting the remote filesystem locally using SSHFS but I found that this created issues of its own. In the end I settled on using Syncthing to sync all of my folders and then just run a local instance of Pycharm (and VS Code) at either end (Ubuntu 20 desktop and Macbook Pro). I’m very happy with this setup so far and would happily recommend it.
Re:Guide to Python Debugging by heavenlyblue on Tuesday June 02, 2020 @04:51AM (#23389050)
You could just use sshfs.
Re:Guide to Python Debugging by chillee on Tuesday June 02, 2020 @05:53AM (#23389380)
Not the same. sshfs is acceptable if you only want to do basic file editing, but is unusable if you want to say use language services or search in your directory.

What you want is for basic text editing to run locally (to minimize latency) and heavy duty services (like language services or search) to run on the remote server where accessing a file only requires a file system call instead of a network call. Then, after gathering the required information it can send that over some kind of pipe to your text editor.

This is the workflow that VSCode has set up and it's very good. In many ways, even if you're used to vim, VSCode's setup is preferable to vim over ssh shell due to lower latency.

Re:Guide to Python Debugging by heavenlyblue on Tuesday June 02, 2020 @07:48AM (#23390004)
I would be happier if python Language Server would actually be better than the tool chain supplied by PyCharm, however they haven’t yet reached that level at all.

Yeah, it’s an efficient setup that doesn’t actually work because it’s lacking in features.

Re:Guide to Python Debugging by greenshackle2 on Tuesday June 02, 2020 @12:18AM (#23387392)
I used the Jetbrains suite for a year or two but I switched back to sublime last year. I have a CLI-centric workflow and I switch between 5-6 different languages, sublime is fast and I can edit whatever languages I want in the same editor. If I want a particular IDE feature there's a good chance I can Ctrl-Shift-P -> Install it with Package Control. I use pudb to debug Python, it works just fine, it lets you drop into an IPython shell too.

But I admit, I work on a weird product stitched from different languages, there's just no IDE that handles that particularly well as far as I know. If I was back to working on Python only I might switch back to PyCharm.

Re:Guide to Python Debugging by pjmlp on Tuesday June 02, 2020 @06:51AM (#23389644)
VSCode and Visual Studio have first class support for Python.

https://visualstudio.microsoft.com/vs/features/python/

https://code.visualstudio.com/docs/python/python-tutorial

Both include quite capable debuggers, and AI powered code completion, intelicode.

https://marketplace.visualstudio.com/items?itemName=VisualSt...

Re:Guide to Python Debugging by zo1 on Tuesday June 02, 2020 @09:05AM (#23390542)
I've tried it, and watched many a colleague "struggle" with it. "First class" is definitely an over-statement from my perspective, even if it might be great compared to what you'd get with plain-text editors. It's type-hinting half the time behaves oddly even with type-hinting present.

But it is getting better, that's for sure. Looking forward to see upcoming iterations.

Re:Guide to Python Debugging by pjmlp on Tuesday June 02, 2020 @10:29AM (#23391376)
My "first class" remark is because it comes from Microsoft itself and isn't a third party plugin.
Re:Guide to Python Debugging by kingosticks on Tuesday June 02, 2020 @11:05AM (#23391778)
Would "first-party" be more accurate? First-class implies it's the best. Which it is not.
Re:Guide to Python Debugging by pjmlp on Wednesday June 03, 2020 @01:31AM (#23400144)
I guess so, not native speaker, thanks.
Re:Guide to Python Debugging by zwieback on Tuesday June 02, 2020 @10:09AM (#23391155)
My goto for Python debugging is always Visual Studio since I'm coming from a Windows C# and C++ background. I find the Python debugger super easy to use if you're already familiar with that environment.

Early on I used Eclipse plugin for similar reasons - unified debugging across multiple platforms.

Re:Guide to Python Debugging by martopix on Tuesday June 02, 2020 @06:21AM (#23389491)
I used PyCharm for a while, and got frustrated at all the tons of things that it tries to do "smartly". There is a shitton of buttons everywhere, and I ended up using breakpoint() anyway because I couldn't figure out how the debugger works properly -- and it doesn't always work (e.g. it didn't break when I was using GPU, for some reason). I'm sure it's excellent, but I just can't be bothered learning all of that. Same for git/github GUI, I prefer atom's, it's so much easier to use.
Re:Guide to Python Debugging by thomk on Tuesday June 02, 2020 @12:01AM (#23387294)
I'm a vscode user and I do like it. I only really use it for the file navigation and color coding though. Autocomplete drives me nuts, so I turned that off completely. I am a pdb fan so i drop in to that when needed. I never really considered PyCharm to be honest, it feels like overkill for me.

However your allegiance sounds interesting. Can you give some reasons why you like it so much?

Re:Guide to Python Debugging by _coveredInBees on Tuesday June 02, 2020 @12:18AM (#23387393)
Not the person you replied to, but I love vscode from when I did a bunch of React + JS stuff a few years back. I also do a ton of software development and machine learning work in Python now and have never managed to move away from Pycharm even though I'd love to be able to have a good enough experience in Vscode so that I could make the switch from Pycharm (the remote development experience in vscode is really neat which is what attracts me).

In no particular order, here are all the things that I love about Pycharm and that Vscode doesn't really compete on at the moment:

1. Integration of IPython shell (if you install IPython in your project venv) with Pycharm is absolutely amazing. Copy-pasting multi-line code just works. It handles all the auto-indent stuff. You have keyboard shortcuts to directly send highlighted sections of code in the editor to the shell. You can even add a plugin (Python code cell) to make the editor work like Spyder/Matlab's cell mode

2. Excellent history browser of all commands entered in the IPython shell. This is extraordinarily handy as I end up doing all my code prototyping directly in the IPython REPL, fix all the edge cases and issues interactively, and then just grab the things that work and graduate them to a script/function/class. This reduces the overall time to code things up and I end up with far less bugs.

3. The code completion/intelligence is unbelievably good. Vscode's intellisense is really really bad in comparison when it comes to Python. With Vscode, the tab completion almost always feels nonsensical with the ordering of things making no sense, and a lot of times just plain wrong things being suggested in tab completion. By contrast, Pycharm's code completion is really smart, and extremely helpful, especially when filling in method signatures, or trying to tab-complete instance attributes/methods. It is smart enough to even parse type-hinting from docstrings and provide code completion accordingly, which vscode fails at.

4. Refactoring is a lot easier. Code inspection tools are pretty powerful

5. Related to the fantastic REPL experience in Pycharm, the debugging experience is hands-down the best Python debugging experience I have found. Also, the integrated variable viewer/explorer is really really nice and makes debugging that much easier/faster

6. PEP8 linting + auto style refactoring is really good in Pycharm. I've found vscode lacking considerably in that department. Pycharm ends up catching a lot of silly mistakes that vscode doesn't (example: forgetting to have your first param be `self` for an instance method).

7. Managing your project venvs and dependencies is also much easier and more robust in Pycharm. Pycharm will alert you the moment any file utilizes imports not installed in your venv and will also highlight missing dependencies in your requirements.txt / setup.py

I'm sure I'll think of more, but these are some of the top things that come to mind. It is a really fantastic product and what's crazy is that all of the above you get with the free community edition! I actually want to purchase the pro version but haven't found any need for it. I was interested in the remote-code development capabilities but unfortunately have been pretty underwhelmed by that aspect of Pycharm. Vscode seems to have a much better UX on that front imho.

Re:Guide to Python Debugging by diarrhea on Tuesday June 02, 2020 @04:21AM (#23388880)
I have used both softwares but stick with VSCode for Python, simply because it is very powerful to have one great editor for all projects, as opposed to one specialized IDE for each.

PyCharm is (AFAIK) monolithic in what it brings to the table, whereas VSCode is more modular. As such, some of your points are true, but not an inherit fault of VSCode, but probably of some settings/underlying tools used. Maybe these can get you a better experience:

3. Agreed with VSCode being lackluster, but switching to MS Language Server [0] has helped this a great deal (while having other downsides, like being slower).

5. VSCode also has a variable viewer/browser and an interactive shell (which cannot be IPython as of today, however [1])

6. Disagree: I forgot what VSCode uses per default, but I switched to `black` for formatting, which works very well (` "python.formatting.provider": "black"` in settings.json). For linting, I switched to `mypy` (which also handles type hints, which is helpful), which can be set via `Python: Select Linter` (CTRL+SHIFT+P). `mypy` seems to be powerful. It definitely catches method 1st arguments not being `self`.

0: https://github.com/microsoft/python-language-server

1: https://stackoverflow.com/questions/47701629/vs-code-run-ipy...

Re:Guide to Python Debugging by _coveredInBees on Tuesday June 02, 2020 @11:57AM (#23392462)
5. They have a variable viewer, but as far as I can tell, it only works when in debugging mode? Not if you are prototyping code in a REPL, which is my workflow when developing code 99% of the time. It is also a bit limited compared to Pycharm. Vscode also does not handle multi-line interactive code nearly as well when working in the REPL in the debugger.

6. Yup, it supports linting, but my point was that Pycharm has better overall support for linting/code-intelligence in that it is better at auto-fixing things or having shortcuts to do so, without having to resort to something as extreme as `black`. It handles multi-line strings, or just general formatting of multi-line stuff perfectly. If I hit enter after `long_var_name = `, it will automatically put in a back-slash so you can continue writing a multi-line expression. I edit in both vscode and pycharm and I usually will still find useful things that the Pycharm linter catches or code-formatting things that Vscode does not (and I have tried a dew different linters though perhaps I should give mypy a shot).

I raise all these as a big proponent of vscode. Nothing would make me happier than to switch to it for all my Python development work as I enjoy the UX a lot more over Pycharm in the code writing/editing side of things. But Pycharm is just superior in so many ways that improves my productivity that I have not been able to make the swap despite several attempts.

Re:Guide to Python Debugging by heavenlyblue on Tuesday June 02, 2020 @04:52AM (#23389055)
You are forgetting that PyCharm is actually IDEA, which is an IDE for all languages out there.
Re:Guide to Python Debugging by markus_zhang on Tuesday June 02, 2020 @01:02AM (#23387630)
Agreed to every point.

Pycharm really shocked me by correctly suggesting most of library related stuffs as ling as I type hint them.

I really enjoyed pycharm and datagrip and wondering how does clion compare with vs.

Re:Guide to Python Debugging by dijit on Tuesday June 02, 2020 @03:53AM (#23388720)
I've been using CLion with the 'rust' plugin, which is quite nice.

But it's not pycharm; the debugger is very much lacking, often failing to resolve or step through.

But this is likely because Rust is not a primary target for the IDE.. I haven't tried any complex C++ in it yet.

Re:Guide to Python Debugging by markus_zhang on Tuesday June 02, 2020 @03:30PM (#23395222)
Thanks. Maybe I'll try it out. The only downside of Jetbrain tools is that they consume a lot of memory, but should be fine on my side.
Re:Guide to Python Debugging by msluyter on Tuesday June 02, 2020 @09:16AM (#23390640)
re: 3, for me another killer feature is syntactically aware selection (alt-up/alt-down). Whatever symbol you're on, alt-up will select increasingly large scopes: symbol -> expression(s) -> statement(s) -> method/function(s) -> class -> file. I feel pretty crippled without it.
Re:Guide to Python Debugging by elcomet on Tuesday June 02, 2020 @06:45AM (#23389599)
Pycharm doesn't work well for remote editing (ie when files are only stored on a distant server, not on your machine and synchronized regularely).

It is very slow over sshfs for instance, whereas other editors work very well (sublime, vscode..). Also, vs code has a very good remote development plugin.

Re:Guide to Python Debugging by dangoljames on Tuesday June 02, 2020 @08:33AM (#23390276)
I'd love to use pycharm, but my current project involves about 85% work on a raspi 4 and while it is quite impressive in many respects, it's just a little less than what pycharm requires in terms of resources.
Re:Guide to Python Debugging by mixmastamyk on Tuesday June 02, 2020 @12:43AM (#23387547)
I write Python with an enhanced text editor every day and don't struggle in the slightest. It's lightening quick as well. I pull out a proper debugger about twice a year.
Re:Guide to Python Debugging by jpitz on Tuesday June 02, 2020 @08:44AM (#23390367)
I do data engineering and lots of systems scripting and Dev ops in python. I write in emacs and find I rarely if ever need a debugger.
Re:Guide to Python Debugging by willbw on Tuesday June 02, 2020 @02:26AM (#23388182)
What are you working on that you never need to pull out a debugger?
Re:Guide to Python Debugging by mixmastamyk on Tuesday June 02, 2020 @10:51AM (#23391628)
I almost always understand my code, maybe I’m good at it, not sure. I use a lot of logging and untangle tech debt at first opportunity.

That said, it isn’t rocket surgery. ETL, web services, and FS/image tools lately.

Re:Guide to Python Debugging by Seirdy on Tuesday June 02, 2020 @02:06PM (#23394011)
PyCharm is an open-core editor. The FOSS edition is a second-class citizen. I make a point not to run any proprietary software on my machine besides occasional whitelisted JS, firmware, and patent-encumbered media codecs (which still have free implementations).

Neovim's master branch has a built-in Language Server Protocol client and Treesitter implementation, both of which have pre-made configs for Python development. Jumping between symbols, viewing floating documentation/signatures/git-blame, asynchronous completion, diagnostics, etc. are all readily available with a fraction of the battery/CPU/memory usage on a fully-FOSS platform that works on even more platforms (BSD, over SSH, etc). Jumping between symbols and listing usage of an object in a project with hundreds of files has never been faster. All this starts up in milliseconds. Being able to use an editor advanced enoguh for a large project but lightweight enough to launch with a keybind just to write the HN comment is amazing.

Here are some screenshots of the LSP and treesitter in action:

Floating documentation and diagnostics: https://i.imgur.com/0TYi5en.png

A small taste of Treesitter's highlighting + navigation features: https://i.imgur.com/wrSpJ2X.png

Completion from the nvim-lsp source: https://i.imgur.com/CzAAJuC.png

Neovim also has FZF integration to fuzzy-search thousands of symbols, files, reference, etc. Since it's the same FZF that I use everywhere else underneath, the interface is quite familiar.

TLDR: Neovim is a lightweight text editor that integrates with external tools to transform itself into an IDE.