News for hackers. Stuff that matters.
Re:What’s New in Python 3.8 by geofft on Monday October 14, 2019 @07:28PM (#21253983)
> The "walrus operator" will occasionally be useful, but I doubt I will find many effective uses for it.

The primary one I want is

    if m := re.match(...):
        print(m.group(1))
and

    while s := network_service.read():
        process(s)
both of which are both clearer and less error-prone than their non-walrus variants.

The other one that I would have found useful an hour ago is in interactive exploration with comprehensions. I frequently take a look at some data with [i for i in data if i[something].something...], and being able to quickly give a name to something in my conditional, as in {i: x for i in data if x := i[something]}, helps maintain the focus on data and not syntax. Obviously it will get rewritten to have clearer variable names, at the least, when it gets committed as real code, and almost certainly rewritten to be a normal non-comprehension for loop.

Like comprehensions, I expect the walrus operator to be valuable when used occasionally, and annoying if overused. There's no real language-level solution to bad taste. In Python 3 you can now do [print(i) for i in...], and I occasionally do at the REPL, but you shouldn't do it in real code and that's not an argument against the language supporting comprehensions.

Re:What’s New in Python 3.8 by drdaeman on Monday October 14, 2019 @08:11PM (#21254277)
Coming from Perl, I used to want this badly, but then I thought that there's absolutely nothing wrong with

    m = re.match(...)
    if m is not None:
        pass
Now, I wonder what you meant saying that single-line version is less error-prone, because I don't think so. I believe they're exactly the same in this regard, except for a bizarre case when someone would bastardize the code by putting some irrelevant lines between the assignment and the comparison, obfuscating the logic.

Also, as I tend to use `if m is not None:` rather than just `if m:` (because I typically don't want to run all that subtle `__nonzero__` magic), it would - subjectively - look less pretty in the one-line version: `if (m := re.match(...)) is not None:`.

Use in the generator expressions to create aliases/shortcuts and avoid repetition is great, though. I love this use case.

Re:What’s New in Python 3.8 by 3PS on Monday October 14, 2019 @08:31PM (#21254396)
The issue, in my opinion, is when you want something like this:

    while m := f(many, args):
        # do stuff with m
Now, if you're writing this in Python 3.7, you often end up with some code duplication:

    m = f(many, args)
    while m:
        # do stuff with m
        m = f(many, args) # duplicate
Or something like this:

    while True:
        m = f(many, args)
        if not m:
            break
        # do stuff with m
Personally, I consider the last version to be the most elegant, but still something of an anti-pattern. So I welcome this new operator.
Re:What’s New in Python 3.8 by hermitdev on Monday October 14, 2019 @09:41PM (#21254814)
I literally wrote your last example twice last week to hash files in 4k chunks. Those 4 lines would be reduced to 1 with the walrus operator. I welcome it, as well.

Edit: 1 line, not 2 (excluding the "do stuff")

Re:What’s New in Python 3.8 by cellularmitosis on Tuesday October 15, 2019 @12:07PM (#21260013)
This comes up in recursive descent implementations as well https://gist.github.com/cellularmitosis/53913ad229bad5d0a3bb...
Re:What’s New in Python 3.8 by ygrechuk on Monday October 14, 2019 @10:22PM (#21255033)
Why not something like this:

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if m:
              yield m
          else:
              raise StopIteration
  ...
  for m in f_iter(many, args):
      # do stuff with m
This way you’re isolating all the initialization logic, error handling, etc. And you can focus on your domain logic in your client code.
Re:What’s New in Python 3.8 by bvrmn on Tuesday October 15, 2019 @10:54AM (#21259059)
Python has built-in facilities for that, there is no need to write wrapper by yourself.

   from functools import partial
   with open('mydata.db', 'rb') as f:
       for block in iter(partial(f.read, 64), b''):
           process_block(block)
Re:What’s New in Python 3.8 by ciupicri on Tuesday October 15, 2019 @10:03AM (#21258420)
Instead of raising StopIteration you could also just return from the function. You could also make it a bit shorter if you swap the if branches.

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if not m:
              return
          yield m
Re:What’s New in Python 3.8 by zb on Tuesday October 15, 2019 @10:12AM (#21258523)
Instead of raising StopIteration you have to return from the function, otherwise you'll just get a RuntimeError starting with Python 3.7.
Re:What’s New in Python 3.8 by ciupicri on Tuesday October 15, 2019 @11:20AM (#21259375)
You're right. The [changelog] for Python 3.7 states:

> [bpo-32670]: Enforce [PEP 479] for all code. This means that manually raising a StopIteration exception from a generator is prohibited for all code, regardless of whether ‘from __future__ import generator_stop’ was used or not.

[changelog]: https://docs.python.org/3.7/whatsnew/changelog.html#id115

[bpo-32670]: https://bugs.python.org/issue32670

[PEP 479]: https://www.python.org/dev/peps/pep-0479/

Re:What’s New in Python 3.8 by repsilat on Monday October 14, 2019 @10:37PM (#21255107)
And less reasonably, something similar for regular expressions:

    for m in filter([re.match(...)], bool):
        ...
;-)
Re:What’s New in Python 3.8 by mlevental on Tuesday October 15, 2019 @12:03AM (#21255459)
so i should write a whole other function that iterates over a list instead of being happy that := exists?
Re:What’s New in Python 3.8 by throwaway744678 on Tuesday October 15, 2019 @03:47AM (#21256368)
Yes! No pain, no gain!
Re:What’s New in Python 3.8 by pyrale on Tuesday October 15, 2019 @03:51AM (#21256383)
The last version clearly exposes that you have an infinite loop. This is something hidden by the `while(evaluate)` expression.

Also if it comes to pure form I would have preferred `while (evaluate) as x:` out of establishing a parallel with existing syntax, but that's not very important.

Re:What’s New in Python 3.8 by pdpi on Tuesday October 15, 2019 @04:34AM (#21256590)
On the contrary: the loop isn’t fundamentally infinite, it’s just an artefact of the language that you have to write it that way. The walrus lets you put the termination where it belongs.
Re:What’s New in Python 3.8 by pyrale on Tuesday October 15, 2019 @07:44AM (#21257489)
I don't understand that the walrus has any significance with regards to the `while` loop termination. it's the nature of the evaluated expression that makes a difference, more precisely, whether it will return something that is cast to False.

In that sense, while loops over expressions always require knowledge of the expression to understand the semantic. On the other hand, something like:

    while True:
       with x as y:
           print(y)
would not require understanding of x to understand the intent. Therefore it is not semantically equivalent to:

    while x as y:
        print(y)
Note that in both cases, `:=` is strictly equivalent to `as` as it is defined for `with`. That is a matter of personal preference.
Re:What’s New in Python 3.8 by xapata on Tuesday October 15, 2019 @01:43AM (#21255840)
That last one isn't an anti-pattern. That's the pattern concluded best for do-while in the discussion of PEP-315.

https://www.python.org/dev/peps/pep-0315/#notice

Re:What’s New in Python 3.8 by eesmith on Tuesday October 15, 2019 @06:22AM (#21257119)
Think about if you match multiple patterns:

    for line in f:
      if (m:= pat1.search(line)) is not None:
        ... do stuff ..
      elif (m:= pat2.search(line)) is not None:
        ... do other stuff ..
      elif (m:= pat3.search(line)) is not None:
         ... do something else ..
In older Python that's:

    for line in f:
      m = pat1.search(line)
      if m is not None:
        ... do stuff ..
      else:
        m = pat2.search(line)
        if m is not None:
          ... do other stuff ..
        else:
           m = pat3.search(line)
           if m is not None:
             ... do something else ..
I think the newer makes it clear that it's supposed to be a simple elif chain, where all branches following the same structure.

There are other ways to structure it, but the alternatives I can think of also have their own cumbersome complexities.

Re:What’s New in Python 3.8 by drdaeman on Wednesday October 16, 2019 @12:33AM (#21267061)
If actions are different, I tend to put them in functions (independent testable pieces, yay!) and iterate over a mapping:

    LINE_ACTIONS = (
        (re.compile("pattern 1"), do_stuff),
        (re.compile("pattern 2"), do_other_stuff),
        (re.compile("pattern 3"), do_something_else),
    )
    ...
    for pattern, action in LINE_ACTIONS:
        m = pattern.search(line)
        if m is not None:
            action(m)
            break
Or even use a method registry pattern that would auto-populate LINE_ACTIONS by just declaring the fuctions:

    @action("pattern 1")
    def do_stuff(m):
       ...
Alternatively, I might just break the processing into a function:

    def _process(line):
        m = pat1.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat2.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat3.search(line)
        if m is not None:
            ... do stuff ...
            return

    _process(line)
Of course, this depends on the purpose. Could be completely inadequate in some situations.
Re:What’s New in Python 3.8 by eesmith on Wednesday October 16, 2019 @05:50AM (#21268664)
Indeed, I've also used this pattern.

I've found that I don't like using it. As you write, it's "inadequate in some situations", which I consider as part of the cumbersome complexities I mentioned.

For examples, do_stuff() and do_other_stuff() may need to share variables, and do_something_else() might need the line number to report an error while the others don't.

This can be handled with shared state/nonlocal, and by passing in more parameters to the generic handler API, but these add complexity.

Or, different parts of the file may have different line dispatch processing (eg, a header block followed by a data block followed by a footer block) where one of the handlers must indicate the transition to a different processor.

Also, function dispatch in CPython is slow. While the regex tests are also slow, it can also be important to consider the (using a hypothetical number) 5% overhead for dispatching over inline code.

Re:What’s New in Python 3.8 by kstrauser on Tuesday October 15, 2019 @12:22PM (#21260223)
That's... lovely. I propose making this the standard example of why you might want to use the walrus operator.
Re:What’s New in Python 3.8 by Alex3917 on Monday October 14, 2019 @11:46PM (#21255377)
If you only have one or two regex like that then there is no difference in readability. If you have ten in a row though then suddenly your function no longer fits on the screen and it’s more difficult to understand what’s going on.
Re:What’s New in Python 3.8 by bandrami on Tuesday October 15, 2019 @02:26AM (#21256014)
I mean, what you really want in this case is Perl's =~ operator, as well as the implicit $_ variable.
Re:What’s New in Python 3.8 by est on Monday October 14, 2019 @09:32PM (#21254765)
IMHO the whole `re.match` is a design flaw in stdlib.

re.match() should always return a match object, but instead .group(1) will return None. Then we can write one-liners easier without the walrus operator.

Re:What’s New in Python 3.8 by nine_k on Monday October 14, 2019 @09:45PM (#21254833)
Sometimes you just check for a match (e.g.of entire string) and define no groups.

The problem is not in re.match but in None being a relatively poor Maybe / Optional (still ways better than C-style `null` of many other languages).

Re:What’s New in Python 3.8 by geofft on Tuesday October 15, 2019 @07:36AM (#21257455)
Could just have been

    if re.match(...).matched:
But yeah, agree that Optional is what I want here.

    if let Some(m) := re.match(...)
        m.group(1)...

    m = re.match(...)
    m.group(1) # TypeError because I haven't unwrapped the Optional
In current Python, that last line sometimes throws a runtime error.
Re:What’s New in Python 3.8 by est on Monday October 14, 2019 @09:46PM (#21254836)
> Sometimes you just check for a match

re.match vs re.search

Re:What’s New in Python 3.8 by Liquid_Fire on Tuesday October 15, 2019 @07:23AM (#21257389)
re.search is different. re.match matches from the beginning of the string, while re.search matches from anywhere in the string.
Re:What’s New in Python 3.8 by eesmith on Tuesday October 15, 2019 @06:32AM (#21257162)
I think you mean .group(0) as .group(1) can already return None:

    >>> import re
    >>> pat = re.compile("(A+)|(B+)")
    >>> m = pat.search("HUBBLE")
    >>> m.group(1) is None
    True
    >>> m.group(2)
    'BB'
    >>> m.group(0)
    'BB'
I don't see how using your proposed change would help.
Re:What’s New in Python 3.8 by est on Wednesday October 16, 2019 @01:08AM (#21267239)
I mean re.search() should always return a MatchObject, dont return None.
Re:What’s New in Python 3.8 by Elrac on Tuesday October 15, 2019 @06:30AM (#21257152)
Looking at your wish for exploratory evaluations in for-comprehensions, I was reminded of how Clojure does this. I found it quite elegant.

The inside of the for-braces allow for keyword-based sub-clauses: There's

- ":when" which will suppress output for elements not matching a filter expression,

- ":while" which will stop when elements stop matching a filter, and

- ":let" which will let you bind some new values in mid-loop.

I rarely need these features, but when I do I find them really really helpful. Maybe someday someone will consider doing something similar in Python.

Re:What’s New in Python 3.8 by hchasestevens on Tuesday October 15, 2019 @07:22AM (#21257387)
Your second use case is actually already covered in Python, assuming you have some kind of sentinel value you can break on:

  for s in iter(network_service.read, ''):
      process(s)
Re:What’s New in Python 3.8 by xg15 on Tuesday October 15, 2019 @02:54AM (#21256137)
I think there are a lot of use-cases for some kind of dedicated "if-with-outputs" syntax, where you have some extra variables available inside the if-block if the condition matched. Such a syntax could cover a lot of the problems of double execution but also prevent hard to understand code like the walrus operator.

... I've got no idea how the syntax could look though.

Re:What’s New in Python 3.8 by Sean1708 on Tuesday October 15, 2019 @03:17AM (#21256251)
How would this hypothetical syntax differ from the walrus operator? I.e. what is hard to understand with the walrus operator that wouldn't be with this new hypothetical syntax and why?
Re:What’s New in Python 3.8 by xg15 on Wednesday October 16, 2019 @02:22AM (#21267604)
I think it could avoid cases where you "abuse" the walrus operator and make code harder to understand than necessary. E.g. things like:

  if (value := func()) and another_condition:
    ...
Or:

  if another_condition or (value := func()):
    ...
Or:

  x=1
  my_list = [x := x+1, x := x-1]
etc.
Re:What’s New in Python 3.8 by Sean1708 on Wednesday October 16, 2019 @07:06AM (#21269079)
Ah ok, so you'd essentially allow it in fewer contexts? I would definitely have supported that, although I do think combining walrus with other boolean expressions in if and while is super useful and not particularly hard to understand.
Re:What’s New in Python 3.8 by mixmastamyk on Tuesday October 15, 2019 @05:32PM (#21264074)
It comes up somewhat rarely but it is useful at times. But of those rare times, it is exceedingly rare to require any more complexity than you've shown above, such as having two assignments in one statement.

That was the reasoning given for using ":=" instead of "as", to allow more complexity. I still think it was a mistake.

Re:What’s New in Python 3.8 by kolinko on Tuesday October 15, 2019 @07:04AM (#21257313)
> while s := network_service.read(): > process(s)

Watch out with this one though. In this case it's good, but if you receive numbers from such function, it will break loop not just on None but on 0 as well. Easy to forget.

In many cases you'll need to do:

    while (s:= ....) is not None: