News for hackers. Stuff that matters.
Re:Data flow syntax: The missing piece for the success of fu... by icebraining on Friday July 17, 2015 @12:10AM (#9901246)
You can be sure it doesn't execute twice because the generation is lazy/on-demand. When you create a generator, nothing gets actually done, it just sits there waiting for you to pull elements from it, and which point it'll actually generate one element (and wait for you to request another).

This is in contrast with list comprehensions, which have a similar syntax, but which generate a full list immediately after being declared.

Re:Data flow syntax: The missing piece for the success of fu... by m_mueller on Friday July 17, 2015 @12:52AM (#9901311)
I know that this is true in general, but this case here seems tricky to me. Look at the following line:

> yield (s[len(s)//2:], s[:len(s)//2])

I'm aware that the generator is being called lazily, but here we're actually calling each yield twice - once the left side is thrown away, once the right side downstream. Now, are you still sure that left and right is generated only once?

Re:Data flow syntax: The missing piece for the success of fu... by icebraining on Friday July 17, 2015 @01:18AM (#9901365)
Oh, right, I missed that. No, it's actually generating both every time. CPython can't eliminate one of the sides, since the calls (either to len() or to the slicing method of 's') might have side effects.

In this case, I think it'd make more sense to divide gen_splitted_his into two functions (or one which accepted a parameter), for generating one or the other.

Re:Data flow syntax: The missing piece for the success of fu... by m_mueller on Friday July 17, 2015 @01:44AM (#9901424)
I have been toying around now with this second python example for a bit.

Here's my conclusion: Not only is it not efficient, it doesn't even work. The generator cannot be rewound after it has been iterated over for lowercase - the uppercase version will never be executed. You essentially run into the problem described here: http://stackoverflow.com/questions/1271320/reseting-generato... .

It only works once you actually generate twice

> lc_his = gen_lc_his(gen_splitted_his(('hi for the %d:th time!' % i for i in xrange(10))))

> uc_his = gen_uc_his(gen_splitted_his(('hi for the %d:th time!' % i for i in xrange(10))))

It seems to me for this toy example, generators are simply not the right tool.

Re:Data flow syntax: The missing piece for the success of fu... by icebraining on Friday July 17, 2015 @01:53AM (#9901442)
Yes, but I think the author realizes that, which is why (s)he wrote above "this will produce an error, since running a generator function twice is not allowed, and this would not re-use the same items in memory anyway, which is kind of the main point with stream processing".

The solution being proposed at the end is actually not using generators at all.

Re:Data flow syntax: The missing piece for the success of fu... by m_mueller on Friday July 17, 2015 @02:05AM (#9901458)
Oh I see, I missed that this "limitation" also applies for his DPI example that I was toying with.