tag:blogger.com,1999:blog-8699431508730375743.post5096427249718206614..comments2016-07-20T07:15:28.987-07:00Comments on The History of Python: From List Comprehensions to Generator ExpressionsGuido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-8699431508730375743.post-24834416435512837172010-06-30T14:15:11.133-07:002010-06-30T14:15:11.133-07:00It seems like it shouldn't be bad for generato...It seems like it shouldn't be bad for generator expressions to be a small constant factor slower than list comprehensions. Aren't generator expressions primarily a memory optimization, and only secondarily possibly a speed optimization? I guess it's bad if you write "sum(x*x for x in L)" and it's slower than "sum([x*x for x in L])"...HilbertAstronauthttps://www.blogger.com/profile/11443786031975040593noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-7856819366907464492010-06-29T20:06:28.550-07:002010-06-29T20:06:28.550-07:00@James Brown:
It can be difficult to determine wh...@James Brown:<br /><br />It can be difficult to determine what qualifies as an implementation detail and what qualifies as a language feature, especially in the absence of a standard. While I think we have a great understanding of where that line lies with Python today, there is one subject where the distinction is fairly clear, that is, when talking about speed. If we are talking about the time order of an operation, then it is possibly part of the language (list indexing is specified as an O(1) operation in python). Otherwise, the time taken for any expression to execute is obviously not considered part of the language; as long as evaluation of some expression terminates, it can take as long as it likes.wlesliehttps://www.blogger.com/profile/12093160811053013157noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-34549491522790234122010-06-29T15:04:11.372-07:002010-06-29T15:04:11.372-07:00@Nick: Thanks for the detailed timings. I had time...@Nick: Thanks for the detailed timings. I had timed something similar for a much smaller sequence, so my numbers probably include more per-loop setup overhead.Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-41231740725503010982010-06-29T15:00:05.627-07:002010-06-29T15:00:05.627-07:00Oops, there should be a "./" at the star...Oops, there should be a "./" at the start of last timeit snippet as well (and there was in the shell where I was running the test).Anonymoushttps://www.blogger.com/profile/14824694805745746190noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-15863738901665489272010-06-29T14:58:42.671-07:002010-06-29T14:58:42.671-07:00Are you sure about your performance figures at the...Are you sure about your performance figures at the end there? Comprehensions are still much faster than generator expressions in Py3k for me (which makes sense, since the comprehensions still do everything inline in one function, while the generator expression has to keep popping in and out of the generator frame from C code). To avoid the name lookup confounding the relative timings, I used the following timeit snippets:<br /><br />./python -m timeit -s "seq = [1]*1000" "[x for x in seq]; list"<br />./python -m timeit -s "seq = [1]*1000" "{x for x in seq}; list<br />python -m timeit -s "seq = [1]*1000" "list(x for x in seq)"<br /><br />Those timings were in the vicinity of 45-50 us, 60-70 us, 75-85 us.<br /><br />The raw speed of the operations is also pretty similar between 2.7 and 3.2 for me (neither being noticeably slower or faster than the other just eyeballing the timeit results). Although both seem a little faster than 2.6, so maybe the speedups were backported along with dict and set comprehensions (which seems likely, since it should be the same code that handles it all in both branches now).<br /><br />One thing that *is* much faster in Py3k is a module level list comprehension, since those now automatically benefit from function local variable access optimisations for their loop variables.Anonymoushttps://www.blogger.com/profile/14824694805745746190noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-81910413017339351292010-06-29T14:22:04.034-07:002010-06-29T14:22:04.034-07:00@David Mertz: Very clever, but I *did* say that th...@David Mertz: Very clever, but I *did* say that there was a universal set implied by the context. The math books on my shelves show lots of examples where the universal set is omitted from the notation. Also, {x ∈ ℚ | x > 10} differs from list comprehensions because the latter have to repeat "x" twice: [x for x in Q if x > 10].<br /><br />@James Brown: list comprehensions *are* map/filter. As for reduce and tail recursion, that discussion is closed.Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-40899827969892826922010-06-29T11:57:47.032-07:002010-06-29T11:57:47.032-07:00Correct me if I'm wrong, but wouldn't you ...Correct me if I'm wrong, but wouldn't you be able to support map/reduce just as efficiently as list comprehensions/generator expressions if you simply re-used the stack frame (a la tail recursion)?James Brownhttps://www.blogger.com/profile/03689879611011141899noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-63780249984555410752010-06-29T11:03:05.380-07:002010-06-29T11:03:05.380-07:00The parallel with set theory is even closer than y...The parallel with set theory is even closer than you suggest, Guido. If you could write in set theory simply:<br /><br /> { x | x > 10 }<br /><br />Then that would expose you to Russell's Paradox, and you could also write:<br /><br /> { x | x ∉ x }<br /><br />And that would be very naughty indeed. :-)<br /><br />So in real set notation, one must be so clean as to write, e.g.:<br /><br /> { x ∈ ℚ | x > 10 }<br /><br />Which is, after all, the same as Python list/generator comprehensions (other than a slight spelling difference).David Mertzhttps://www.blogger.com/profile/03057134173589181346noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-633603052932757112010-06-29T10:36:27.413-07:002010-06-29T10:36:27.413-07:00very much informative! thanksvery much informative! thanksNithinhttps://www.blogger.com/profile/08620669284660131333noreply@blogger.comtag:blogger.com,1999:blog-8699431508730375743.post-32080537538081193022010-06-29T10:29:06.744-07:002010-06-29T10:29:06.744-07:00Very interesting and refreshing. Thanks!Very interesting and refreshing. Thanks!Nicolas Grillyhttps://www.blogger.com/profile/16959696672419872463noreply@blogger.com