Re:Composer – Disable GC when computing deps and refs
by sillysaurus3 on Tuesday December 02, 2014 @11:02AM
(#8687727)
As to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue.
That sounds like a bug in the profiler, not with Composer. Observing internal time is pretty important for any profiler.
Re:Composer – Disable GC when computing deps and refs
by Seldaek on Tuesday December 02, 2014 @11:05AM
(#8687741)
Yes it is definitely a failure of the tooling and I hear it is actually being worked on.
Re:Composer – Disable GC when computing deps and refs
by haaaaaaryf on Tuesday December 02, 2014 @10:11AM
(#8687456)
Couldn't I also speed up Composer by using Toran Proxy? ( https://toranproxy.com/ )
Re:Composer – Disable GC when computing deps and refs
by Seldaek on Tuesday December 02, 2014 @10:25AM
(#8687529)
That would help speed up the network part of the install/update yes. The linked patch on the other hand hopefully gets the CPU part to a decent level for most people.
Re:Composer – Disable GC when computing deps and refs
by icehawk219 on Tuesday December 02, 2014 @11:26AM
(#8687875)
Out of curiosity what tools do you use for profiling and finding these sorts of things? Plain old xdebug, xhprof, or other things? I'm going to have to jump into debugging a fairly large Symfony application within the next couple months and am on the look out for good tools to help me along.
Re:Composer – Disable GC when computing deps and refs
by Seldaek on Tuesday December 02, 2014 @11:50AM
(#8688029)
Traditionally xdebug/xhprof were pretty decent, but xhprof has been a bit abandoned since facebook uses HHVM now.
There are two new commercial condenters though that came out in the last few months: Blackfire.io and QafooLabs.com
Both have announced support for showing GC time in profiles as a result of today's noise :) https://twitter.com/beberlei/status/539816149303955456 https://twitter.com/symfony_en/status/539815082881187841
Re:Composer – Disable GC when computing deps and refs
by jexus2 on Wednesday December 03, 2014 @03:09AM
(#8692478)
Xhprof is maintained by Phacility. They did pick up on this[1], but it's not as easy as one might think.
Re:Composer – Disable GC when computing deps and refs
by Seldaek on Wednesday December 03, 2014 @06:10AM
(#8693031)
Thanks for the correction, I wasn't aware that was part of what phabricator extracted out of facebook :)
Re:Composer – Disable GC when computing deps and refs
by icehawk219 on Tuesday December 02, 2014 @12:32PM
(#8688386)
Thanks for the info, those two will be very helpful.
Re:Composer – Disable GC when computing deps and refs
by munificent on Tuesday December 02, 2014 @05:21PM
(#8690393)
> It keeps trying and trying as objects are allocated
Worse. GC gets triggered just by assigning references, allocation isn't even needed.
Disabling GC just kills the advanced GC but leaves the basic reference counting approach to freeing memory, so Composer can keep trucking without using much more memory as the GC wasn't really collecting anything. The memory reduction many people report is rather due to some other improvements we have made yesterday.
As to why the problem went unnoticed for so long, it seems that the GC is not able to be observed by profilers, so whenever we looked at profiles to improve things we obviously did not spot the issue. In most cases though this isn't an issue and I would NOT recommend everyone disables GC on their project :) GC is very useful in many cases especially long running workers, but the Composer solver falls out of the use cases it's made for.