Including files on NFS mount slow with APC enabled
in [General]
|
Prev: erratic results from ftp_rawlist
Next: mysqldump
From: Mark Hunting on 16 Aug 2010 09:21 I am struggling with the performance of some websites that use a lot of includes (using include_once). The files are on a NFS mount (NFSv4), and I use APC to speed things up. APC has enough memory, and I see all included files are in the APC cache. apc.include_once_override is enabled. This is on a Ubuntu Linux 10.04 server. What surprises me is that using strace I see apache open()ing all included files. I would think this is not necessary as APC has these files in its cache. Opening a file takes 1-3 ms, the websites include 100-200 files, so this costs about half a second for each request. I tried a lot to prevent this but my options are exhausted. Is it normal that all these files are open()ed each time, and if so how can I speed up these includes? Part of the trace is below, look especially at the first line where 2ms are lost while this file is in the APC cache: open("/[removed]/library/Zend/Application.php", O_RDONLY) = 1440 <0.002035> fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000137> fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000124> fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000133> mmap(NULL, 11365, PROT_READ, MAP_SHARED, 1440, 0) = 0x7faf3f068000 <0.000395> stat("/[removed]/library/Zend/Application.php", {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000219> munmap(0x7faf3f068000, 11365) = 0 <0.000151> close(1440) = 0 <0.000845> Thanks, Mark
From: Jonathan Tapicer on 16 Aug 2010 13:21 Hi, APC checks by default if every included file was modified doing a stat call. You can disable it, setting apc.stat to 0 (http://www.php.net/manual/en/apc.configuration.php#ini.apc.stat). Try if that improves the performance. Of course, you should manually delete the APC opcode cache every time you modify a PHP file, since APC won't detect that it was modified. Regards, Jonathan On Mon, Aug 16, 2010 at 10:21 AM, Mark Hunting <mark(a)netexpo.nl> wrote: > I am struggling with the performance of some websites that use a lot of > includes (using include_once). The files are on a NFS mount (NFSv4), and > I use APC to speed things up. APC has enough memory, and I see all > included files are in the APC cache. apc.include_once_override is > enabled. This is on a Ubuntu Linux 10.04 server. > > What surprises me is that using strace I see apache open()ing all > included files. I would think this is not necessary as APC has these > files in its cache. Opening a file takes 1-3 ms, the websites include > 100-200 files, so this costs about half a second for each request. I > tried a lot to prevent this but my options are exhausted. Is it normal > that all these files are open()ed each time, and if so how can I speed > up these includes? > > Part of the trace is below, look especially at the first line where 2ms > are lost while this file is in the APC cache: > > open("/[removed]/library/Zend/Application.php", O_RDONLY) = 1440 <0.002035> > fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000137> > fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000124> > fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000133> > mmap(NULL, 11365, PROT_READ, MAP_SHARED, 1440, 0) = 0x7faf3f068000 > <0.000395> > stat("/[removed]/library/Zend/Application.php", {st_mode=S_IFREG|0755, > st_size=11365, ...}) = 0 <0.000219> > munmap(0x7faf3f068000, 11365) = 0 <0.000151> > close(1440) = 0 <0.000845> > > Thanks, > Mark > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > >
From: Mark Hunting on 16 Aug 2010 14:10 Thanks for your answer. I have tested this option before, and it indeed disables the stat() operation. However, it doesn't disable the open() operation, which is about 10x slower than the stat() call (see my example trace). On 08/16/2010 07:21 PM, Jonathan Tapicer wrote: > Hi, > > APC checks by default if every included file was modified doing a stat > call. You can disable it, setting apc.stat to 0 > (http://www.php.net/manual/en/apc.configuration.php#ini.apc.stat). Try > if that improves the performance. Of course, you should manually > delete the APC opcode cache every time you modify a PHP file, since > APC won't detect that it was modified. > > Regards, > > Jonathan > > On Mon, Aug 16, 2010 at 10:21 AM, Mark Hunting <mark(a)netexpo.nl> wrote: > >> I am struggling with the performance of some websites that use a lot of >> includes (using include_once). The files are on a NFS mount (NFSv4), and >> I use APC to speed things up. APC has enough memory, and I see all >> included files are in the APC cache. apc.include_once_override is >> enabled. This is on a Ubuntu Linux 10.04 server. >> >> What surprises me is that using strace I see apache open()ing all >> included files. I would think this is not necessary as APC has these >> files in its cache. Opening a file takes 1-3 ms, the websites include >> 100-200 files, so this costs about half a second for each request. I >> tried a lot to prevent this but my options are exhausted. Is it normal >> that all these files are open()ed each time, and if so how can I speed >> up these includes? >> >> Part of the trace is below, look especially at the first line where 2ms >> are lost while this file is in the APC cache: >> >> open("/[removed]/library/Zend/Application.php", O_RDONLY) = 1440 <0.002035> >> fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000137> >> fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000124> >> fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000133> >> mmap(NULL, 11365, PROT_READ, MAP_SHARED, 1440, 0) = 0x7faf3f068000 >> <0.000395> >> stat("/[removed]/library/Zend/Application.php", {st_mode=S_IFREG|0755, >> st_size=11365, ...}) = 0 <0.000219> >> munmap(0x7faf3f068000, 11365) = 0 <0.000151> >> close(1440) = 0 <0.000845> >> >> Thanks, >> Mark >> >> -- >> PHP General Mailing List (http://www.php.net/) >> To unsubscribe, visit: http://www.php.net/unsub.php >> >> >> >
From: Mark Hunting on 17 Aug 2010 04:08 I now notice that when I replace include_once with include the open() call disappears. That's very nice, but why does include_once need to open the file, even when apc.include_once_override is enabled? Is this a bug? On 08/16/2010 03:21 PM, Mark Hunting wrote: > I am struggling with the performance of some websites that use a lot of > includes (using include_once). The files are on a NFS mount (NFSv4), and > I use APC to speed things up. APC has enough memory, and I see all > included files are in the APC cache. apc.include_once_override is > enabled. This is on a Ubuntu Linux 10.04 server. > > What surprises me is that using strace I see apache open()ing all > included files. I would think this is not necessary as APC has these > files in its cache. Opening a file takes 1-3 ms, the websites include > 100-200 files, so this costs about half a second for each request. I > tried a lot to prevent this but my options are exhausted. Is it normal > that all these files are open()ed each time, and if so how can I speed > up these includes? > > Part of the trace is below, look especially at the first line where 2ms > are lost while this file is in the APC cache: > > open("/[removed]/library/Zend/Application.php", O_RDONLY) = 1440 <0.002035> > fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000137> > fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000124> > fstat(1440, {st_mode=S_IFREG|0755, st_size=11365, ...}) = 0 <0.000133> > mmap(NULL, 11365, PROT_READ, MAP_SHARED, 1440, 0) = 0x7faf3f068000 > <0.000395> > stat("/[removed]/library/Zend/Application.php", {st_mode=S_IFREG|0755, > st_size=11365, ...}) = 0 <0.000219> > munmap(0x7faf3f068000, 11365) = 0 <0.000151> > close(1440) = 0 <0.000845> > > Thanks, > Mark > >
From: Colin Guthrie on 17 Aug 2010 07:13
'Twas brillig, and Mark Hunting at 17/08/10 09:08 did gyre and gimble: > I now notice that when I replace include_once with include the open() > call disappears. That's very nice, but why does include_once need to > open the file, even when apc.include_once_override is enabled? Is this a > bug? I don't know the internals of APC but that smells like a bug to me. Can you post the bug number here if you report one? Cheers Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited [http://www.tribalogic.net/] Open Source: Mandriva Linux Contributor [http://www.mandriva.com/] PulseAudio Hacker [http://www.pulseaudio.org/] Trac Hacker [http://trac.edgewall.org/] |