Prev: Smtpd module
Next: py3 tkinter Text accepts what bytes?
From: Alexandre Fayolle on 23 Apr 2010 07:59
On Friday 23 April 2010 08:36:01 Mark Hammond wrote:
> On 22/04/2010 7:23 AM, Alexandre Fayolle wrote:
> > Hi everyone,
> >
> > I have a production server running a Windows Service written in Python,
> > which uses python 2.5.4 (yes I know it is old, but I am somewhat stuck
> > with this for now) and pywin32 214.
> >
> > Given a set of manipulations, I get a stack overflow in the service, and
> > a bad crash. The same operation when running without
> > win32serviceutil.ServiceFramework does not trigger the bug. I'm looking
> > for some debugging tools (debug build of the interpreter and the pywin32
> > libraries) that some good soul could have kept from a previous debugging
> > session to try to get a C stack trace and understand what causes this.
>
> I expect the problem might be that pythonservice.exe isn't linked with a
> large enough stack - python itself builds with a larger than default
> stack. That difference could cause a recursive function to hard-crash
> before Python itself detected the recursion as being too deep.
>
> You could test this by playing with the sys.setrecursionlimit function -
> at some limit I expect you would find the hard-crash would be replaced
> with a max recursion exception. Let me know if that is the case and
> I'll be sure to adjust the stack size for the next pywin32 build.
>
> > Any hint towards what could cause that stack overflow would be welcome
> > too. The application is multithreaded (and uses pyro and twisted). I can
> > provide more information for the curious.
>
> Some parts of twisted are written such that server responses are
> processed recursively, and very large input can cause the recursion
> limit to be hit. I'm still occasionally bitten by this in the IMAP
> client code...

Thank you for this very helpful information. I've found on my side that
upgrading to python2.6 will prevent the crash from happening in the reported
conditions, and used this as a fix (currently running extensive tests to be on
the safe side).

Runnning 2.6 will among other things enable me to use a recent VS compiler to
rebuild stuff if required.

I'm very interested in a pywin32 build with a larger stack for
pythonservice.exe, as this would seriously increase my confidence level and
improve my sleep quality :-)

--
Alexandre Fayolle LOGILAB, Paris (France)
Formations Python, CubicWeb, Debian : http://www.logilab.fr/formations
Développement logiciel sur mesure : http://www.logilab.fr/services
Informatique scientifique: http://www.logilab.fr/science
From: Mark Hammond on 23 Apr 2010 02:36
On 22/04/2010 7:23 AM, Alexandre Fayolle wrote:
> Hi everyone,
>
> I have a production server running a Windows Service written in Python, which
> uses python 2.5.4 (yes I know it is old, but I am somewhat stuck with this for
> now) and pywin32 214.
>
> Given a set of manipulations, I get a stack overflow in the service, and a bad
> crash. The same operation when running without
> win32serviceutil.ServiceFramework does not trigger the bug. I'm looking for
> some debugging tools (debug build of the interpreter and the pywin32
> libraries) that some good soul could have kept from a previous debugging
> session to try to get a C stack trace and understand what causes this.

I expect the problem might be that pythonservice.exe isn't linked with a
large enough stack - python itself builds with a larger than default
stack. That difference could cause a recursive function to hard-crash
before Python itself detected the recursion as being too deep.

You could test this by playing with the sys.setrecursionlimit function -
at some limit I expect you would find the hard-crash would be replaced
with a max recursion exception. Let me know if that is the case and
I'll be sure to adjust the stack size for the next pywin32 build.

> Any hint towards what could cause that stack overflow would be welcome too.
> The application is multithreaded (and uses pyro and twisted). I can provide
> more information for the curious.

Some parts of twisted are written such that server responses are
processed recursively, and very large input can cause the recursion
limit to be hit. I'm still occasionally bitten by this in the IMAP
client code...

HTH,

Mark
 | 
Pages: 1
Prev: Smtpd module
Next: py3 tkinter Text accepts what bytes?