Prev: Modify an old Device Driver for MISSIL 4302HD Mp4 Player?
Next: Warnings with Inf2Cat.exe for new INF files
From: John Bond on 21 Dec 2009 06:24
Max, that sounded like a good idea, but it quickly failed and it was clear
that since the lock was used twice in some places, it was tough to "get it
right" easily. But in general that is a very good idea. JB

"Maxim S. Shatskih" wrote:

> > response from the DSP. The DPC acquires the two locks at the very beginning
> > A then B, then releases them B then A at the end.
>
> Try to rewrite the code to use 1 and only 1 lock.
>
> --
> Maxim S. Shatskih
> Windows DDK MVP
> maxim(a)storagecraft.com
> http://www.storagecraft.com
>
> .
>
From: John Bond on 21 Dec 2009 06:29
Scott, thanks! I realized that I had not yet converted to WDFSPINLOCKs so
the captures I get today will let me use !wdfkd.wdfspinlock.

Question about "!running -t", it told me there were three processors and all
were idle for a hyperthreaded processor crash dump... I took the dump off the
crashed computer and was looking at it on my laptop. Could it have been
counting the laptop's cpu? Why did it say that all three were idle, was that
due to the Ctrl-Scroll-Scroll induced crash?

-JB

"Scott Noone" wrote:

> Have you looked at the other processors in the system to see if you can find
> the CPU holding the spinlock? "!running -t" is useful here.
>
> If there's no other CPU out there doing something with the lock held then
> you likely have an error path that does not drop the lock correctly. You can
> also try !wdfkd.wdfspinlock on the lock handle to try to get some debugging
> info (I've never used this so I'm not sure what kind of information it
> outputs).
>
> -scott
>
> --
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
>
>
> "John Bond" <johnbond(a)newsgroup.nospam> wrote in message
> news:56EDAF0C-777F-42F4-A705-08734D182B4D(a)microsoft.com...
> > Using KMDF 1.7 to update an OLD NT 4.0 driver that worked on Win2000 sp4
> > to
> > now work on Win XP sp3, I was able to get the driver to the testing phase
> > in
> > 6 weeks while learning the KMDF (the KMDF is one huge step forward in
> > driver
> > development). In very rare instances, the driver locks up for users
> > signed
> > in as administrators, but the lock-up is easily reproduced in less than
> > 1/2
> > hour. We force a blue screen (Ctrl, Scroll, Scroll) and sure enough it is
> > frozen in the DPC after getting the first spinlock and hung waiting for
> > the
> > second spinlock.
> >
> > Looks like a classical getting-locks-out-of-order system freeze. For the
> > last 3 days I have studied the code and don't see the cause of the
> > lock-up.
> >
> > ****Question***** How can I use WinDbg to find and look into the data
> > sctructures stored in my driver to help me determine who might be holding
> > the
> > other lock? The driver originally used KSPIN_LOCKs, and I have crash
> > dumps
> > for that. The lastest test versions (shots in the dark) use WDFSPINLOCKs
> > (was hoping I would get some WDF warning somewhere, wishful thinking).
> >
> > More details: The driver is for an audio interface card with three ports,
> > two for headsets one for 4-wire trunk. The driver uses IOCTLs heavily to
> > control the DSP on the board. There are three kernel threads (one per
> > channel/audio interface). The DLL sends IOCTLs to 1) send commands to the
> > DSP's three channels, 2) change states of the driver, 3) poll the driver
> > for
> > events collected from the DSP. The locks keep the DPC and the channel
> > threads and the IOCTLS from using the 1) DSP interface and 2) channel
> > state
> > data simultaneously. Most locks are held for just a few assignment
> > statements (atomic state changes) or for sending a command or getting a
> > response from the DSP. The DPC acquires the two locks at the very
> > beginning
> > A then B, then releases them B then A at the end.
> >
> > Thanks in advance, John Bond
> >
>
>
> .
>
From: Scott Noone on 21 Dec 2009 12:22
> Question about "!running -t", it told me there were three processors and
> all
> were idle for a hyperthreaded processor crash dump

I suspect that it really showed three idle and one running (quad proc,
right?). That means that you don't likely have a race here since you're
deadlocking on a single CPU.

I'd start looking for error paths that don't drop the spinlock, since
they're not recursively acquireable a dangling lock can deadlock like this.
!wdfspinlock might be helpful here.

(And the advice of using fewer locks certainly makes things simpler, but in
a world where Windows supports 256 processors breaking locks up is sometimes
necessary for high perf!)

-scott

--
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com


"John Bond" <johnbond(a)newsgroup.nospam> wrote in message
news:6C684507-6473-4AAF-841C-2ED051F4AF53(a)microsoft.com...
> Scott, thanks! I realized that I had not yet converted to WDFSPINLOCKs so
> the captures I get today will let me use !wdfkd.wdfspinlock.
>
> Question about "!running -t", it told me there were three processors and
> all
> were idle for a hyperthreaded processor crash dump... I took the dump off
> the
> crashed computer and was looking at it on my laptop. Could it have been
> counting the laptop's cpu? Why did it say that all three were idle, was
> that
> due to the Ctrl-Scroll-Scroll induced crash?
>
> -JB
>
> "Scott Noone" wrote:
>
>> Have you looked at the other processors in the system to see if you can
>> find
>> the CPU holding the spinlock? "!running -t" is useful here.
>>
>> If there's no other CPU out there doing something with the lock held then
>> you likely have an error path that does not drop the lock correctly. You
>> can
>> also try !wdfkd.wdfspinlock on the lock handle to try to get some
>> debugging
>> info (I've never used this so I'm not sure what kind of information it
>> outputs).
>>
>> -scott
>>
>> --
>> Scott Noone
>> Consulting Associate
>> OSR Open Systems Resources, Inc.
>> http://www.osronline.com
>>
>>
>>
>>
>> "John Bond" <johnbond(a)newsgroup.nospam> wrote in message
>> news:56EDAF0C-777F-42F4-A705-08734D182B4D(a)microsoft.com...
>> > Using KMDF 1.7 to update an OLD NT 4.0 driver that worked on Win2000
>> > sp4
>> > to
>> > now work on Win XP sp3, I was able to get the driver to the testing
>> > phase
>> > in
>> > 6 weeks while learning the KMDF (the KMDF is one huge step forward in
>> > driver
>> > development). In very rare instances, the driver locks up for users
>> > signed
>> > in as administrators, but the lock-up is easily reproduced in less than
>> > 1/2
>> > hour. We force a blue screen (Ctrl, Scroll, Scroll) and sure enough it
>> > is
>> > frozen in the DPC after getting the first spinlock and hung waiting for
>> > the
>> > second spinlock.
>> >
>> > Looks like a classical getting-locks-out-of-order system freeze. For
>> > the
>> > last 3 days I have studied the code and don't see the cause of the
>> > lock-up.
>> >
>> > ****Question***** How can I use WinDbg to find and look into the data
>> > sctructures stored in my driver to help me determine who might be
>> > holding
>> > the
>> > other lock? The driver originally used KSPIN_LOCKs, and I have crash
>> > dumps
>> > for that. The lastest test versions (shots in the dark) use
>> > WDFSPINLOCKs
>> > (was hoping I would get some WDF warning somewhere, wishful thinking).
>> >
>> > More details: The driver is for an audio interface card with three
>> > ports,
>> > two for headsets one for 4-wire trunk. The driver uses IOCTLs heavily
>> > to
>> > control the DSP on the board. There are three kernel threads (one per
>> > channel/audio interface). The DLL sends IOCTLs to 1) send commands to
>> > the
>> > DSP's three channels, 2) change states of the driver, 3) poll the
>> > driver
>> > for
>> > events collected from the DSP. The locks keep the DPC and the channel
>> > threads and the IOCTLS from using the 1) DSP interface and 2) channel
>> > state
>> > data simultaneously. Most locks are held for just a few assignment
>> > statements (atomic state changes) or for sending a command or getting a
>> > response from the DSP. The DPC acquires the two locks at the very
>> > beginning
>> > A then B, then releases them B then A at the end.
>> >
>> > Thanks in advance, John Bond
>> >
>>
>>
>> .
>>
From: Doron Holan [MSFT] on 21 Dec 2009 16:54
you should enable for the io veifier for deadlock detection as well as the
KMDF verifier (which will turn on ownership tracking for the WDFSPINLOCK as
well as acquisition / release history)

d

--

This posting is provided "AS IS" with no warranties, and confers no rights.


"Scott Noone" <snoone(a)osr.com> wrote in message
news:C2C6E1B5-EEAD-443F-B150-7783C8206BB0(a)microsoft.com...
>> Question about "!running -t", it told me there were three processors and
>> all
>> were idle for a hyperthreaded processor crash dump
>
> I suspect that it really showed three idle and one running (quad proc,
> right?). That means that you don't likely have a race here since you're
> deadlocking on a single CPU.
>
> I'd start looking for error paths that don't drop the spinlock, since
> they're not recursively acquireable a dangling lock can deadlock like
> this. !wdfspinlock might be helpful here.
>
> (And the advice of using fewer locks certainly makes things simpler, but
> in a world where Windows supports 256 processors breaking locks up is
> sometimes necessary for high perf!)
>
> -scott
>
> --
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
> "John Bond" <johnbond(a)newsgroup.nospam> wrote in message
> news:6C684507-6473-4AAF-841C-2ED051F4AF53(a)microsoft.com...
>> Scott, thanks! I realized that I had not yet converted to WDFSPINLOCKs
>> so
>> the captures I get today will let me use !wdfkd.wdfspinlock.
>>
>> Question about "!running -t", it told me there were three processors and
>> all
>> were idle for a hyperthreaded processor crash dump... I took the dump off
>> the
>> crashed computer and was looking at it on my laptop. Could it have been
>> counting the laptop's cpu? Why did it say that all three were idle, was
>> that
>> due to the Ctrl-Scroll-Scroll induced crash?
>>
>> -JB
>>
>> "Scott Noone" wrote:
>>
>>> Have you looked at the other processors in the system to see if you can
>>> find
>>> the CPU holding the spinlock? "!running -t" is useful here.
>>>
>>> If there's no other CPU out there doing something with the lock held
>>> then
>>> you likely have an error path that does not drop the lock correctly. You
>>> can
>>> also try !wdfkd.wdfspinlock on the lock handle to try to get some
>>> debugging
>>> info (I've never used this so I'm not sure what kind of information it
>>> outputs).
>>>
>>> -scott
>>>
>>> --
>>> Scott Noone
>>> Consulting Associate
>>> OSR Open Systems Resources, Inc.
>>> http://www.osronline.com
>>>
>>>
>>>
>>>
>>> "John Bond" <johnbond(a)newsgroup.nospam> wrote in message
>>> news:56EDAF0C-777F-42F4-A705-08734D182B4D(a)microsoft.com...
>>> > Using KMDF 1.7 to update an OLD NT 4.0 driver that worked on Win2000
>>> > sp4
>>> > to
>>> > now work on Win XP sp3, I was able to get the driver to the testing
>>> > phase
>>> > in
>>> > 6 weeks while learning the KMDF (the KMDF is one huge step forward in
>>> > driver
>>> > development). In very rare instances, the driver locks up for users
>>> > signed
>>> > in as administrators, but the lock-up is easily reproduced in less
>>> > than
>>> > 1/2
>>> > hour. We force a blue screen (Ctrl, Scroll, Scroll) and sure enough
>>> > it is
>>> > frozen in the DPC after getting the first spinlock and hung waiting
>>> > for
>>> > the
>>> > second spinlock.
>>> >
>>> > Looks like a classical getting-locks-out-of-order system freeze. For
>>> > the
>>> > last 3 days I have studied the code and don't see the cause of the
>>> > lock-up.
>>> >
>>> > ****Question***** How can I use WinDbg to find and look into the data
>>> > sctructures stored in my driver to help me determine who might be
>>> > holding
>>> > the
>>> > other lock? The driver originally used KSPIN_LOCKs, and I have crash
>>> > dumps
>>> > for that. The lastest test versions (shots in the dark) use
>>> > WDFSPINLOCKs
>>> > (was hoping I would get some WDF warning somewhere, wishful thinking).
>>> >
>>> > More details: The driver is for an audio interface card with three
>>> > ports,
>>> > two for headsets one for 4-wire trunk. The driver uses IOCTLs heavily
>>> > to
>>> > control the DSP on the board. There are three kernel threads (one per
>>> > channel/audio interface). The DLL sends IOCTLs to 1) send commands to
>>> > the
>>> > DSP's three channels, 2) change states of the driver, 3) poll the
>>> > driver
>>> > for
>>> > events collected from the DSP. The locks keep the DPC and the channel
>>> > threads and the IOCTLS from using the 1) DSP interface and 2) channel
>>> > state
>>> > data simultaneously. Most locks are held for just a few assignment
>>> > statements (atomic state changes) or for sending a command or getting
>>> > a
>>> > response from the DSP. The DPC acquires the two locks at the very
>>> > beginning
>>> > A then B, then releases them B then A at the end.
>>> >
>>> > Thanks in advance, John Bond
>>> >
>>>
>>>
>>> .
>>>
From: alberto on 22 Dec 2009 11:01

A few precautions at code writing time may
go a long way to insure against synchronization issues.

1. One lock, one unlock. One-to-one. Strictly nested, unless you're
using a time-honored computer science algorithm such as producer-
consumer.
2. No critical section back doors, in or out.
3. Try to get lock and unlock such that the whole critical section is
visible within one single source code window.
4. Locks are meant to protect data in processor-accessible memory.
Shouldn't be used for anything else.
5. Only use the critical section to transact with the data being
protected. Keep everything else out of the critical section.
6. Do all synchronization through macros. This allows adding debug
code subarchitecturally to the macros, without disrupting the driver.
7. No point using a sync construct to protect a piece of data if your
i/o hardware can asynchronously touch that data.
8. Not every i/o ends with an interrupt! Be particularly critical with
things that get grabbed at i/o time and released at interrupt time.
9. A small circular history buffer is cheap to implement and to run,
and it goes a long way towards helping to solve race conditions.
10. Pay attention to underflows. Will a faulty duplicate interrupt
release more semaphore items than you grabbed ?
11. If you can afford it, fork a kernel thread to watch your sync
constructs and break deadlocks if necessary.
12. Sometimes it may be tough to know the difference between a
deadlock and a bus hang; adding debug code or a log file may help.

If you feel the need to abandon these and other safety practices, be
very sure you know what you're doing, and try to minimize those
ocasions. I use these guidelines when I write my programs, and I
haven't been disappointed!

Hope this helps,


Alberto.



On Dec 21, 12:22 pm, "Scott Noone" <sno...(a)osr.com> wrote:
> > Question about "!running -t", it told me there were three processors and
> > all
> > were idle for a hyperthreaded processor crash dump
>
> I suspect that it really showed three idle and one running (quad proc,
> right?). That means that you don't likely have a race here since you're
> deadlocking on a single CPU.
>
> I'd start looking for error paths that don't drop the spinlock, since
> they're not recursively acquireable a dangling lock can deadlock like this.
> !wdfspinlock might be helpful here.
>
> (And the advice of using fewer locks certainly makes things simpler, but in
> a world where Windows supports 256 processors breaking locks up is sometimes
> necessary for high perf!)
>
> -scott
>
> --
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.http://www.osronline.com
>
> "John Bond" <johnb...(a)newsgroup.nospam> wrote in message
>
> news:6C684507-6473-4AAF-841C-2ED051F4AF53(a)microsoft.com...
>
>
>
> > Scott, thanks!  I realized that I had not yet converted to WDFSPINLOCKs so
> > the captures I get today will let me use !wdfkd.wdfspinlock.
>
> > Question about "!running -t", it told me there were three processors and
> > all
> > were idle for a hyperthreaded processor crash dump... I took the dump off
> > the
> > crashed computer and was looking at it on my laptop.  Could it have been
> > counting the laptop's cpu?  Why did it say that all three were idle, was
> > that
> > due to the Ctrl-Scroll-Scroll induced crash?
>
> > -JB
>
> > "Scott Noone" wrote:
>
> >> Have you looked at the other processors in the system to see if you can
> >> find
> >> the CPU holding the spinlock? "!running -t" is useful here.
>
> >> If there's no other CPU out there doing something with the lock held then
> >> you likely have an error path that does not drop the lock correctly. You
> >> can
> >> also try !wdfkd.wdfspinlock on the lock handle to try to get some
> >> debugging
> >> info (I've never used this so I'm not sure what kind of information it
> >> outputs).
>
> >> -scott
>
> >> --
> >> Scott Noone
> >> Consulting Associate
> >> OSR Open Systems Resources, Inc.
> >>http://www.osronline.com
>
> >> "John Bond" <johnb...(a)newsgroup.nospam> wrote in message
> >>news:56EDAF0C-777F-42F4-A705-08734D182B4D(a)microsoft.com...
> >> > Using KMDF 1.7 to update an OLD NT 4.0 driver that worked on Win2000
> >> > sp4
> >> > to
> >> > now work on Win XP sp3, I was able to get the driver to the testing
> >> > phase
> >> > in
> >> > 6 weeks while learning the KMDF (the KMDF is one huge step forward in
> >> > driver
> >> > development).  In very rare instances, the driver locks up for users
> >> > signed
> >> > in as administrators, but the lock-up is easily reproduced in less than
> >> > 1/2
> >> > hour.  We force a blue screen (Ctrl, Scroll, Scroll) and sure enough it
> >> > is
> >> > frozen in the DPC after getting the first spinlock and hung waiting for
> >> > the
> >> > second spinlock.
>
> >> > Looks like a classical getting-locks-out-of-order system freeze.  For
> >> > the
> >> > last 3 days I have studied the code and don't see the cause of the
> >> > lock-up.
>
> >> > ****Question*****  How can I use WinDbg to find and look into the data
> >> > sctructures stored in my driver to help me determine who might be
> >> > holding
> >> > the
> >> > other lock?  The driver originally used KSPIN_LOCKs, and I have crash
> >> > dumps
> >> > for that.  The lastest test versions (shots in the dark) use
> >> > WDFSPINLOCKs
> >> > (was hoping I would get some WDF warning somewhere, wishful thinking).
>
> >> > More details:  The driver is for an audio interface card with three
> >> > ports,
> >> > two for headsets one for 4-wire trunk.  The driver uses IOCTLs heavily
> >> > to
> >> > control the DSP on the board.  There are three kernel threads (one per
> >> > channel/audio interface).  The DLL sends IOCTLs to 1) send commands to
> >> > the
> >> > DSP's three channels, 2) change states of the driver, 3) poll the
> >> > driver
> >> > for
> >> > events collected from the DSP.  The locks keep the DPC and the channel
> >> > threads and the IOCTLS from using the 1) DSP interface and 2) channel
> >> > state
> >> > data simultaneously.  Most locks are held for just a few assignment
> >> > statements (atomic state changes) or for sending a command or getting a
> >> > response from the DSP.  The DPC acquires the two locks at the very
> >> > beginning
> >> > A then B, then releases them B then A at the end.
>
> >> > Thanks in advance, John Bond
>
> >> .- Hide quoted text -
>
> - Show quoted text -

 |  Next  |  Last
Pages: 1 2
Prev: Modify an old Device Driver for MISSIL 4302HD Mp4 Player?
Next: Warnings with Inf2Cat.exe for new INF files