Thursday, July 12, 2012

0x9F: DRIVER_POWER_STATE_FAILURE

Alright, let's get down to some business. As I said, I am now going to start going through successful analysis posts of mine and posting them here for others to learn from, read up on, etc. Also just for personal reference as well! Some will be very easy and I will just explain what I did, and some will be difficult, etc.

Now, this post is going to be about bugcheck 0x9F: DRIVER_POWER_STATE_FAILURE. 9F bugchecks are personally my favorite as in most cases they're very easy to solve, and I will explain why. In most cases, a 9F will tell you the driver culprit right in the "probably caused by", however in some cases it will shoot an incorrect fault, or it will relay what the bucket ID is in that specific dump.

Before we get into all of that though, here's that basic definition of a 0x9F:

A device driver is in an invalid or inconsistent power state from either shutdown or going into or returning from hibernate or standby modes.
I recently dealt with a case in which the user was reporting he/she was receiving 0x9F's BSOD(s). I opened the dump in WinDbg:

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 9F, {3, fffffa800a727060, fffff80004ba93d8, fffffa8006a62010}

*** WARNING: Unable to verify timestamp for asmthub3.sys
*** ERROR: Module load completed but symbols could not be loaded for asmthub3.sys
*** WARNING: Unable to verify timestamp for win32k.sys
*** ERROR: Module load completed but symbols could not be loaded for win32k.sys
Probably caused by : asmthub3.sys

Followup: MachineOwner
---------

0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

DRIVER_POWER_STATE_FAILURE (9f)
A driver is causing an inconsistent power state.
Arguments:
Arg1: 0000000000000003, A device object has been blocking an Irp for too long a time
Arg2: fffffa800a727060, Physical Device Object of the stack
Arg3: fffff80004ba93d8, Functional Device Object of the stack
Arg4: fffffa8006a62010, The blocked IRP

Debugging Details:
------------------


DRVPOWERSTATE_SUBCODE:  3

DRIVER_OBJECT: fffffa800670ac10

IMAGE_NAME:  asmthub3.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  4eb203d0

MODULE_NAME: asmthub3

FAULTING_MODULE: fffff88005f25000 asmthub3

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0x9F

PROCESS_NAME:  System

CURRENT_IRQL:  2

STACK_TEXT: 
fffff800`04ba9388 fffff800`033046c2 : 00000000`0000009f 00000000`00000003 fffffa80`0a727060 fffff800`04ba93d8 : nt!KeBugCheckEx
fffff800`04ba9390 fffff800`032a4e3c : fffff800`04ba94c0 fffff800`04ba94c0 00000000`00000000 00000000`00000002 : nt! ?? ::FNODOBFM::`string'+0x34050
fffff800`04ba9430 fffff800`032a4cd6 : fffff800`03433f70 00000000`0000ea7d 00000000`00000000 00000000`00000000 : nt!KiProcessTimerDpcTable+0x6c
fffff800`04ba94a0 fffff800`032a4bbe : 00000002`2e2dc26d fffff800`04ba9b18 00000000`0000ea7d fffff800`03410228 : nt!KiProcessExpiredTimerList+0xc6
fffff800`04ba9af0 fffff800`032a49a7 : 00000000`bedfc7c2 00000000`0000ea7d 00000000`bedfc78b 00000000`0000007d : nt!KiTimerExpiration+0x1be
fffff800`04ba9b90 fffff800`03291eca : fffff800`0340ce80 fffff800`0341acc0 00000000`00000000 fffff880`00000000 : nt!KiRetireDpcList+0x277
fffff800`04ba9c40 00000000`00000000 : fffff800`04baa000 fffff800`04ba4000 fffff800`04ba9c00 00000000`00000000 : nt!KiIdleLoop+0x5a


STACK_COMMAND:  kb

FOLLOWUP_NAME:  MachineOwner

FAILURE_BUCKET_ID:  X64_0x9F_3_IMAGE_asmthub3.sys

BUCKET_ID:  X64_0x9F_3_IMAGE_asmthub3.sys

Followup: MachineOwner
---------
As you can see, this is a fairly straightforward  0x9F. It says the culprit right there in the probably caused, which is asmthub3.sys (ASMedia USB 3.0 Hub driver). All that needed to be done was link the user to the latest chipset / utility drivers on the motherboard page and the issue was solved after updating the driver.

Now, let's assume that the following dump file was not so straightforward. Let's pretend that when we opened it up, rather than the probably caused by faulty displaying the guilty driver, it said for example "usbhub.sys".

What we would do then is we would take a look at the 4th argument is there was a blocked IRP, and then run an !irp on the address of the 4th argument.

So, for example, in the following dump: !irp fffffa8006a62010

After running that, we then get the following:

0: kd> !irp fffffa8006a62010
Irp is active with 8 stacks 7 is current (= 0xfffffa8006a62290)
 No Mdl: No System Buffer: Thread 00000000:  Irp stack trace. 
     cmd  flg cl Device   File     Completion-Context
 [  0, 0]   0  0 00000000 00000000 00000000-00000000   

            Args: 00000000 00000000 00000000 00000000
 [  0, 0]   0  0 00000000 00000000 00000000-00000000   

            Args: 00000000 00000000 00000000 00000000
 [  0, 0]   0  0 00000000 00000000 00000000-00000000   

            Args: 00000000 00000000 00000000 00000000
 [  0, 0]   0  0 00000000 00000000 00000000-00000000   

            Args: 00000000 00000000 00000000 00000000
 [  0, 0]   0  0 00000000 00000000 00000000-00000000   

            Args: 00000000 00000000 00000000 00000000
 [  0, 0]   0  0 00000000 00000000 00000000-00000000   

            Args: 00000000 00000000 00000000 00000000
>[ 16, 2]   0 e1 fffffa800a740790 00000000 fffff80003285ce0-fffffa800b934340 Success Error Cancel pending
           \Driver\asmthub3    nt!IopUnloadSafeCompletion
            Args: 00016600 00000001 00000004 00000005
 [  0, 0]   0  0 00000000 00000000 00000000-fffffa8006958dc0   

            Args: 00000000 00000000 00000000 00000000
As you can see, there's a (>). This symbol indicates what driver was active at the time of the crash, and the asmthub3 driver is listed there. That's what you'd do if there was a false probably caused by fault. However, sometimes you get 0x9F's that don't have a blocked up IRP AND an incorrect / false fault. Well, you'd then have to obviously go through some other dumps or take a look at the loaded modules list and see if there are any obvious troublesome 3rd party drivers that may have caused it, etc. You can also get 9F bugchecks that deal with locks and such, but we'll get into that at another time.

Generally, 0x9F's are very easy. I learned how to solve most 0x9F's by reading VirGnarus' posts across various BSOD communities.

3 comments:

  1. This is all very nice and informative, however it assumes that the only thing you want to do is find out which driver caused the 0x9F. I agree this is very easy, and your description is top notch.

    However, if you are the developer of said driver, and wish to debug it, it's not so easy, as there is no readily visible information on the stack to show you which part of the driver code it was stuck in. As you see, the only things on the stack in your case are generic NT functions. The ASMedia driver is nowhere to be seen. This is the case most of the time.

    ReplyDelete