Tuesday, September 30, 2014

Registers (x86)

As I discussed towards the end of my last post regarding stacks, my next post was likely going to be about registers. Well, here we are! I had originally planned on discussing both x86 and x64 registers in a single post, but this posed two main problems. The first problem was this would have been a very long post! The second problem, which is a considerably larger one, is I don't know x64 assembly/architecture as much as I'd like to feel confident in making a post about it. The good news is whenever I am brushed up enough in regards to x64's architecture to write a detailed post, I can simply jump right in as I did all the dirty work right here in this post! Happy days.

Memory Hierarchy

First off, it's important to discuss and understand what a register is. Before we get into that however, let's have a look at my favorite image of the memory hierarchy:


(thanks to COMPUTER SCIENCE E-1 for this great image)

It's safe to say there are probably thousands of images regarding the memory hierarchy throughout CS books, documents, and presentations, but this one takes the cake for me! It's about as good as it visually gets for a hierarchy image, and although it doesn't display a few key points, I can do that myself right here in this post.

What are the key points I'm talking about that are missing from this image? Well, from the bottom>top, we're going from slowest to fastest. If we're coming from the top>bottom, we're going from the fastest to the slowest (in regards to read and write access time). It's absolutely imperative to also understand that the faster we get, the more expensive we get, and the slower we get, the less expensive we get (in regards to USD). With that now known, you can imagine that the read/write from a removal media device (such as USB) is slower than the read/write from your hard drive, but is less expensive.

As this is a post strictly about registers, I won't go into the complexities and intricacies of each part of the hierarchy, and will instead focus on the registers themselves. As far as access time goes, let's compare registers and the hard drive as an example:

Registers - 1-2ns (nanoseconds)

Hard Drive - 5-20ms (milliseconds)

-- It's all dependent on the architecture of the processor, really. These are rough #'s.

Cue the amazing Grace Hopper!


Why are registers so fast? Registers are actually circuits which are built/wired (literally) into the Arithmetic logic unit (ALU), which is also widely considered the fundamental building block of a CPU. With that said, we really can't get any closer, which means there's also no data transfer overhead as there are barely any clock cycles required. Also, a CPU's instruction set tends to work with registers more than it does with actual memory locations.

Speaking of clock cycles, here's a chart displaying the cycles regarding the memory hierarchy:


(thanks to HLNAND for this great image)

As we can see, a register only takes one clock cycle.

What is a register?
 
Now that we understand the basic fundamentals behind the memory hierarchy and where the register resides on the hierarchy, we can discuss what a register actually is! In its most basic definition, a register is used to store small pieces of data that the processor is actively working on. There are many different registers and categories of registers, all of which essentially do something different, however you can generally break registers down into two types. For example, regarding the first type, we have the General purpose register (GPR), which essentially stores data and performs arithmetic based on an instruction (addition, subtraction, multiplication, etc). Once the arithmetic is finished (or the manipulation of data/memory is finished), it's entirely up to what the instruction is set to do. It can either store it back into memory with the same instruction, a different one, etc.

Regarding the second type, we have the Special purpose register (SPR) which as the name implies has a special meaning and specific purpose. For example, the SP (Stack Pointer) register is a SPR in addition to being a GPR regarding the IA-32 architecture. This register is used to have the CPU store the address of the last program request in a stack. Among other things, as new requests are coming in, they push down the older ones in the stack, with the most recent request always residing at the top of the stack.

It's important to note that at every clock tick, there are specific values regarding registers. The values stored in a specific register may have been updated on a tick, so the values may not be the same as they were prior to the tick. For example, when an interrupt fires, register values are copied to a stack and stay on that stack while an Interrupt Service Routine (ISR) is being executed by the CPU. Once the interrupt is properly handled, the original register values are loaded back from the stack so they can continue to service the instruction they were previously working with.

What I described above is known as context switching, which is essentially the jump of instructions from CPU > ISR. Although unrelated yet interesting to note, in some special cases regarding 0x101 bug checks depending on what actually caused the bug check, you may need to have knowledge of context switching to properly debug.

With all of the above said, there's about a dozen different ways I could go at this point. I could go on to talk about the register file, the many different and various categories of registers, etc. However, let's jump ahead to register renaming as that's a pretty important topic.

Register Renaming

Register renaming is essentially a form of pipelining that deals with data dependencies between instructions by renaming their register operands. The way renaming works is, it will go ahead and replace the architectural register (user-accessible registers, or more easily just known to us as 'the registers') names (value names) with a new value name for each instruction destination operand.

Thanks to register renaming, we can also successfully perform what is known as Out-of-order execution (OOE). How exactly does it allow OOE to be performed? Register renaming entirely eliminates name dependencies between instructions, and recognizes true dependencies. True dependencies occur when an instruction depends on the result of a subsequent instruction.

Given the above is now explained, here's a good time to explain data hazards. Data hazards occur when instructions that exhibit data dependence modify data in different stages of a pipeline. Ignoring a data hazard can lead to what is known as a race condition, which is when the order of the data that was outputted was not the intended order. We have three main data hazards:

  • Read-after-write (RAW), also known as a true dependency.
  • Write-after-read (WAR), also known as an anti-dependency.
  • Write-after-write (WAW), also known as an output dependency.

1. What is RAW? Let's take two instruction locations (l1, l2). A prime RAW example is when l2 tries to read a source before l1 writes to it. l2 is attempting to refer to a result that hasn't been calculated or retrieved yet by l1.

2. What is WAR? Let's once again take l1 & l2. l2 tries to write a destination before it is read by l1. This is a problem in concurrent execution, which notes of course they must work concurrently and not sequentially. If they do work sequentially, then we have a data hazard like so.

3. What is WAW? Taking l1 & l2 one last time, l2 tries to write an operand before it is written by l1.

With register renaming, since we're ultimately maintaining a status bit for each value that indicates whether or not it has been completed, it allows the execution of two instruction operations to be performed out of order when there are no true data dependencies between them. This removes WAR/WAW, and of course leaves RAW intact as discussed above.


(Excerpt from the following .pdf)

x86 Registers

In WinDbg, by using the r command we can go ahead and dump the registers from the context of the thread that caused the crash. For example:

 kd> r  
 eax=818f4920 ebx=86664d90 ecx=818fb9c0 edx=000002d0 esi=818f493c edi=00000000  
 eip=818c07dd esp=a1e72cb0 ebp=a1e72ccc iopl=0     nv up ei pl nz na pe nc  
 cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000       efl=00000206  

x86 (IA-32) has eight GPRs, which are:

  • EAX
  • EBX
  • ECX
  • EDX
  • ESI
  • EDI
  • EBP
  • ESP

Great, so there's our eight GPRs. Now we can go ahead and break them down:

  • EAX - The 'A' in the EAX register implies it's the accumulator register for operands and results data.
  • EBX - The 'B' in the EBX register implies it's the pointer to the data in the DS segment. DS is the current data segment. It also means 'base register'.
  • ECX - The 'C' in the ECX register implies it's the counter for storing loop and string operations.
  • EDX - The 'D' in the EDX register implies it's the I/O pointer.
  • ESI - The 'SI' in the ESI register implies it's the Source Index, which is a pointer to data in the segment pointed to by the DS register.
  • EDI - The 'DI' in the EDI register implies it's the Destination Index, which is a pointer to data (or a destination) in the segment pointed to by the ES register. It's essentially the counterpart to the ESI register, for lack of a better word.
  • EBP - The 'BP' in the EBP register implies it's the Base Pointer, which is the pointer to data on the stack.
  • ESP - The 'SP' in the ESP register implies it's the Stack Pointer, which is used to detect the location of the last item put on the stack.

Whew, alright! So there's just a few things I'd like to quickly explain as well as we haven't covered all of the bases:

  • EAX - The 'AX' in the EAX register is used to address only the lower 16 bits of the register. If we were to reference all 32 bits, we'd use all of EAX, and not just AX.
  • EIP - I didn't forget this guy, don't worry! The 'IP' in the EIP register implies it's the Instruction Pointer, which can also actually be called the 'program counter'. It contains the offset in the current code segment for the next instruction that will be executed. It's also interesting to note that EIP cannot be accessed by software, and is explicitly controlled by control-transfer instructions such as JMP, CALL, JC, and RET. Aside from control-transfer instructions, interrupts and exceptions can also access EIP directly.

Okay, so I can't mention control-transfer instructions and then not explain them. I mean, I could... but I wouldn't be happy. Control-transfer instructions specifically control the flow of program execution. There are quite a few control-transfer instructions, but I will discuss the ones I mentioned that can directly access EIP:

  • JMP - Jump. JMP transfers program control to a different point in the instruction stream without recording any return information. The destination operand specifies the address of the instruction being jumped to. This operand can be an immediate value, a GPR, or a memory location. The JMP instruction can be used to actually execute four different types of jumps:
1. Near jump - A jump to an instruction within the segment currently pointed to by the CS register. It can also at times be referred to as an intrasegment jump.

2. Short jump - A type of near jump in that the jump range is limited to -128 to +127 from the current EIP value.

3. Far jump - A jump to an instruction that's located in a different segment than the current code segment, but at the same privilege level. It can also at times be referred to as an intersegment jump.

4. Task switch - A jump to an instruction that's located in a different task. Note that a task switch can only be accomplished in protected-mode, which not to fly too off the handle here, but it's necessary to explain. Rather than explaining it here though as it's a bit large, I'll explain it below in a short while.
  • CALL - Call procedure. CALL pushes the current code location onto the hardware supported stack in memory, and then performs an unconditional jump to the code location indicated by the label operand. Unlike the simple jump instructions I listed above, the call instruction saves the location to return to when the subroutine completes. 
  • JC - Jump if carry flag is set. 
  • Ret - Return. This instruction transfers control to the return address located on the stack, which is usually placed on the stack by a call instruction that we discussed above. It then performs an unconditional jump to the retrieved code location. For example:

 call <label>  
 ret  
I mentioned intersegment/intrasegment jumps above. Intersegment jumps can transfer control to a statement in a different code segment, while intrasegment jumps are always between statements in the same code segment.

Now that we have the above control-transfer instructions explained, let's discuss protected-mode as I mentioned above. Before further discussing protected-mode and the similarly named but very different real-mode, we'll need to do a bit of a history lesson.

Way before my time, back in the late 70's (76-78), Intel's 16-bit 8086 processor was released. It was 16-bit because its internal registers + internal/external data buses were 16 bits wide. A 20-bit external address bus meant this beast could address a whopping 1 MB's of memory! 1 MB may not seem like anything these days, but it was actually considered more than overkill around this time. Due to this being the case, the max linear address space was limited to a mere 64 KB. Aside from 1 MB being overkill, this was also because the internal registers were only 16 bits wide.

There were two problems here:

1. Programming over 64 KB boundaries meant adjusting segment registers.

2. As time went on, applications were being released that made this mere 64 KB seem like the measly number it is today.

What was the saving grace? Intel's 80286 processor was released in 1982! Well, what's so great about this processor that solve the above two problems? The 80286 processor had two operating modes, as opposed to the 8086 which only had one. The operating modes were:

1. Real-Mode (backwards compatible 8086 mode).

2. Protected-Mode.

The 80286 processor had a 24-bit address bus, which could address up to 16 MB! That's 15 more MB than its predecessor. There's too much good stuff here, so let's discuss the problems. The problems were certainly problems, and they were considerably big ones:

- DOS apps couldn't easily be ported to protected-mode granted that most/if not all DOS apps were developed in a way that made them incompatible.

- The 80286 processor couldn't successfully revert back to the backwards compatible real-mode without a CPU reset. Later on however in 1984, IBM added circuitry that allowed a special series of instructions to successfully cause a revert without initiating a CPU reset. This method while certainly being a great feat, posed quite the performance penalty. Later on it was discovered that initiating a triple fault was a much faster and cleaner way, but there was still yet to be a 'painless' method of transition.

With the above said, the successor was released which is the 80386. The 80386 had a 32-bit address bus, which allowed for 4 GB of memory access. 1 MB > 16 MB > 4 GB, quite the increase! In addition, the segment size was increased to 32-bits, which meant there wasn't a need to switch between multiple segments to access the full address space of 4 GB.

Whew! With all of this known, what is protected-mode actually so great for at this point? Protected-mode allows for virtual memory, paging, ability to finally painlessly switch back to real-mode without a CPU reset, etc.

How do we actually get into protected-mode? The Global Descriptor Table (GDT) needs to be created with a minimum of three entries. These three entries are a null descriptor, a code segment descriptor, and a data segment descriptor. Afterwards, the PE bit needs to be set in the CR0 register and a far JMP needs to be made to clear the prefect input queue (PIQ).

 // Set PE bit  
 mov eax, cr0  
 or eax, 1  
 mov cr0, eax  
 // Far JMP (Remember CS = Code Segment)  
 jmp cs:@pm  
 @pm:  
 // Now in protected-mode :)  

So we got to most of the registers from the x86 register dump excerpt, but we're missing these ones:


 nv up ei pl nz na pe nc  

These are the current contents of the FLAGS register (there are 20[?] different flags), which is the status register for x86.

  • nv - No overflow.
  • up - Up.
  • ei - Enable interrupt.
  • pl - Plus (I believe).
  • nz - Not zero.
  • na - Not auxiliary carry.
  • pe - Parity even.
  • nc - No carry.

Disassembly Example

Let's now dump a stack from a random x86 crash dump to show an example of some of the registers we talked about, some assembly code, and instructions:

 ChildEBP RetAddr   
 a1e72ccc 81a8eec4 nt!KeBugCheckEx+0x1e  
 a1e72cf0 81a1e85f nt!PspCatchCriticalBreak+0x73  
 a1e72d20 81a1e806 nt!PspTerminateAllThreads+0x2c  
 a1e72d54 8185c986 nt!NtTerminateProcess+0x1c1  
 a1e72d54 77725d14 nt!KiSystemServicePostCall  
 0028f0d0 00000000 0x77725d14  

Let's go ahead and disassemble the nt!PspTerminateAllThreads+0x2c kernel function:

 nt!PspTerminateAllThreads+0x2c:  
 81a1e85f 8b450c     mov   eax,dword ptr [ebp+0Ch]  // Move value stored at memory address in the ebp+0Ch register to the eax register.
 81a1e862 8b4048     mov   eax,dword ptr [eax+48h]  // Move value stored at memory address in the eax+48h register to the eax register.
 81a1e865 8945f0     mov   dword ptr [ebp-10h],eax  // Move contents of eax register to the ebp-10h register.
 81a1e868 6a00      push  0  // Push a 32-bit zero on the stack.
 81a1e86a 8bc7      mov   eax,edi  // Move contents of the edi register into the eax register.
 81a1e86c c745fc22010000 mov   dword ptr [ebp-4],122h  // Store the 32-bit value 122h to the ebp-4 register.
 81a1e873 e8bf010000   call  nt!PspGetPreviousProcessThread (81a1ea37) // Call function name nt!PspGetPreviousProcessThread.
 81a1e878 8b5d14     mov   ebx,dword ptr [ebp+14h]  // Moving value stored at memory address in the ebp+14h register to the ebx register.

Hope you enjoyed reading!

References

Intel 80386 Reference Programmer's Manual
X86 Assembly/High-Level Languages
A Guide to Programming Intel IA32 PC Architecture
Segment Registers: Real mode vs. Protected mode

Saturday, September 20, 2014

Stacks, stacks, and more stacks!

If you've ever debugged a crash dump before, user-mode or kernel, you've very likely seen a stack. If you debug crash dumps quite often, you've seen and traversed thousands of stacks, maybe more! For a long time I knew the basics behind a stack and what I was looking at/through for the most part, but really beyond that I didn't know much for quite awhile until I dug deep into dozens of documents and books as time went on. With that said, let's discuss stacks and their mechanics!

The stack excerpt below is from an x86 0xF4 crash dump. The cause of the bug check and such is irrelevant, and this stack is purely for explanatory purposes. With that said, turn off your tingly BSOD senses for a moment and look at the functions and such for what they are, and not what they could have done to play a role in the crash : )

 kd> k  
 ChildEBP RetAddr   
 a1e72ccc 81a8eec4 nt!KeBugCheckEx+0x1e  
 a1e72cf0 81a1e85f nt!PspCatchCriticalBreak+0x73  
 a1e72d20 81a1e806 nt!PspTerminateAllThreads+0x2c  
 a1e72d54 8185c986 nt!NtTerminateProcess+0x1c1  
 a1e72d54 77725d14 nt!KiSystemServicePostCall  
 0028f0d0 00000000 0x77725d14  

k* will perform a stack unwind, or more specifically will display the stack backtrace. * denotes the placeholder for the various possible parameters that we can use, such as - kb, kv, kc, kd, the list goes on. Each of those all does something different, but for now we'll stick to plain old k for example purposes.

Let's break this down one at a time!

ChildEBP

ChildEBP - First we have our ChildEBP, otherwise known as the 'base pointer', which is essentially nothing more than a pointer to a stack frame that has been set up by a piece of code (a reference to a stack of memory). As an example, let's take a look at the following command:

 dd a1e72cf0 l1  

dd - This is a variation of plain old d, which simply means dump. If we however use dd, this will display/dump dword(s).

a1e72cf0 - This is a pointer address from our stack, specifically from nt!PspCatchCriticalBreak+0x73.

L1 - I've always been under the impression that this tells WinDbg to print a line based on what we're looking to do as far as the command goes. I've searched the WinDbg help doc far and wide for a conclusive answer, but I haven't found one yet. There are billions of people out there far smarter than I, and hopefully one of them reads my blog to tell me what it really does!

EDIT: Hilariously enough, someone smarter than me did happen to come along. Thanks to Blair Foster, I now understand what L1 really does.
its a size/range specifier. In the case of d* its used as size. So, dd L1 displays 1 dword, db L1 is 1 byte, etc.
Thanks, Blair!

If we go ahead and run this command, here's what we get:

 kd> dd a1e72cf0 l1  
 a1e72cf0 a1e72d20  

Look familiar? a1e72d20 is another pointer address from our stack, but more specifically the pointer from the previous function in the stack - nt!PspTerminateAllThreads+0x2c. With this known, we can conclude that the ChildEBP also stores the previous function address.

 a1e72d20 81a1e806 nt!PspTerminateAllThreads+0x2c  

In this specific example, we don't have a good stack to show us what would happen if we went all the way back to what function spawned/began the stack. Taking a quick look through some x64 dumps (didn't have any good x86 ones), I dug up a quick stack. Try not to get overwhelmed with the differences between x86 and x64 as far as stack goes (even though there aren't really any). It's just more bits and bobs!

 3: kd> k  
 Child-SP          RetAddr             
 fffff880`033dfa58 fffff800`0302da3b nt!KeBugCheckEx  
 fffff880`033dfa60 fffff800`031f0463 hal!HalBugCheckSystem+0x1e3  
 fffff880`033dfaa0 fffff800`0302d700 nt!WheaReportHwError+0x263  
 fffff880`033dfb00 fffff800`0302d052 hal!HalpMcaReportError+0x4c  
 fffff880`033dfc50 fffff800`0302cf0d hal!HalpMceHandler+0x9e  
 fffff880`033dfc90 fffff800`03020e88 hal!HalpMceHandlerWithRendezvous+0x55  
 fffff880`033dfcc0 fffff800`030d84ac hal!HalHandleMcheck+0x40  
 fffff880`033dfcf0 fffff800`030d8313 nt!KxMcheckAbort+0x6c  
 fffff880`033dfe30 fffff800`030d07b8 nt!KiMcheckAbort+0x153  
 fffff880`09e9f638 00000000`00000000 nt!memcpy+0x208  

Just by looking at the stack we can see it begins right at nt!memcpy+0x208, but let's show it the way we've been demonstrating:

 3: kd> dd fffff880`033dfe30 l1  
 fffff880`033dfe30 00000000  

We can see at address fffff880`033dfe30, the return value was zero. This tells us it wasn't called from another prior function, etc, and is the very beginning of the stack. Given the function itself is nt!memcpy, this is understandable.

RetAddr

RetAddr - Now that we understand our base pointer, we have our RetAddr, which implies return address. This is done because quite simply the CPU without being told what to do is pretty useless. We need to let it know what address to return to after it's done successfully executing the code, because all it's concerned about is instructions and executing their code. It has no idea how to handle them without this stuff set in place, or what to do after it's done. With that said, this is why we have our return address.

If we take another look at our x64 stack excerpt:

 3: kd> k  
 Child-SP          RetAddr             
 fffff880`033dfa58 fffff800`0302da3b nt!KeBugCheckEx  
 fffff880`033dfa60 fffff800`031f0463 hal!HalBugCheckSystem+0x1e3  
 fffff880`033dfaa0 fffff800`0302d700 nt!WheaReportHwError+0x263  
 fffff880`033dfb00 fffff800`0302d052 hal!HalpMcaReportError+0x4c  
 fffff880`033dfc50 fffff800`0302cf0d hal!HalpMceHandler+0x9e  
 fffff880`033dfc90 fffff800`03020e88 hal!HalpMceHandlerWithRendezvous+0x55  
 fffff880`033dfcc0 fffff800`030d84ac hal!HalHandleMcheck+0x40  
 fffff880`033dfcf0 fffff800`030d8313 nt!KxMcheckAbort+0x6c  
 fffff880`033dfe30 fffff800`030d07b8 nt!KiMcheckAbort+0x153  
 fffff880`09e9f638 00000000`00000000 nt!memcpy+0x208  

According to the stack, HalpMceHandlerWithRendezvous was called by HalHandleMcheck. We know that when the work HalpMceHandlerWithRendezvous was doing was finished, it would return back to HalHandleMcheck. We can confirm this by taking a look at the return memory addresses and using the ln command:

 3: kd> ln fffff800`03020e88  
 (fffff800`03020e48)  hal!HalHandleMcheck+0x40  

ln - This command will display the function/routine at or near the given address.

Bingo! Execution will resume afterwards in HalHandleMcheck, specifically 64 bytes from the start of the function.

Functions/Call Site

Functions/Call Site - Last but not least, we have our third column (it's not labeled above in any of the stack excerpts). Interestingly enough, x64 stacks do label their function columns, specifically as Call Site. In the above stack x64 excerpts however, I removed them to avoid confusion! : )

Here is what a non-edited x64 stack looks like:

 3: kd> k  
 Child-SP          RetAddr           Call Site  
 fffff880`033dfa58 fffff800`0302da3b nt!KeBugCheckEx  
 fffff880`033dfa60 fffff800`031f0463 hal!HalBugCheckSystem+0x1e3  
 fffff880`033dfaa0 fffff800`0302d700 nt!WheaReportHwError+0x263  
 fffff880`033dfb00 fffff800`0302d052 hal!HalpMcaReportError+0x4c  
 fffff880`033dfc50 fffff800`0302cf0d hal!HalpMceHandler+0x9e  
 fffff880`033dfc90 fffff800`03020e88 hal!HalpMceHandlerWithRendezvous+0x55  
 fffff880`033dfcc0 fffff800`030d84ac hal!HalHandleMcheck+0x40  
 fffff880`033dfcf0 fffff800`030d8313 nt!KxMcheckAbort+0x6c  
 fffff880`033dfe30 fffff800`030d07b8 nt!KiMcheckAbort+0x153  
 fffff880`09e9f638 00000000`00000000 nt!memcpy+0x208  

As we can see, there's Call Site! In x86 crash dumps however, it's not labeled. I don't need to paste the stack excerpt we used from above as I didn't edit it, so you can just scroll back up and take a quick look.

It should be no mystery that this third and final column displays the function (and routine) names throughout the stack. Do note that they are in direct correlation with the current loaded symbols. If you don't have proper symbols, you're going to get an unresolved function (or routine), and it's going to look like junk.

--------------------

Thanks for reading! In my next post I plan on going into the x86 registers (maybe x64 although it'll be a lot of work -- may save that for a future post).

Friday, September 19, 2014

Thermal Zones

Today we're going to be taking a look into thermal zones, which are essentially different physical regions of the hardware platform that are partitioned. This act of partitioning is done so that when a sensor detects that a thermal zone is overheating, it will either use passive or active cooling to cool the devices in the specific thermal zone. I'm still learning about them in-depth as they're relatively confusing.

 ACPI thermal zone ACPI\ThermalZone\TZS0 has been enumerated.  
 _PSV = 371K  
 _TC1 = 0  
 _TC2 = 50  
 _TSP = 0ms  
 _AC0 = 0K  
 _AC1 = 0K  
 _AC2 = 0K  
 _AC3 = 0K  
 _AC4 = 0K  
 _AC5 = 0K  
 _AC6 = 0K  
 _AC7 = 0K  
 _AC8 = 0K  
 _AC9 = 0K  
 _CRT = 373K  
 _HOT = 0K  
 _PSL - see event data.  

Having a look at the above event data, this is the thermal zone data for the TZS0 sensor. Without having other more detailed event data logs aside from this, I can only assume this is the sensor for the CPU. The reason for this is the _PSL child object is used to list the processors in the thermal zone. If we saw _TZD instead of _PSL, this would list the non-processor devices in the thermal zone. For the moment however, we'll just assume the entire system was overheating as this was a really hot laptop. Let's go down the line:

PSV

_PSV Indicates the temperature at which the operating system started passive cooling control. In our case here, this was 371K (K = Kelvin). 371 Kelvin is 97.85 Celsius. The system we're dealing with in this post is an Acer Aspire 5740G, which according to tech specs houses an i5 430m for its processor. If we consult the manual:

We can see the max temperature is 105C regarding the CPU core, and 100C for the integrated graphics + IMC. With this said, the CPU started throttling (passive countermeasure to overheating) at 97.85C. How did the CPU know when to start throttling? PROCHOT#!

PROCHOT# is Intel's thermal throttle activity bit:


Note it states - The TCC will remain active until the system deasserts PROCHOT#. In fancy software engineer talk, this simply means that so long as the temperatures continue to rise (or doesn't drop), it will continue to lower power consumption from the CPU until it's in a safe spot. The downside to this is in extreme situations of overheating (like this one here), we trip because we hung around at an unsafe operational temperature for too long (or got higher) and shut down to prevent permanent damage.

TC1/2

_TC1/2 - These are both known as the Thermal Constants, which are essentially objects used to evaluate the constants _TC1/2 for use in the passive cooling formula.

 Performance [%]= _TC1 * ( Tn -Tn-1 ) + _TC2 * (Tn. - Tt)  

The return value is an integer containing Thermal Constant #'s 1 or 2. In our case, our return value was 50 for _TC2.

TSP

_TSP - Evaluates to a thermal sampling period (in tenths of seconds) used by Operating System-directed configuration and Power Management (OSPM) to implement the passive cooling equation. This value, along with _TC1 and _TC2, will enable OSPM to provide the proper hysteresis required by the system to accomplish an effective passive cooling.

In our case, this was 0ms.

AC(x)

_AC(x) - This optional object, if present under a thermal zone, returns the temperature trip point at which Operating System-directed configuration and Power Management (OSPM) must start or stop active cooling, where (x) is a value between 0 and 9 that designates multiple active cooling levels of the thermal zone.

In our case, we can see it does go 0-9, and all is listed as 0 Kelvin.

CRT

_CRT - Indicates the temperature at which the operating system will shut down as it simply cannot throttle (passive) or use fans (active) to succeed in cooling these temperatures in time before permanent damage. In this case, we can see it's 373k (373 Kelvin) when this occurred, which is 99.85 Celsius. Unsure as to why this is 99.85 and not 100 in our case, but I digress. Possibly it rounded up?

HOT

_HOT - Indicates the return value (temperature) at which Operating System-directed configuration and Power Management (OSPM) may choose to transition the system into the S4 sleeping state.

--------------------

In any case, all we know is that this laptop was having some serious overheating problems. It was occurring mostly overnight when the system was being awoken from sleep to perform a defrag. The defrag was enough to push it to throttle/shutdown temperatures, which was also throwing bug checks as Windows wasn't happy it couldn't defrag.

Thanks for reading, and hopefully more info on thermal zones in the near future.

References

ACPI-defined devices
Advanced Configuration and Power Interface Specification
Intel® Atom™ Processor 330 Series Datasheet

Sunday, September 7, 2014

Rootkit Debugging (runtime2 postmortem) - SwishDbgExt, SysecLabs script, etc.

Today we're going to be doing some rootkit debugging, specifically regarding runtime2, with a bit of a twist! I have a ton of rootkit debugging posts coming in the next few weeks, as I've decided to break them up rather than throwing them together in one giant mess of a post.

I've shown various scenarios in which I've debugged a rootkit before (0x7A, etc), but this time we're going to use various extensions to help us, other methods, and overall go a lot more in-depth. The postmortem runtime2 rootkit KMD that will be used in this post was generated by niemiro, a good friend from Sysnative forums. He aimed to make it a good example of some things a rootkit/malware developer can do to make things not as obvious when you resort to methods such as hooking the SSDT, which is rather old and very detectable these days.

 CRITICAL_OBJECT_TERMINATION (f4)  
 A process or thread crucial to system operation has unexpectedly exited or been  
 terminated.  
 Several processes and threads are necessary for the operation of the  
 system; when they are terminated (for any reason), the system can no  
 longer function.  
 Arguments:  
 Arg1: 00000003, Process  
 Arg2: 86664d90, Terminating object  
 Arg3: 86664edc, Process image file name  
 Arg4: 819e91f0, Explanatory message (ascii)  

Right, so here's our bug check. Most if not all of these 'older' rootkits will use Direct Kernel Object Manipulation (DKOM) to hook low-level routines/functions within the System Service Dispatch Table (SSDT). When this is occurring, assuming the developer of the rootkit didn't do a very good job in writing their rootkit, a lot can go wrong when disabling write protection, carelessly swapping memory, and inserting hooks. Malware scans from many AV programs can also cause crashes when they detect that the SSDT is hooked under certain circumstances. The rootkit can also intentionally call a bug check if written this way when it has detected a scan has initiated.

However, if the driver is well written, you may not crash at all. What do you do if you're suspicious of a rootkit infection/hooking, yet a bug check isn't occurring naturally due to proper programming of the rootkit? This is when in some cases (like in this specific example here), you may need to take matters into your own hands and either:
  • Force a bug check.
  • Live kernel debugging session.
  • Run an ARK (Anti-Rootkit) tool. This is no fun... we want to have fun and learn : )
The first option is much less effective than the second, and that's due to the fact that the rootkit may not be doing any obvious hooking of the SSDT, etc, at the time of you forcing the crash. If you're investigating during a live session, it's much more effective and there are a lot more commands you can use. However, for learning purposes, I will show both.

1st argument - The value in the 2nd argument (0x3) implies it was a process as opposed to a thread that unexpectedly terminated. If it was a thread however, it would have instead been 0x6.

2nd argument - The value in the 2nd argument (86664d90) is a pointer to the _EPROCESS object that unexpectedly terminated. We can dump it using !object, which I will show below:

 kd> !object 86664d90  
 Object: 86664d90 Type: (841537e8) Process  
   ObjectHeader: 86664d78 (old version)  
   HandleCount: 4 PointerCount: 127  

3rd argument - The value in the 3rd argument (86664edc) is the process image file name. Essentially, it's the process name that unexpectedly terminated. We can dump it by using dc, which I will show below:

 kd> dc 86664edc  
 86664edc 73727363 78652e73 00000065 00000000 csrss.exe.......  
 86664eec 00000000 00000000 00000000 8605a278 ............x...  
 86664efc 869356d8 00000000 00000000 0000000a .V..............  
 86664f0c 8c04d631 00000000 00000000 7ffdc000 1...............  
 86664f1c 00000000 00000099 00000000 00000000 ................  
 86664f2c 00000000 000005c6 00000000 0003cc50 ............P...  
 86664f3c 00000000 00000000 00000000 00006d77 ............wm..  
 86664f4c 00000000 00000000 00000162 00000000 ........b.......  

The process that unexpectedly terminated was csrss.exe, which is the Client/Server Runtime Subsystem.

For some extra food for thought, we can get a lot of the information above manually if we'd like to. If we take the 2nd argument (_EPROCESS object) and run the following:

 dt _EPROCESS 86664d90 imageFileName  

dt will display the type, which will show us the offset, etc. More specifically it displays information about a local variable, global variable or data type:

 kd> dt _EPROCESS 86664d90 imageFileName  
 nt!_EPROCESS  
   +0x14c ImageFileName : [16] "csrss.exe"  

If we take the _EPROCESS object and add it to the offset (+0x14c), we get the following:

 kd> ? 86664d90+0x14c  
 Evaluate expression: -2040115492 = 86664edc  

Look familiar? It's our 3rd argument, the process image name (csrss.exe).

4th argument - The value in the 3rd argument (819e91f0) is the explanatory message regarding the reason for the bug check, specifically in ASCII. To dump this, we'll need to use the dc command as we used earlier on the 3rd argument:

 kd> dc 819e91f0  
 819e91f0 6d726554 74616e69 20676e69 74697263 Terminating crit  
 819e9200 6c616369 6f727020 73736563 25783020 ical process 0x%   

As we can see, this is reiterating the 1st argument.

--------------------

So far all we know is that this bug check occurred because csrss.exe, a critical Windows process, unexpectedly terminated. Why? If you weren't able to tell whilst reading through by now, we purposely terminated it via Task Manager to force the bug check.With that said, for this post we're choosing the first method (forcing a bug check).

Pretending we ourselves didn't purposely force the bug check for a moment, what would we at this point do if this was a crash dump we were looking at from a system that isn't ours, and/or from our doing? Among many things, one of the first few things I do in 'not so obvious' 0xF4's is I check the summary and stats regarding virtual memory on the system at the time of the crash:

 kd> !vm  
 *** Virtual Memory Usage ***  
      Physical Memory:   917370 (  3669480 Kb)  
      Page File: \??\C:\pagefile.sys  
       Current:  3976680 Kb Free Space:  3976676 Kb  
       Minimum:  3976680 Kb Maximum:   4193280 Kb  
      Available Pages:   769568 (  3078272 Kb)  
      ResAvail Pages:    869504 (  3478016 Kb)  
      Locked IO Pages:      0 (     0 Kb)  
      Free System PTEs:   348283 (  1393132 Kb)  
      Modified Pages:    11884 (   47536 Kb)  
      Modified PF Pages:   11830 (   47320 Kb)  
      NonPagedPool Usage:   8265 (   33060 Kb)  
      NonPagedPool Max:   522998 (  2091992 Kb)  
      PagedPool 0 Usage:   5501 (   22004 Kb)  
      PagedPool 1 Usage:   10013 (   40052 Kb)  
      PagedPool 2 Usage:    623 (   2492 Kb)  
      PagedPool 3 Usage:    631 (   2524 Kb)  
      PagedPool 4 Usage:    726 (   2904 Kb)  
      PagedPool Usage:    17494 (   69976 Kb)  
      PagedPool Maximum:  523264 (  2093056 Kb)  
      Session Commit:     3758 (   15032 Kb)  
      Shared Commit:     9951 (   39804 Kb)  
      Special Pool:       0 (     0 Kb)  
      Shared Process:     1260 (   5040 Kb)  
      Pages For MDLs:      2 (     8 Kb)  
      PagedPool Commit:   17550 (   70200 Kb)  
      Driver Commit:     2335 (   9340 Kb)  
      Committed pages:   115256 (  461024 Kb)  
      Commit limit:    1882649 (  7530596 Kb)  

We can see right away that we don't have insufficient non-paged pool, which is a pretty popular cause of most 0xF4's as it at that point cannot handle I/O operations, etc. It's generally due to buggy drivers causing pool related memory leaks, etc.

With the above said, assuming we are looking at a crash dump that wasn't ours, we can almost entirely rule out this specific 0xF4 being caused by a buggy driver (MSFT or 3rd party). To be extra sure, double-check the modules list and see if there's anything that jumps out as problematic.

--------------------

At this point I would start becoming suspicious of a rootkit, as I would not right away suggest the hard disk (whether HDD or SSD) is the immediate problem given we'd probably see obvious NT_STATUS codes for that. For example, possibly 0xc0000006. Either that, or an entirely different bug check (perhaps 0x7A). With that said, now we get to have some fun!

There are many ways to go about detecting a rootkit hooking the SSDT, and we will discuss extensions/scripts for the moment:

1st Method - SwishDbgExt

The first method we will be using in this postmortem debugging example is the wonderful SwishDbgExt, which was developed/created by a friend of mine (Matt Suiche). I've made various contributions to the help file considering the love I have gathered for this extension. It makes a lot of our lives as debuggers much easier.

Once you have the extension loaded, we're going to be using the !ms_ssdt command. This command displays the System Service Dispatch Table, which is extremely helpful in the investigation of suspected rootkit hooks through using what is known as Direct Kernel Object Manipulation (DKOM).

-- Chopping some of the SSDT output as it's fairly large, let's skip to what's important:

   |-------|--------------------|--------------------------------------------------------|---------|--------|  
   | Index | Address      | Name                          | Patched | Hooked |  
   |-------|--------------------|--------------------------------------------------------|---------|--------|  
  *** ERROR: Module load completed but symbols could not be loaded for Gjglly.sys  
   |  126 | 0xFFFFFFFF91F054C2 | Gjglly                         |     |    |  
   |  127 | 0xFFFFFFFF81A34F80 | nt!NtDeviceIoControlFile                |     |    |  
   |  128 | 0xFFFFFFFF81949B44 | nt!NtDisplayString                   |     |    |  
   |  129 | 0xFFFFFFFF81A2117F | nt!NtDuplicateObject                  |     |    |  
   |  130 | 0xFFFFFFFF81A18134 | nt!NtDuplicateToken                  |     |    |  
   |  131 | 0xFFFFFFFF81AB14E8 | nt!NtEnumerateBootEntries               |     |    |  
   |  132 | 0xFFFFFFFF81AB278A | nt!NtEnumerateDriverEntries              |     |    |  
   |  133 | 0xFFFFFFFF91F04FFA | Gjglly                         |     |    |  
   |  134 | 0xFFFFFFFF81AB10B7 | nt!NtEnumerateSystemEnvironmentValuesEx        |     |    |  
   |  135 | 0xFFFFFFFF81A9F073 | nt!NtEnumerateTransactionObject            |     |    |  
   |  136 | 0xFFFFFFFF91F051B6 | Gjglly                         |     |    |  
   |  137 | 0xFFFFFFFF81A802D5 | nt!NtExtendSection                   |     |    |  
   |  138 | 0xFFFFFFFF819A113A | nt!NtFilterToken                    |     |    |  
   |  139 | 0xFFFFFFFF819B39FC | nt!NtFindAtom                     |     |    |  
   |  140 | 0xFFFFFFFF819DCA86 | nt!NtFlushBuffersFile                 |     |    |  
   |  141 | 0xFFFFFFFF819AD0F6 | nt!NtFlushInstructionCache               |     |    |  
   |  142 | 0xFFFFFFFF819781EB | nt!NtFlushKey                     |     |    |  
   |  143 | 0xFFFFFFFF818B11C1 | nt!NtFlushProcessWriteBuffers             |     |    |  
   |  144 | 0xFFFFFFFF819C175B | nt!NtFlushVirtualMemory                |     |    |  
   |  145 | 0xFFFFFFFF81A82D64 | nt!NtFlushWriteBuffer                 |     | Yes  |  

We can see a module (Gjglly.sys), and nt!NtFlushWriteBuffer is hooked. Let's not jump to conclusions just yet as this could be a completely legitimate hook.

First of all, what's Gjglly.sys? This is a driver in relation to AntiDebugLIB.
An advanced software encryption tool.

AntiDebugLIB is a useful tool that was designed in order to assist software developers protect their applications against advanced reverse engineering and software cracking. It offers a powerful advanced license control which allow developers to distribute trial versions of their applications securely.
This is a legitimate driver, but we can also at the same time not jump to conclusions on it being safe as said above (hint: it's not - I will discuss later).

2nd Method - Script

The second method we will be using in this postmortem debugging example is an older script developed by Lionel d'Hauenens of Laboskopia. This script will only work with the x86 WinDbg client. Regardless, as we know, it's always best to debug a crash dump in the client based off of the system's architecture. Given the system that generated this crash dump was 32-bit, we'll be debugging it with the x86 WinDbg client.

This script is incredibly helpful in not only checking the SSDT for hooking, but also detecting what is known as a Shadow SSDT hook. You can find a great article on Shadow SSDT hooking from my very good friend Harry - Shadow SSDT Hooking with Windbg.

Nevertheless, if you don't understand French instructions, to install the script, simply drag and drop the script folder into your x86 Debuggers folder.

Once it's installed, type the following command into WinDbg with your postmortem rootkit crash dump:

 kd> $$><script\@@init_cmd.wdbg  

So long as you installed the script properly, you should see the following:

 SysecLabs Windbg Script : Ok :)  
 ('al' for display all commands)  

Once you see that which has verified the script is loaded properly, type the following command:

 !!display_system_call  

Here's a few excerpts:

 007D  0003  OK  nt!NtDeleteObjectAuditAlarm (81a46500)  
 007E  0002 HOOK-> *** ERROR: Module load completed but symbols could not be loaded for Gjglly.sys  
 007F  000A  OK  nt!NtDeviceIoControlFile (81a34f80)  
 0080  0001  OK  nt!NtDisplayString (81949b44)  
 0081  0007  OK  nt!NtDuplicateObject (81a2117f)  
 0082  0006  OK  nt!NtDuplicateToken (81a18134)  
 0083  0002  OK  nt!NtEnumerateBootEntries (81ab14e8)  
 0084  0002  OK  nt!NtEnumerateDriverEntries (81ab278a)  
 0085  0006 HOOK-> Gjglly+0x1ffa (91f04ffa)  
 0086  0003  OK  nt!NtEnumerateSystemEnvironmentValuesEx (81ab10b7)  
 0087  0005  OK  nt!NtEnumerateTransactionObject (81a9f073)  
 0088  0006 HOOK-> Gjglly+0x21b6 (91f051b6)  

 00BC  0003  OK  nt!NtOpenJobObject (81a912df)  
 00BD  0003 HOOK-> Gjglly+0x1f4c (91f04f4c)  
 00BE  0004  OK  nt!NtOpenKeyTransacted (819727e1)  

 0143  0001  OK  nt!NtSetUuidSeed (8195977b)  
 0144  0006 HOOK-> Gjglly+0x2372 (91f05372)  
 0145  0005  OK  nt!NtSetVolumeInformationFile (81a6c5de)  

Here's Gjglly.sys again, and we can see it's hooking. At this point, we would be suspicious enough to run an ARK tool such as GMER. Of course, if we ran GMER at this point, it would show that the system is in fact infected with runtime2, and Gjglly.sys is our rootkit driver. Normally, by itself and by default, the runtime2 driver is runtime2.sys, not Gjglly.sys. In this specific scenario, niemiro used a different loader to inject the driver than the original, and renamed runtime2.sys to Gjglly.sys (a legitimate module name). Although GMER if run would still say the system is infected with a rootkit, and it would label Gjglly.sys as the rootkit driver, it's done not to trick various ARK tools, but the user.

If you've ever been infected with runtime2, you know it additionally drops startdrv.exe in the Temp directory of the current Windows install. startdrv.exe is the part that infuriates those who become infected with runtime2, because it's the part that immediately makes you suspicious of an infection as it's a trojan dropper. This is the specific part of the rootkit that will cause slowness, garbage popups, etc.

In this specific case, niemiro intentionally corrupted startdrv.exe which stopped it from executing entirely. You may be saying to yourself, what's the point of a rootkit's protection driver with no execution of payload, etc, by its dropped trojan? Well, there really is no point (unless your goal was to use something else that's malicious and not as obvious as a trojan dropper)! This was just an example to show how you can better hide a rootkit (protection driver, really) if you wanted to. Although ARK tools would still pick it up, it's not anywhere near as obvious to the user.

Thanks for reading, and I will get around to doing a live debugging of runtime2 as soon as I can.

--------------------

References

Hunting rootkits with Windbg.
Some notes about System Service Dispatch Table hook.