PageHeap Notes

Most of the corruptions in heap blocks can be discovered in one of two ways:

1. Full-page heap: Placing a non-accessible page at the end of the allocation.
2. Normal page heap: Checking fill patterns when the block gets freed.

Full-Page Heap

Normal Page Heap

To enable system-wide normal page heap

1. Type the following at the command line:
   gflags -r +hpa
2. Restart your computer.

To disable system-wide normal page heap

1. Type the following at the command line:
   gflags -r -hpa
2. Restart your computer.

Note

• No other GFlags settings are useful when you enable page heap. If other settings that appear to be related to the heap are enabled, then page heap bugs can be introduced due to conflicts between page heap manager and these "harmless" heap flags.

The Windows heap managers (all versions) have always guaranteed that the heap allocations have a start address that is 8-byte aligned (on 64-bit platforms the alignment is 16-bytes). The page heap manager makes the same guarantee. This is impossible, however, if you want to have the end-of-the-allocation exactly at the end of a page. The exact end-of-page allocation is needed so that an off-by-one-byte error will trigger a read or write into the non-accessible page and cause an immediate fault.

If the user-requested size for the block is not 8-byte aligned, then page heap cannot meet both the "start address 8-byte aligned" and the "end address page aligned" constraints. The solution is to meet the first constraint and make the start of the block 8-byte aligned. Then use a small fill pattern between the end of the block and the start of the non-accessible page. This fill pattern can be from 0 bytes through 7 bytes in length on 32-bit architectures. The fill pattern is checked upon free.

If immediate fault detection is needed for these allocations that otherwise will have a fill pattern at the end, then make the page heap manager ignore the 8-byte alignment rule, and always align the end of the allocation at a page boundary by using the /unaligned and /full parameters. For more information, see the /unaligned parameter.

Note

• Some applications make assumptions about 8-byte alignment and they will stop working correctly with the /unaligned parameter. Internet Explorer is one such example.

The core full-page heap implementation commits two pages for any allocation smaller than one page. One page is used for the user allocation, and the other is made non-accessible at the end of the buffer.

End-of-buffer overruns can be detected by using a zone of reserved virtual space, instead of a non-accessible committed page. An access violation exception occurs when the process touches that reserved virtual space. This approach can reduce memory consumption by up to 50 percent. For more information, see the /decommit parameter.

You can control page heap manager so that some allocations are deliberately failed. This is useful in simulating low memory conditions without actually using all system resources.

Specify a number from 1 through 10000 to represent the probability that an allocation will fail. Using a probability of 10000 ensures that 100 percent of allocations will fail. A probability of 2000 specifies that approximately 20 percent of allocations will fail.

The page heap manager takes special care to avoid fault injection in both the first five seconds of the process’ life, and the Windows NT loader code paths (for exampole, LoadLibrary, FreeLibrary). If five seconds is not sufficient to allow your process to complete startup, then ou can specify a longer timeout at the beginning of the process. For more information, see the /fault parameter.

When you use the /fault parameter and the process being tested has a bug, an exception will be raised. Generally, the reason for this is that the allocate operation returned a NULL, and the application later tries to access the allocated memory. Since the allocate failed, however, the memory cannot be accessed, and therefore an access violation occurs.

The other reason that an exception is raised is that the application tries to deal with the allocation failure, but does not release some resources. This manifests as a memory leak and is more difficult to debug.

In order to help diagnose these failures, the page heap manager keeps a history of stack traces from the moment of fault injection. These traces can be displayed with the following debugger command:

!heap –p –f [NUMBER-OF-TRACES]

By default the extension will display only the last four traces.

Some applications are difficult to start from a command prompt, or they are spawned from other processes. For these applications, specify that, whenever they are started, a debugger will automatically be attached to them. This is useful if page heap is enabled for that process and heap failures occur. For more information, see the /debug parameter.

PageHeap is effective when used to verify any memory allocation process, including C++ style allocations new and delete, as long as the custom allocation/free functions eventually call into NT heap management interfaces (that is, RtlAllocateHeap, RtlFreeHeap). The following functions are guaranteed to do that:

• Functions like HeapAlloc, HeapFree, HeapReAlloc: These functions are exported by kernel32.dll and call directly into the NT heap interfaces.
• Functions like GlobalAlloc, GlobalFree, GlobalReAlloc: These functions are exported by kernel32.dll and call either directly or indirectly into the NT heap interfaces.
• Functions like LocalAlloc, LocalFree, LocalReAlloc: These functions are exported by kernel32.dll and call either directly or indirectly into the NT heap interfaces.
• Functions malloc, free, realloc, msize, expand: These functions are exported by msvcrt.dll and call either directly or indirectly into NT heap. This has not always been the case. The C run-time used to have a different heap implementation, but the current C run-time calls directly into NT heap.
• Operators new, delete, new[ ] , delete[ ] : These functions are exported by msvcrt.dll and call either directly or indirectly into NT heap.

• Any other allocation/free set of functions probably is a custom scheme and is not guaranteed to call either directly or indirectly into NT heap. Only source code inspection or running under debugger can reveal the actual implementation.

  Avoid using static linking. Some applications have been statically linked to old versions of the C runtime. These old versions do not call NT heap APIs, and PageHeap cannot be used to verify these allocations. Dynamic linking ensures that you get the latest C runtime library (msvcrt.dll).

PageHeap detects most heap-related bugs; however, it is focused on heap corruption problems and not focused on leaks. PageHeap has limited success with finding heap leaks, although it has functionality to target this.

One of the advantages of PageHeap is that many errors are detected when they happen. For example, an off-by-one-byte error at the end of a dynamically allocated buffer might cause an instant access violation. There are few types of errors that cannot be detected when they occur. In those cases, the error report is delayed until the block is freed.

• Invalid heap pointer: All Win32 and Windows NT level heap interfaces take as a first parameter a pointer to the heap where the operation should happen. The page heap manager detects an invalid heap pointer at the moment the call is made.
• Invalid heap block pointer: After a block is allocated, it can be used as a parameter for several heap interfaces, notably the free() class of interfaces. The page heap manager immediately detects an invalid heap block pointer. See Debugging Page Heap Failures for a way to determine if the invalid address is a few bytes off, or is completely incorrect.
• Multithreaded unsynchronized access to the heap: Some applications call into a heap from multiple threads. This type of scenario requires setting a flag (by the user) which will trigger acquiring a heap lock. The page heap manager will detect this type of violation when two threads attempt to call simultaneously into the heap.
• Assumptions about reallocation of a block at the same address: A reallocation operation is not guaranteed to return the same address. This is especially true when the reallocation reduces the size of the block. Some applications assume that reallocation will return the same address. The page heap manager always allocates a new block during a reallocation and frees the old block. The free block is protected for read/write access, and therefore any access to it will raise an access violation.
• Double free: This bug, where the same heap blocks are freed several times, is common in some applications. This is detected immediately by the page heap manager because, on the second free, the block will not have the proper prefix header and cannot be found among the allocated blocks. See Debugging Page Heap Failures for ways to analyze the stack trace of the first free operation. This error can be a variant of the reallocation problem because, when the application frees what it thinks it is the address of the block, that block was already freed as part of the reallocation.
• Access of block after free: Freed memory blocks are kept for a short time by the page heap manager in a pool of protected memory. Any access to those blocks will raise an access violation. Based on the "locality" principle, most of the problems should be caught if the free protected pool is large enough. If the freed block is still in the protected pool, the bug is caught instantly. However, if the memory was reused, then there is less chance of finding the bug or identifying the code that caused it.
• Access after the end of allocated block: The page heap manager places an inaccessible page immediately following the allocated block. Any access past the end of the block will raise an access violation. Some applications expect allocations to be 8-byte aligned. This feature has been supported since Windows NT 3.5 heap managers. A request size that is not 8-byte aligned will still get an 8-byte aligned address, but this leaves a few bytes after the end of the block that are still accessible. If the application corrupts only those few bytes, then the error will be caught only by checking the block suffix pattern when the block is freed.
• Access before the start of allocated block: The page heap manager can be instructed through a settable flag to place the inaccessible page at the beginning of the block, rather than at the end. Any access before the start of the block will raise an access violation.

With full-page heap enabled for end-of-buffer-overrun bug detection, an application will access violate at the moment it tries to access memory past the end of the page. These failures are easy to debug because the current stack trace points directly to the failing code.

If normal page heap is used, or if the corruption happens in the small fill pattern at the end of the buffer for alignment reasons, then the corruption will be detected only when the block is freed. In these cases, additional debugging is required to identify the problem.

The Structure of a Page Heap Block

The page heap manager places a header at the beginning of all allocations, both full and normal. This header contains valuable information: the owning heap, user requested size, and in some cases, stack trace for the allocation.

The information block has the following structure:

DPH_BLOCK_INFORMATION

ULONG			 StartStamp;
PVOID		 Heap;
SIZE_T		 RequestedSize;
SIZE_T		 ActualSize;
LIST_ENTRY	FreeQueue;
PVOID		 StackTrace;
ULONG		 EndStamp;

The Heap field stores the owning heap. The user-requested size for the block is in RequestedSize. The stack can be obtained by issuing a dds debugger command at StackTrace address. The best way to get a dump of this structure is to use the following dd debugger command:

dd ADDRESS L20

Or if you have the latest NT debugger package and PDB symbols for ntdll.dll, then use the following:

dt ntdll!_DPH_BLOCK_INFORMATION ADDRESS

The StackTrace field does not always contain a non-null value. Stack trace detection is only supported on x86 platforms, but even on x86 machines the stack trace detection algorithms are not completely reliable. If the block is an allocated block, then the stack trace is for the allocation moment. If the block was freed, then the stack trace is for the free moment.

The Page Heap Debugger Extension

Typical Debug Scenarios