Skypher

Quoting http://msdn.microsoft.com/en-us/library/aa930622.aspx:
typedef struct tagBITMAPINFOHEADER {
DWORD biSize;
LONG biWidth;
LONG biHeight;
WORD biPlanes;
WORD biBitCount
DWORD biCompression;
DWORD biSizeImage;
LONG biXPelsPerMeter;
LONG biYPelsPerMeter;
DWORD biClrUsed;
DWORD biClrImportant;
} BITMAPINFOHEADER;

“If the bitmap is a packed bitmap (a bitmap in which the bitmap array immediately follows the BITMAPINFO header and is referenced by a single pointer), the biClrUsed member must be either zero or the actual size of the color table.”

ANI files stores each frame of the animated cursor as a packed bitmap inside the ANI file. The DWORD biClrUsed member of the BITMAPINFOHEADER of each such bitmap can be used to cause the Windows API functions to allocate any number of bytes and attempt to copy a similarly large ammount of data from the file in memory to the newly allocated memory. The code does not check that such a large ammount of data is available and this can cause the copy operation to read beyong the bounds of the memory allocated for the ANI file. This problem affects Windows XP and Vista but not the newer Windows 7.

This can cause a read access violation if a small ANI file specifies a large value for biClrUsed: the memory copy operation runs beyond the memory allocated for the file data and into unallocated memory. MSIE is an example of an application that uses these Windows APIs to read ANI files, it is also the application that this problem was first detect in. However, there could be other applications that use these API functions.

In MSIE, this read AV is handled by an exception handler, so it does not cause the browser to crash. The read access violation is therefore only visible if a debugger is attached to MSIE. However, this problem can also be used to cause the application to allocate excessive amounts of memory and become unresponsive, using 100% CPU. For MSIE, this seems to affect the x86 version only and not the x64 version. I cannot explain, not have I investigated the exact cause of this difference between these two versions.

Microsoft has confirmed the issue and based on the impact has decided to fix this in Service Packs for the various versions of Windows affected. They cannot provide release estimates for these Service Packs at this time. If you maintain an application that depends on the Windows APIs to load ANI files, you may want to investigate how this issue affects your application and do additional checks on the contents of ANI files before calling vulnerable Windows API functions.

Full details, including case history and repro for this particular bug can be found here.