Subclassing windows

[donkey]

Subclassing is a powerful tool in the kit of any Windows programmer, it allows you to modify the behavior of a control by intercepting and processing messages sent to it. Essentially you are injecting your own message handler between the message pump and the Windows default handler for that control. The old way to set up a subclass was to change the GWL_WNDPROC field of the window using SetWindowLong. However that required that you saved the old Wndproc address in some way and called it when your subclassing procedure was done processing the message. This was either done by storing it in a global variable or storing it in the GWL_USERDATA field of the window. Both have their disadvantages, using a global variable makes your program all the more vulnerable to buffer overflow exploits, and using GWL_USERDATA takes that rather useful field away. Here's an example of the Get/SetWindowLong method of subclassing:

Code Select Expand

invoke GetDlgItem,[hwnd],IDC_COMBOBOX
// Store the handle in a nonvolatile register
mov ebx,eax
// get the old (default) window proc
invoke GetWindowLong,ebx,GWL_WNDPROC
// Store this address in GWL_USERDATA
invoke SetWindowLong,ebx,GWL_USERDATA,eax
// Set the window proc to our subclass procedure
invoke SetWindowLong,ebx,GWL_WNDPROC,offset ComboEditSubClass

Code Select Expand

ComboEditSubClass FRAME hwnd, uMsg, wParam, lParam

// The only thing we're interested in is if there
// has been any typing in the ComboBoxes edit control
cmp D[uMsg],WM_COMMAND
jne >>.DEFPROC
mov ecx,[wParam]
shr ecx,16
movzx edx,W[wParam]
cmp ecx,EN_CHANGE
jne >>.DEFPROC
// Need the handle of the main window
// GetParent will only get the handle of the
// ComboBox so we use GetAncestor to get the
// root window.
invoke GetAncestor, [hwnd], GA_ROOT
invoke PostMessage, eax, [uMsg], [wParam], [lParam]
.DEFPROC

invoke GetWindowLong,[hwnd],GWL_USERDATA
invoke CallWindowProc,eax,[hwnd], [uMsg], [wParam], [lParam]
ret
endf

Not bad but sloppy and using a nice little storage place that you can directly attach to a window, a convenient little thing that shouldn't be so easily discarded.

But beginning with XP there is another way, using SetWindowSubclass / DefSubclassProc, at least MSDN says it starts with Windows XP, it actually started way back in Windows 98 and using the CCUSEORDINALS switch in the headers you can use it on Win98 systems. Here's an example of how it works:

The set up:

First we need a handle for the control, GetDlgItem will provide one. We pass that to the SetWindowSubclass API along with the address of the Subclass procedure. The last two parameters are the unique ID for the subclass and an application defined DWORD value (dwRefData) that will be passed to the Subclass procedure. In this case since this is the only subclass we'll use an ID of 0 and we have no special information to pass to the procedure so we'll set that to 0 as well.

Code Select Expand

// We need to know if the edit control in the combo box
// has had its text changed. Subclassing it allows us to
// relay the message to the main window
invoke GetDlgItem,[hwnd],IDC_COMBOBOX
// Since we are supporting XP or greater we'll use SetWindowSubclass
// however, it is polite to support older OS versions so we'll define
// CCUSEORDINALS, that will allow OS versions down to Comctl32 v4.72 (Win98)
invoke SetWindowSubclass,eax,offset ComboEditSubClass,0,0

Now the sublcass procedure. This has changed a bit from the old GWL_WNDPROC procedure that was simply a normal WndProc. We have to add the ID and reference data (application defined DWORD) to the FRAME since it is now expecting 6 parameters. In general you can ignore the ID and RefData. The major change is that you no longer need to keep track of the old window procedure for the particular control, the sublclass does that for you. You only have to call DefSubclassProc with the normal parameters and Windows takes care of the rest for you. You can also chain subclass procedures, though I haven't tried this it does seem kind of interesting.

Code Select Expand

ComboEditSubClass FRAME hwnd, uMsg, wParam, lParam, uIdSubclass, dwRefData

// The only thing we're interested in is if there
// has been any typing in the ComboBoxes edit control
cmp D[uMsg],WM_COMMAND
jne >>.DEFPROC
mov ecx,[wParam]
shr ecx,16
movzx edx,W[wParam]
cmp ecx,EN_CHANGE
jne >>.DEFPROC
// Need the handle of the main window
// GetParent will only get the handle of the
// ComboBox so we use GetAncestor to get the
// root window.
invoke GetAncestor, [hwnd], GA_ROOT
invoke PostMessage, eax, [uMsg], [wParam], [lParam]
.DEFPROC
// Call the default handler for this control
invoke DefSubclassProc, [hwnd], [uMsg], [wParam], [lParam]
ret
endf

All in all a much cleaner way to deal with subclassing.

Now, if you're a MASM user and reading this first let me say wrong forum :) Actually you can use it directly starting with XP and if you need to use it as an ordinal import you will have to do it via LoadLibrary (comctl32.dll) and GetProcAddress (SetWindowSubclass = Ordinal 410, DefSubclassProc Ordinal 413).

Code Select Expand

;MASM
invoke LoadLibrary, ADDR szComCtl32
mov hComCtrl, eax
invoke GetProcAddress, hComCtrl, 410
mov pSetWindowSubclass, eax
invoke GetProcAddress, hComCtrl, 413
mov pDefSubclassProc , eax

; Using push/push/call because prototypes suck...

push 0
push 0
push offset ComboEditSubClass
call pSetWindowSubclass

push lParam
push wParam
push uMsg
push hwnd
call pDefSubclassProc

Ofcourse with GoAsm the LoadLibrary/GetProcAddress method is not necessary, the simple CCUSEORDINALS switch will do all the work for you and call the ordinal version rather than the named function.

Edgar

[Vortex]

Hi donkey,

Thanks for the info. Japheth proposed a method to export functions by ordinal in Masm :

http://www.masm32.com/board/index.php?topic=5838.msg43572#msg43572

[donkey]

Hi donkey,

Thanks for the info. Japheth proposed a method to export functions by ordinal in Masm :

http://www.masm32.com/board/index.php?topic=5838.msg43572#msg43572

Hi Vortex,

But how would you import them without LoadLibrary/GetProcAddress ? Not sure if there's an easy way to do that in MASM, can you build an import library using NONAME ?

Edgar

[Vortex]

Hi donkey,

Here is how it works :

Create first the .def file. No need to decorate all the functions. DefSubclassProc could be also specified without NONAME below :

Code Select Expand

LIBRARY comctl32
EXPORTS

AddMRUStringW
CreateMRUListW
.
.
"_DefSubclassProc@16" @413 NONAME
.
.
"_SetWindowSubclass@16" @410 NONAME
.
.

Build the import library :

Code Select Expand

\masm32\bin\polib /OUT:comctl32.lib /DEF:comctl32.def /MACHINE:x86

An example project. The exit box is subclassed to make it passive :

Code Select Expand



.386
.model flat,stdcall
option casemap:none

include    \masm32\include\windows.inc
include    \masm32\include\kernel32.inc
include    \masm32\include\user32.inc
include    \masm32\include\gdi32.inc

includelib \masm32\lib\kernel32.lib
includelib \masm32\lib\user32.lib
includelib \masm32\lib\gdi32.lib
includelib comctl32.lib

DlgProc PROTO :DWORD,:DWORD,:DWORD,:DWORD
EditBox PROTO :DWORD,:DWORD,:DWORD,:DWORD,:DWORD,:DWORD

SetWindowSubclass PROTO :DWORD,:DWORD,:DWORD,:DWORD
DefSubclassProc   PROTO :DWORD,:DWORD,:DWORD,:DWORD

IDC_EDIT = 4001


.data
Resource    db 'MYDIALOG',0

.code

start:

    invoke  GetModuleHandle,0
    invoke  DialogBoxParam,eax,ADDR Resource,NULL,ADDR DlgProc,0
    invoke  ExitProcess,eax

DlgProc PROC hWnd:HWND,uMsg:UINT,wParam:WPARAM,lParam:LPARAM

    .IF uMsg==WM_INITDIALOG
   
        invoke  GetDlgItem,hWnd,IDC_EDIT
        invoke  SetWindowSubclass,eax,ADDR EditBox,0,0

    .ELSEIF uMsg==WM_CLOSE

        invoke  EndDialog,hWnd,NULL

    .ELSE

        mov     eax,FALSE
        ret

    .ENDIF

    mov     eax,TRUE
    ret

DlgProc ENDP

EditBox PROC hWnd:HWND,uMsg:UINT,wParam:WPARAM,lParam:LPARAM,uIdSubClass:DWORD,dwRefData:DWORD

    .IF uMsg == WM_SETFOCUS || uMsg == WM_CONTEXTMENU || uMsg == WM_CHAR

        xor     eax,eax
        ret

    .ENDIF

    invoke  DefSubclassProc,hWnd,uMsg,wParam,lParam
    ret

EditBox ENDP

END start


Here is the Ollydbg and dumpbin.exe report :

Code Select Expand


004010C0 JMP DWORD PTR DS:[<&comctl32.#410>] ; comctl32.SetWindowSubclass
004010C6 JMP DWORD PTR DS:[<&comctl32.#413>] ; comctl32.DefSubclassProc


Code Select Expand

comctl32.dll
             4020A0 Import Address Table
             402078 Import Name Table
                  0 time date stamp
                  0 Index of first forwarder reference

                   Ordinal   410
                   Ordinal   413

[xandaz]

   This is new to me Donkey. Nice. I guess i'll try using that. Thanks and bye

[xandaz]

    well... i tried SetWindowSubclass and DefSubclassProc but errors on me.I guess i'll just use the old method. You wouldnt know why a subclasses toolbar wouldnt respond to WM_COMMAND messages would you? thanks

[donkey]

WM_COMMAND is a notification sent to the parent so it would not be sent to subclassed control.