Annoyance: toolbar & mouse cursors

[NoCforMe]

I have an annoying little problem. I have a toolbar that changes the mouse cursor according to what button is selected. That part works fine. (The tool bar is the only child so far of my main window, and it's horizontal at the top.)

The problem is that my special selected cursors show even when the mouse is over the toolbar itself and the non-client area above it (the main window's title bar). I'd like them to revert to the standard arrow anytime it's outside the client area. (I'm using SetCursor() to set them.)

Do I need to subclass the toolbar control and explicitly set the cursor to the standard system one when it's inside the control? or is there some other way to solve this?

[jj2007]

Yes. No. Maybe ::)

With so little of your code visible to us there are too many possibilities as to what is wrong with it.

[NoCforMe]

You have to understand that this is in what has become a somewhat messy and not-easy-to-follow program (at least to someone else other than me). I'll try to post the relevant extracts (mixed pseudocode and real code):

Code Select Expand



o register window class
o create main window:

INVOKE CreateWindowEx, WS_EX_OVERLAPPEDWINDOW, ADDR MainClassName, ADDR MainTitleText,
$mainWinAttrs, wX, wY, $mainWindowWidth, $mainWindowHeight,
NULL, NULL, hInst, NULL
MOV MainWinHandle, EAX

o message loop, etc.

===== Main window proc: =====

case WM_CREATE:

; Create toolbar window:
INVOKE CreateWindowEx, WS_EX_OVERLAPPEDWINDOW, ADDR ToolbarClassName, NULL,
$toolbarAttrs, 0, 0, $toolbarWidth, $toolbarHeight,
hWin, NULL, InstanceHandle, NULL
MOV ToolbarHandle, EAX

; INVOKE SetClassLong, ToolbarHandle, GCL_HCURSOR, NULL ;Doesn't help ...

; Set up toolbar:
; Set size of TBUTTON structure:
INVOKE SendMessage, ToolbarHandle, TB_BUTTONSTRUCTSIZE, SIZEOF TBBUTTON, 0
; Set size of  bitmap:
INVOKE SendMessage, ToolbarHandle, TB_SETBITMAPSIZE, 0, $buttonWidth OR ($buttonHeight SHL 16)
; Add bitmap:
MOV EAX, InstanceHandle
MOV ToolbarAddBitmapStruct.hInst, EAX
INVOKE SendMessage, ToolbarHandle, TB_ADDBITMAP, $numToolbarButtons, OFFSET ToolbarAddBitmapStruct
; Add buttons:
INVOKE SendMessage, ToolbarHandle, TB_ADDBUTTONS, $numToolbarButtons, OFFSET ToolbarButtonStructs
; Set button size:
INVOKE SendMessage, ToolbarHandle, TB_SETBUTTONSIZE, 0, $buttonWidth OR ($buttonHeight SHL 16)

; Now set up tooltip stuff:
INVOKE SendMessage, ToolbarHandle, TB_SETEXTENDEDSTYLE, 0, TBSTYLE_EX_MIXEDBUTTONS
XOR EAX, EAX
RET

case WM_SETCURSOR:
CMP EditingCursor, NULL
JE dodefault
INVOKE SetCursor, EditingCursor
MOV EAX, TRUE ;Halt further processing of msg.
RET

case WM_NOTIFY:
MOV EDX, lParam ;--> NMTTDISPINFO struct.
CMP [EDX + NMHDR.code], TTN_NEEDTEXT ;aka TTN_GETDISPINFO
JNE dodefault
MOV EAX, [EDX + NMHDR.idFrom] ;Could be the ID of one of our controls.
CMP EAX, $TBBwholeNote
JB dodefault
CMP EAX, $TBBeighthRest
JA dodefault

; At this point, we have a TTN_NEEDTEXT message for one of our buttons:
SUB EAX, $TBBwholeNote ;Get offset from 1st button ID:
SHL EAX, 2 ;convert to DWORD offset
MOV EAX, [EAX + OFFSET ToolTipTextPtrs]
MOV [EDX + NMTTDISPINFO.lpszText], EAX ;Ptr. to tooltip text.
MOV [EDX + NMTTDISPINFO.uFlags], TTF_DI_SETITEM ;Don't bother me again!
RET ;No particular return value needed.



EditingCursor is a handle to a cursor that gets set by various WM_COMMAND receptors. Cursor selection works perfectly.

I think that's pretty much all that's relevant to this problem. If you want to see more, let me know. (I suppose I could attach the whole file.)

Again, this works fine, except that my custom cursors show where I don't want them to.

[jj2007]

Check WM_NCHITTEST.

[NoCforMe]

Check WM_NCHITTEST.

OK, that looks good. But how do I use it?

MSDN says:

The return value of the DefWindowProc function is one of the following values, indicating the position of the cursor hot spot.

So should I call DefWindowProc() with this message and then look at its return value?

Well, I went ahead and tried that:

Code Select Expand



case WM_NCHITTEST:
INVOKE DefWindowProc, hWin, uMsg, wParam, lParam
CMP EAX, HTCLIENT ;Is mouse in client area?
JE ignore ;Yes, ignore.
; Mouse in a non-client area: see if we have one of our editing cursors active:
CMP EditingCursor, NULL
JE ignore
INVOKE LoadCursor, InstanceHandle, IDC_ARROW
INVOKE SetCursor, EAX
XOR EAX, EAX
ignore: RET



But it doesn't work. In fact, it makes things worse: now the cursor not only doesn't change to the standard arrow, but it can't be used to close the application using the "X" button (the cursor is only active in the client area and the toolbar). Besides, that test is wrong, since I also need to determine if I'm within the toolbar and change the cursor there.

I understand why I'm having this annoying problem: it's because the toolbar is within the client area of the parent window, and I'm changing the cursor within the client. Before, when I was using a child window instead, the cursor worked perfectly, since the child window was completely separate from the parent and the toolbar (which was then a sibling window). But of course I had that other problem (still unresolved) of the contents of the child window not appearing until it was resized. So I'm in kind of a bind here ...

[jj2007]

No, it's the other way round: You return a value telling the system which part of your window is beneath the cursor.

  CASE WM_NCHITTEST
   movsx eax, word ptr lParam+2   ; Y
   push eax
   movsx edx, word ptr lParam   ; X
   push edx
   invoke ScreenToClient, hWnd, esp   ; convert to client coords
   pop eax      ; X
   pop edx      ; Y
   .if eax<80      ; cursor will change when you move right
      mov eax, HTCAPTION   ; arbitrary test: click on left side lets you move the
      ret         ; window as if you had clicked into the title bar
   .endif

[dedndave]

WM_SETCURSOR

when you receive this message, you can get the mouse position and select the appropriate cursor
more correctly, see if the cursor needs to be changed

[NoCforMe]

jj: Thanks, but arrrrrgh: how would I even use that message? I'd have to do all kinds of research (getting client rects, etc.) on my windows to determine just where the heck the cursor was! I don't think this is the way to go. (Unless I'm missing something obvious here.) How do you even determine whether the mouse is on the left resize border, or in the minimize button?

I really need to get that child window working correctly, as that would make all of this messing around with the mouse cursor unnecessary.

[jj2007]

What happens if you use my example above but with .if edx<9 ? Meaning "tell the OS it's a caption if the mouse is over the top edge of the toolbar"...

[MichaelW]

I think this at least shows what is happening.

Code Select Expand


;===================================================================================
; Build as a console app.
;===================================================================================
include \masm32\include\masm32rt.inc
;===================================================================================

printf MACRO format:REQ, args:VARARG
    IFNB <args>
        invoke crt_printf, cfm$(format), args
    ELSE
        invoke crt_printf, cfm$(format)
    ENDIF
    EXITM <>
ENDM

;===================================================================================

IDC_TB    equ 101
IDS_NEW   equ 200
IDS_OPEN  equ 201
IDS_SAVE  equ 202
IDS_CUT   equ 203
IDS_COPY  equ 204
IDS_PASTE equ 205

;===================================================================================
.data
    hInstance   HMODULE     0
    hDlg        HWND        0
    hwndTB      HWND        0
    pPrevTBProc WNDPROC     0
    hTT         HANDLE      0
    tbb         TBBUTTON    7 dup(<>)
    iccex       INITCOMMONCONTROLSEX <SIZEOF INITCOMMONCONTROLSEX, ICC_BAR_CLASSES>
    rcTB        RECT        <>
    msg         MSG         <>
    tbab        TBADDBITMAP <>
    pt          POINT       <>
.code
;===================================================================================

SubclassProc proc hwnd:HWND, uMsg:UINT, wParam:WPARAM, lParam:LPARAM

    SWITCH uMsg

      CASE WM_NCHITTEST

        printf("over tool bar\n")

    ENDSW

    invoke CallWindowProc, pPrevTBProc, hwnd, uMsg, wParam, lParam

    ret
SubclassProc endp


;===================================================================================

DialogProc proc hwndDlg:HWND, uMsg:UINT, wParam:WPARAM, lParam:LPARAM

    SWITCH uMsg

      CASE WM_INITDIALOG

        invoke GetDlgItem, hwndDlg, IDC_TB
        mov hwndTB, eax

        ;--------------------------------------------------------
        ; Specify the structure size so the system can determine
        ; which version of the common control DLL is being used.
        ;--------------------------------------------------------

        invoke SendMessage, hwndTB, TB_BUTTONSTRUCTSIZE,
                            SIZEOF TBBUTTON, 0

        ;----------------------------------------------------
        ; Add the system-defined bitmap button images to the
        ; list of available images.
        ;----------------------------------------------------

        mov tbab.hInst, HINST_COMMCTRL
        mov tbab.nID, IDB_STD_LARGE_COLOR
        ;mov tbab.nID, IDB_STD_SMALL_COLOR
        ;mov tbab.nID, IDB_VIEW_LARGE_COLOR
        ;mov tbab.nID, IDB_VIEW_SMALL_COLOR

        invoke SendMessage, hwndTB, TB_ADDBITMAP, 0, ADDR tbab

        ;------------------------------------------------------
        ; For each button, specify the button image index, the
        ; associated command identifier, and the button state
        ; and style. Note that this code depends on the unused
        ; members of the TBBUTTON structure being initialized
        ; to zero.
        ;------------------------------------------------------

        mov tbb[0*SIZEOF TBBUTTON].iBitmap, STD_FILENEW
        mov tbb[0*SIZEOF TBBUTTON].idCommand, IDS_NEW
        mov tbb[0*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[0*SIZEOF TBBUTTON].fsStyle, TBSTYLE_BUTTON
        mov tbb[1*SIZEOF TBBUTTON].iBitmap, STD_FILEOPEN
        mov tbb[1*SIZEOF TBBUTTON].idCommand, IDS_OPEN
        mov tbb[1*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[1*SIZEOF TBBUTTON].fsStyle, TBSTYLE_BUTTON
        mov tbb[2*SIZEOF TBBUTTON].iBitmap, STD_FILESAVE
        mov tbb[2*SIZEOF TBBUTTON].idCommand, IDS_SAVE
        mov tbb[2*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[2*SIZEOF TBBUTTON].fsStyle, TBSTYLE_BUTTON
        mov tbb[3*SIZEOF TBBUTTON].iBitmap, 0
        mov tbb[3*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[3*SIZEOF TBBUTTON].fsStyle, TBSTYLE_SEP
        mov tbb[4*SIZEOF TBBUTTON].iBitmap, STD_CUT
        mov tbb[4*SIZEOF TBBUTTON].idCommand, IDS_CUT
        mov tbb[4*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[4*SIZEOF TBBUTTON].fsStyle, TBSTYLE_BUTTON
        mov tbb[5*SIZEOF TBBUTTON].iBitmap, STD_COPY
        mov tbb[5*SIZEOF TBBUTTON].idCommand, IDS_COPY
        mov tbb[5*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[5*SIZEOF TBBUTTON].fsStyle, TBSTYLE_BUTTON
        mov tbb[6*SIZEOF TBBUTTON].iBitmap, STD_PASTE
        mov tbb[6*SIZEOF TBBUTTON].idCommand, IDS_PASTE
        mov tbb[6*SIZEOF TBBUTTON].fsState, TBSTATE_ENABLED
        mov tbb[6*SIZEOF TBBUTTON].fsStyle, TBSTYLE_BUTTON

        ;---------------------------------
        ; Add the buttons to the toolbar.
        ;---------------------------------

        invoke SendMessage, hwndTB, TB_ADDBUTTONS, 7, ADDR tbb
        invoke SendMessage, hwndTB, TB_AUTOSIZE, 0, 0

        ;------------------------------------------------------------
        ; Get and save the handle to the ToolTip control associated
        ; with the toolbar (see the TBSTYLE_TOOLTIPS documentation).
        ;------------------------------------------------------------

        invoke SendMessage, hwndTB, TB_GETTOOLTIPS, 0, 0
        mov hTT, eax

        ;--------------------------------------------------------
        ; Subclass the TB window so we can monitor its messages.
        ;--------------------------------------------------------

        invoke SetWindowLong, hwndTB, GWL_WNDPROC, SubclassProc
        mov pPrevTBProc, eax

      CASE WM_COMMAND

        movzx eax, WORD PTR wParam

        SWITCH eax
          CASE IDS_NEW
            invoke MessageBox, hwndDlg, chr$("New"), chr$(" ") , 0
          CASE IDS_OPEN
            invoke MessageBox, hwndDlg, chr$("Open"), chr$(" ") , 0
          CASE IDS_SAVE
            invoke MessageBox, hwndDlg, chr$("Save"), chr$(" ") , 0
          CASE IDS_CUT
            invoke MessageBox, hwndDlg, chr$("Cut"), chr$(" ") , 0
          CASE IDS_COPY
            invoke MessageBox, hwndDlg, chr$("Copy"), chr$(" ") , 0
          CASE IDS_PASTE
            invoke MessageBox, hwndDlg, chr$("Paste"), chr$(" ") , 0
          CASE IDCANCEL
            invoke DestroyWindow, hwndDlg
        ENDSW

      CASE WM_NOTIFY

        push ebx
        mov ebx, lParam
        mov eax, hTT
        .IF [ebx].NMHDR.hwndFrom == eax
          .IF [ebx].NMHDR.code == TTN_GETDISPINFO

            ;----------------------------------------------------
            ; We now know that lParam is actually a pointer to a
            ; NMTTDISPINFO structure, and the ToolTip control is
            ; requesting information that it needs to display a
            ; tooltip. Note that the hdr member of the structure
            ; is a NMHDR structure.
            ;----------------------------------------------------

            SWITCH [ebx].NMTTDISPINFO.hdr.idFrom
              CASE IDS_NEW
                mov [ebx].NMTTDISPINFO.lpszText, chr$("New")
              CASE IDS_OPEN
                mov [ebx].NMTTDISPINFO.lpszText, chr$("Open")
              CASE IDS_SAVE
                mov [ebx].NMTTDISPINFO.lpszText, chr$("Save")
              CASE IDS_CUT
                mov [ebx].NMTTDISPINFO.lpszText, chr$("Cut")
              CASE IDS_COPY
                mov [ebx].NMTTDISPINFO.lpszText, chr$("Copy")
              CASE IDS_PASTE
                mov [ebx].NMTTDISPINFO.lpszText, chr$("Paste")
            ENDSW

            ;-------------------------------------------
            ; This causes the ToolTip control to retain
            ; the information after the first request.
            ;-------------------------------------------

            or [ebx].NMTTDISPINFO.uFlags, TTF_DI_SETITEM

          .ENDIF
        .ENDIF
        pop ebx

      CASE WM_SIZE

        invoke GetWindowRect, hwndTB, ADDR rcTB
        mov ecx, rcTB.bottom
        sub ecx, rcTB.top
        inc ecx
        movzx eax, WORD PTR lParam

        invoke MoveWindow, hwndTB, 0, 0, eax, ecx, FALSE

      CASE WM_NCHITTEST

          movsx eax, WORD PTR lParam+2
          mov pt.y, eax
          invoke ScreenToClient, hwndDlg, ADDR pt
          printf("%d\n",pt.y)
          .IF SDWORD PTR pt.y >= 0
            printf("over client area\n")
          .ENDIF

      CASE WM_CLOSE

        invoke DestroyWindow, hwndDlg

      CASE WM_DESTROY

        invoke PostQuitMessage, NULL

    ENDSW

    xor eax, eax
    ret

DialogProc endp

;===================================================================================
start:
;===================================================================================

    invoke InitCommonControlsEx, ADDR iccex

    invoke GetModuleHandle, NULL
    mov hInstance, eax

    Dialog "Test", \
           "MS Sans Serif",10, \
           WS_OVERLAPPEDWINDOW or WS_VISIBLE or DS_CENTER, \
           1,0,0,200,150,1024
    DlgComCtl "ToolbarWindow32",TBSTYLE_TOOLTIPS or TBSTYLE_FLAT,0,0,0,0,IDC_TB

    CallModelessDialog hInstance,0,DialogProc,NULL
    mov hDlg, eax

  msgLoop:

    invoke GetMessage, ADDR msg, 0, 0, 0
    .IF eax != 0
      invoke IsDialogMessage, hDlg, ADDR msg
      .IF eax == 0
        invoke TranslateMessage, ADDR msg
        invoke DispatchMessage, ADDR msg
      .ENDIF
      jmp msgLoop
    .ENDIF

    invoke ExitProcess, eax

;===================================================================================

end start