Azimut Project

[jdoe]

Instead of opening multiple topics for the same goal, I thought that one should be enough.

I'm working on a Windows SDK/DDK translation for MASM (Prototypes/Constants/Structures) and I need from time to time few answers that some of them were already gotten elsewhere on this board.

I have now all the set of SDK/DKK (2000/XP/2003/Vista/2008) and what I'll have to offer when completed is all of them merged together.

At this day, I have all the prototypes for the DLLs (*.lib and *.inc). The include files, as an extra feature that I can't live without, is taking care of the UNICODE definition to make it easier to switch from ANSI to UNICODE.

I'll be thankful to those taking time to give me answers and in case some of them may seem obvious, believe me, they are not for someone that is not a C programmer. I can't expect my deduction to be flawless.

Regards

------------------------------------------------------------------------

Version 0.3
Import libraries and prototypes
Resource Header File
Unicode support (TCHAR and String Macro)

http://pages.videotron.com/jdoe/index.html

[jdoe]

There are structures that needs to be aligned. The program build correctly but fails at run-time if the alignment is not corrected.

I know one of these cases that I came across when coding a COM shell extension. It is FORMATETC that need to be DWORD aligned.

Code Select Expand



FORMATETC STRUCT DWORD
   cfFormat                   WORD ?
   ptd                        DWORD ?
   dwAspect                   DWORD ?
   lindex                     DWORD ?
   tymed                      DWORD ?
FORMATETC ENDS




Do you know more of these structures and the alignment required ?

[GregL]

jdoe,

You see where the problem is right? If DWORD alignment is not specified the cfFormat field will throw off the alignment for the rest of the fields. If DWORD alignment is specified MASM will insert (2) padding bytes to restore the alignment of the rest of the fields. A C compiler does this for you, so C programmers  usually don't have to deal with it. Intel x86 processors will read misaligned data but will have substantial performance losses. Some CPUs will not read misaligned data.

Read - The MASM Programmer's Guide - Chapter 5 - Defining and Using Complex Data Types - Alignment Value and Offsets for Structures.

Here's another article  from MSDN.

The rule of thumb is:

  REAL10 should be 16 byte aligned
  REAL8 and QWORD should be 8 byte aligned
  REAL4 and DWORD should be 4 byte aligned
  WORD should be 2 byte aligned
  BYTE doesn't need any alignment

[jdoe]

You see where the problem is right? 

Yes, but what I don't understand is why in most case misalignment don't fails and in particular case it does ?

I'll look at the link you post for some clues. Thanks.

[GregL]

jdoe,

Oh ..., sorry, I thought you were asking how to determine which structures needed alignment adjustments. Well, I bet it's a pointer problem.

[jdoe]

Oh ... , I thought you were asking how to determine which structures needed alignment. Well, I bet it's a pointer problem.

You were not mistaken but you'll see my point with this example.

Code Select Expand



URL_COMPONENTSA STRUCT
    dwStructSize          DWORD ?
    lpszScheme            DWORD ?
    dwSchemeLength        DWORD ?
    nScheme               DWORD ?
    lpszHostName          DWORD ?
    dwHostNameLength      DWORD ?
    nPort                 WORD ?
    lpszUserName          DWORD ?
    dwUserNameLength      DWORD ?
    lpszPassword          DWORD ?
    dwPasswordLength      DWORD ?
    lpszUrlPath           DWORD ?
    dwUrlPathLength       DWORD ?
    lpszExtraInfo         DWORD ?
    dwExtraInfoLength     DWORD ?
URL_COMPONENTSA ENDS




nPort make the rest to not be aligned but it will not fails. Why ?

[GregL]

jdoe,

Without seeing the code, my only guess is it that has something to do with a pointer to the FORMATETC structure, as the structure data is actually changing memory location with the alignment.

Wait, FORMATETC is failing when it's not DWORD aligned, ..., I don't know.

[MichaelW]

It fails in my tests.

Code Select Expand


; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    include \masm32\include\masm32rt.inc
    include \masm32\macros\ucmacros.asm
    includelib winhttp.lib

    WinHttpCreateUrl PROTO :DWORD,:DWORD,:DWORD,:DWORD

    WinHttpCrackUrl  PROTO :DWORD,:DWORD,:DWORD,:DWORD

    _URL_COMPONENTSA STRUCT DWORD
      dwStructSize       dd ?
      lpszScheme         dd ?
      dwSchemeLength     dd ?
      nScheme            dd ?
      lpszHostName       dd ?
      dwHostNameLength   dd ?
      nPort              dw ?
      lpszUserName       dd ?
      dwUserNameLength   dd ?
      lpszPassword       dd ?
      dwPasswordLength   dd ?
      lpszUrlPath        dd ?
      dwUrlPathLength    dd ?
      lpszExtraInfo      dd ?
      dwExtraInfoLength  dd ?
    _URL_COMPONENTSA ENDS
   
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    .data
   
      WSTR url1,"http://www.masm32.com/board/index.php?topic=7866.msg57747#msg57747"

      urlComp   _URL_COMPONENTSA <>
      url2      dw 100 dup(0)
      urlLen    dd 0     
    .code
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
start:
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    invoke RtlZeroMemory, ADDR urlComp, SIZEOF urlComp
    mov urlComp.dwStructSize,     SIZEOF urlComp
    mov urlComp.dwSchemeLength,   -1
    mov urlComp.dwHostNameLength, -1
    mov urlComp.dwUrlPathLength,  -1
    mov urlComp.dwExtraInfoLength,-1

    invoke crt_wcslen, ADDR url1
    mov urlLen, eax
    print ustr$(eax),13,10
    invoke crt_printf, chr$("%S%c%c"), ADDR url1, "<", 10

    inc urlLen          ; must be set to acceptable length

    invoke WinHttpCrackUrl, ADDR url1, urlLen, 0, ADDR urlComp
    print ustr$(eax),13,10

    invoke WinHttpCreateUrl, ADDR urlComp, 0, ADDR url2, ADDR urlLen
    print ustr$(eax),13,10
   
    print ustr$(urlLen),13,10
    invoke crt_printf, chr$("%S%c%c%c"), ADDR url2, "<", 10, 10

    inkey "Press any key to exit..."
    exit
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
end start

Results as posted:

Code Select Expand


66
http://www.masm32.com/board/index.php?topic=7866.msg57747#msg57747<
1
1
66
http://www.masm32.com/board/index.php?topic=7866.msg57747#msg57747<

Results with DWORD following STRUCT commented:

Code Select Expand


66
http://www.masm32.com/board/index.php?topic=7866.msg57747#msg57747<
0
0
67
<


[jdoe]

Without seeing the code, my only guess is it that has something to do with a pointer to the FORMATETC structure, as the structure data is actually changing memory location with the alignment.
Wait, FORMATETC is failing when it's not DWORD aligned, ..., I don't know.

Greg,

This the part of the code I was talking about that needs the FORMATETC structure.

Code Select Expand



IShellExtInit_Initialize PROC p_this:DWORD, p_pidlFolder:DWORD, p_pdtobj:DWORD, p_hkeyProgID:DWORD

    LOCAL l_fetc:FORMATETC
    LOCAL l_stgm:STGMEDIUM
    LOCAL l_dwMemSize:DWORD
    LOCAL l_hResult:DWORD

    mov l_hResult, E_FAIL

    ;
    ; We must have a IDataObject pointer to get paths
    ;

    .if (p_pdtobj != NULL)

        mov l_fetc.cfFormat, CF_HDROP
        mov l_fetc.ptd, NULL
        mov l_fetc.dwAspect, DVASPECT_CONTENT
        mov l_fetc.lindex, -1
        mov l_fetc.tymed, TYMED_HGLOBAL

        ;
        ; Get selected paths
        ; hGlobal of STGMEDIUM points to a DROPFILES structure
        ;

        CallMethod GetData, IDataObject, p_pdtobj, addr l_fetc, addr l_stgm
        .if (eax == S_OK)

        ...

        .endif




If "STRUCT DWORD" is not used then the GetData method of the IDataObject interface fails. Must be aligned. No workaround.

------------------

MichaelW,

Sorry, maybe I made you work for nothing on this one. Shame on me because I took a structure randomly that had a word in between just to try to point you where I don't understand. What adding "STRUCT DWORD" is doing to the URL_COMPONENTSA structure when the first member is already a DWORD. The nPort member can't be patch with 2 bytes at this location because it will add 2 bytes to the structure size.


Look...

Code Select Expand



SHFILEOPSTRUCT STRUCT
   hwnd                                DWORD ?
   wFunc                               DWORD ?
   pFrom                               DWORD ?
   pTo                                 DWORD ?
   fFlags                              WORD ?
   fAnyOperationsAborted               DWORD ?
   hNameMappings                       DWORD ?
   lpszProgressTitle                   DWORD ?
SHFILEOPSTRUCT ENDS





I'm using this one since very long time and never add problems with it, without alignment and never fails. What is the difference ?

Wow, my question was having a list of structure that really needs to be aligned and Now I'm confused about structures alignment.

:'(

[MichaelW]

Windows expects the structure member alignments to match the alignments implemented by the C/C++ compilers. Depending on the structure and what you are doing with it, having some of the members misaligned may have no effect. For the URL_COMPONENTS structure, when used with the WinHttpCrackUrl and/or WinHttpCreateUrl function, it does have an effect, and these two apps demonstrate why.

Code Select Expand


typedef unsigned long       DWORD;
typedef unsigned short WCHAR;
typedef WCHAR *LPWSTR, *PWSTR;
typedef int INTERNET_SCHEME, * LPINTERNET_SCHEME;
typedef unsigned short      WORD;
typedef WORD INTERNET_PORT;
typedef struct
{
    DWORD   dwStructSize;       // size of this structure. Used in version check
    LPWSTR  lpszScheme;         // pointer to scheme name
    DWORD   dwSchemeLength;     // length of scheme name
    INTERNET_SCHEME nScheme;    // enumerated scheme type (if known)
    LPWSTR  lpszHostName;       // pointer to host name
    DWORD   dwHostNameLength;   // length of host name
    INTERNET_PORT nPort;        // converted port number
    LPWSTR  lpszUserName;       // pointer to user name
    DWORD   dwUserNameLength;   // length of user name
    LPWSTR  lpszPassword;       // pointer to password
    DWORD   dwPasswordLength;   // length of password
    LPWSTR  lpszUrlPath;        // pointer to URL-path
    DWORD   dwUrlPathLength;    // length of URL-path
    LPWSTR  lpszExtraInfo;      // pointer to extra information (e.g. ?foo or #foo)
    DWORD   dwExtraInfoLength;  // length of extra information
}
URL_COMPONENTS, * LPURL_COMPONENTS;

int main()
{
  URL_COMPONENTS urlComp;
  printf("sizeof(urlComp) %d%c",sizeof(urlComp),10);
  printf("&urlComp.dwHostNameLength %x%c",&urlComp.dwHostNameLength,10);
  printf("&urlComp.nPort %x%c",&urlComp.nPort,10);
  printf("&urlComp.lpszUserName %x%c",&urlComp.lpszUserName,10);
  while( !kbhit() );
  return 0;
}


Code Select Expand


sizeof(urlComp) 60
&urlComp.dwHostNameLength 12febc
&urlComp.nPort 12fec0
&urlComp.lpszUserName 12fec4


Code Select Expand


; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    include \masm32\include\masm32rt.inc

    _URL_COMPONENTSA STRUCT
      dwStructSize       dd ?
      lpszScheme         dd ?
      dwSchemeLength     dd ?
      nScheme            dd ?
      lpszHostName       dd ?
      dwHostNameLength   dd ?
      nPort              dw ?
      lpszUserName       dd ?
      dwUserNameLength   dd ?
      lpszPassword       dd ?
      dwPasswordLength   dd ?
      lpszUrlPath        dd ?
      dwUrlPathLength    dd ?
      lpszExtraInfo      dd ?
      dwExtraInfoLength  dd ?
    _URL_COMPONENTSA ENDS

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    .data
      urlComp   _URL_COMPONENTSA <>
    .code
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
start:
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
    print chr$("SIZEOF(_URL_COMPONENTSA) ")
    print ustr$(SIZEOF(_URL_COMPONENTSA)),13,10
    print chr$("OFFSET urlComp.dwHostNameLength ")
    print uhex$(OFFSET urlComp.dwHostNameLength),13,10
    print chr$("OFFSET urlComp.nPort ")
    print uhex$(OFFSET urlComp.nPort),13,10
    print chr$("OFFSET urlComp.lpszUserName ")
    print uhex$(OFFSET urlComp.lpszUserName),13,10

    inkey "Press any key to exit..."
    exit
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
end start


Code Select Expand


SIZEOF(_URL_COMPONENTSA) 58
OFFSET urlComp.dwHostNameLength 00403014
OFFSET urlComp.nPort 00403018
OFFSET urlComp.lpszUserName 0040301A


[jdoe]

MichaelW,

So the alignment is done on each members of a structure.

Now it means that to be safe, all structures must be at least DWORD aligned if one of the member is less than a DWORD. But if one member is a REAL8. Hmmm...

I'll sleep on that... it's 3h30 here.


Thanks for your time Michael. Really appreciated.



:U

[japheth]

Hello,

Instead of opening multiple topics for the same goal, I thought that one should be enough.

I'm working on a Windows SDK/DDK translation for MASM (Prototypes/Constants/Structures) and I need from time to time few answers that some of them were already gotten elsewhere on this board.

I have now all the set of SDK/DKK (2000/XP/2003/Vista) and what I'll have to offer when completed is all of them merged together.

are you aware of http://www.japheth.de/Win32Inc.html?

Do you know more of these structures and the alignment required ?

Yes. Which come into my mind are some in OAIDL (OLE automation), and for console, the INPUT_RECORD also needs dword alignment.

Thanks for the hint about the WORD member in URL_COMPONENTS! It's missing in Win32Inc.

Regards

Japheth

[MichaelW]

The URL_COMPONENTS in the current MASM32 windows.inc is correct.

Code Select Expand


URL_COMPONENTSA STRUCT dword
dwStructSize       dd ?
lpszScheme         dd ?
dwSchemeLength     dd ?
nScheme            dd ?
lpszHostName       dd ?
dwHostNameLength   dd ?
nPort              dw ?
lpszUserName       dd ?
dwUserNameLength   dd ?
lpszPassword       dd ?
dwPasswordLength   dd ?
lpszUrlPath        dd ?
dwUrlPathLength    dd ?
lpszExtraInfo      dd ?
dwExtraInfoLength  dd ?
URL_COMPONENTSA ENDS
URL_COMPONENTS equ <URL_COMPONENTSA>


[GregL]

jdoe,

Windows expects the structure member alignments to match the alignments implemented by the C/C++ compilers.

That's what it boils down to.

[jdoe]

Finally, I found what in the C header files, tells which structures needs to be aligned and how.

So the statement I made before...

Now it means that to be safe, all structures must be at least DWORD aligned if one of the member is less than a DWORD

...is wrong.

----------

I need to check for "pshpack1.h, pshpack2.h, pshpack4.h, pshpack8.h", to know which structures (and how) needs to be aligned.

Code Select Expand



/*
* Set up Structure Packing to be 4 bytes
* for all wininet structures
*/
#if defined(_WIN64)
#include <pshpack8.h>
#else
#include <pshpack4.h>
#endif




Thanks for your clues.