lib2inc

[zcoder]

I have been trying my hand at making a lib2inc that I hope will do ALL lib types.
as you know, it is hard to make one that can read C, style libs and a few others.

I am trying. I need some people to help see if it works ok, it seems to work but
you never know.

it's size is about 7K so it's really opimized and I disabled the wildcard option untill
I feel that it is stable, This app will do TASM and MASM style proto's

Thanks

Zcoder....

[attachment deleted by admin]

[ToutEnMasm]

Hello,
Without the source it's difficult to help you.
                   ToutEnMasm

[PBrennick]

You don't need the source to test the program.
Paul

[zcoder]

I found a small bug, and fixed it.
Now this is not to say that it's not bugy, this is not a stable version untill I am sure it has been tested well.

I have optimized it alittle more, so please tell me what you think on the speed as well as size.
The program reads the LIB file and does everything in one pass, gets rid of dupes and then
sorts them into aphabetic order, then creates the proto's and I still managed to do it all in about
7K.

Thanks again.

Zcoder....

[PBrennick]

ZCoder,
I have tested your program using my static library (cuz I know exactly what is in it) and it did an excellent job.  There is a soft error, however.  I told it to output to test.inc, instead it created a file called list.txt, why is this?

It is not a serious problem, but it sure surprised me.  I ran it in a folder with a lot of files and at first, I thought it had failed!

Paul

[Mark Jones]

Hi Dan, have you looked at the disassembly?

One thing that can be streamlined is the display strings, they are all handled individually, i.e.:

Code Select Expand


00401048   .  68 52304000   PUSH Lib2Inc.00403052                            ; /Arg1 = 00403052 ASCII CR,LF,CR,LF,"Lib2inc version 3.0.9 All rights reserved",CR,LF,""
0040104D   .  E8 3E060000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
00401052   .  68 82304000   PUSH Lib2Inc.00403082                            ; /Arg1 = 00403082 ASCII "Copyright(c) Dan Ferguson 2000-2006",CR,LF,CR,LF,""
00401057   .  E8 34060000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
...and...
00401077   .  68 AA304000   PUSH Lib2Inc.004030AA                            ; /Arg1 = 004030AA ASCII CR,LF,"No arguments found.",CR,LF,""
0040107C   .  E8 0F060000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
00401081   >  68 C2304000   PUSH Lib2Inc.004030C2                            ; /Arg1 = 004030C2 ASCII CR,LF,"usage: LIB2INC [options] [path]libname.lib [[path]outfilename.inc]",CR,LF,""
00401086   .  E8 05060000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
0040108B   .  68 09314000   PUSH Lib2Inc.00403109                            ; /Arg1 = 00403109 ASCII "options:",CR,LF,""
00401090   .  E8 FB050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
00401095   .  68 14314000   PUSH Lib2Inc.00403114                            ; /Arg1 = 00403114 ASCII "         /M = MASM style PROTO's(default)",CR,LF,""
0040109A   .  E8 F1050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
0040109F   .  68 40314000   PUSH Lib2Inc.00403140                            ; /Arg1 = 00403140 ASCII "         /T = TASM style PROTO's",CR,LF,""
004010A4   .  E8 E7050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010A9   .  68 63314000   PUSH Lib2Inc.00403163                            ; /Arg1 = 00403163 ASCII "         /I = include libname in INC file.",CR,LF,""
004010AE   .  E8 DD050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010B3   .  68 90314000   PUSH Lib2Inc.00403190                            ; /Arg1 = 00403190 ASCII "         /h = this help message.",CR,LF,""
004010B8   .  E8 D3050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010BD   .  68 B3314000   PUSH Lib2Inc.004031B3                            ; /Arg1 = 004031B3 ASCII "         /? = this help message.",CR,LF,""
004010C2   .  E8 C9050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010C7   .  68 D6314000   PUSH Lib2Inc.004031D6                            ; /Arg1 = 004031D6 ASCII "         /E = no echo.",CR,LF,""
004010CC   .  E8 BF050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010D1   .  68 EF314000   PUSH Lib2Inc.004031EF                            ; /Arg1 = 004031EF ASCII "         Wild cards accepted -> *.lib *.inc",CR,LF,""
004010D6   .  E8 B5050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010DB   .  68 1D324000   PUSH Lib2Inc.0040321D                            ; /Arg1 = 0040321D ASCII "         You may place all options anywhere as well as",CR,LF,""
004010E0   .  E8 AB050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010E5   .  68 56324000   PUSH Lib2Inc.00403256                            ; /Arg1 = 00403256 ASCII "         the filenames. But you may NOT use wild cards",CR,LF,""
004010EA   .  E8 A1050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690
004010EF   .  68 8F324000   PUSH Lib2Inc.0040328F                            ; /Arg1 = 0040328F ASCII "         for extensions",CR,LF,""
004010F4   .  E8 97050000   CALL Lib2Inc.00401690                            ; \Lib2Inc.00401690


All those calls are to:

Code Select Expand


00401690  /$  55            PUSH EBP
00401691  |.  8BEC          MOV EBP,ESP
00401693  |.  83C4 F4       ADD ESP,-0C
00401696  |.  53            PUSH EBX
00401697  |.  51            PUSH ECX
00401698  |.  57            PUSH EDI                                         ;  ntdll.7C910738
00401699  |.  56            PUSH ESI
0040169A  |.  57            PUSH EDI                                         ;  ntdll.7C910738
0040169B  |.  833D 00344000>CMP DWORD PTR DS:[403400],0
004016A2  |.  75 0C         JNZ SHORT Lib2Inc.004016B0
004016A4  |.  6A F5         PUSH -0B                                         ; /DevType = STD_OUTPUT_HANDLE
004016A6  |.  E8 83070000   CALL <JMP.&kernel32.GetStdHandle>                ; \GetStdHandle
004016AB  |.  A3 00344000   MOV DWORD PTR DS:[403400],EAX
004016B0  |>  FF75 08       PUSH [ARG.1]                                     ; /String = "j"
004016B3  |.  E8 6A070000   CALL <JMP.&kernel32.lstrlenA>                    ; \lstrlenA
004016B8  |.  8945 F4       MOV [LOCAL.3],EAX
004016BB  |.  6A 00         PUSH 0                                           ; /pOverlapped = NULL
004016BD  |.  8D45 F8       LEA EAX,[LOCAL.2]                                ; |
004016C0  |.  50            PUSH EAX                                         ; |pBytesWritten = NULL
004016C1  |.  FF75 F4       PUSH [LOCAL.3]                                   ; |nBytesToWrite = 7C8399F3 (2088999411.)
004016C4  |.  FF75 08       PUSH [ARG.1]                                     ; |Buffer = Lib2Inc.<ModuleEntryPoint>
004016C7  |.  FF35 00344000 PUSH DWORD PTR DS:[403400]                       ; |hFile = NULL
004016CD  |.  E8 62070000   CALL <JMP.&kernel32.WriteFile>                   ; \WriteFile
004016D2  |.  8B45 F8       MOV EAX,[LOCAL.2]                                ;  kernel32.7C816D58
004016D5  |.  5F            POP EDI                                          ;  kernel32.7C816D4F
004016D6  |.  5E            POP ESI                                          ;  kernel32.7C816D4F
004016D7  |.  5F            POP EDI                                          ;  kernel32.7C816D4F
004016D8  |.  59            POP ECX                                          ;  kernel32.7C816D4F
004016D9  |.  5B            POP EBX                                          ;  kernel32.7C816D4F
004016DA  |.  C9            LEAVE
004016DB  \.  C2 0400       RETN 4


How about reformatting the strings to remove most of the nulls, then optimize the call to WriteFile? Notes:

* The GetCurrentDirectory does not necessarily return the same directory as which the .exe is ran from. Use GetModuleFileName instead and strip off the file part.
* List.txt appears to be hard-coded.
* Some of the procs appear to be aligned on 16-byte boundaries. If this is not needed, reduce it to four, two, or one and it will save space.
* lstrcmp/lstrcpy/lstrlen are quite slow. Try these:

Code Select Expand


align 16
szCompMJ3 proc szDest:DWORD,szSource:DWORD
    mov ecx,szSource                ; only slightly changed
    mov edx,szDest
@@:
    mov al,byte ptr[ecx]            ; fetch bytes
    inc ecx                         ; prevent stall
    mov ah,byte ptr[edx]
    inc edx
    test al,ah                      ; null?
    jz @good
    cmp al,ah
    jnz @bad                        ; match?
    jmp @B
@good:
    mov eax,0
    ret
@bad:
    mov eax,1
    ret
szCompMJ3 endp


align 16
szCopyMJ3 proc uses esi edi szDest:DWORD,szSource:DWORD
    mov esi,szSource
    mov edi,szDest
    cld                             ; copy forwards
@@:
    cmp byte ptr [esi+03],00        ; better branching
    jz CopyFour
    cmp byte ptr [esi+02],00
    jz CopyThree
    cmp byte ptr [esi+01],00
    jz CopyTwo
    cmp byte ptr [esi+00],00
    jz CopyOne
    movsd                           ; else move DW ESI --> EDI & increment
    jmp @B

CopyFour:                           ; further modified from above
    movsd
    ret
CopyTwo:
    movsw
    ret
CopyThree:
    movsw
CopyOne:
    movsb
    ret
szCopyMJ3 endp


align 4
szLen5 proc src:DWORD     ; unknown author
    mov     ecx, [esp+1*4]
    test    ecx, 3
    jz      @max8

@bucle:
    mov     al, [ecx]
    add     ecx, 1
    test    al, al
    jz      @lb1

    test    ecx, 3
    jnz     @bucle
align 4
@max8:
    mov     eax, [ecx]
    mov     edx, 7EFEFEFFh
    add     edx, eax
    xor     eax, 0FFFFFFFFh
    xor     eax, edx
    add     ecx, 4
    test    eax, 81010100h
    jz      @max8

    mov     eax, [ecx-4]
    test    al, al
    jz      @lb4

    test    ah, ah
    jz      @lb3

    test    eax, 0FF0000h
    jz      @lb2

    test    eax, 0FF000000h
    jnz     @max8

@lb1:
    lea     eax, [ecx-1]
    mov     ecx, [esp+1*4]
    sub     eax, ecx
    ret     1*4

@lb2:
    lea     eax, [ecx-2]
    mov     ecx, [esp+1*4]
    sub     eax, ecx
    ret     1*4

@lb3:
    lea     eax, [ecx-3]
    mov     ecx, [esp+1*4]
    sub     eax, ecx
    ret     1*4

@lb4:
    lea     eax, [ecx-4]
    mov     ecx, [esp+1*4]
    sub     eax, ecx
    ret     1*4
szLen5 endp


Timing results comparison from my (unreleased prototype) test bench app:

Code Select Expand


Timing TestBed app for MASM32. Fail Warn Pass Error (Bound, Null, High ASCII)
Processor: AMD Athlon(tm) XP 2500+

64-byte String Compare Routines, read-aligned 32/16/8/4/3/2/1 --- TTszComp.asm
lstrcmpi:  748, 1564, 1551, 1565, 1551, 1551, 1565 (ecx,edx)
lstrcmp:   1548, 1559, 1548, 1548, 1561, 1550, 1561 (ecx,edx)
szCompMJ1: 404, 404, 404, 404, 404, 404, 404 (esi,edi) cmpsb
szCompMJ2: 339, 339, 339, 339, 339, 339, 339 (esi,edi) byte mov/cmp
szCompMJ3: 338, 338, 338, 338, 338, 338, 338 (ecx,edx) byte mov/cmp
szComp01:  363, 350, 350, 350, 350, 350, 350 (esi,edi,ecx) repz cmpsb

64-byte String Copy Routines, read-aligned 32/16/8/4/3/2/1 ------ TTszCopy.asm
lstrcpy:   234, 234, 234, 234, 234, 234, 234 (eax,ecx,edx)
szCopyMJ1: 230, 230, 230, 230, 230, 230, 230 (esi,edi,al) byte mov
szCopyMJ2: 165, 165, 165, 165, 165, 165, 166 (esi,edi,al) movsd
szCopyMJ3: 132, 158, 158, 158, 158, 158, 158 (esi,edi) movsd/b
szCopySH1: 161, 153, 153, 153, 153, 153, 153 (esi,edi,ebx) 4x unroll
szCopyLI1: BNH113, BNH122, BNH102, BNH119, BNH115, BNH84, BNH98 (ebx) mov/add/lea

64-byte String Length Routines, read-aligned 32/16/8/4/3/2/1 ----- TTszLen.asm
szLen1:    92, 92, 92, 92, 92, 92, 92 (?)
szLen2:    95, 95, 95, 95, 95, 95, 95 (?)
szLen3:    111, 111, 111, 113, 119, 115, 119 (?)
szLen4:    120, 120, 120, 120, 144, 120, 127 (?)
szLen5:    79, 79, 79, 79, 79, 81, 89 (?)
szLen6:    82, 82, 82, 82, 82, 86, 82 (?)
szLen7:    168, 168, 168, 168, 168, 168, 168 (?)

Press enter to exit...


[zcoder]

PBrennick,
oh, I should have said something about that list.txt  :bg
I coded it that way for now so it would not hurt any ones real inc files while
the app is being tested. I would feel real bad if it damaged anyones files.

I made the app take as command line arg's some switch's and "filein" and "fileout"
you can mix them anyway you wish, all that is required is that "filein" and "fileout"
end in eather .inc or .lib this way those two arg's can be in any order you wish.

example c:\masm32\include\incfile.inc /M c:\masm32\lib\libname.lib /E
the above example has the inc(outfile) first this can be done this way
becouse all that is required is that the extention be present this way
you can have the args in any order.

Couse .lib is assumed as input while .inc is assumed as output, like
wise for the switch's they can come before, or in the middle or at the
end. does not matter.

If you use LFN ie: c:\my include files\myinc.inc they must be placed in quotes
"c:\my include files\myinc.inc"

wild cards are eccepted but I have it disabled for now.


Mark Jones,
Thanks for all the info I will check into alot of what you said. thing is I did try align 1 , 2, 4 ect and it did not
change in size, I am not sure why, maybe becouse the project is made up of OBJ's??????
maybe I am doing something wrong? could you explain this alignment stuff to me more?

About the GetCurrentDirectory, I have tested that on alot of progs and had no problems(unless this is a OS difference)
but for safty reasons to make sure it always does I can change that as you suggested.

On the lstrcmp, I can use a good relacement, but there is one case in my code where I need not only to know if
the strings match, but I need to know which string is greater then or less then becouse I wrote a small quik
simple bubble sort algo. As you know bubble sorts are not that good to use, but I took that route anyway couse
it seemed to fit good in this proccess.

On the lstrlen, there is times I wanted to keep long code out if the speed would not really be noticed, but after I
did some tests myself, I noticed a longer time with very large LIBs so sence the app is around 7K I guess I have room
for a faster len , even if it's larger code. :bg

On the lstrcpy I have had some rare times when some of those replacement algo's have triped on strings, I am not sure
why, maybe that was awhile back and that has been fixed, I will check into that too.

I will try some faster lstr algo's and post it again when I get that done. :bg

And Thanks PBrennick, Mark Jones, for the feed back on this.
it sure helps out alot, and helps me to get this thing into shape.

Thanks alot guys. I have gained alot of needed info.

Zcoder....

[PBrennick]

zCoder,

To help you understand, I will explain what Mark means about the possability of GetCurrentDirectory failing, you could probably test it forever and never see a failure.  A reliable test to see it always fail, first time every time, is to path to the executable.  If I have an EXE in c:\folder1\someapp.exe I can run the program from folder2 by including the complete path to the EXE.  In this example, folder2 will always be returned (properly) as the defaut directory instead of folder1 which contains the EXE.  This will confirm that Mark is correct.

BTW:  echo on does not work, either.  It just echos the switches and not the output.  echoing the output if the user wants 'echo on' would make your app work in a proper fashion.

Paul

[zooba]

I think Paul just mentioned what I was about to, sending the output to STDOUT if there is no file provided would be useful, especially for piping into sort/find commands.

Also, I ran it on my ASM Runtime library and a few imported functions were shown (ie. from other DLLs) as well as my own with no way of telling them apart programatically (short of comparing all the entries to another list).

Finally, on the GetCurrentDirectory, I would prefer it to use GetCurrentDirectory rather than GetModuleFileName, as in general I keep one copy of tools like this (generally in MASM32\TOOLS\<tool directory>) and call them from the project dir with fully qualified paths:

Code Select Expand

C:\MASM32\PROJECTS\ASMRT\> \masm32\tools\lib2inc\lib2inc.exe ASMRT.LIB ASMRT.INC

Using GetModuleFileName in this case will cause file-not-found errors, because of course the files aren't stored with the EXE, they're in my project dir! In general, I think it's probably best to use the command line as the user gives it and let Windows find the files (ie. using PATH variable, etc.)

Very nice work :bg

Cheers,

Zooba :U

[zcoder]

PBrennick,
Your right, I did the test just as you said, and it failed everytime, sence this is so I will change it.
I am not sure why someone would do that, but I will fix that anyway,
(This might happen in a IDE where the IDE invokes these from another folder.) <-- good point.

As for the echo I will modify that to make it comply. it does sound like a good idea.
So far all the suggestions have been GOOD ones and well worth looking into.
I have fixed the output so it does go to the file you name, just be carefull not to
name a path and file where your real inc files are untill I can deam this as stable...

I have not posted the new changes yet, I have other things I need to change that was suggested
I have changed all that was suggested eccept for the lstrcmp and lstrcpy I need to find or make good
fast replacements for those.

Zcoder....

[PBrennick]

zCoder,
If you create replacements to those algos, please consider contributing them to The GeneSys Project.  I also noticed the point that zooba just made.

I guess it would be difficult to determine what is a call to a proc and what is a call to an external API?  All API calls are listed as if they are internal to the library when, in fact, they are not.

As you know, I am partnered with Vortex in The GeneSys Project so I will always support and use his tools.  I am just helping you here as it is good to have as many choices as possible.  I bring this point up because it is to be noted that Vortex's lib2inc tool correctly determines what should be put in the generated include file.  His tool does not put APIs in the include file so there has to be a way for you to figure this out

Paul

[zcoder]

PBrennick,

GeneSys Project? what is that? I have not heard of that  :bg silly me.

zooba,
good point also. I think that I could aslo put in finding path's but can someone tell me how?
how to you get the ENV path's and how would one do this so it complys with all compilers?
I have never done anything that worked with paths.

Zcoder....

[PBrennick]

The GeneSys Project has its own subforum in this forum.  It is a development system used to help beginners in assembly programming to quickly come up to speed.  If you do not wish to contribute to the library that is
not a problem.

Also, reread my previous post where I have talked about you creating PROTOs for APIs.

Paul

[zcoder]

PBrennick,
As for the API's from other dll's going in the output I have thought about it and I
think I have a way to limit those from the output. I will try it out and see how it works.


Zcoder....

[PBrennick]

If you have trouble with that, let me know and I will ask Vortex to help you.

Paul