The MASM Forum Archive 2004 to 2012

General Forums => The Workshop => Topic started by: Serj on July 26, 2007, 09:18:44 AM

Title: Assembler Enhancement Initiative
Post by: Serj on July 26, 2007, 09:18:44 AM
Hi, guys. I wanna tell you about some of my thoughts about assemler.
So, I often use standalone assemblers to develope programs entirely in this beautifull language and several years of practice brought me to an idea of complete enchancement of assembler. Today we have three main chioces of how to develope a program:
- use standalone assembler.
- use Highlevel language such as VC++ etc.
- use HL and its built-in assembler.

Let's talk about the first way. Let's say honistly that assembler is very very inconvenient development instrument in comparison with HL compilers. There are some concrete problems:

- almost no support for creating and management of namespaces.
- week support for type checking
- week mechanisms for detecting errors at build-time
- week support for describing complex data and code structures

This problems make it almost impossible to develope large serious projects using assembler so tried to enchance it. I wrote a macrolibrary but discovered tons of bugs in MASM's  preprocessor so it's still just a project. I've considered closely almost all existing win32 assemblers and i'm absolutely sure we all need a new powerfull one that will make it so easy to develope applications that you will forget about all HLs)) I hope you aree with me. So, let's begin a serious discussion. I've got big experience and advanced ideas but i started this topic no to discuss them but just to see you are READY. Drop a line about what features do you think assembler needs, what other problerms do you see and may be something else. I really hope you all are enthusiastic and we will soon make this world bettrer)..
Title: Re: Assembler Enhancement Initiative
Post by: ramguru on July 26, 2007, 10:11:37 AM
I'm pretty satisfied with poasm or masm their suit my needs even for big projects. Assembler isn't meant to be clearly comprehensible, it's for people who are not afraid to spend more time coding who are opened for challenge...and if you're not ready to accept one, better look at stuff that M$ is offering, like that VS2008 which is migrating to "speech to pe_coff" pattern
Title: Re: Assembler Enhancement Initiative
Post by: hutch-- on July 26, 2007, 12:49:37 PM
Serj,

Welcome on board. You have some interesting ideas but I am not sure how you would put an assembler together to deal with the problems you see.

Quote
- almost no support for creating and management of namespaces.
- week support for type checking
- week mechanisms for detecting errors at build-time
- week support for describing complex data and code structures

In procedural programming "namespace" is more or less a "so what" consideration.
Type checking is as much a pain as an advantage but MASM can tighten that up if you want to write that style of code.
MASM manages build time assembler errors without much trouble.
Complex data structures are usually written as libraries and yes you can do them in assembler.

The preprocessor in MASM is old and quirky but it manages to do many useful things when macros are written properly.

One area that really could do with big improvements is a far more poweful pre-processor that can be done as a plugin to the assembler as it would allow the programmer to properly customise the pre-processor capacity.
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 26, 2007, 01:54:09 PM
I'll explain then my thoughts more carefully.

1) Namespaces - procedures can almost never be nested. What d'you think about nested procedures? I think it'd be an easy way to group code pieces that make similar job.
2)Type checking is a pain only when it requires casts too often. Actually casting can be hidden from programmist (when type sizes are equal and so on) and you will only need to use casting when it really helps avoid seriuos mistakes. But there's another application for type checking. Only by means of casting and type checking you can tell assembler, that some variable is not just a double word but it is a pointer to a quite complex structure.
3)I think most runtime errors are caused by using wrong values as pointers, so to catch such errors at build-time all working with pointers should be done in very very accurate manner. Actually there's a way to put working with pointers behind the scenes.
4)Well, and what will you do if you want to create several instances of that data structure? Will you try to load the library several times??)) MASM's structures dont provide means of discribing some kinds of complex data strutures, but let's discuss it more closely in future.
5)The only way to make the language's grammar very flexible and extensible is to use macros. It is part of my plan. I'll explain it when we will talk about compilation principles. I wanna object that in most complex cases macros written absolutely properly may produce errors. I've spent millions of hours trying to catch them but I give it up when I saw they are caused with internal bugs.

I'm glad to see there are some people at this forum that think like me)) I'm waiting for your new ideas and after some time when everybody who is really interested in what I suggest to do will say his word, we will start to discuss the subject more closely.
Title: Re: Assembler Enhancement Initiative
Post by: Vortex on July 26, 2007, 06:29:28 PM
Hi Serj,

I would like to introduce here a well known and  flexible method to build applications :

- use HL and object modules ( MS COFF or OMF ) created by assemblers.

You can easily create optimized routines in assembly where high execution speed is required. Those routines should be assembled to build libraries to support high-level languages. The flexibiliy of C / C++ and the power of assembly is a well know combination. As an example, I can mention about C start-up modules coded in asm to reduce the size of executables built with C compilers :

http://www.masm32.com/board/index.php?topic=166.0

Despite those problems you mentioned, Masm is a very powerfull assembler. You can even do OOP with Masm, ObjAsm32 is a good example :

http://objasm32.tripod.com
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 26, 2007, 10:19:29 PM
Vortex, I can't but to agree with you that C/C++ and assembler is a very good combination. But I'm talking about a bit another thing. You sad that C/C++ are flexible. Yes they are. They're far more flexible than assembler, so if you are interested in rapid development of your applications you have to write some code in C/C++/VC++ and other part in assembler so in fact you cannot just sit and code, you have to warry about HL support in your assembler code. But haven't you ever dreamed about such an assembler that could do all the work alone, providing C++'s flexibility and power.

I often met people that just hate HLs 'caz they produce horrible code for quite simple and handy HL constructiaons. But if you bring HL elements into assembler you have a chance to know what code that elements produce and make sure that code is quite optimized. I know almost all methods of building applications using assembler in combination with other instruments but dont' you want to make all programs with only one tool? I want! And guess you too.)) I just wanna stop this shit and make a really powerfull and flexible assembler so that people can use it without those problem they have to put up with today

P.S. Sorry, but OOP in MASM is just perversion.) This is my private opinion and I dont want to hurt anyone but beleave I know what I'm talking about.
Title: Re: Assembler Enhancement Initiative
Post by: GregL on July 27, 2007, 03:44:32 AM
Serj,

I don't know, but I just don't see all the problems with MASM that you do. Granted, there are cases where it's not the best tool to use, but you can do anything with it. It sounds like what you want is a high-level language. I am pretty satisfied with MASM as it is too. If I want all the stuff your talking about I'll use a high-level language. No offense, but that's how I feel about it.

Title: Re: Assembler Enhancement Initiative
Post by: hutch-- on July 27, 2007, 05:27:18 AM
Serj,

I would have to agree with Greg here, while I understand your criticisms and have heard most of them before, to rectify these comments as problems would be to no longer end up with an assembler but a high level language. The rough distinction is between bottom up as with an assembler versus top down with a compiler that has an inline assembler built into it.

A large number of people who write assembler are also highly fluent in numerous compilers and have a pretty good idea of what does what so they tend to chose to use an assembler on a very well informed basis whether its for components (object modules) or for specialised small apps and/or DLLs for languages that cannot produce them properly.

The OOP view has its place in high level languages if you have the tolerance to suffer the design but it does not have a place built into an assembler as it would no longer be an assembler. Building OOP design with an assembler is a different matter and a number of people have done it in reliable ways but the day an asembler shifts from delivering a 1 to 1 correlation between written code and generated opcodes it becomes just another compiler.

Where I suggest there is room for smart enhancements is in the control of a pre-processor. The one in most C compilers is a pile of appalling toothless junk that MASM eats alive yet as I mentioned earlier, the pre-processor in MASM is quirky and old and you need to know a lot about it to get the more complex macros going properly.

Randy Hyde has taken a different approach, he uses an assembler as a back end and effectively produces a complex pre-processor for it with what he calls a high level assembler. Not all agree with him but its a similar idea to what you appear to have in mind.
Title: Re: Assembler Enhancement Initiative
Post by: Shantanu Gadgil on July 27, 2007, 07:26:38 AM
Quote from: Greg on July 27, 2007, 03:44:32 AM
Serj,
...
... but you can do anything with it. ...

I have faced problems with nested structures and having predfined nested structures.

Some months back I was working with MASM and predefined nested structures thing and hit a wall with the way you can define them.
The same thing worked OK in POASM btw! :)

Later in the day I saw that, that is an inherent issue with MASM and there was nothing much I could about it!!! (Just use POASM, I guess)  :toothy

Given that this predefined nested structure thing  is a snap in C code it could very well have been possible in MASM!!!

Leaving aside the reasons of "why the feature is not implemented" etc, this is something I have faced issues with!

Regards,
Shantanu

Title: Re: Assembler Enhancement Initiative
Post by: japheth on July 27, 2007, 09:41:49 AM
> P.S. Sorry, but OOP in MASM is just perversion.) This is my private opinion and I dont want to hurt anyone but beleave I know what I'm talking about.

> The OOP view has its place in high level languages if you have the tolerance to suffer the design but it does not have a place built into an assembler as it would no longer be an assembler.

I have some doubts that you 2 know what you are talking about. Implementing some support for OOP in an assembler is the same thing than implementing the "LOCAL" or "PROC" directives, for example. And these directives are useful and don't make MASM a compiler.
Title: Re: Assembler Enhancement Initiative
Post by: hutch-- on July 27, 2007, 09:55:56 AM
You are probably confusing the difference between a bare assembler and a macro assembler ratrher than the difference between an assembler and a compiler.
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 27, 2007, 11:05:38 AM
Well, guys, I think you don't get me absolutely right. I don't want to make a HL compiler. I think you just were misled with my statements about handy features HLs have. hutch--, 1 to 1 correlation is the basement of any assembler, it's the basement of my ideas too. When I talk about HL features I, actaully, dont mean they should be hardcoded in compiler. I think assembler should provide some basic way to describe code and data as it is, and most assemblers do it quite well, but a powerfull assembler should also provide a programmer with means to indroduce HL constructions that of course should be emplemented as macros. Hence, you have absolute control of abstraction level you work at: if you want to use bare assembler you use it, if you want to insert some ready code you use macros. But let's look at it from a bit another point of view. We can make an assembler that will provide bare coding mechanisms as well as a way to extend lang.'s grammar with macros, but not only extensions could be emplemented as macros, bare coding mechanisms can also be based on macros. Just imagine, you've just created a very cool assembler and the next day Intel releases a new processor with absolutely new architecture and of course new set of commands and ,in addition, Microsoft reports they already support it, so you're gonna rebuild your assembler to support new commands etc. But if all commands in your assembler are just macros you only need to correct them and add some new ones. So the only language assembler must support in this case is a macrolanguage, that provides some means to define code and data as it is, so it's not assembler which produces code but flexible macros that can be easily changed without changing assembler's code. What d'you think of it, hutch--? Ah!?

At last I want to show you a snippet that I was testing while working on the first version of my macrolibrary. I place code that was really assembled in comments. I don't wanna you think that I suggest to build an assembler with support of this macrolanguage, it was just a project and no more.

...

image  ; .code

    object main():default   ; $main:

    protobj first(int32 par1, int32 par2, int32 par3)

       mov eax,val(par1)   ; mov eax,dword ptr [ebp+8]
   
       ....

       first(1,2,3)   ; push 3/ push 2/ push 1/ call $main$first

        ....

        invoke ExitProcess,0

        object first(int32 par1, int32 par2, int32 par3)   ; $main$first:/ push ebp/ mov ebp,esp
           
        ...

            return 0   ; leave/ xor eax,eax/ ret 12

        endobj first

    endobj main

endim   ; end $main


I hope you won't say that I suggest to make a HL compiler instead of assembler any more))
Title: Re: Assembler Enhancement Initiative
Post by: TNick on July 27, 2007, 12:19:47 PM
You may do what you want, but if I will see a type checking in masm I will hang myself. LEAVE my DWORDS in peace!!!! :)

Nick
Title: Re: Assembler Enhancement Initiative
Post by: Tedd on July 27, 2007, 12:33:01 PM
Serj, I've had similar ideas to this so it is something I've also considered and thought about - I'll explain a few points and reasons why you will meet a lot of resistance trying to advertise this idea to a board of assembler programmers.

Firstly you should realise that HLLs and ASM are separated for a reason. ASM programmers like to know the lower-level details and see exactly what is going on, and control exactly what happens to each and every bit in the cpu. HLL programmers, on the other hand, generally want to ignore these details and get on with writing their code, allowing the compiler to take care of these tiny unimportant details (who cares what registers are used or how a function is called?) That said, many of us here can and do use both for different reasons and even mix the two together.
However, mixing the two immediately causes problems - there is a conflict between wanting to know all of the details (asm) and wanting to hide all of the details (hll.) This results in two types of tool: assemblers with a few extra lightweight HL 'features' (if-else, while-loops, basic structures, etc) and compilers that only really allow the basic insertion of asm code. Which is exactly what leads you (and I) to the idea that it would be very nice to mix the two and have an assembler which provides much better support for HL features, but without taking control away from the programmer. Unfortunately, you can only get so far with this idea without making it extremely awkward to use.
So, you take your plain assembler (supports instructions, symbols, etc - all the basic stuff you'd expect) and add on the expected if-else, while-loop, repeat, structures, unions, etc.. And that's very nice, but it doesn't offer anything over the competition. So you start to add in more useful higher-level things like classes and inheritence.. wow, it's really starting to get better now! All of this can actually be done quite nicely, and with little intrusion to the programmer (though it would be good for the programmer to read the manual so they can understand exactly how it works so they understand about providing the object pointers, which takes up a register.) And of course, now that you've got classes, you must have type-checking - the very point of having classes is to be able to build complex types - uh oh! This is where the conflicts begin.
Type-checking implies type conversion, and that implies inserting code without telling the programmer - unsurprisingly, asm programmers do not like this. In fact, even without this, asm programmers seem not to particularly like even basic type-checking "it's a dword!!" (masm does provide support for 'types' but very few use it - it causes more problems and actually leads to program errors because it sometimes alters the code you write without telling you.) Okay, a compromise, we'll have soft type-checking so it can be easily over-ridden when required, and any type conversions must be either an explicit cast (e.g. int -> dword) which doesn't generate extra code but is just sugar for the compiler, or a true conversion (e.g. float -> int) which obviously must generate extra code and must therefore be an explicit function call (which could be inlined.) There, that sounds a lot better.
Okay, this is really good - more! Well, it's really tedious to have to write out all of the instructions for a simple little expression like "a+b*c/d" so why not just write it like that and have the code generated for you? That sounds very nice, but it would need extra registers for the calculations and I might already have values in those registers. Okay, we'll save all registers used. That's pretty inefficient though, if we don't really need to save half of them. Okay, you can specify which registers to preserve - better? Well, maybe, although it's going to look a bit of a mess with all of these preservation enclosures, and type conversions and whatnot all over the place... This is why it's awkward - to take the features you must lose some control, or end up with some terrible syntax where you must specify constraints that turns out worse than just writing the code yourself.
And so it continues... You can add lots of wonderful time-saving features, making the asm code look more and more like hll code, but as you do you will need to take more and more control away from the asm programmer and generate code that wasn't explicitly requested. This is exactly the reason why many of us like to program in asm - it gives you that explicit control and you don't have a compiler meddling around with your code.

Now, as I said, I do (surprsingly) think this is a good idea. But it must be done in a particular way, for the reasons which (I hope) I've highlighted. And if you're wanting to pitch it as an assembler with HL features, then it must be exactly that and any design decisions must favour the asm side rather than the HLL. That means not hiding anything from the programmer - no implicit code generation, no hidden type casting or conversions, no runtime pointer checking, no compiler telling you that's not allowed or 'fixing' your code, etc. But these things are exactly what make hll features so 'nice' and easy to use - so you must choose which side to lean towards.

From your comments it seems you are more of a HLL programmer who likes to use ASM sometimes. But in order to make this project work you really need to think like an asm programmer - otherwise you will end up with a HLL compiler with some asm features - and that means using asm as a preference for a long time (I'm not saying this in any snobbish way, it's simply that asm programmers generally think in a particular way, and hll programmers generally think in another.) Also from your comments it seems you have not used masm for very long, many of the 'deficiencies' you point out are actually fully supported features - you just don't know how to use them. Also, suggesting that if you cannot find the problems with your macros then it must be caused by internal bugs strikes me as somewhat arrogant; not that I'm saying masm is in any way perfect, you may even be correct, but you are not an masm expert so the problems could be entirely yours - have you asked for help here?

Many program errors are caused by bad use of pointers - you are correct - this is true for HLLs, not so much in asm. The reason is very simply down to the information hiding that characterises HLLs. In asm you know exactly what a pointer is and what it's doing, in a HLL it can be many things depending on the type of the pointer and such. It's not pointers that cause the problems, per se, it's the fact that the programmers using them often do not know the implications of using them and what happens - because this information is mostly hidden from them. Of course they can, amongst other things, cause genuine mistakes for both asm and hll programmers, but asm programmers have the explicit details staring them in the face, and so are more likely to take notice and be careful, whereas hll programmers have a compiler to protect them and sometimes rely on it even when they shouldn't.


ADDENDUM:
I think this highlights my points quite succinctly:
Quote from: TNick on July 27, 2007, 12:19:47 PM
You may do what you want, but if I will see a type checking in masm I will hang myself. LEAVE my DWORDS in peace!!!! :)
Title: Re: Assembler Enhancement Initiative
Post by: hutch-- on July 27, 2007, 12:44:39 PM
Serj,

Perhaps you have not looked at masm lately after various people have done some work in masm's ancient cranky old pre-processor. The MASM32 project has been developing masm macros for years and you can have a good look at them in the last release of the masm32 project. (version 9). The macro file is macros.asm in the macros directory and the4e is a "high level help" help file for them.

Randy Hyde has taken the alternative approach and written his own pre-processor in his project HLA which has an extended capacity beyong the pre-processor in MASM.

Now another problem you will run into is the inbuilt prejdice of many older asm programmers to any form of high level construction. It has been the masm32 project that championed the built in pseudo high level constructions in masm.

If I get what you are sketching out, its a concept from the mainframe days which is a programmable assembler / compiler where the bulk of the work is done as macros. Its an interesting idea and vaguely I think there is still one made that way that exists but its not a common tool any longer.

The problem as I see it is the only interests your idea is serving so far is people who want a high level architecture and there is a simple answer for them, let them use compilers where they have all of the high level attributes. Mention "abstraction" in the C++/OOP/Delphi/VB/SmallTalk etc ..... and you will probably get a cool response from anyone who has escaped from this trash and can write assembler code where they want to.


; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««

main proc

    LOCAL pbuf  :DWORD
    LOCAL buffer[260]:BYTE

    cast buffer, pbuf

    .if rv(GetCL,1,pbuf) != 1
      print "Please enter the name of an ALA programmer",13,10
      print "Your choices are : "
      print "Annie Beth Betov Frank Hutch Randy Wannabee",13,10
      return 0
    .endif

    switch$ pbuf
      case$ "Annie"
        print "Best uplift bra in ALA",13,10
      case$ "Beth"
        print "Programmer and social commentator extrordinaire",13,10
      case$ "Betov"
        print "Betov is a "
        REPEAT 8
          print "chick "
        ENDM
        print "chicken",13,10
      case$ "Frank"
        print "Nature's gentleman",13,10
      case$ "Hutch"
        print "Geriatric API Grinder",13,10
      case$ "Randy"
        print "Master of reality",13,10
      case$ "Wannabee"
        print "Howz the chewin' goin' Wannabee ?",13,10
      else$
        print "Sorry, I don't know that programmer",13,10
    endsw$

    return 0

main endp

; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 27, 2007, 05:08:21 PM
...Well Tedd, I Dont Like When people who dont know me closely try to treat me like a child!!! If you guess about something I haven't told about yet, ask before typing dozens of lines! It's all my emotions, sorry. but I really don't appreciate such treatment.

So, specially for you:
1) I was using assembler for all recent years and I preffer it to HLLs. (I just want to kill you (not right now!!!)))) when you say I dont think like asssembler programmer)
2) Errors in my macrolibrary are really caused with bugs in MASM's preprocessor. I've done some dumping to figure it out and I don't want to listen that I dont know MASM's macro language quite well. I can almost speak it!!))
3) The scenario you described IS JUST YOUR OWN POINT OF VEIW. NOT MINE. Almost everything you described is NOT among my ideas!

So, thats enough for emotions))

1. Type checking
   I think that type casting that causes implicit code insertion or modification is just HLLs' business. Let them do it, DONT even pronouce a sound about it when talking about assembler!! When I say 'type checking' or 'casting' I mean mechanisms that make it possible not just to tell assembler about variable's SIZE but to provide it some additional information about what data structure a variable points to, how many arguments a procedure takes (you DO use invoke syntax) and so on. When asembler knows about it it can make a hit (not issuing an error message and stopping compilation process) at the fact the pointer you provide in a call points to some structure that is not quite big so it can prevent errors when data is erased or your program tries to access memory that was not allocated. And so on.
2. Classes. Well, I consider all today's OOP no more than SHIT. And I don't wanna hear about it any more!!! I suggest using nested procedures I the way I mentioned above. Procedure's prologue in my macrolibrary creates a new namespace so you shouldn't warry about name conflicts any more. Procedure's epilogue closes the namespace. You can use some procedures just to group some others into a container, 'caz procedures may be
empty (not contain their own code) and have many subprocedures. Nesting is unlimited (dont even tell me about local variables!!)
3. I wanna say it loud that I dont want to bring HLLs' features into assembler as they are. I'm against using syntax like a=b*c. I'm not against expressions like this eax=911, 'caz it's just another form of mov eax,911 but nevertheless I don't like such syntax. I'm for common assembler expressions when command's name goes first with a tail of comma-separated arguments appended to it.
...Assembler provides us unlimited freedom. I don't wanna lose it. Please, everybody, next time you want to say I'm gonna introduce constructions that generate uncontrollable code or generate errors where it's ok in MASM, think twise or better ask me before.

hutch--, I've considered closely every feature you mentioned, I'm using masm32 v9 since in was released. Before it I used preivious version. Long ago I used to develope DOS applications in MASM 5.10. When you say
Quoteyou will probably get a cool response from anyone who has escaped from this trash and can write assembler code where they want to.
I dont get you. I think I said it in plain English that assembler must provide features that will make it possible to write pure code when you want to write exactly puse assembler code and use HL constructions when you want to save your time but without loosing control or the like.

Well, guys no offends of course, but get me right, I almost thrown my monitior away through the window when I read everithing you wrote here!!)))))
Title: Re: Assembler Enhancement Initiative
Post by: Vortex on July 27, 2007, 05:50:13 PM
Hi Serj,

Take it easy, Tedd or any other member will not treat you in an other way. Masm forum is a place with mature members and we are here like a family. Nobody would like to hurt you.

eax=911 is a statement accepted in the C-- language, Sphinx C-- is a mixture of asm and C :

http://c--sphinx.narod.ru/indexe.htm

I guess Randall Hyde's HLA is what are you looking for. It has a very powerfull macro engine, more even powerfull than Masm. HLA accepts nested function parameters like HLLs. Why you don't give a try?
Title: Re: Assembler Enhancement Initiative
Post by: hutch-- on July 27, 2007, 10:08:33 PM
Serj,

What you run into here is just a few of many hundreds of people who float around this forum who have heard probably every flavour of HLL based criticism of how and why there is something wrong with assembler that needs to be fixed (IE made into a HLL). Given that the vast majority of programmers floating around here are fluent in at least a number of programming languages as well as various forms of scripting, thy all write assembler when they need it for very good, well informed reasons.

A long time ago when I first laid out MASM32 as a project and this is after a background of years of commercial assembler programming in the DOS market I went out of my way to avoid making assembler programming as a subset of HLL programming. By treating assembler as a language in its own right its real strengths could be capitalised upon without the limitations and irritations of various HLLs and the body of folklore built up around them.

Data types based on "how big" they are instead of some phony name based on multi-port theory, pointers that are known from their documentation so that every pointer in win32 is the same size, 32 bit. No guessing what a range of phony data types were based on the theory of portability and native processor WORD size.

Its not that anyone is trying to take the fun out of programming for you but if anyone inflicted limitations of the ty[pe you have in mind on them theywould throw their monitors (at least the old CRT type) at you.
Title: Re: Assembler Enhancement Initiative
Post by: anunitu on July 27, 2007, 10:32:32 PM
Almost sounds like you might be looking for something like FORTH, a somewhat HLL, that is very low level. A contradiction? Perhaps, perhaps not. Do a search, and check it out.


Anunitu
Title: Re: Assembler Enhancement Initiative
Post by: GregL on July 28, 2007, 04:23:32 AM
Serj,

Well, let's see a bit of what you have in mind.

Title: Re: Assembler Enhancement Initiative
Post by: Betov on July 28, 2007, 08:43:50 AM
Serj wrote:

"... almost impossible to develope large serious projects using assembler..."

In this matter, opinions are nothing. Only fact matters, and one fact that one maight consider is that RosAsm, which has been developped with this target in mind, is actually a monolithic Assembly Source, 4 Megas long. So, it proves, by itself, two things:

* Large Sources are very possible, in Assembly.

* Large Assembly Sources can be developped collectively.

It also makes a demonstration (by the minimalist number of RosAsm users) of the fact that, 99% of the programmers, who introduce themselves as Assembly programmers are, in fact, HLLers, making occasional use of Assembly, and that they will never consider writting anything real in Assembly.


Betov.

< http://rosasm.org >




Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 28, 2007, 11:33:42 AM
Well, either RosAsm or C-- are just another compilers with their own features and look at compilation process. But if I was searching for such tools there'd be no need for me to write another assembler. There is considerable amount of assemblers that are quite flexible and extensible. But as Tedd wrote (sorry, guy, if i hurted you))) the more HL features they introduce the more control they lose. I wanna quite another level of control. I like assembler 'caz (you will probably disagree) it's declarative i.e. when coding in assembler I'm just describing some data structure that will be called a program as it is. I'm just describing. Almost full control. But there are still some limitations. Assembler sources are mostly 'flat' - I mean there are some procedures, some variables put side by side on the same nesting level. I'm also a web-developer, so I get used describe what iI write far more carefully. When I say I need HL features I also mean I need some means for better describing and structuring of my code.

Let's say a program is just a chunk of data (if it makes a confusion you may think that OS's loader treats a program like other data file - it loads it, places some parts at certain addresses and so on). Let's also say we only deal with data chunks when building a program. So, every such chunk has three main characteristics: size, offset from the begining of parent chunk (an absolute one or calculated using alignment value). There can also exist some other characteristics like maximum chunk size, alignment of its sub-chunks etc. So, as I treat assembler as a declarative language I want it to provide easy means to 'mark up' my code. Using all existing assemblers you can only write 'this is a dword variable inside .data segment' or 'this is a procedure that contain tree commands each 2 bytes long' and the like. But I need more. I want to be able to express such phrases like 'this is a set of variables, located inside another one that starts at address 80h from the beging of .data segment. It contains two dwords located at its beginig and one byte at offset 12 from it'. I think its very very important to have absolute control of your code, to be able to describe it entirely or in other words to set all its layout attributes. You may say assembler is already possible to, but you are wrong then (what about nesting of code, what about structuring at all?). I invented a syntax for describing data structures more carefully. I dont consider MASM's types enough descriptive, 'caz they just say what size varible is but no more. Naming types with arbitary (i mean not based on their size) names is common practice. I guess you wont chose a name like '32dwords' or '128bytes' for a structure that's 32 dwords long)) We all get used to see names like 'dword' or 'byte' in variable declarations but it doesnt mean we cannot use another names. So, here's  my syntax. I lay it out for you JUST to think what kind of control i have in mind. I use some type names that you may hate but honistly I prefer them to traditional ones 'caz they tell about variable's size in a more native manner.

object - directive for describing a piece of code that can communicate with others tru call-ret mechanism. Example

object myObj(int32 par1, int32 par2):default at 80h maxsize 128
   .... ; some code goes heree
endobj myObj

With this I tell approximately this 'this is a piece of code that is also the antry point of a program, its maximum size is 128 bytes, it's located at offset 80h from the begining of .code segment and when you call it you should put two integer 32-bit variables onto the stack'

varset - directive for describin' data structures. Example

varset i_hate_bush:bss   ; a set of variables named i_hate_bush
   int32 i   ; 32-bit variable named as 'i'
   int8 hate
   int32 bush at :4  ; dword 'bush' with 4-byte alignment relative to the begining of i_hate_bush variable set
   varset at :32 ; unnamed varset at offset multiple of 32 bytes
     int8 myStr "And what about you?#0" ; a null-terminated string
     int8 str1 "Ah??#0"
     int8 font "Tahoma" maxsize 16 ; a 16-byte string buffer
     varset myStruct:struct ; structure declaration. 'Structure' mean here a template you can use for rapid declaring of large data structures.
        int32 x 15 ; if initial value isl not supplied when instantiatin' 15 will be used instead
        int32 y 16
     endset myStruct
   endset
endset i_hate_bush

There some other smart ones but I think that enough for this time.

What is about compilation process I dont share all today's approaches. Most assemblers just translate soucre to binary code and then probably fix it.
I think that its quite out-of-date. I think its far more smart to buid a tree instead. Its nodes would present separete data chunks and due to perferct structuring abilities they can also store all the info about node nesting. This is exactly what I ment when was talking about namespaces (each node can be named at all chunk's attributes can be attached to it). When holding such a tree in the memomy it's so easy to resolve any reference and fix opcodes. So, when you have a program tree you can easily translate it into any arbitary file format - pe, bin or any other.

I also introduce 'chain typing' if its possible to say so. Imagine you are going to store in a dword variable a pointer to a COM object

ppIActiveScript dd ?

When you want to call a method within that object you should either do it as it requires to or use some macros. So, to call a method you should do something like this

...;push here all the arguments
mov eax,ppIActiveScript
push eax
mov eax,[eax]
call (IActiveScript ptr [eax]).<method_name>  ; Let IActiveScript be a MASM structure that describes the interface

To declare ppIActiveScript to make calls easily I suggest describe pointers in this way

int32*int32*IActiveScript ppIActiveScript

You should read it as 'a 32-bit variable that contains address of another one that contains addr. of IActiveScript structure'. Here IActiveScript is a structure (template) that lets assembler know that it only contains pointers to procedures.
Then you can call a method in a quite easy way

ppIActiveScript.<method_member>(this,....parameter list)  ; 'this' is placeholder for ppIActiveScript

I think it's far more transparent way then that introduced in masm32 package. I should also note that when you use ppIActiveScript you are working with IActiveScript structure. To make it possible to work with the pointer I use root sign '^', so I call int32 a root type and IActiveScript a target type. It may seem to be very complex or too 'high-level', so just forget about it)) I dont insist on this technigue though I find quite convenient.


..I'm not willing to create just another assembler 'caz I'd better sit and code in MASM or NASM or FASM or the like. I hope you wont tell me that structured coding is a bad manner otherwise I will think bad about you)). I dream of a tool that will make it possible to describe any structures of any compexity. So HL features (note that I dont type 'HLL'. HL stands for 'High Level' here) are perfect way to introduce structuring without any loss of control.
Title: Re: Assembler Enhancement Initiative
Post by: ramguru on July 28, 2007, 12:18:11 PM
Your syntax tends to eliminate use of asm instructions, how the hell I know I'm still using an assembler, when I look at Visual_Basic-like code, what Intel Manuals will be for?
Title: Re: Assembler Enhancement Initiative
Post by: zooba on July 28, 2007, 12:43:25 PM
I suggest reading about the STRUCT statement in MASM. It does exactly what you describe. You could also read about the UNION statement, which will let you overlap two variables. You can specify default initialisers for structure members, but they are only used in situations that don't require any code generation.

TYPEDEF will let you rename the built-in types to whatever name you like.

Apologies in advance if producing your unnecessarily complex results requires more thinking than in your scenario, but that is the realm of assembly language programming. The requirement to be fully aware of the abilities of your tools and how to use them to achieve your aim is what we enjoy about it.

Cheers,

Zooba :U
Title: Re: Assembler Enhancement Initiative
Post by: Betov on July 28, 2007, 04:52:45 PM
Serj wrote:

"Well, either RosAsm or C-- are just another compilers with their own features and look at compilation process. But if I was searching for such tools there'd be no need for me to write another assembler".

I was not telling you that RosAsm is the tool you mean to write (it is much evidently not). I was talking about the intention of developping an Assembler for writting real Applications, in real Assembly - which, by the way, would never be achieved by what you are thinking about, but this is another story... -.

What i was just saying is that RosAsm has entirely been developped with that goal in mind, and that its existence is the direct demonstration of the fact that this is doable, and viable.

Now, the problem is, that you would have to face, admitting you would be right, and admiting your project would go somewhere, the fact that you would get, - leet's be optimistic - 10 times less users that RosAsm, because there does simply not exist Programmers who would consider developping Applications in full Assembly. Now, think of this:

10 Times less users than RosAsm, this make about... 1 user: Yourself. :P


Betov.

< http://rosasm.org >

Title: Re: Assembler Enhancement Initiative
Post by: TNick on July 28, 2007, 05:39:05 PM
Men, you're killing him here! :)
No, really, Serj, why don't you write some code so that we can understand you better (for this project, I mean).

Nick
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 28, 2007, 10:14:28 PM
ramguru, there's no great tend to eliminate assembly instructions, I just use HLLs' approach for declaring variables. It makes possible to declare several variables per line. And well it may look like VB, but if you look better you will find some similarity with XML. but I did mentioned I laid it out JUST to show what I mean when talk about structured programming. Generally, I chose this sysntax 'caz its very expressive, what I can't say about assembler's syntax. And dont think I'm gonna replace common semantics for opcodes. They are saint for every assembly developer.

zooba, d'you think I'm so silly that I've never read about STRUCT statement? It's just funny)) But there's a difference you dont see at all: when using STRUCT you should define a structure and then instatiate it but when using my syntax you describe data as it is in place where it lies. Well, I think it's not pleasant to define MASM structures every time you just want to describe structure of some piece of data that doesnt seem to appear at any other place in your code again. What about TYPEDEF, you are absolutely right but read the two first lines of this post again, please. Now I think you see why I dont' use TYPEDEF.

QuoteI was talking about the intention of developping an Assembler for writting real Applications, in real Assembly - which, by the way, would never be achieved by what you are thinking about, but this is another story... -.
--... Sure?? I think you just advertice your project and dont wanna understand what I'm talking about.

TNick, what code d'you want to see exactly? I can lay out several sources and show my approach in action. ...But well, excuse my, it's about time so I should go. I'll write more the next time. Just wait. At last I want to say this: when using most HLLs or some mixtures of HLL and assembler you're just afraid to write what you want where you want (I mean declaring a variable outside a class or method and so on). It's not about my ideas. I say you should write what you want where you want. But if you choose to use some HL constructions you should follow some easy rules they require. Anyway if you know perfectly what they do (what code they generate or they may not generate it at all) you can do absolutely everything.

Note: almost all assemblers require typing some code (mostly directives) before even an empty program can be built. NASM treats absolutely empty sources as full-fleged ones and you use directives and anything else only if you deside you need it. Isn't that beautifull. And another thought. In early versions of NASM they used macros to generate PEs so you just compile your exe like a binary file and macros do all needed work for you. Today there's almost no assembler that wiil allow you to fully describe your programs. You will probably want to let them do some work behind the scenes but I think it's better to have an ability to control final layout generation.
Title: Re: Assembler Enhancement Initiative
Post by: hutch-- on July 29, 2007, 03:18:39 AM
Serj,

Its not that the people around here want to spoil your fun or even a task of the size you have in mind, its just the value judgement system that goes with it about what is virtuous in a computer language and what is not will find very many detractors in a forum like this one.

I personally characterise OOP as OOP(S)[another mistake] and prefer another definition borrowed from a friend of mine, HOP = Hardware Oriented Programming, write things that agree with the processor to make it small and fast. Abstraction belongs in "Alice in Wonderland" and objects are nothing new, we have all been writing them for years in procedural programming.

Hardware in x86 produces traditional procedural programming from the inbuilt CALL / RET nature of branching and alternatives to procedural programming in x86 have to add padding to the is basic hardware capacity to produce the level of abstraction that many in HLLs want.

What offends a vast number of assembler programmers when they want low level capacity is the notion that they don't have access to the bare bones of assembler programming when they want it in anything else but an assembler. The other thing you will find is a highly ingrained independent streak in assembler programmers which is very hard to bypass on the alter of new and inovative ideas.

Most want a capacity that is reliable and accessible where they can write any damned thing they like without a second hand set of asumptions inflicted by a software author on them.

My generation actually LIKE code written like an assembler dump and have in the past written a lot of code that way. The general change has been to ensure that assembler programming can and will be understood by the next generation.
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 29, 2007, 11:08:15 AM
hutch--, I've been thinking a lot about what makes assembler special, what are its pros and cons. And, yeah, I agree with you at many points. I'm not against what you call 'code written like an assambler dump'. I just wanna add some means for structured programming to go side by side with old-style code. 'Flat' style coding is mostly applicable for development of small programms. And its exactly what you mostly deal with as I see. But I think assembler has too pure abilities to describe big and complex programs. Writing something complex code 'like an assembler dump' looks like trying to write a document that contains pictures, nested lists and so on without any markup. But even when trying to write a small reusable component that majority of advanced programs consist of you often need some more than common assemblers can offer 'cause the component's structure may appear to be very complex. I guess strutured programming is just another way of thinking. When you try to put your brilliant idea to some code you try to describe its 'structure' step by step, figuring out what you really have in mind. It's the point you start thinking in a 'structured' way even not realising it.

Well, may be it's difference in ages that makes it so difficult for us to come in terms but I want you to see that I'm just trying to bring assembler to the next level, to make it a tool not just for elite. Almost anyone here knows that only a small precent of all programmers use assembler as a stand-alone full-fledged development tool. I want to break this rule that assembler is often considered too complex to be worth learnig. The more time pass the more assembler shifts to a tool just for underground developres, hackers and so on. Why not to make it possible to develope assmbly products more easily. Of course I see that today there are very few people that would like to start using assembler the way we do but I think if we make it more structural it will become more understandible thus more popular.
And there's another pain. The more 'advanced' products M$ realeases the more these products lose in quality, speed and robustness. And it's not about M$ only. But today there's no power that will make people think more before the start coding. Assembler teaches us to but it's really too complex today. I dont wanna say it's complex for me, but I've never met a man of my age (I real life I mean) that would use assembler (no matter standalone or built-in), most of them dont even know what it is at all or are frightened with its complexity. Lasy creatures))).. I'm sure assembler needs encancements. If you dont agree, it just means you 're satisfied with it's today's abilities. But what about those who want more... I need a tool that will make it possible to build programs easily but without loss of control and the like. There's no such a tool today, so this is why I started this topic, why I spend time explaining you my ideas. The world shifts to structured describin' of everything and programming is not an exception. So, why not to bring it into assembler?? Please tell me.
I will appreciate it very much if you tell me what features d'you want assembler to provide, 'caz as I see it's no more than a habit to code in well-known old style that is the real reason of your criticism
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 29, 2007, 11:12:39 PM
I think, guys you will understand me better if you keep in mind that HL constructions I use are more just markup than something that generates uncontrolable code. My 'object' and 'return' generate code in exactly the same way 'proc' and 'ret' do. I dont insist upon the names I used for my syntax, it's not significant. I just wanted to show what structured coding may look like. So, give it a thought one more time. Nobody's gonna steal your dwords)) Of course you can keep on coding the way you get used to and use tools of yor choice. But before you write the next line of code, please think just a bit about the other way of coding and may be about the future of assembly programming...
Title: Re: Assembler Enhancement Initiative
Post by: jdoe on July 30, 2007, 12:42:15 AM
Serj,

HLLs makes fast developments because they input an abstraction layer to the final assembly as for HL syntax you could think of into an assembler language. The problems you are talking about are somekind of coding design problems. For me, working on an abstraction layer for assembly language is more like trying to make an HLL.
You haven't show clearly what you have in mind and it is difficult for me to understand what's your point and what you want to achieved and this topics goes on without being clear.

Title: Re: Assembler Enhancement Initiative
Post by: TNick on July 30, 2007, 04:52:27 AM
Thanks for not stealing our dword's.  :bg What I was trying to say in my previous reply was that:
cmp  Interested_in_what_others_think, TRUE
jne   @F
   mov Have_other_opinions, 1
@@:
cmp  Searching_for_coworkers, TRUE
jne  L1
  mov   NrOfReplies, -1*ReqNrOfCoWorkers
  @@:
  ; maybe this is an infinite loop, so a counter should be added :)
  INVOKE WaitForMultipleObjects,NrOfMASM32ForumMembers,addr HArray,FALSE,1000*60*60*24
  cmp      eax,WAIT_TIMEOUT
  je         @B
  add      NrOfReplies, 1
  jnz       @B
L1:
INVOKE DoTheJob
INVOKE PostAlfaVersion


Regards,
Nick
Title: Re: Assembler Enhancement Initiative
Post by: Sameer on July 30, 2007, 05:50:09 AM
Serj mentioned in his first post
QuoteI've got big experience and advanced ideas but i started this topic no to discuss them but just to see you are READY

so its not a question of weather you should go ahead with your project or not and if you can come of with something useful, programmers will definitely use it.
Title: Re: Assembler Enhancement Initiative
Post by: TNick on July 30, 2007, 06:48:14 AM
jmp L1
Title: Re: Assembler Enhancement Initiative
Post by: Serj on July 30, 2007, 09:17:02 AM
Deal :bg
Title: Re: Assembler Enhancement Initiative
Post by: Howard on August 03, 2007, 01:52:24 AM
Serj

I'll sure be watching with interest to see what you come up with...

Howard
Title: Re: Assembler Enhancement Initiative
Post by: Mark Jones on August 18, 2007, 08:37:18 PM
Well if I can add my two cents... I think many of us have just assumed you were new to MASM32 Serj because you are new to this forum. The forum here is like a small community - everyone knows everyone else, except we don't know you yet. We also cannot read minds, so please try to be more concise about what you are saying. I think it would help me if you could demonstrate a comparison between existing code and your proposition. I know you are only talking hypotheses at the moment, but please give us a physical demonstration of what you are trying to accomplish, because many of us are not understanding you fully. This might be due to many of us being so accustomed to writing code a certain way, that we can't envision what you are talking about.

Also, I agree with some of Tedd's initial observations. The more HLL-like you try and make assembler, the less control (and more abstraction) you seem to have. I've tried a few various object-oriented assemblers over the years and while they may be fine for some people, personally I've always found them too much hassle for the investment. It always seems like you spend twice as long writing all the markup and "support" code than it takes to just write it in assembler. There is something truly magnificent to be said about the simplicity of raw assembler. Therefore I think a large part of helping to overcome the limitations of assembler is simply to learn more about it. (Not that I'm saying anyone needs to learn assembler, only that I've gotten better at it just by learning more about it.)

Anything that makes assembler language longer or more "syntaxial" seems counter-productive to me. I really like the nearly 1:1 ratio of code to binary. If you could make a language that takes that ratio down to 0.75:1, I'm all ears! :wink

Quote from: TNick on July 30, 2007, 06:48:14 AM
jmp L1

:lol