Create a Compiler

[nathanpc]

Hello,
I'm learning Assembly and i want to build a compiler, but i don't know where i can start, then i'm asking here for some good tutorials.

Thanks,
Nathan Paulino Campos

[BlackVortex]

First learn assembly   :green

[nathanpc]

I know Assembly(Nasm), but now i'm learning Assembly(MASM). :U

[rags]

Assembly on the x86 is the same whether you are using Nasm or Masm, only the syntax differs.

[Ossa]

Hi,

writing a compiler is a very large and difficult task... I would recommend the following:

• Get a copy of the Dragon Book (Compilers: Principles, Techniques, and Tools)
• If you are interested in linkers, I would also get a copy of Linkers & Loaders
• I would also advocate the use of standard file formats, so you probably want a copy of all of the file format specifications for the various formats, which is partially easy and partially difficult to find (I spent a good month to accumulate all of the info I wanted)... I would attach a ZIP with them, but it is nearly 3MB - so message me and I will email it to you if you want it
• More controversially, I would also recommend *NOT* writing this in ASM but rather your favourite HLL. Why? Because unless you are very comfortable with ASM and your chosen assembler, I would suggest that you will spend more time fretting about learning the idiosyncrasies of it than programming your compiler. However, if you are very comfortable with it, then by all means try.
• If you don't want to buy books then a I suggest that find out about: Parsing (Regular Languages, Context-Free Grammar, Finite State Automatons, Recursive Decent Parsers, LL, LR(0), LR(1), SLR, LALR(1) parsers), Compiler Structure, Syntax Directed translation... the list goes on - just go down the index of the Dragon Book)

Anyway, hope that helps,
Ossa

[BATSoftware]

I wrote two, unless you are a software engineer with a course in compliler design, your arent going to do it. If you are good at self-learning pick up a college-level compiler design textbook and start there.

[mitchi]

First learn assembly   :green

I wish I could quote this 2 times.

[ecube]

As i've mentioned in another similar thread, why bother? writing a assembler is a massive task, and with great assembler like GoAsm etc.. already available it's pretty much a waste of time. I recommend instead you write a preprocessor, which is alot easier and allows you to add any functionality you want.

[hutch--]

Nathan,

There is much you can learn writing a compiler but as mentioned above, it is a massive and complex task and the more you know about it first, the better chance you have of getting something to work. I agree with Ossa here, assembler is an excellent tool but probably not the best one to write a compiler with.  language like C or C++ is probably the right choice as it has been around in a standardised form for a long time and is within boundaries portable across operating system versions, this will be important to you as the world slowly changes from 32 bit to 64 bit.

Where assembler WILL be very useful to you is in the backend design as you must eventually produce the output format of opcodes specific to the hardware and OS version you write for. Among the later development in compilers is componentisation, pre-processor, optimisation of the notation into lower level parts, opcode conversion and subsequent optimisation of the opcode format. In addition to the reference material Ossa has mentioned, Wikipedia has some very good articles on compiler design and code optimisation.

One of my side hobbies is writing interpreters and it is an interesting task in that you design both ends, the notation, parsing it and error checking it while loading at one end and the engine to run the result at the other end. I did that last one a few years ago to test out a number of ideas for parsing a particular notation and while I got it up to speed, about 3 million instructions a second to test the main instruction loop, it was extremely UNportable. There was also another problem that with a script engine of the type, there were potential security problems in someone feeding a malicious script to it so I restricted it to no internet functionality and used it as the script engine in my editor so that it could not be autorun from the command line, it could only be started by a user inside the editor.

[Ossa]

As i've mentioned in another similar thread, why bother?

Because it is a challenge. Because you'll learn a lot from doing so. Because it might be fun. Because it will reinforce many more basic concepts. Because it is a large project and large projects require a different kind of ability than do small ones. Because maybe no assembler has the features that you want (configurable name mangling, multiple output formats (maybe you want a COFF object file and a DWARF debug file - I don't know why but maybe you do), native OOP support, ability to handle both MASM and FASM syntax using command line switches, ...). Because not doing so "because something similar already exists" is not always a good reason... there were plenty of "perfectly good" search engines already established before Google was created.

If you feel like writing something as big and complex like an assembler, there are always a myriad of reasons to do so; you should always be aware of their complexity, however, and be prepared to stay in the game for the long-haul.

Ossa

[mitchi]

Can we rank things by difficulty levels here?

- Learning Assembly
- Writing An Assembler
- Writing a Compiler
- Writing your OS (added for fun)

[Arhk]

I find it commendable that someone isn't to intimidated to make such a goal. (That is if he does realize the probable work to be involved)
Besides isn't it better to try & encounter some failure & then success  than to not try at all, just because, everyone says there are already perfectly fine version of a product doesn't mean anything. AltaVista was before Google & Dunkin' Donuts was there before Starbucks (& of course even Microsoft has a similar story). It's not if you fall down but if you get back up, I'm sure even Bill Gates broke sweat making Microsoft what it is now.
~
To say something is impossible or improbable just because there's competition is lackadaisical. Where's the passion for one's art...? If you lack a passion for your craft you insult your fellow more passionate craftsmen  regardless of the comparison of skill - passion for your craft can quickly dissipate gaps in proficiency.  It works for me playing violin & so far with me & technology.
~

- Writing your OS (added for fun)

I plan on doing the last one (from scratch *added fun*).
nothing is ridiculous if you're willing to make it your life's work *simply; be dedicated & hard-working & nothing is out of reach.

~

*responding to the initial topic* Also you should probably learn Assembly coding first

[/b]
::) but if it's what you wanna do do it, my mom basically tells me what I do with computer's is a waste of time but I still learn coding regardless.
_{just making this clear I'm not 30 & living with my mom, I'm 14}

[ecube]

at the end of the day is a very high chance if he does waste the years on writing his own compiler it won't be to the level of existing ones, much less better. Thats not doubting his ability, thats just speaking the truth, guys like Jeremy etc who've written their own compilers, are extremely intelligent, and they didn't do it overnight.

If its a quest in learning I recommend you just go read up on how they're made, and even look at/build upon existing open source assembler like nasm, fasm etc... Or what I original said and just write a preprocessor(so you can get to the real fun/creative stuff, instead of the complicated/boring catchup), you can literally create your very own language ontop of asm, or just add a few features, but save yourself from all the grueling backend,grunt work stuff that others have already put their blood,sweat,tears,time in making.

or...you could listen to others here, who act like i'm trying to kill ambitions, and go ahead and waste your life DIY'n. I'm sure you'll make the right decision ;D

[dedndave]

my first experience with assembly language was actually machine language
i wrote code on paper and programmed it into eprom in hexidecimal
the 8008 instruction set wasn't that big
it isn't hard to imagine the desire to write an assembler
and, a simple one is not all that difficult
what becomes hard is to manage large programs and, of course, macros
and newer processors have much larger and varied instruction sets
if you want to support all the different processors, it becomes a bit of a nightmare

[nathanpc]

Can we rank things by difficulty levels here?

- Learning Assembly
- Writing An Assembler
- Writing a Compiler
- Writing your OS (added for fun)

About the OS development, i'm developing one but in Nasm! :)