Help with segmentation

[fafastrungen]

Hi, I don't understand how segmentation applies. I know that it is a way to divide the memory and protect it in protected mode, but I don't know how can I obtain a segment.
For example if I have 256 MB memory, I could declare a variable in this way (pascal code):

MEMORY :Array[0..268435456] of Byte

and I could access to memory in this way (we assume I want to access to 15000 index):

MEMORY[ 15000 ] := 150

with this previous declaration, I'm accessing memory in position 15000 and set a value of 150. Until now everything is clear for me, but the problem is when I can't access to memory positions bigger than 1 MB due to address bus with 20 bits in older pc's. So, how can I get the position 15000 in the SEGMENT:OFFSET format ? and how can I know how big could be a segment ?
I know how to get a real address from a SEGMENT:OFFSET format, but I don't know the inverse operation

This is the way I understand all this, so thanks a lot for any enlightenment.

[gabor]

Hello!


First of all if you don't have to bother with segmentation then don't do it! Segmentation sucks and is really old technique.

About your question: are you using real or protected mode, or the hybrid real flat?
Let's start with pmode! In protected mode are the memory spaces of seperate processes guarded, as far as I know the segment size is not binded (It can have a minimum and a maximum size). Usually the segments have set up their maximum as size. The addressing is done by index registers or in pmode by every register (not just esi, edi, ebx) so the maximum size is 4GB.

In real mode the segment size is limited to 64kB, the addresses are combined from segment and offset, this is 20 bits as you mentioned. But, and about this I don't know everything, there is the A20 Gate wich controlls the 1MB memory bank selection. The theory is that this 1MB addressing space is assigned to different memory spaces, the program can select which memory chunk is taken.

Real Flat is a trixky one. To start real flat one hase to switch into pmode, set the segments as it is desired, for example to have maximum size, not just 64k then switch back into real mode. So in this mode you will have bigger than 64k segments, but the CPU still works in real mode.

I hope I didn't mess things up and did not tell to much stupid stuff!

Greets, Gábor

[BogdanOntanu]

You understand wrongly.

In real mode you can ONLY access as much as 1M byte of memory and that is ALL.
The segments of real mode are there just to help you access more than 64K of RAM.

You need to go to protected mode (at least briefly) in order to be able to access more than 1M of memory.

That is why real mode programming is considered obsolete today.
I suggest that you seek another OS that providesa interfaces in protected mode.
There are many: Windows, Linux/UNIXes, etc

Real mode is still of some use when accessing BIOS in the startup stages of an OS and as the first mode of CPU start...
Eventually it can be used for very small programs that need only limited resources from a computer.

IF you need 256M of RAM THEN real mode and real mode segments are NOT a very good answer / solution.

[BogdanOntanu]

Segmentation sucks and is really old technique.

In real mode it may be obsolete...

However segmentation is a great advancement in computer concepts and technology.
It is new compared to the paging and virtual memory concepts and in protected mode it's implementation was "almost" very good.

It has been dropped in the new implementations of 64bits CPU because of course nobody uses it...
All humans did prefered the old stuff paging and label it as "new" and "innovative" ...

Such is human race... and indeed for new 64bits systems one should "stick with paging from now on" :D

[fafastrungen]

First, many thanks to both.
I'm learning assembler and I'm starting to use this language, in fact, I don't want to use segmentation and is much better for me if I can forget it. I want to start from cero, no OS in background, I just want to make a program, compile it and use it from a diskette when I boot the pc.
So, let's forget segmentation but, how do I access to memory (we're speaking about 486 machines or upper!) ?
Recovering my example, how do I do this in assembler:

PASCAL:
-----------
var
Buf    :Array[0..14999] of Byte


I guess this is correct in assembler:

Buf  byte 15000 dup (?)

My doubt is, if I do the previous declaration in assembler, will I get an error due to 1 MB memory limitation?
As far as I know, 486 machines and uppers has a 32 address bus, so memory limitations is 4GB, so in theory I could declare an array with a maximum size of the total amount of memory in the system.

Thanks again.

[BogdanOntanu]

Well,

Making a "simple"programm that does NOT depend
or base on any other OS or API or programming language / library
is in FACT MUCH HARDER TO DO and requires MORE KNOWLEDGE.

IMHO your best chance is to start using ASM in Windows using the standard Win32 API.

If you want to avoid the GUI API you can try and create only console like applications.

And YES in modern win32ASM you could eventually define a huge array like that... even much greater...

However it is far better to allocate that memory dynamically at run time because it is done this way by the PE loader anyway and you have the chance to handle errors and adapt your program IF the system does not have so much memory as you need.

Your assumptions that 386/486 are automagically able to use 4G without the help of an OS and some API are utterly wrong ...

[fafastrungen]

I'm not making an OS, I just want to learn the base of the systems, I'm not interesting in assembler under windows becouse I want to reech deep in the system, in fact, I'm trying to avoid the use of BIOS/DOS's ints and use the IN and OUT functions to access hardware, this is the way I want to learn becouse in this way I'll learn how a system works.
But returning to my original question, it's not clear for me yet if the system recognize at boot time the total amount of memory and I can use it. I need to know if the total amount of memory I have in the system is fully available for me.
For example, I have a PC with 256 MB, this 256 MB is fully available when I program ? or I can only use the 1 Mb at maximun ?
Supose I make a little program and I install it in the boot sector of a diskette (so no OS in background), now supose this little program write a byte in the last index position of system memory, is this possible or not ?

Thanks for your help.

[BogdanOntanu]

This was clarely answered above :D ...

NO you can NOT access more than 1M of RAM even if you have 1Terrabyte installed in the PC.

The reason is that the CPU starts in "Real mode" and it will refuse to access more than 1M until you get into "Protected Mode" (and then into "Long Mode" for 64bits) 

IF you would have read the Intel manuals ...honestly you would have known by now... besides any tutorial on the net teaches this.

Going to "protected mode" and beyond and doing IN/OUT is exactly how an OS does  things...
now you can name it as you like... :P besides who has said that you want to make an OS ?

[Dinosaur]

fafastrungen

Seeing as you are interested in real mode Dos to access hardware direct, there is a work around.
It is a small program called "Unreal". You simply run it once at startup, and according to the author
"it messes with the segments". It does not leave anything in memory once it is finished.
Google for it.

Then you can address directly gigabytes of memory in dos real mode.
It is very reliable, and I currently have this running on machines in industry for 5 years or more.
I currently access 64Mbyte in real mode by loading the memory address directly into registers.

;--------------------------------------
;WipeMem puts 0 in Unreal Flat Memory
;   29-12-2004
;--------------------------------------
   .MODEL MEDIUM
   .486
   .CODE
   PUBLIC   WipeMem
   ;------------------
WipeMem    PROC    FAR
   PUSH    BP
   MOV     BP,SP
   PUSH   DS
   ;------------------
   xor      eax,eax                  
   mov      ds,ax         ;DS = AX = 0  ..Use LINEAR ADDRESSING !
   mov      eax,100000h      ;Start is beginning of 2nd MegaByte
   mov      ecx,700000h      ;do it up to start of 8th Megabyte
Clear:
   mov      word ptr DS:[EAX],00h   ;Clear memory !!!
   add      eax,2
   loopd   Clear
   ;------------------
   POP     DS
   POP     BP
   RET     2
WipeMem      ENDP
;----------------------
   END

This is an example of how to put zeroes in that memory, (inefficient ,I know but this was my earliest attempt)

[fafastrungen]

Thanks for the help, I know now that I can't access more than the first MB in memory.
I'll look for the code in google and I'll study what it does.

[Mark Jones]

Hello fafastrungen, please understand that even if you are able to access >1MB ram, that isn't going to make your program any easier to create.

Also, if you are looking for a real-mode operating system, save yourself years of development and try QNX. :thumbu

See, windows provides API functions to do things. If you want to print a string to a console window you can call the WriteFile function, simple enough. In DOS, there are a series of "interrupt routines" which are just calls to code snippets which perform set tasks, so if you want to print a stirng in DOS you set a few registers and INT 09 or whatever. But in REAL MODE, you have NONE of this. You must do everything yourself. And writing code which prints a string in REAL mode, is not a trivial task. I can see why you would want to start learning this way, but it is not practical. This is why Bogdan suggested you start writing windows programs, then maybe someday try a real-mode application. Good luck. :U

[fafastrungen]

Thanks Mark for your advice and I understand perfectly what all you say, but what I want is try to understand what happens when I press a key in the system or what happens when I want to read a file, I'm very tired of using things like VB, Delphi, Visual C, etc., now I want to know what is behind all this, thats why I took this way, I'm just trying to understand the things at very low level.
I know that I can use the bitmap object of Delphi to manage graphics and I know that the cls command in DOS clear the screen, but I want to know how it clears the screen or how bitmap object manage graphics, I'm very tired of just drop text boxes onto a form.
I'm making things in windows with WinAsm and fasm, but again, in this way I just call the CreteWindow function and nothing more, and this is not the level of knowledge I'm looking for.

[sinsi]

1. Get an old 486
2. Install DOS 5
3. Use EDLIN to write your program
4. Use MASM 1.25 and LINK 2.05 to make your EXE
5. Use DEBUG to see if it works
6. Get really frustrated
7. Disassemble the ROM BIOS and video BIOS to see how things work
8. Finally feel superior to others because you can program the VGA registers
9. Realise that you are now a dinosaur...

[tenkey]

No, they're not a dinosaur - they are a preservationist!

[ChrisLeslie]

fafastrungen,

Unlike some others I think there are some benefits to learning a little bit of DOS assembly to learn the basics if you desire to get a little bit closer to the machine hardware. Some simple .com programs, using the FASM package if you wish or masm tiny model, will provide an easy start as you will be in a flat 64K limited mode. The DOS and BIOS ints are a bit of a black box at first but at least you have some chance of inspecting the code if you wish. There will also always be people willing to help with specific DOS problems in these asm forums. As far a segmented programs are concerned, there is not much point in pursuing this as you will be very unlikely to write any beginning DOS programs that will get anywhere near to 64K. If you do, then you will be advised to go over to Windows/Unix etc. Unfortunately, the various APIs that do most of the IO work etc will always be a black box, but there is nothing you can do about it, just use them.

btw -  When writing DOS programs under a Windows environment remember that although you may be writing and executing a real mode program, you are still limited by the constraints of the OS as far as memory access is concerned. You will not be able to access physical memory locations that are outside of your allotment.  eg,  mov esi,[0]  mov al,[esi] will assemble/compile but not execute because you are not permitted to access location 0. All memory references should be by means of labels for beginners.

Chris