Dinamicaly driver linking brings problems like "the egg and the hen" what will be first!?

Because I will use a driver for dinamicaly load and links drivers! but the driver that does that job can't do IT for itself!.

So, that is one of the reasons for that I am thinking in doing a PRE-KERNEL.

The pre-kernel will be like "GOD" creating the egg.... and using the egg doing the HEN!

But that is not the reason why i wrote this forum-thread!.

---------------------------------------------------------------------------------

If I realy will do a pre-kernel.... IT has good sides... and BAD sides!.

GOOD sides:
----------------

As the MAIN kernel will be ALWAYS in RAM, allows to use a bit of kernel space for global cpu-variables. And I was already thinking about storing there the CPROC and the ADR hooks for RECOVER the speed that it losed when I changed it to multi-threading support! (that famous 5%).

As the kernel in ROM will be a PRE-KERNEL, isn't necesary to have all the functions that normaly the kernel has... so, I can use that space for others purposes.

As the ADR system will be in kernel space and not in HIMEM... there will be 1kb+ available space for kernel purposes, I not knows in what to use that space.

The high-kernel can now have code of functions (&h4000 space), instead drivers and libraries packed (the eggs).

BAD sides:
-------------

Will be &h3A00 space for kernel code... instead &h4000 (in low kernel).

If the kernel is in ROM... it always will use 32kb RAM for kernel purposes... like the kernel loaded from DISK MODE. Unlike now, that if it is in ROM is 32kb more of free RAM. . <----- But what mean IT, maybe the 256kb that have a lot of ppl in his msx2 will be very very tight including in ROM mode.

16kb thread's kernel data + 32kb kernel + 48kb libs+drivers mounted (with GUI + object libraries) +
16kb thread's shell data + 16kb shell code size

= 128KB used... and the minimun application is 16kb thread's appl. data + 16kb code appl... (and that without opening any data file... like text.txt that if you opens one it will use 16kb as minimun in RAM). And if we programs that notepad appl as multi-instance that probably it will be... it needs extra 16kb per each instance... and if each instance opens a data file so each instance is 16kb more....

so, as much you can open 3 times the notepad! and nothing more.

So, everybody already knows.... 512kb is the recomended!!!!!!! 256 is too tight!.

Two weeks more of delay! ... ( a lot of work to do).

AND MORE DELAY in each POWER ON or hard RESET
-----------------------------------------------------------------------------

in short... it is 5% maybe 10% more SPEED or more free RAM SPACE?

what you want?

and the PRE-KERNEL idea is from GuyveR800

Another good side of the pre-kernel: is that if it works ok for you! you not needs to re-burn the kernel in ROM again if there is a kernel upgrade then. Unless the bugs were in the eggs, but if they were in eggs... is possible to overwrite them... enlarging the boot process.

I think Multi-threading support would be revolutionary and would be worth the extra RAM.
Certainly since most active MSX-users do have a 256 KB or more RAM nowadays.
And all the emulators support > 512 RAM & adjusting the emulation-speed.

You could make a non-multi-threading version for 256 KB, but even then you still need a lot of
RAM for the power users (opening more than just 3 notepads), so I'd set the requirements to 512 KB.
Saving a 32 KB of free RAM whenever every app-instance needs at least 16 KB is no agument IMHO,
although others could disagree, mentioning the 256 KB RAM users being 'left out'.
Perhaps you could make a MemManager which assigns special 4 KB blocks/chunks for 'drivers / TSRs' instead
of a whole-whooping 16 KB ? Didn't MCM's MemMan 2.x support 4 KB block assigments too ?
But you probably thought all that through I guess ...

Since I know GuyveR800 has had this kernel (and pre-kernel) idea (TNIOS anyone ?) for a long time,
I wonder wether you're implementing (enforced) user-mode / kernel-mode too ?
Seems impossible to me without hardware support. But that's just a brainwave of mine ;-)

About the somewhat longer delay on restart; how much delay are we talking about, an extra 3 second when loading from disk ?

It sounds really cool; keep up the good work !

I think your memory layout needs some work. I don't really get how this idea changes the amount of free memory in the system. But your example makes me think you're wasting a lot of ram. Take for instance hde 16kb thread kernel data. How many threads can you store there compared to the amount of free pages you expect to get? If all this stuff you're storing in RAM is really necessary, I predict the system will slow down far more than you expect.
Try to conserve some more memory. For instance a "notepad" should really only need 32kb if a substantial file is loaded. I don't think the code will need an entire 16kb.

Having a part of the kernel (preferably all the stuff needed for general operation) together with the data in a single page seems like a good idea. It reduces unnecessary page/slot switching and would be a start in packing things together.

I think that flyguille deliberately wants to seperate code and data into seperated segments (which is generally a good idea).
Using the hardware-supported 16 KB pages is easy (especially for multi-threading ;-), but I agree with Edwin on the 'wasting part'.
You might consider support for assigning whole 16 KB pages (for Multi-threading-supporting apps) and support for smaller (non-multi-threaded apps) assignments ? The design would be more complex, but it could combine the best of both worlds ...

Quote:

I think Multi-threading support would be revolutionary and would be worth the extra RAM.

I agree

Quote:

Certainly since most active MSX-users do have a 256 KB or more RAM nowadays. And all the emulators support > 512 RAM & adjusting the emulation-speed.

I hope it get out of emulators and will be used in real MSXs.

Quote:

You could make a non-multi-threading version for 256 KB, but even then you still need a lot of RAM for the power users (opening more than just 3 notepads), so I'd set the requirements to 512 KB.

That isn't possible now... it is all designed for multithread unlike the BETA versions.

Quote:

Saving a 32 KB of free RAM whenever every app-instance needs at least 16 KB is no aRgument IMHO, although others could disagree, mentioning the 256 KB RAM users being 'left out'.

I agree

Quote:

Perhaps you could make a MemManager which assigns special 4 KB blocks/chunks for 'drivers / TSRs' instead of a whole-whooping 16 KB ? Didn't MCM's MemMan 2.x support 4 KB block assigments too ? But you probably thought all that through I guess ...

That is done... in BETA versions it is the INSTALL.EXEC utility... it merge drivers in a single PAGE saving each single byte.! In final version as it turn very necesary to BOOT it will be supported from a library inside an egg in kernel.

Quote:

Since I know GuyveR800 has had this kernel (and pre-kernel) idea (TNIOS anyone ?) for a long time, I wonder wether you're implementing (enforced) user-mode / kernel-mode too ?

You mean how it distinguished the kernel code from appl code?.... testing the IP register at the interrupt routine...

Quote:

Seems impossible to me without hardware support. But that's just a brainwave of mine ;-)

It is hard... one lose in speed, but impossible NOT!.

Quote:

About the somewhat longer delay on restart; how much delay are we talking about, an extra 3 second when loading from disk ?

I not knows... that depends on the device0 mounted. Maybe more

Quote:

It sounds really cool; keep up the good work !

THANKS!

Quote:

I think your memory layout needs some work. I don't really get how this idea changes the amount of free memory in the system. But your example makes me think you're wasting a lot of ram. Take for instance hde 16kb thread kernel data. How many threads can you store there compared to the amount of free pages you expect to get? If all this stuff you're storing in RAM is really necessary,

The 16kb used for each extra-instance is 100% necesary. INSIDE IS:

A HIMEM area. where is stored all dinamicaly inside.

and inside HIMEM.....

CPU stack
PRG stack
DAT stack
Command line buffer
Local memory subdirectory
local file handlers
local data of each drivers/library installed in the instance
a temp buffer
a diskcache
a alphanumeric space for variables
INBUFFER for broadcast system
local instance variables of the application

AND I AM FORGETTING 3 items more that now i just doesn't remembe.

OUTSIDE of HIMEM but inside the same PAGE IS:

TEEM (access to all the kernel functions)
kernel's local instance variable
kernel's local instance hooks.

And any application needs free space in HIMEM for... parameters or data structure to libraries or drivers...
By example... all the objects (buttons, listbox, checkbox, windows)... have all its properties stored in this HIMEM that each application has. So also is needed free RAM space in HIMEM.

SO YES IT IS 100% necesary.... this page is always in PAGE3 from the point of view of the application.

Also a applications needs a minimun ONE page for its CODE!

And if this application needs to open a DATA FILE, it needs other PAGE.

But if it is an multi-instance appl. the several instance will share the CODE page... only needs an extra PAGE3 (called PROC-DATASYS).
Now, if 3 instances fills an 256kb MSX... in 512kb MSX will be enought space for a lot of extra stuff..

Quote:

I predict the system will slow down far more than you expect.

No, the speed is not related to the amount of thread directly... well maybe a bit. It have to do more with the timing attributes that each thread has!. But, offcourse, no one user has 10 applications running at the same time in WINXP you do?.

Quote:

Try to conserve some more memory. For instance a "notepad" should really only need 32kb if a substantial file is loaded. I don't think the code will need an entire 16kb.

That doesn't allow multiinstance code.

Quote:

Having a part of the kernel (preferably all the stuff needed for general operation) together with the data in a single page seems like a good idea. It reduces unnecessary page/slot switching and would be a start in packing things together.

I agreed, that is the 5% speed lose!

Quote:

I think that flyguille deliberately wants to seperate code and data into seperated segments (which is generally a good idea). Using the hardware-supported 16 KB pages is easy (especially for multi-threading ;-), but I agree with Edwin on the 'wasting part'. You might consider support for assigning whole 16 KB pages (for Multi-threading-supporting apps) and support for smaller (non-multi-threaded apps) assignments ? The design would be more complex, but it could combine the best of both worlds ...

we have not spoken of multi-threading appl. in this forumthread... that is other thing. We have spoken about a multi-threading O.S.

Anyway... if we want to execute an application without creating a new thread... the only one way is to use some pre-existant thread... like the kernel thread... but that makes that the application shares a lot of data structure...... and will make a lot of unsolvable conflicts... that is no way to be possible...

and by other hand i already are thinking in using the kernel thread for other active purposes

and to get a closer idea

usuarios.arnet.com.ar/flyguille/EnMyProg.asm

but it will be upgrade later because it has typos "lenght" instead "length"

that is the appl. header.

already available in downloads of www.mnbios.com.ar

how many ppl of the active community already has 512 ....
and how many ppl of the active community has only 256?

if it is 30% for 512kb vs 60% for 256kb. mmmm the decision turns too hard to take!

(I have 512 k
Somehow I still can't escape the feeling you've been "copying" too much ideas from other OS systems. We're talking MSX here. A 3 and somemore MHZ machine with small memory layout. Not even mentioning the relative slow video acces. (don't get me started on aux storage)
Baring in mind this is the year 2005, about 10 years after MSX was official declared DEAD and less software companies were interested in publishing softs. (not even the amateur scene).
I might sound negative but your plans are somewhat too ambitious I think. It's too overloaded, too heavy, too unrealistic. But hell they might have said that to Linus Torval (what's his name) too. But atleast that guy programmed on a machine that had potential. (and see what the PC stands for nowadays)

Don't get me wrong Fly, it's a hella good idea and job to be done. And not the least "to be proud of" project too!
but euhrm, what's next. . . when your OS is finished. WHAT THEN? I'm curious.

'aving said that.
-------

A pre-kernel is nothing more than some fancy pancy load routines if ya'd ask me. The chicken and the egg problem is always prominant and is a nice one to solve (if at all) What we've done (in the fuzz OS called F-kernel a.k.a. MIDAS): a small pre-kernel (if you want) which does the following:

- Set up a loading environment and load up all necessary files (from lib if you want) (not yet booting any module)
(this is a stripped DISK module)
- check memory and initialize the memory manager (memory chunks are min. 512 bytes. and max 16k.)
- initialize all other modules (which use the memory manager to initialize their own working space)
- execute project (if any)

The kernel is devided on 2 memory maps. the primairy one are the base functions (the REAL OS you might say) and always prominent in page 0 including interrupt handling, memory/page/slot switching, keyboard manager and some more.
The workhorses (all the other modules like File and Data Retrieval System, Memory Manager, Disk routines etc) are stored in a secondairy mapper on page 1 which is called through the BASE functions.

So the user ends up having atleast 32k in a row available at all times. Yes the kernel takes about 32k (including hi-mem)

The pre-kernel (loader I like to call it) takes about an amount of 4k of total RAM (MAX)

hope this gives a clue.
good luck

Quote:

(I have 512 k Somehow I still can't escape the feeling you've been "copying" too much ideas from other OS systems.

like winXP?... I personaly don't saw any MSX OS. more far than MSXDOS1.
To remove a lot of STARTUP code from the main kernel was all the time in my mind... but, before I didn't see how necesary is it.
Now, in which way to remove that... GuyveR800... comments about a prekernel.... not now... since several months before...
I was always against to lose those 32kb of free ram in ROM mode.... but, there is other question... how many ppl will plug an alternative rom in slot 0, in his real msx in order to have that extra RAM memory?. And if they do that.... if they can do that... surely they also can upgrade his msx to 512k or not?

So maybe nobody uses that extra free RAM anyway.... because:
1: in real MSX they will loads from disk all the MNBIOS.
2: in emulators they can increase always the RAM size.

So, my feelings about if I uses more RAM for OS purposes the guys with 256kb will see it useless or they won't be able to run bigs applications are a bit groundless.

Quote:

We're talking MSX here. A 3 and somemore MHZ machine with small memory layout. Not even mentioning the relative slow video acces.

Well, maybe you thinks that a MSX running Space Manbow is a miracle!?
Don't get me wrong... 3.58mhz are the minimun specs.
But anyway, now if I does this improvements... I only can thinks that the more complex GUI will be the only one slowdown factor than the speed of the BETA versions. But, including the new GUI has parts where the speed is a bit compensated.... by example... the CRT matrix isn't more in VRAM... it is now in RAM. That is a bit
improve... to compensate the more complex clipping.

Quote:

(don't get me started on aux storage) Baring in mind this is the year 2005, about 10 years after MSX was official declared DEAD and less software companies were interested in publishing softs. (not even the amateur scene).

You develops thinking in get the attemption of commercials soft companies? I not. Well maybe for the PDA market, but i am interested in hard companies.

Quote:

I might sound negative but your plans are somewhat too ambitious I think.

Agree... can't be more ambitious.... proud of that!

Quote:

It's too overloaded, too heavy, too unrealistic.

overloaded <> HEAVY.

Quote:

But hell they might have said that to Linus Torval (what's his name) too. But atleast that guy programmed on a machine that had potential. (and see what the PC stands for nowadays)

PC world wins! and what?

Quote:

Don't get me wrong Fly, it's a hella good idea and job to be done. And not the least "to be proud of" project too! but euhrm, what's next. . . when your OS is finished. WHAT THEN? I'm curious.

I personaly GAMES & TOOLS!

Quote:

'aving said that. ------- A pre-kernel is nothing more than some fancy pancy load routines if ya'd ask me.

In the case of MNBIOS the pre-kernel needs all the functions of the normal kernel except the VDP/PSG/KEYB/CLOCK/TIMERS functions.
Is to say it needs all the main file's system engine...
It needs the same native's drivers/libraries used then in the main kernel.
It needs all the threading engine... and the function that can create new threads.

And include the addresses of the pre-kernel with the main kernel needs to match.. atleast in the first 8kb.. so it is the same code... that is necesary in way to be able then to remove all the startup routines and remplace it with more functions.

I am more thinking IF will be possible to put all the code inside one PAGE (atleast the 95% of the functions that the BETA version has) and nothing will be OUT of space... than the speed.... an usable speed is guarantee... if it isn't all the proyect dosn't serves.

Quote:

The chicken and the egg problem is always prominant and is a nice one to solve (if at all) What we've done (in the fuzz OS called F-kernel a.k.a. MIDAS): a small pre-kernel (if you want) which does the following: - Set up a loading environment and load up all necessary files (from lib if you want) (not yet booting any module) (this is a stripped DISK module) - check memory and initialize the memory manager (memory chunks are min. 512 bytes. and max 16k.) - initialize all other modules (which use the memory manager to initialize their own working space) - execute project (if any) The kernel is devided on 2 memory maps. the primairy one are the base functions (the REAL OS you might say) and always prominent in page 0 including interrupt handling, memory/page/slot switching, keyboard manager and some more. The workhorses (all the other modules like File and Data Retrieval System, Memory Manager, Disk routines etc) are stored in a secondairy mapper on page 1 which is called through the BASE functions. So the user ends up having atleast 32k in a row available at all times. Yes the kernel takes about 32k (including hi-mem) The pre-kernel (loader I like to call it) takes about an amount of 4k of total RAM (MAX)

4k? nothing more?.... ah.. maybe you are using the DISKROM for disk I/O access?. I am not

Quote:

hope this gives a clue. good luck

Luck no!... TIME TIME TIME!

yup, using DISKROM to boot the kernel. why not. it's there already, might aswell abuse it in order to get the kernel loaded. ofcourse it'll only work on fat systems which, in your case, isn't an option?!

Quote:

yup, using DISKROM to boot the kernel. why not. it's there already, might aswell abuse it in order to get the kernel loaded. ofcourse it'll only work on fat systems which, in your case, isn't an option?!

MNBIOS is FAT 12, 16 and 32 based filesystem...

what everybody see as negative was the 12x4 filenames instead 8.3 that is the reason that i will downgrade the filename.