|
|
Acorn Arcade forums: Programming: Backporting OS_DynamicArea to RO3.1
| |
Backporting OS_DynamicArea to RO3.1 |
| |
sirbod (11:02 3/4/2013)
arawnsley (11:07 3/4/2013)
sirbod (11:18 3/4/2013)
nunfetishist (12:05 3/4/2013)
sirbod (18:49 3/4/2013)
sirbod (22:59 4/4/2013)
sirbod (09:11 5/4/2013)
sirbod (22:11 19/4/2013)
sirbod (07:09 4/4/2013)
sirbod (12:29 3/4/2013)
Phlamethrower (13:09 3/4/2013)
swirlythingy (13:41 3/4/2013)
Phlamethrower (13:57 3/4/2013)
swirlythingy (14:43 3/4/2013)
Phlamethrower (16:08 3/4/2013)
sirbod (16:33 3/4/2013)
|
| |
|
Jon Abbott |
Message #122238, posted by sirbod at 11:02, 3/4/2013 |
Member
Posts: 563
|
I need to backport this to RO3.1, to enable ADFFS to use DA's instead of RMA for floppy images. More specifically, I need to implement:
OS_DynamicArea 0, where R1=-1 (ie allocate an area number)
OS_DynamicArea 1, where R1=our area (ie remove area)
OS_ChangeDynamicArea, where R0=our area
Does anyone have any hints / tips?
Has it already been done?
Implementing the SWI hook is already done, so its simply a matter of allocating and freeing memory blocks.
Q1. How do I choose free MemMap entries on RO3.1? Do I simply allocate from the top down, how can I check if they're actually free or have been previously allocated by the OS?
Q2. Should I issue a Service_MemoryMoved after making changes?
[Edited by sirbod at 12:11, 3/4/2013] |
| |
 [ Log in to reply ] |
| |
|
Andrew Rawnsley |
Message #122239, posted by arawnsley at 11:07, 3/4/2013, in reply to message #122238 |
R-Comp chap
Posts: 600
|
AFAIK most apps that need DA type behaviour on 3.1 tended to use RMA anyway, if flex() / wimpslot wasn't sufficient.
This would suggest to me that two codepaths would be the best bet - a DA-based one for 3.5+ and an RMA-based one for 3.1 |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122240, posted by sirbod at 11:18, 3/4/2013, in reply to message #122239 |
Member
Posts: 563
|
This would suggest to me that two codepaths would be the best bet - a DA-based one for 3.5+ and an RMA-based one for 3.1 That's the problem, RMA isn't the best method on RO3.1 because a lot of modules crash on RMTidy, so it effectively legally leaks memory in large chunks.
Example: ADFFS allocates 700k for disc 1, game loads and installs several modules and then requests disc 2. ADFFS releases 700k from disc 1 image and requests 800k for disc 2. It won't fit on the 700k hole, so attempts to allocate another 800k to RMA, leaving a leak of 700k. If disc 3 subsequently needs 820k, that's 1.5mb leaked...see the problem I'm facing!
The only way to fix this for RO3.1 (which is really the target as most games only run on RO3.1) is to implement dynamic memory allocation. Backporting OS_DynamicArea seems like the simplest of the solutions I've considered. |
| |
[ Log in to reply ] |
| |
|
Rob Kendrick |
Message #122241, posted by nunfetishist at 12:05, 3/4/2013, in reply to message #122240 |
 Today's phish is trout a la creme.
Posts: 524
|
The only way to fix this for RO3.1 (which is really the target as most games only run on RO3.1) is to implement dynamic memory allocation. Backporting OS_DynamicArea seems like the simplest of the solutions I've considered. It isn't that simple: where in the memory map are you going to shove them?
Solutions I've seen in the past involved shoving data in the sprite area or, if you're a WIMP task, shoving it into another always-idle task and swapping the data in and out using Wimp_TransferArea |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122242, posted by sirbod at 12:29, 3/4/2013, in reply to message #122238 |
Member
Posts: 563
|
Looking at memory allocation at the Supervisor, every page is allocated. Scanning from logical &8000 up will find the end of the application space, but telling if they're in use is an interesting problem.
It doesn't look like there's an obvious way to tell if a page is used by an application, so I can only presume RO just nabs blocks from the top of application space and presumes nothing is using them. I'll need to see what OS_ChangeDynamicArea does currently when it allocates pages.
Q3. Can I use Wimp_SlotSize outside of the Wimp, whilst at the Supervisor?
There is the possibility of setting the next slot to the memory size required, then spawn an application to allocate it and pass the address back to ADFFS. To free I then simply close the application.
This has the added benefit of leaving the OS to allocate the memory. However, this will only work if RO3.1 doesn't page swap.
Q4. Does RO3.1 page swap (ie all apps run from 8000 for example)? Or are all applications in distinct memory addresses? |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #122243, posted by Phlamethrower at 13:09, 3/4/2013, in reply to message #122242 |
 Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
It doesn't look like there's an obvious way to tell if a page is used by an application, so I can only presume RO just nabs blocks from the top of application space and presumes nothing is using them. Pretty much, yeah. However it does issue Service_Memory before moving any pages, so if something is using application space it can prevent the move from happening if it wants.
For RISC OS 3.5+ there's also the free pool DA, which the OS will prefer to take pages from before resorting to taking them from application space.
Q4. Does RO3.1 page swap (ie all apps run from 8000 for example)? Or are all applications in distinct memory addresses? The only version of RISC OS that doesn't do page swapping is Arthur. |
| |
[ Log in to reply ] |
| |
|
Martin Bazley |
Message #122244, posted by swirlythingy at 13:41, 3/4/2013, in reply to message #122242 |
Posts: 460
|
Q3. Can I use Wimp_SlotSize outside of the Wimp, whilst at the Supervisor? Yes. Wimp_SlotSize doesn't need Wimp_Initialise to have been called. It's a handy way of rolling your own dynamic memory allocation in BASIC.
There is the possibility of setting the next slot to the memory size required, then spawn an application to allocate it and pass the address back to ADFFS. To free I then simply close the application. When you say "pass the address", I presume you mean the physical address. When the application is not swapped in, AIUI the memory isn't mapped to any logical address. (Of course RO3.1 page swaps! How did you manage to get this far without knowing that?)
On 26-bit only machines, mapping it to something stupidly high and hoping nothing else wanted that address is probably safe, since addresses above 64MB didn't really get used (and you won't be executing code in it). This will probably break on newer (or even RiscPC-era) machines, but I presume you're just going to use OS_DynamicArea for those anyway.
As long as RISC OS never tries to swap your application in - there might be a way to physically prevent that if you dig - it should be none the wiser. It might even work regardless - I don't know if physical memory is allowed to be mapped to more than one logical address or not, but I suspect not.
I'm no expert on this, but check out OS_FindMemMapEntries and OS_SetMemMapEntries. Possibly OS_Memory as well (I can't remember if that was an RO3.5 thing or not).
[Edited by swirlythingy at 14:45, 3/4/2013] |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #122245, posted by Phlamethrower at 13:57, 3/4/2013, in reply to message #122244 |
Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
On 26-bit only machines, mapping it to something stupidly high and hoping nothing else wanted that address is probably safe, since addresses above 64MB didn't really get used (and you won't be executing code in it). Addresses above 64MB "didn't really get used" on pre-RiscPC machines because it was impossible to use them. All you'll get is the CPU throwing an address exception. |
| |
[ Log in to reply ] |
| |
|
Martin Bazley |
Message #122246, posted by swirlythingy at 14:43, 3/4/2013, in reply to message #122245 |
Posts: 460
|
On 26-bit only machines, mapping it to something stupidly high and hoping nothing else wanted that address is probably safe, since addresses above 64MB didn't really get used (and you won't be executing code in it). Addresses above 64MB "didn't really get used" on pre-RiscPC machines because it was impossible to use them. All you'll get is the CPU throwing an address exception.
I thought that only applied to code (i.e. PC flag bits restrictions), not data? Mind you, I did say I wasn't an expert.
Is it legal to use them on RiscPC, but not pre-RiscPC? |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #122247, posted by Phlamethrower at 16:08, 3/4/2013, in reply to message #122246 |
Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
On 26-bit only machines, mapping it to something stupidly high and hoping nothing else wanted that address is probably safe, since addresses above 64MB didn't really get used (and you won't be executing code in it). Addresses above 64MB "didn't really get used" on pre-RiscPC machines because it was impossible to use them. All you'll get is the CPU throwing an address exception.
I thought that only applied to code (i.e. PC flag bits restrictions), not data?
Nope, both code and data (at least as far as ArcEm is concerned! I realised I wasn't entirely sure myself).
Anyway, if it didn't abort you'd still be stuck with the fact that MEMC only does address translation on the first 32MB of logical address space.
One option could be to avoid allocating logical address space and use the physical mappings directly (via the 32MB-64MB address range). But that might require a bit more complexity in your code as you may not be able to get contiguous physical pages.
Is it legal to use them on RiscPC, but not pre-RiscPC? Yes, addresses above 64MB are perfectly legal on a RiscPC (as long as you're not intending to use code above 64MB while in a 26bit mode, of course) |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122248, posted by sirbod at 16:33, 3/4/2013, in reply to message #122238 |
Member
Posts: 563
|
To be clear, this is only being implemented for RO3.1, RO3.5+ will use the OS provided OS_DynamicArea
Wimp_SlotSize, R0 + &8000 does indeed point to the end of user memory when the Wimp is shutdown.
That covers things when the Wimp is shutdown. If I end up taking memory in use by the app, it should simply Abort. That's acceptable - well, its not, but probably the best case when memory runs out.
The problem comes when the Wimp is launched, provided it only takes memory logically mapped to &8000+ there's not a problem. If on the other hand it takes all memory not offically allocated to a known DA, I've got a problem. I'll have to test and see what happens.
The last issue is what happens to allocated memory allocated by the Wimp to tasks after the Wimp shuts down. I'm guessing it will re-allocate any task memory back to application space. I could in theory check after the Wimp has closed and remap my pages back, rejigging &8000+ if necessary, so there's a continuous block. I'd then have to correct Wimp_SlotSize. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122252, posted by sirbod at 18:49, 3/4/2013, in reply to message #122241 |
Member
Posts: 563
|
Backporting OS_DynamicArea seems like the simplest of the solutions I've considered. It isn't that simple: where in the memory map are you going to shove them?
Solutions I've seen in the past involved shoving data in the sprite area or, if you're a WIMP task, shoving it into another always-idle task and swapping the data in and out using Wimp_TransferArea
No idea where it will end up, I'd need a 2mb gap somewhere.
It is actually deceptively simple, a few lines to allocate memory and a few more to remap them. ADFFS already hooks into the SWI vector so it only requires a check on the OS version to implement for RO3.1 only.
Implementing within the Sprite Area was ruled out early on, as it vastly complicates things. Quite a lot of games alter the sprite memory area, not to mention the issues around storing non-sprite data within sprites and any potential issues around hitting limits.
What you suggest for Wimp is feasible, although Wimp_TransferArea probablywon't be used, I'll simply remap memory to the ADFFS buffer area and leave a stub block behind to deal with Wimp polling.
If I can find a way to grab some Wimp free memory without breaking things, that would be simpler. I don't think there's a way to do it though.
I'll tackle the problem as three distinct paths:
1. Supervisor prompt (Wimp not running)
2. Wimp running
3. Transition as Wimp starts up or shuts down |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122255, posted by sirbod at 07:09, 4/4/2013, in reply to message #122241 |
Member
Posts: 563
|
where in the memory map are you going to shove them? There appears to be two gaps that are suitable candidates:
1900000 - 1BFFFFF (3MB)
1D00000 - 1EFFFFF (2MB)
EDIT: 1D00000 looks like the only viable option
[Edited by sirbod at 09:04, 4/4/2013] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122265, posted by sirbod at 22:59, 4/4/2013, in reply to message #122252 |
Member
Posts: 563
|
I'll tackle the problem as three distinct paths:
1. Supervisor prompt (Wimp not running)
2. Wimp running
3. Transition as Wimp starts up or shuts down I now have 1 and 3 implemented, however...
Q. Is there an official way (on RO3.1) to notify the OS that MemLimit and AplWorkSize have changed?
The RO source alters DA 6, but that just returns an error so may not be implemented on RO3.1.
For the time being, I'm loading the values from &11C and &570 respectively and reducing them by the amount relocated.
Doing this I can go from Supervisor to Wimp and back and everything behaves as it should. The Wimp Free slot reduces by the correct amount and BASIC shows the correct HIMEM.
I just need to figure out how to handle things whilst the Wimp is active. The RO source uses a "magic number" for the Service_Memory call, but again it's returning an error, so may not be relevant to RO3.1
[Edited by sirbod at 00:00, 5/4/2013] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122269, posted by sirbod at 09:11, 5/4/2013, in reply to message #122265 |
Member
Posts: 563
|
Is it possible to issue a Message_SlotSize so your app displays a slot size, without actually allocating one via Wimp_SlotSize?
I'd like to display the memory allocated to ADFFS in the tasks list.
EDIT: Answering my own question, yes it is possible to set your slot size without issuing Wimp_SlotSize.
[Edited by sirbod at 11:11, 5/4/2013] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #122312, posted by sirbod at 22:11, 19/4/2013, in reply to message #122265 |
Member
Posts: 563
|
I've near enough implemented all of this, mostly with official SWI's. When the WIMP's in control, I've had to resort to changing the WIMP's private module space* as there's no way to reallocate free memory, without it being allocated to either a running task or an existing DA.
There's one side effect though, the "System heap/stack" slot increases by the amount of memory taken, so I presume that the WIMP subtracts all memory slots from the memory size and displays the result in this slot...or I've simply missed something. You can still drag this slot up, and even down if you've previously increased it, so I presume it unallocates memory associated to this slot.
I now need to add all the error handling and unrack code to hand memory back to the WIMP/application space, then get some people to test once it's in ADFFS.
The end result should mean that there's no longer any RMA being wasted due to modules being loaded between swapping floppy discs**, and it enables me to start on the delta save code so that hiscores etc are written back to a delta change file in a write-thru or write-back fashion, depending on the flush setting of the mounted floppy.
The one thing I'm not sure on however, is how RO3.1 determines if memory can be moved to a DA when outside of the WIMP. It can't simply take memory from the top of the application space, as that could potentially cause chaos when memory is tight. Unfortunately, this is when ADFFS will be allocating memory most of the time, so I will need to disassemble OS_ChangeDynamicArea to see what it does.
*orig_memorylimit and orig_applicationspacesize have to be reduced by the amount taken. This ensures that application space is the correct size when the WIMP quits.
**Chuck Rock is a good example, where it simply fails to load the 2nd floppy on a 4mb Arc as it can't allocate enough RMA, due to modules being loaded by the game and a huge RMA gap when it releases the 1st floppy block. |
| |
[ Log in to reply ] |
| |
|
Acorn Arcade forums: Programming: Backporting OS_DynamicArea to RO3.1 | |
|
