Acorn Arcade forums: Programming: Filtered image scaling
| |
Filtered image scaling |
| |
This is a long thread. Click here to view the threaded list. |
| |
|
Adrian Lees |
Message #78414, posted by adrianl at 12:20, 7/8/2006 |
Member
Posts: 1637
|
I've decided there's not enough activity on this forum, so....
I'm part-way through writing smooth (filtered) image resampling functions for use in NetSurf (probably to be integrated into Tinct) when images are resized by the html source, or when the page zoom setting is not 100%. I currently have 4 filters:
trapezoidal downsampling (below, say, 50%)
nearest neighbour (used for 100% plotting)
bilinear interpolation(up to, say, 200% plotting)
bicubic interpolation (above, say, 200%)
The filtering algorithms will, of course, be optimised in ARM assembler, using the DSP extensions of ARMv5 where appropriate and available - hey, I'm an addict! - and I suspect it'll be beneficial to overlap the CPU activity (which, especially in the case of bicubic interpolation is pretty intensive) with DMA plotting to screen from a couple of buffers into which the CPU renders.
Oh, if I can get the info and get some code working, there may be merit to using the NVidia card to upsample images when running on an Iyonix.
Now to my question - is this of any use to other people? What can we do with it? Besides its intended use in NetSurf, it has obvious application in Geminus (plotting scaled JPEGs and Sprites to that the image quality of all OS-using applications is improved) and Cino (in the unlikely event of having enough spare CPU cycles to resample the output image!).
Anyone want to steer its development?
[Edited by adrianl at 13:23, 7/8/2006] |
| |
 [ Log in to reply ] |
| |
|
Phil Mellor |
Message #78428, posted by monkeyson2 at 17:07, 7/8/2006, in reply to message #78414 |
 Please don't let them make me be a monkey butler
Posts: 12380
|
Er... dunno. Once you can replace the OS sprite calls with it, loads of things would benefit straight away. I can't think of any application that could specifically use it without having to write the application too. Scaling images nicely in Paint would be nice, instead of having to go via ChangeFSI just to get anti-aliasing.
Using the graphics card acceleration sounds good. Would that only work for sprites being plotted directly to the screen; you couldn't use it if output was switched to another sprite or printer? |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78430, posted by adrianl at 17:40, 7/8/2006, in reply to message #78428 |
Member
Posts: 1637
|
Using the graphics card acceleration sounds good. Would that only work for sprites being plotted directly to the screen; you couldn't use it if output was switched to another sprite or printer? Correct, in practice. If the NV's chipset can access main memory (as opposed to video memory), then I don't know how, and reading back from the NV's memory is very slow even with DMA. |
| |
[ Log in to reply ] |
| |
|
ninjah |
Message #78431, posted by ninj at 17:47, 7/8/2006, in reply to message #78428 |
Member
Posts: 288
|
I agree that if the OS SpriteOp calls could be rewritten to give this functionality, that would be great. I can't quite see how you'd plug it into paint though, since paint doesn't really handle fancy-pants things like /rescaling/. I'm more interested in what it could do for the likes of Draw, Artworks and Compo. Although I don't know which of those use the OS_SpriteOps and which use their own routines. Basically though, anything that renders images to screen would be the sort of thing you'd want to benefit from this, and the majority of those would be caught by an OS_SpriteOp catcher.
I'm slightly worried by the use of 'nearest neighbour' between 50 and 100 percent though. Does this mean dropping pixels, in effect? I was also unable to find anything on trapezoidal downsampling on wikipedia. |
| |
[ Log in to reply ] |
| |
|
Phil Mellor |
Message #78433, posted by monkeyson2 at 18:06, 7/8/2006, in reply to message #78431 |
Please don't let them make me be a monkey butler
Posts: 12380
|
I can't quite see how you'd plug it into paint though, since paint doesn't really handle fancy-pants things like /rescaling/. Edit -> Scale X. Edit -> Scale Y.
Could you get it to do other things, like blurring, sharpening, noise removal? Transformations - matrix, distortions, perspective? Layer masking effects - multiply, colour dodge, hue? Turn it into an all singing, all dancing Photoshop module? |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78434, posted by adrianl at 18:21, 7/8/2006, in reply to message #78431 |
Member
Posts: 1637
|
I'm slightly worried by the use of 'nearest neighbour' between 50 and 100 percent though. Don't be, for two reasons...because I'm actually using bilinear interpolation for 50%-200% atm, excluding 100% where I can just copy directly (trivial case of nearest neighbour; this is the only place it's used currently).
Secondly, the transition points between the different filters haven't been fixed atm; I'll set them according to subjective assessment or do some research on what 'the experts' think.
I was also unable to find anything on trapezoidal downsampling on wikipedia. Indeed. It took me a while to find out what it is and how it works, but it does seem to be quite a bit better than rectangular (mere averaging over all the source pixels that contribute to a destination pixel) in that it gives fewer 'jaggies.'
In effect it's a weighted sum of all the source pixels contributing, but with the edge pixels being less significant. If you think of the source image as a horizontal plane and each pixel's contribution to the current output pixel being the height above the plane then you get a shape like the keycap on a keyboard (probably called a trapezoid, funnily enough  , with sloping edges and a flat top. The source pixels that lie under the slopes have a reduced contribution to the destination pixel being produced, those that lie outsides the keycap make no contribution at all.
[Edited by adrianl at 19:40, 7/8/2006]
[Edited by adrianl at 02:40, 21/4/2007] |
| |
[ Log in to reply ] |
| |
|
ninjah |
Message #78436, posted by ninj at 18:49, 7/8/2006, in reply to message #78434 |
Member
Posts: 288
|
Ah, right. I recall reading that antialiasing for fonts uses something that looks rather like a gaussian curve in both the x and y directions. But wikipedia suggests this is unnecessary when the source image is based on a rectangular grid - of course outline fonts are (at least in theory, I don't know about practice) of infinite resolution. |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78666, posted by adrianl at 16:46, 10/8/2006, in reply to message #78436 |
Member
Posts: 1637
|
There's an early prototype (full of redraw bugs and the odd crash oweing to my laziness! ) here. You will, however, need an Iyonix. That code uses a basic asm implementation for the bicubic filtering, has a rectangular filter in place of trapezoidal (cheaper but not as good; prob will use trapz instead) and DMA plotting overlapped with CPU activity, which is actually beneficial on the bilinear filtering now that it's faster. Many optimisations still to be done.... yada, yada, yada....
WARNING: DO save your work before trying the above code and do NOT be surprised/litigious if it freezes your computer. It contains some shortcuts to get prototype code working sooner and an idea of the performance attainable without worrying about all the corner cases.
[Edited by adrianl at 17:52, 10/8/2006] |
| |
[ Log in to reply ] |
| |
|
richard cheng |
Message #78667, posted by richcheng at 17:25, 10/8/2006, in reply to message #78414 |
Posts: 655
|
Now to my question - is this of any use to other people? What can we do with it? If it does a better job of scaling than ChangeFSI then I'd get great use out of it as a standalone app*. Thumbnails scaled in CFSI look, to be frank, crappy.
*Especially if it's got a command-line interface, or is otherwise scriptable/batch-job allowing.
[Edited by richcheng at 18:26, 10/8/2006] |
| |
[ Log in to reply ] |
| |
|
Richard Goodwin |
Message #78685, posted by rich at 07:19, 11/8/2006, in reply to message #78667 |
 Dictator for life
Posts: 6828
|
If it does a better job of scaling than ChangeFSI then I'd get great use out of it as a standalone app*.
...
*Especially if it's got a command-line interface, or is otherwise scriptable/batch-job allowing. ^ This.
________
 Cheers,
Rich.
|
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78700, posted by adrianl at 11:30, 11/8/2006, in reply to message #78685 |
Member
Posts: 1637
|
Latest code is 10cs vs 12cs for the code that's on the website (bilinear filtering) but I'm not sure that it's possible to optimise it any further on the IOP321 
The problem is that if I read all the source pixels that I need, do nothing with them and write junk to the screen, it's still taking 10ms, ie. the Evil IOP321 Memory Bus strikes again! This is with prefetching, of course, but past experience has shown that the bus really doesn't cope well with simultaneous reading and writing.
*thinks*
Hmm, maybe I need to prefetch a whole cache worth's of data before processing, allocate dirty cache lines for the destination addresses, process data, write data to destination and repeat.
More nastiness to follow later perhaps... I am not satisfied with the speed at the moment and it's frustrating because I can definitely tune the filter kernel quite a bit more (doing so will probably help on other RO machines).
[Edited by adrianl at 12:30, 11/8/2006]
[Edited by adrianl at 12:46, 11/8/2006] |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78702, posted by adrianl at 11:33, 11/8/2006, in reply to message #78685 |
Member
Posts: 1637
|
If it does a better job of scaling than ChangeFSI then I'd get great use out of it as a standalone app*.
...
*Especially if it's got a command-line interface, or is otherwise scriptable/batch-job allowing. ^ This.
Well, I haven't compared with ChangeFSI yet but the scaling does seem to produce good images, such that when I used 150% scale on NS last night with Tinct just doing nearest-neighbour scaling, it made me wince. The trapezoidal filter seems to be especially good for making thumbnails; much better than I'm used to seeing from RO apps/SpriteExtend/Geminus atm. |
| |
[ Log in to reply ] |
| |
|
richard cheng |
Message #78707, posted by richcheng at 12:48, 11/8/2006, in reply to message #78702 |
Posts: 655
|
Cool! I'll give you a fiver for it. |
| |
[ Log in to reply ] |
| |
|
Tony Haines |
Message #78723, posted by Loris at 17:50, 11/8/2006, in reply to message #78414 |
 Ha ha, me mine, mwahahahaha
Posts: 1025
|
Hey Adrian. This is only tangentially related, but I think I heard you were fiddling around with JPEGs...
Wat I want is an app which will convert a JPEG to a sprite, let you fiddle with it an hand it back - and produce a new JPEG which is only different where you've edited it. I want to avoid the quality loss associated with re-JPEGing the image, where I haven't changed it.
This would be good for blurring backgrounds, adding text and so on.
I've looked into JPEG compression, but can't find the nerve to do it myself. |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78725, posted by adrianl at 18:54, 11/8/2006, in reply to message #78723 |
Member
Posts: 1637
|
Hey shAdrian. This is only tangentially related, but I think I heard you were fiddling around with JPEGs... I wrote a JPEG decoder and released it as a Geminus feature, yes.
What I want is an app which will convert a JPEG to a sprite, let you fiddle with it an hand it back - and produce a new JPEG which is only different where you've edited it. I want to avoid the quality loss associated with re-JPEGing the image, where I haven't changed it. Interesting. It's certainly technically possible, I'd imagine that you want unrestricted use of the output sprite, ie. you don't want this program to provide (probably very limited) editing facilities.
Ergo, what's really required is not a decoder, but a JPEG encoder that accepts both a JPEG and a modified sprite; it can then decode the JPEG to produce a sprite, difference the two and re-encode only the changed macroblocks.
I've never written a complete JPEG encoder, as it happens but I've always fancied it, especially since developing my optimised decoder (the JPEG algorithm is approximately symmetric, so an encoder should be almost as fast as my decoder, ie. about 20fps at DVD resolution, say. Actually, make that 10fps because I suspect the IOP321's memory bus could make a good attempt at crippling the encoding performance, oweing to the need to read in a lot of uncompressed pixel data).
I've looked into JPEG compression, but can't find the nerve to do it myself. Mmm, it is quite complicated, especially if you have no control over the format of the files you're receiving.
[Edited by adrianl at 19:55, 11/8/2006] |
| |
[ Log in to reply ] |
| |
|
Tony Haines |
Message #78736, posted by Loris at 10:28, 12/8/2006, in reply to message #78725 |
Ha ha, me mine, mwahahahaha
Posts: 1025
|
What I want is an app which will convert a JPEG to a sprite, let you fiddle with it and hand it back - and produce a new JPEG which is only different where you've edited it. I want to avoid the quality loss associated with re-JPEGing the image, where I haven't changed it. Interesting. It's certainly technically possible, I'd imagine that you want unrestricted use of the output sprite, ie. you don't want this program to provide (probably very limited) editing facilities.
Yes - why re-invent the wheel. Also, if you can pass it off to another program, the user can use their favourite app, or some custom tool.
Ergo, what's really required is not a decoder, but a JPEG encoder that accepts both a JPEG and a modified sprite; it can then decode the JPEG to produce a sprite, difference the two and re-encode only the changed macroblocks. Essentially, yes. I think you'd want to create the sprite yourself in the first place to make sure that there are no differences in the decoding algorithm, like variations in rounding errors. Then the app could potentially hang on to it (or let you re-supply it) to save having to do it again - if that were faster.
I've never written a complete JPEG encoder, as it happens but I've always fancied it, especially since developing my optimised decoder (the JPEG algorithm is approximately symmetric, so an encoder should be almost as fast as my decoder, ie. about 20fps at DVD resolution, say. Actually, make that 10fps because I suspect the IOP321's memory bus could make a good attempt at crippling the encoding performance, oweing to the need to read in a lot of uncompressed pixel data). The speed isn't all that important compared to the result quality. Of course the faster the better, optimising is good, but I wait for ChangeFSI now.
Go on then - what do you say? I know you like a challenge.
I'm sure quite a few people would buy it if you priced it low. |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #78737, posted by Phlamethrower at 10:43, 12/8/2006, in reply to message #78736 |
 Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 15100
|
Go on then - what do you say? I know you like a challenge.
I'm sure quite a few people would buy it if you priced it low. It's certainly possible to do, but a quick search doesn't reveal any examples where the source code is available 
http://www.betterjpeg.com/jpeg-plug-in.htm |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78738, posted by adrianl at 16:05, 12/8/2006, in reply to message #78737 |
Member
Posts: 1637
|
Thinking about this a bit more, it's actually quite easy starting with my existing JPEG decoder; it needs some code to encode macroblocks using the tables already available in the decoder and some trivial differencing code to work on the two lots of sprite data. That's all.
My JPEG decoder already extracts information on each macroblock because it greatly accelerates redraws by cacheing this information and then diving into the middle of the JPEG bitstream to decode only the bits it needs.
[Edited by adrianl at 17:05, 12/8/2006] |
| |
[ Log in to reply ] |
| |
|
Tony Haines |
Message #78774, posted by Loris at 13:50, 15/8/2006, in reply to message #78738 |
Ha ha, me mine, mwahahahaha
Posts: 1025
|
Thinking about this a bit more, it's actually quite easy starting with my existing JPEG decoder; it needs some code to encode macroblocks using the tables already available in the decoder and some trivial differencing code to work on the two lots of sprite data. That's all. Um... does that mean that you'll do it, or that it is too easy to be interesting?
Please make it! |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78800, posted by adrianl at 12:03, 16/8/2006, in reply to message #78774 |
Member
Posts: 1637
|
Thinking about this a bit more, it's actually quite easy starting with my existing JPEG decoder; it needs some code to encode macroblocks using the tables already available in the decoder and some trivial differencing code to work on the two lots of sprite data. That's all. Um... does that mean that you'll do it, or that it is too easy to be interesting?
Please make it!
It means I will add it to my list of interesting projects and probably at least prototype it at some point, but I have oodles of other work to be doing too. |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #78993, posted by adrianl at 08:26, 22/8/2006, in reply to message #78800 |
Member
Posts: 1637
|
Latest filtered scaling code is on drobe (coz aemulor.com appears to have gone gluteus maximus over pectoralis major) for any Ix owners who want to play:
http://adrianl.drobe.co.uk/ns/scaling.zip
Oh, space bar toggles between the various filter kernels, Select zooms in, Adjust zooms out.
[Edited by adrianl at 09:26, 22/8/2006] |
| |
[ Log in to reply ] |
| |
|
Ian Cook |
Message #79035, posted by ilcook at 18:32, 22/8/2006, in reply to message #78993 |
 Resident idiot
Posts: 1077
|
Latest filtered scaling code is on drobe (coz aemulor.com appears to have gone gluteus maximus over pectoralis major) for any Ix owners who want to play:
http://adrianl.drobe.co.uk/ns/scaling.zip
Oh, space bar toggles between the various filter kernels, Select zooms in, Adjust zooms out. All I get is. Forbidden You don't have permission to access /ns/scaling.zip  |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #79068, posted by adrianl at 02:38, 23/8/2006, in reply to message #79035 |
Member
Posts: 1637
|
That appears to be some silliness/politics between drobe and TIB. If you hit return in the URL bar after clicking on the link, or type it in manually it works. I think Drobe is rejecting any referrals from TIB which is not very helpful for users trying to host stuff on the webspace they kindly make available |
| |
[ Log in to reply ] |
| |
|
Phil Mellor |
Message #79069, posted by monkeyson2 at 02:45, 23/8/2006, in reply to message #79068 |
Please don't let them make me be a monkey butler
Posts: 12380
|
That appears to be some silliness/politics between drobe and TIB. If you hit return in the URL bar after clicking on the link, or type it in manually it works. I think Drobe is rejecting any referrals from TIB which is not very helpful for users trying to host stuff on the webspace they kindly make available /me *points at the 'add attachment' link. |
| |
[ Log in to reply ] |
| |
|
VinceH |
Message #79076, posted by VincceH at 08:00, 23/8/2006, in reply to message #79068 |
 Lowering the tone since the dawn of time
Posts: 1600
|
I think Drobe is rejecting any referrals from TIB which is not very helpful for users trying to host stuff on the webspace they kindly make available Are you sure it isn't just direct links to images and downloadable files from anywhere outside of Drobe? |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #79084, posted by adrianl at 11:30, 23/8/2006, in reply to message #79076 |
Member
Posts: 1637
|
Well, I don't know what the rule is, but something's not right. It's back in its original home now too. |
| |
[ Log in to reply ] |
| |
|
ninjah |
Message #79106, posted by ninj at 17:44, 23/8/2006, in reply to message #79076 |
Member
Posts: 288
|
I think Drobe is rejecting any referrals from TIB which is not very helpful for users trying to host stuff on the webspace they kindly make available Are you sure it isn't just direct links to images and downloadable files from anywhere outside of Drobe?
I've just tested disabling referrers...
/remembers to turn referrers back on...
You're right - it's not TIB/Drobe nastiness. It doesn't like any referrers at all whether from here or elsewhere. Presumably that restriction isn't applied to HTML file extensions - presumably it's to stop you using your webspace for file storage.
Naughty Adrian! |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #79265, posted by adrianl at 06:37, 27/8/2006, in reply to message #78993 |
Member
Posts: 1637
|
Both URLs now updated with newer, though still incomplete, code that can actually be driven from the command line to batch process files so, plz Richard, let me know if there's anything that you'd like improved in any way.
Edit: Oh, it probably won't work on non-Iyonix machines atm, at least for some of the filter kernels because they take advantage of the ARMv5TE extensions and I don't yet have substitute routines for earlier processors.
[Edited by adrianl at 07:40, 27/8/2006] |
| |
[ Log in to reply ] |
| |
|
Adrian Lees |
Message #79412, posted by adrianl at 23:58, 28/8/2006, in reply to message #79265 |
Member
Posts: 1637
|
Merged into Geminus, at least for now, so that it's available to all apps. Screenshots and verbiage: http://adrian.aemulor.com/scaling.html |
| |
[ Log in to reply ] |
| |
|
Phil Mellor |
Message #79415, posted by monkeyson2 at 00:05, 29/8/2006, in reply to message #79412 |
Please don't let them make me be a monkey butler
Posts: 12380
|
Neat! |
| |
[ Log in to reply ] |
| |
| Pages (2): 1
> >|
|
Acorn Arcade forums: Programming: Filtered image scaling |
|
