Quote:

Yeah well.. F-kernel wasn't designed for 64K RAM, was it... About speed.. now that Bitbuster is more optimized, I think TCF is takes about the same time to decompress. The decompressor is 153 bytes, no additional memory required.

Hmm, do you read 1 byte at the time from disk if no mem. required? What does it do with speed? I'm curious, sounds cool.

Just the standard DOS 128 byte block size.

#0080?

~Grauw

figures (technically it DOES require additional mem. since you can't just figure that every body uses DOS.. but hell)
We decided to use a buffer of 16k. sinces we allready had this (temp) map available in the kernel. A small buffer is cool, but slowes down the decomp. time on the other hand (more diskaction) (tho, RT has a 'templen' byte which tells him the bufferlenght, so it could be downsized, but then you also need to compress it with that buflen)

Well, MSD build in a comp.type.id a user can decide for him/her self which to use... there are a vast amound of great comps out there (we just proved it)

Everybody that has a diskdrive has DOS (or BDOS at least, and that's what counts here). Anyway, a different sized buffer could be used, there's no recompression needed. There's no difference in fileformat compared to RAM, VRAM or direct from disk data.

Anyways, though multiple compression schemes could ofcourse be used in G9B, it would be kinda 'ok' if TCF is the default one, if you ask me . The decompressor being only 153 bytes, it could be added easily, and there is a (PC-)compressor already available...

So, ah, well... I dunno. Maybe before TCF is finalized someone else will come up with even better compression . And there's still the speed issue we didn't measure.

~Grauw

We tested some speeds with a file ~16k, and:

BB (new): 1264038 clock ticks (0,35 s)
BB: 1435048 clock ticks (0,40 s)
TCF (new): 1498766 clock ticks (0,42 s)
apack: 1590929 clock ticks (0,45 s)
TCF: 1860128 clock ticks (0,52 s)

Also, compressed file sizes for that particular file (originally 16391 bytes):

apack: 5187
BB: 5321
TCF: 5345

As you can see, if you compare these results with the compression size tests of earlier, it differs quite a bit depending on the input.

Anyways, so, which is the fastest is clear ^_^. However also remember that this is only one single test case (though the gaps are pretty large to overcome... ). And it's been a nice evening discussing (de)compression on IRC, so surely things will still change for the better! ^_^.


~Grauw

Could you test this with some yuv or 16bit rgb pictures too?

The TCF decompressor used in this test was size-optimized. I've just spent 15 minutes optimizing the decompressor for speed. As I do not have the testcase, I can't post results right now, but I'm sure they'll be quite a bit better for TCF

Still... All this talk about compression engines is nice, but we're actually talking G9B here.
That means there should be a MSX compressor, which isn't available for any of the 3 formats. (for apack there's not even sourcecode for the compressor available)
Or isn't that necessary?

Since we're talking graphics here, I think maybe a difference compression (like T&E Soft's CMP) would be a better idea anyway. Difference compression is easy to program on MSX, unlike any of the LZ packers we've tested until now.

I'm thinking: XOR each line with the previous, do a MTF (Move To Front) on the output and compress with static huffman.

New TNI Compression Format benchmark (same test, optimized engine):

1498766 clock ticks (0,42 s)

I edited this result into the earlier post (compression ratio is the same), and also recalculated all times in s as I made a small miscalculation (I multiplied the Z80's 3.57 MHz two times with 1024 instead of 1000 -_-;.


~Grauw

Btw, these tests were conducted in openMSX using the debugdevice. The given emutime translate to clock cycli when divided by 24. Aso, all three tests were run with interrupts disabled.

~Grauw

I just found out that my timing for bitbuster wasn't correct, due to a difference between having DOS2 inserted and having it not inserted (you could also blame it on my bad coding though )

The number of cycles for that test should be 1435048, making it just a little faster than TCF. However, I got some tips from sjoerd which makes it easier & faster to decode the match length values. Using this optimization, the decompression takes 1264038 cycles. I also tested in R800 mode, in which it only takes 135887 cycles. That is more than 9 times as fast as in z80 mode!

You tested R800 on openMSX? Because I don't think openMSX has accurate timings for R800 yet. There's all kinds of stalls and penalties in R800 which are poorly documented.

uhm yeah, that's tested on openMSX since I don't have a TR.

I updated the test results in the message above.

~Grauw