new hybrid mode found

bluemsx and openmsx do character masking wrong

Those bits are just simply a mask operation on the "ascii" values of nametable entries.
VDP(3) = VDP(3) XOR 16 masks the highest bit of the "ascii".

I got this from this cool TI99 users video, nicely describing the effect: http://www.youtube.com/watch?v=XJljSJqzDR0

I checked on Hit Bit 75 with 9929 PAL, it works like the TI99 in the video.
Like the video shows at 12:05 - character masking got no effect on the sprites.

I would never have expected that I would ever learn something totaly new about the 9918!
I think the feature is underrated because it is not clear how this enables switching between charsets.
e.g. when the "ascii" values always got top 3 bits set, then with the mask register you can switch between 8 charsets a 32 chars.

Here is a test app.

It got nametable contents different than the video to make clear that this time the mask is not about the scanline Y value but about the ascii value. Hit a key to switch to the other halve of the charset.

10 SCREEN 2,2
20 SPRITE $(0)="ldscktjhklstcdhpxksjdhksjhdskjhskjhsd"
30 FOR I = 0 TO 31
40 PUT SPRITE I,(I*8, I*5),I AND 15,0
50 NEXT
60 ON STRIG GOSUB 200
70 STRIG(0) ON
80 OPEN "grp:" AS #1
90 FOR I = 0 TO 2
100 LINE (255,I*64+63)-(0,I*64),I,BF : PRESET(0,I*64),I
110 FOR C = 0 TO 255 : PRINT#1,CHR$(C); : NEXT
120 NEXT
130 FOR I = 0 TO 127
140 VPOKE(6144+I),I+128
150 VPOKE(6144+I+256),I+128
160 VPOKE(6144+I+512),I+128
170 NEXT
180 GOSUB 200
190 GOTO 190
200 VDP(3) = VDP(3) XOR 16
210 RETURN

That's pretty cool! =)
Does it work on V9938 too?

I tried your test on a TMS9129, and it didn't mask characters, just the colors.
I suppose this issue only happens on TMS99xx chips then.

@ Maggoo:
This may or may not count for every systemconfig. So to put it in Wiki, it should be 100% clear how this works and in what circumstance.

I agree on that 100% facts on vdp , registers and such could be in the wiki. 
But what are the boundries for the wiki?