Sounds like a good idea snout. The only problem would be security, you wouldn't want an 'intruder' to connect to the GameStats server mimicking an MSX emulator, or even worse: vice versa. If you manage to get around that problem, I'm all for it.

BiFi: could you explain why you think it's logical an input recording won't certainly playback exactly as recorded?

*edit* I can come up with one thing: faulty emulation of the MSX clock chip "RTC?", caused by the PC clock emulating the MSX clock.

Yes, why wouldn't it be able to output 100% the same, BiFi? The EmuTime timestamps are logged with the event....

The RTC is a good point indeed: when recording, the RTC clock should be saved, I suppose and when replaying the clock should not be synced to the PC clock. Or something. But in practice, there is no software (AFAIK) that depends on the RTC clock's time.

Regarding the security: this user could then submit that he is playing a certain game A LOT. But still, he will need to login, so it is known that he did it.

the RTC is one of them indeed, but there's also the moment you start recording or replaying and what to think about randomizer stuff in a game?

Even when recording the RTC shouldn't even been synced to the real RTC of the emulating PC/Mac/Sun/....
Everything that is related to something of the 'outside world' could break the replay-ability of the recording.
So you should discouple the RTC and save it's current value when you start recording, and the same would go for every content of sram, currently used machine + Z80 or R800 is active if appropriate (for not CPU aware games)+inserted extensions + inserted disks + inserted cartrdiges etc etc since you never know if there would be some bizare programmer who used this as a seed for the random function.

I was thinking more in the sense of requesting/sending lots of random data to mess up or overload the GameStats server, and like I said, it would be even worse if an intruder managed to connect to the emulator, attacking the user's computer.

BiFi: as far as I'm aware, 'randomizer stuff in a game' are simply calculations done on the Z80, and mind that the Z80 is not random at all, just fast enough to do run complex algorithms on it so end-users might think it's actually random.

Quote:

'randomizer stuff in a game' are simply calculations done on the Z80, and mind that the Z80 is not random at all, just fast enough to do run complex algorithms on it so end-users might think it's actually random.

But it is the determination of the seed that is important. Remember the A=RND(-TIME) you see in almost all MSX-BASIC games...

Quote:

The only problem would be security, you wouldn't want an 'intruder' to connect to the GameStats server mimicking an MSX emulator, or even worse: vice versa. If you manage to get around that problem, I'm all for it.

Although the security issues needs to be addressed, I think there aren't too many possible issues if the system is designed well. With the services discussed so far its really not that hard. I would prefer a POST/GET type interface to the server, similar to a standard webserver, i.e. an emulator could request updates to the database, screenshots and game info which is sent back to the emulator. For stats and highscore I suggest ssl or something similar which requires a secure login and only registered users could send stats. Anonymous stats is also possible without security issues although then its easier for someone to make a 'fake' emulator and post stats that are made up.

I think the biggest problem is DoS attacks on the server but this is a well known problem that all web services like online banking suffers from. There are decent solutions to this problem that are easy to implement.
For the emulator the only problem is spoofing attacks and if a user connects to a malicious server. This is really not an issue since users will connect to well known servers.
Wiretapping is also a potential problem but the reqests to the rom database is really nothing of interest. And if we use ssl (or similar) we have the same security as online banks in the US. And if ssl is good enough for banks its definately good enough for this purpose.

Except the stats part, the online rom database is nothing different from a regular web server. The emulator requests information which is sent back to the emulator. People that think this is a security issue should stop using the web immediately.

@BiFi, The information from the database will NOT be viewed in a browser window. Requesting the information from an online server is nothing different from requesting information from the local softwaredb.xml database. The service only adds value to the users, e.g. the ability to view screenshots and detailed information about games. The presentation is decided by each emulator.

So there will not be any URL's you need to click to get the information and no commercial popups or something like that. To the user it will look exactly as it does now. The difference is that we'll get the possibility to update the softwaredb.xml from within the emulator (if the user wants of course) and show additional info.


So the whole idea is not to boost the number of web hits on the generation-msx or MRC websites. Its to create one place where all game info can be stored instead of spreading the info at different locations. This will definately make management of game information be easier and as a bonus it will provide additional value to the users, like screenshots.

The only time the generation-msx or MRC (or other webistes) needs to be used is if someone wants to look at the statistics in a web browser. But for users that doesn't have a web browser, it will work just fine. Database updates or screenshots will not require a web browser.

after ranting some non-MSX-relations off my mind @ the newspost, I'd just like to add that indeed it will not be possible to recognize all games. Multiple games per disk, for instance, are a great example of a no-way-Jose-situation. Still, it -will- be possible to recognize all ROMs, and the vast majority of disk games (especially when allowing multiple hashes per games, which is only a logical thing to do). Provided, of course, that mars2k doesn't evilly change a few bytes per game ^_^.

@Vampier and mars2000you: I do understand your concern though. You have put a LOT of time and effort into making an excellent database with rom info and I definatley understand that you want to protect your rights. The work you are doing is protected so noone can steal it and noone will. The XML format is GPLed and I'm not sure what license the content of the database is under. I was hoping that you could continue your great work and merge it with the generation-msx database. This of course requires you full access and the same flexibility and desicion making and development as you have now. The only real difference is that there will be more people managing it (like sander).
Having one database has several advantages. The biggest advantage is that all information about games will be stored in one location instead of having two or more databases. This will improve maintainablility and also make it easier for other applications to use the information. Having it online also makes it easier for emulators (and other applications) to access the information.

dvik: thanks for your thorough analysis of the security stuff and confirming the things I've been saying. I hope it did arrive this time. About what you said on the database: indeed, that's exactly the idea. I think that Sandy Pleyte (owner of generation-msx.nl) would be more than happy to have more people improving the data in the database.

BiFi: as turbor and jonemaan already indicated:
- the moment is indeed interesting. The simplest way is to record only from the beginning (from MSX power on). The next best thing is to start from a save state. In both cases the state is 100% determined and 100% reproducible at 100% accuracy (if save state is implemented well, of course). So, when you start to replay, you have to start at the proper state, so with the proper save state or from power on, if you used that.
- randomness is not an issue, because an emulated MSX is 100% deterministic (try it: make a program that autostarts when you boot the emulator (so that it requires no external inputs), whatever you do, it will give exactly the same output, every time!), at least if you take all external inputs with you in the logging. So, if you use the RTC as random seed, you also need to take the RTC in your save state as turbor (and I) already indicated.

Any more concerns?

nope, but prepare for exceptions to all these things... past, present and future...

In blueMSX, the RTC is only synced to PC time when the emulation is started from the beginning. After that it is synced to emutime. This is a (almost) requirement when running the emulator at other speeds than 100%. So if you run the emulator at max speed, the RTC will tick at max speed as well. Also the RTC time is saved in the save state. So if you load a save state, you'll notice that the RTC time is the time when you saved the state.

There are also other possible random inputs, like uarts and parallel ports and sampling (video/audio) but if these are logged as well, it shouldn't cause any randomness.

I can't think of any other random inputs but I may have forgotten something. But if all random inputs are logged the replay should be 100% deterministic. Its just a matter of making sure all random inputs are logged (or handled, like the RTC in blueMSX).

Quote:

Remember the A=RND(-TIME) you see in almost all MSX-BASIC games...

... And which often is not random at all, since TIME will be set to 0 at the start of the BASIC program

AFAIK TIME==JIFFY, or isn't it? AFAIK it's not set to 0 at the start of a basic program, but you can manually do so. In any case, TIME is 100% deterministic, because JIFFY is (If I'm correct, of course.).

BiFi: OK, we will have a problem with warp, time travel, black holes and other annoying universe features, indeed

Quote:

Quote: Remember the A=RND(-TIME) you see in almost all MSX-BASIC games... ... And which often is not random at all, since TIME will be set to 0 at the start of the BASIC program

Just tested it on an emulated turbo-r:

1 print time
run
487
Ok
run
515
Ok
run
541
Ok

So out of curiosity, on which machine config is the time set to zero?
Or do you mean the start of the BASIC interpreter ?