On January 12 the /trade add command appeared to stop working properly. It appeared to accept the trade and take the commission but then immediately return the items to you inventory.
Further investigation has revealed a problem that is far bigger than the trade command (which is entirely innocent in this particular case) and has required some extensive investigations and major file and system changes. The Trade command failure just gave me a small window of opportunity to fix it before catastrophic system failure occured. It is possible that the crash on that night was also related but I have no real way of telling.
The problem is field size.
The game operates on a system of TICKS where a tick is one game cycle and there are 18 TICKs per real second. Every time the game runs through a loop of the game cycle it increments the global tick counter by 1. So if the server is shut the tick counter does not progress and it is as if time has stopped. If the server is underloaded or too fast, it waits until 1/18th of a second has passed before doing another game loop and incrementing the TICK counter.
The problem is that the tick counter is an integer which in C is defined as 4 bytes which can hold a number of the size +/- 2147483647. The game has been running for about 3.7 years and on 12 January came within 14 days of exceeding that size. The trade dates look out 14 days and when you add the 14 days (converted to ticks) to the current tick counter it goes negative.
So what?
What this means that within 14 days (of 12 Jan) the tick counter will exceed its limit and either crash the game (I hope not) or go negative (which is what I expect). Then every date stamp that is measured in ticks will be invalid. I do not know or could even guess the implications of this event and I do not really want to try to find out.
Where to from here?
I have attempted to find every field in the game files that stores or refers to the TICK data types. I have found about 55 of them in various places or cases.
Opition 1
I would need to convert all the data fields from type int (4 bytes) to type long long (8 bytes). This involves major file conversions and data conversions. Luckily I did this recently to increase the number of data fields so at least I would have a slight head start.
I would need to check and convert all the programs that refer to TICK data dields to ensure that they all have the correct data type of long long for internal calculations. This is a far bigger and very error prone job.
I would need to check the client to ensure it does not use any TICK data fields from the server. If it does then I have a major problem as the client is in C++ which does not appear to support 8 byte integers (lets hope that this is not the case).
Then I would need extensive testing on the Testserver.
Then I need to shut the production server and apply the conversions and testing there too.
Option 1 turned out not to be a viable proposal after further investigation within the timeframe available.
Option 2
If there are too many places where the data exists and some of them physically can not be converted or the client also has the same data issues then I would need to convert the data stored by say deleting 3 years worth of ticks of the data values where appropriate. This will require alot of context specific analysis but no data size conversions.
Option 2 was selected.
When?
The trial convesrion program has been completed and is being checked in the Testserver.
Assuming that this does not show any issues then it will be implemented soon in the main server. I can not wait the full 14 days as there may be other parts of the game that are forward looking, like character expiry dates etc that will cause problems before the end date. The trade issue was very lucky as it bought time before the end.
The 4BB is very much like the Year 2000 bug (Y2K). If nothing is done or it is not done well then disaster will occur and blood will flow. If it is done well then people will wonder what all the fuss was about. I aim to leave you wondering.
If you notice anything strange, now or over the next 14 days or so, please report it immediately as 4BB may be biting.
Thu Jan 13, 2011 5:22 pm