Ideas on a combination of efforts on parallelism and multiple players

As I said, the goal is to have multithreading and multiplayer working. And if you know me, you know how lazy I am. So I`m thinking about combining these two things:

One basic idea is to split things up like this:
1. server runs game mechanics, no visual stuff at all
2. client only handles input, output

This makes synchronisation very easy, but high bandwidth is needed and the client multicore cpu isn`t really doing anything.

In my vision, the network also has 1 main server (the one to connect to) and n clients (just I/O). But additionally, there are m help servers. The help servers get specific tasks from the main server that need computation and are appropriate for parallelism. Tasks I have in mind are:
– AI for one specific player
– unit/group AI for one player
– pathfinding

Using this approach, even in singleplayer, multicore cpus could be used: the client does IO, the main server is a kind of process manager, and the help servers do computations – all on one machine. In a multiplayer environment, the main server has to distribute tasks to different machines depending on their load, processing power and bandwidth – not trivial at all. However, the classic solution – one server does all but IO – is possible as a fall-back.

Iยดm not sure how difficult this will be to implement, I think it has not been used in game development yet. Possible reasons can be the debugging problems in distributed systems and the overhead involved when using multiple processes – not threads!

Multithreading started

After my studies have taken my brain for quite some time, now I am able to combine both! In April I will take an exam on “Betriebssysteme, Rechnernetze und verteilte Systeme” (operating systems, computer networks and distributed systems).

So I decided to apply my new knowledge to BlueWar, starting with thread support and (I hope my time will suffice) a revival of the multiplayer mode will follow.

As a first step, I upgraded the BlueWar Dependencies:

  • Ogre 1.61 Source compiled with THREAD_SUPPORT 1
  • ETM 2.3.1
  • NaviLibrary 1.8 Pre-Release (SVN Awesomium) see here
  • MouseCursor see here (new! because Navi removed cursor)
  • PagedGeometry 1.05
  • boost 1.37 (new!) – for now I only use boost::thread, as it seemed a great, simple solution for threads and is used by Ogre
Von BlueWar Screenshots

Then, a careful review of the loading process was inevitably:

When a map is selected in the main menu (new!) and loaded, the loading screen (new!) has to be updated while the background thread is building the new scene. I found lots of small interdependencies between the two, which were no problem in a single-threaded environment, as the CameraController. The “CameraController::Init” method attached the Camera to a viewport of the main window, which made things horrible if the main thread was currently rendering! I moved the viewport code to the firstUpdate. Another example is the FrameListener, which must not be added while the main thread is accessing the FrameListener list.

Now I hope that the major raze conditions are found, the only rendering lock required at this moment is the FrameListener initialization. Another lock is used by the InputManager so that its a quasi-monitor. This way I can be sure that no one is going to add listeners while using them – yes I know, this is the reader-writer-problem and it would be possible to have multiple readers at a time, but I really don`t have that many calls to the InputManager, so forgive me!

As a test, I also moved the Navi updates to a worker thread. This works pretty good with some locks (although I`m quite sure it can be either enhanced or dropped, because Navi itself already uses a background thread). I still have to look further into the details of boost::thread::sleep, because when I do

static bool navi_update_continue;
void navi_update()
{
	NaviLibrary::NaviManager& mgr = NaviLibrary::NaviManager::Get();
	navi_update_continue = true;

	boost::xtime update_interval;
	update_interval.sec = 0;
	update_interval.nsec = 50000000;

	while(navi_update_continue)
	{
		MUTEX::scope_lock lock(MUTEX::Navi);
		mgr.Update();
		boost::this_thread::sleep(update_interval);
	}
}

the thread sleeps not at all (at least not noticably).

The next vision for me is to use multiple threads during gameplay, and I am quite positive about that after this fast success! I promise not to let you wait another year before the next post ๐Ÿ˜€

Here we go again

I know it’s been a long time, let me tell you one thing. Yes, come closer. It’s a secret. This project is not dead! It lives brighter and higher than ever before! Let me show off the new assets:

1. GUI-System

After a facelift, the Falagard-extensions from CEGUI 0.5 are used, which means great new features: unified coordinates and a xml-based skin system. While porting the code to the new CEGUI version, I also improved the GUI-Code a lot: It’s much more OO, flexible and very easy to extend.
Additionally I combined CEGUI with BetaCairo to reduce the performance hit that was caused by the minimap. I can’t exactly measure the performance gain because there were much more changes going on until BlueWar was compileable again, but now there is no performance penalty per minimap icon any longer, which will definitely pay off in the future.
As a skin I chose the Vanilla scheme with some custom widgets (ImageButtons) as Vanilla doesn’t provide much of them. I hope you like it!

2. Abstract systems

After starting to read the great book “modern c++ design” by Andrei Alexandrescu I realised that I could implement a generic System with Plugins / PlugInHolders with different features as caching, simple listing, … / Factories for different Plugins / …
This is now used for nearly everything: the application itself, gui, players, objects, explosions, and others I can’t remember atm. I know that the book presents much more sophisticated models, but my understanding isn’t perfect right now and the current systems seem to work ‘good enough’.

3. Field occupation visualisation

As I got familiar with BetaCairo, I also used it for a feature I wanted to have since the introduction of the Battlefield (that stores all the game fields and their occupation data): Visualise it with a transparent plane that’s spanned across the whole terrain. It’s very subtle due to transparency but gives the necessary information for debugging or (I’m not sure yet) to the player in build mode.

Happy new year

Hello you! What a pleasure to see you again! REALLY! Although there were no new posts during the last 2 months, I`ve been coding a lot and rewrote hundreds (or thousands of lines) of BlueWar.

Why did I do that? Because it wasn`t perfect. When I began the development of BlueWar, I was a little silly boy that wanted to know how real games were made. That boy had no idea of how all those models in a game were organized, the most complex data structure he knew was a list.

Although I had refactored my code a lot since the very beginning, there still were some design decision left from that past decade which I wanted to eliminate:

data structure:

  1. file types
    Although most files were logically named and most class definitions were separated from their declarations, all the code was in .h files that got included in one WinMain .cpp. This allowed much faster complete-rebuild-times, but also forbid partly rebuilding.
  2. class hierarchy
    The object base class was extended by objectfight (game objects that are able to fight). There was another class for moving objects (objectmove). Now this design produced many problems: there could be no classic RTS unit, because a unit hast to move AND fight. So objectmove was changed to inherit objectfight, but this meant that the game would not be able to handle civil moving units. What a shame!

The first problem was (uneasily) solved by (just) renaming lots and lots of source code files and (even more embarrassing) adding thousands of #include directions to all those files. This took quite some time and revealed even more problems: because there was more than one code file, static variables that were defined in a header file were not unique! I know this is a child`s mistake, but hey, I quickly fixed it by using really unique statics (defined in cpp`s) and some Singletons as I don`t really like the length of code that Singletons produce:
compare “GameManager::getSingletonPtr()->doSomething()” with “GameManager.doSomething()”!

Perhaps I will try to combine both by overloading or “#define” the “->” operator, but I will need to investigate further.

To introduce a solution for the second problem I implemented a plugin system (you know already, as you read my last post, didn`t you? ;)). PlugIns take over the responsibity of the old inheritances of different object classes. So there is an unlimited number of possibilities to combine different plugins for different behaviours, functions, etc.

While implementing various plugins, I realized that there had to be a simple way for plugins to communicate to each other. So I introduced PlugInInterfaces, abstract base classes for different pluginTypes. The object class holds smart pointers to these interfaces and plugins that are inheriting from the interface classes can register themselves as a “FightInterface”, “MoveInterface”, “AI interface”, or whatever. The interfaces are now also used by “outsiders” as the ActionManager, who calls “attack” to the “FightInterface”.

To enhance the amount of new code even more (is there really some “old” code left?), I also switched to a new CVS version of Ogre, CEGUI and plsm2, which isn`t as easy as the last time due to thousands of interface breaking changes. Main problems are the new OIS (solved by now), and CEGUI 0.5 (I`m still fighting to let my code cooperate with the new coordinate system).

Best wishes for a great year 2007!

And HAPPY CODING for those of you who believe in code…

YASU

Also known as Yet Another Small Update. I started my job as a “Zivi” in a hospital some months ago and so I wasn`t able to code that much. Nevertheless there have been some substantial improvements to BlueWar:

1. new object PlugIns

Some new plugins now implement features that were part of the core object class in the past. Examples are the fieldAllocator (which marks a square on the battlefield as “allocated”, so that pathfinding can happen), collision checks and HUD info icons.

2. Profiling

There were huge slow-downs during gameplay causing the game to hang for several seconds. This was a great opportunity to integrate a profiling mechanism. Now the game checks the period of time each component needs to update und logs every irregular value.

This showed quite clearly that the pathfinding system was much too slow to handle a big map (especially since the computer player uses pathfinding excessively), so I introduced a maximal path length that is set via a new global settings xml file.

There are two systems that need profiling in a game: the CPU and the GPU. The CPU is profiled by the mechanism I described above, the GPU needed something different: I found NVPerfHUD which is a great tool. It displays the usage of the GPU on the screen while running the game. As a bonus, it is also able to debug the scene by going through every render stage manually.

The result was as expected: Because I use a lot of single entities for the buildings and units the batch count is too high so that the GPU cant work properly (this is shown in better detail in many articles on the web, including this one).

3. Bugfixes

Thousands ๐Ÿ˜‰

This was not that much of an update, but hey, there is a real life ๐Ÿ˜‰

Now that I am used to the Job I hope that I can speed up the development and finish something.

SVN access

Yes, it`s online! BlueWar is now available through the sourceforge servers. The source code is available in the SVN repository and the datafiles (which are required to run the game) are on the sourceforge download site. Please refer to the new “downloads” area on this site for instructions on how to get, compile and run the game!

Nevertheless you should be aware that this is no real release yet (not even public beta). Although there is a lot of code, many things just don`t work. I know this and will release a binary, directly executable version as soon as I think BlueWar is ready for version 1.0!

It`s a long way…

I have to admit that I wasn`t that hard-working during the last weeks, mostly due to the hot German summer and the soccer world championship. “Deutschland ist Weltmeister!” We were all waiting for this sentence but we won`t hear it during the next 4 years :(. Nevertheless I am honoured because of the way we lost. Nobody started to fight violently, neither on the football field (as Argentina did) nor in the streets (nobody will ever forget what German hooligans did in France). Instead everyone wants our coach, Klinsmann, to stay and improve our team, which is a great contrast to the generally pessimistic attitude in Germany.

But this blog is about BlueWar, so I have to write some lines about it, right? Right!

There were lots of tasks on my ToDo list, for example the pathfinding system. I recognized that there were lots of pointers to the same units in the main update queue, which made the units search for a path multiple times (and “multiple” means a lot, one time I checked, there were 50 duplicates!). This caused severe lags, which are solved now (I hope) by preventing a second pointer to be added. Only one (probably very) big problem is left: the computer player causes lots of “moveto”s and the pathfinding algo wastes too much time to cope with “no path possible” situations. I read an article written by a developer of Age of Empires which also mentioned this problem. They solved it using different approaches which I don`t understand completely, nevertheless I hope to improve the situation in BlueWar by limiting the time spend on pathfinding every frame.

As promised in an earlier post I worked on new PlugIns, especially weapons. PlugIns now have different priorities which influence the order in which they receive commands. This empowers the developer to create special queues like “update_targets”: if this command is executed by the AI plugin, at first the radar updates the object`s “enemiesInSight” list. Then this list can be edited by a enemy-priority plugIn to prefer a special enemy or type. After that, weapons receive the command and choose an appropriate enemy to attack.

There are only 2 weapon types at this moment (missle launcher with different missle types + machine gun) but the code for these weapon plugins is really small and it will be very easy to create new ones in the future. I also want to emphasize how easy it is to plug these weapons into a totally different object with just one xml line! And you are not limited to one weapon, of course! Take the fighter aircraft, for example. It has 2 machine guns in the front under the cockpit and 2 missle launchers (1 under every wing). This works brilliantly ๐Ÿ˜€

OK, this was a bit too much self-celebration, but I hope you got the idea.

Last, but definately not least: BlueWar is an accepted OpenSource project at sourceforge.net (https://sourceforge.net/projects/bluewar)! I chose an LGPL license which is not as strict as the GPL. Now I (and possibly more people who want to contribute) have access to CVS or SVN (I still have to decide) which will ensure a stable development (I hope).