Why the server project needed a client

Hey folks, as promised in the recap, here's the first post about the client fork. Not about how it works - that comes later - but about why we maintain a game client at all.

The short answer: because for the last ten years the client was the one part of the system we couldn't change, and almost every ugly thing in OpenMU's network code exists because of that.

The client is the specification

When you write a server emulator, you don't design a protocol. You obey one. The client is a compiled binary from another decade, it expects certain bytes in a certain order, and if you send something it doesn't like, it either ignores you or crashes. That's the whole design space.

A good part of this blog is a monument to that: how the packet encryption works, how SimpleModulus was analyzed, the network analyzer I built to see what's going over the wire, and the packet structures which I ended up generating from data because there are so many of them.

None of that work is wasted - the original client is still supported, and supporting it stays a goal. But everything above is the cost of not owning one end of the connection.

What that costs, in bytes

Three examples, all from the server code as it is today.

Damage doesn't fit. The hit packet carries the damage as a 16 bit value. That's a hard ceiling of 65535 per hit. Modern servers hand out far bigger numbers than that, so the workaround has been in OpenMU for years: send the packet more than once until the damage is used up.

// do/while, so that a 'miss' with 0 damage sends a message, too.
do
{
    var healthDamage = (ushort)Math.Min(0xFFFF, remainingHealthDamage);
    var shieldDamage = (ushort)Math.Min(0xFFFF, remainingShieldDamage);

    await connection.SendObjectHitAsync(/* ... */).ConfigureAwait(false);

    remainingShieldDamage -= shieldDamage;
    remainingHealthDamage -= healthDamage;
}
while (remainingHealthDamage > 0 || remainingShieldDamage > 0);

A 200.000 damage hit is four packets. Experience gain has exactly the same problem and exactly the same loop. It works, and it's nonsense.

An item is a 12 byte bit puzzle. In Season 6 an item is serialized into 12 bytes, and there is no room in them. The item level sits in four bits of byte 1, so it can't go beyond 15. The option level is split up: two bits go into byte 1, the third bit is smuggled into bit 6 of the "excellent" byte. My comment in that code has been there for years:

// The item option level is splitted into 2 parts. Webzen... :-/

To support more than 256 items per group, the 9th bit of the item number is parked in yet another free bit. The item group takes the high nibble of byte 5, and the guardian option flag lives in the low one. Every bit in those 12 bytes is spoken for. If you want to add a new item property to your server - go find a bit.

For comparison, the same item in the old versions: 3 bytes in 0.75, 4 bytes in 0.95. The character appearance, which decides what everyone around you looks like, is 9, 11 and 18 bytes respectively. The whole visual state of a player fits into 18 bytes, and that's why some things simply cannot be shown.

The same packet has different codes per language. Not per version - per localized binary. The hit packet is 0x11 for the English client, 0xD6 for Japanese, 0xDC for Vietnamese, 0xDF for Korean and Filipino, 0xD0 for Chinese and 0xD2 for Thai. So the server doesn't only have to know which season you're running, it has to know which regional build you downloaded.

And the bugs you can't fix

The other half of the cost isn't the protocol, it's everything the client decides on its own. If the Dark Lord's raven is drawn wrong when he walks, if an item label is broken, if a window doesn't show a value the server has known all along: there is nothing you can do about it from the server side. You can't add a UI element for a feature you invented. You can't show a number the client has no field for. You can only work around it, and the usual workaround is a chat message.

That's the real reason. Not the missing bits - the missing say.

So I took the sources

In March 2023 I started to clean up the Season 5.2 sources which were made public, with the goal of getting them to Season 6 Episode 3. This doesn't replace the original client, full support for the original Season 6 client stays a goal of OpenMU. Most people running an OpenMU server will keep using the original client, and that's fine. The point is that there is now one client we can change, so we can find out what a MU server looks like when nothing forces it into 12 bytes.

What opened up

The nice surprise was how little the server needed for it. OpenMU has had a plugin system since 2019, and its whole purpose is that view implementations are selected by the client version at runtime - that's how 0.75, 0.95 and Season 6 clients can be served by the same server. So the new client just became another version: it announces itself as season 106, and about two dozen view plugins and serializers are registered for it, right next to the old ones. No fork of the server, no #if, no second code path in the game logic. The game logic doesn't even know.

What those extended plugins do differently:

• Damage and experience are sent once, with the values they actually have. The loops above are gone.
• The item serializer is dynamic, 5 to 15 bytes, with a flags byte that says which parts follow: option, luck, skill, excellent, ancient, harmony, guardian, sockets. The item number is a real 16 bit number, the level is a real byte. No bit smuggling, and there's room to grow.
• The appearance got 27 bytes instead of 18, which is what makes it possible to show things the old format had no space for.
• Monsters have a health bar after you hit them.
• The server tells the client which chat commands exist, so the command window can list them instead of expecting players to memorize them.
• Character list, stats, level, master stats, quests, mail, player shops - all got extended versions.

Was it worth it?

Honestly, it doubled the amount of code I look after, and the second half is a C++ codebase from 2005 with all the charm that implies. I've spent evenings on wchar_t widths and fclose calls instead of on game features.

But the ceiling is gone. When we now want damage to exceed 16 bit, we change a packet definition and both ends follow, instead of writing a loop that lies to the player. That's worth a lot of evenings.

Next time I'll write about how the client got rid of its C++ network code entirely and loads a .NET Native AOT library instead - which is the part where "one packet definition, both ends" actually becomes true.