/ Rust

Seeking Compilation-Independent Type IDs in Rust

In Gunship, my game framework/engine, I use a type maps to store user-defined managers as trait objects while still allowing users (safely) to retrieve them as their concrete type. The key to making this work is to be able to generate a unique ID for each type automatically. In C++ I could achieve this through a nasty combination of the curiously recurring template pattern and function statics, which worked but forced all component managers to inherit from a shared base class. Rust, on the other hand, has the type_id() compiler intrinsic that does this automatically and has been nicely stabilized in the std library through TypeID.

Initially that was enough, being able to statically generate type IDs made my code clean and since type_id() just spits out a u64 using TypeId as a key in a HashMap is about as cheap as I could hope for. But like in all good software projects I just had to go and add another feature that mucks it all up.

Hotloading vs TypeId

One of the cool features that I've been able to add to Gunship early on is code hotloading. The way it works is that the entire game and engine are compiled together into a dynamic library (.dll on windows) rather than an executable. Then I have a static "loader" executable that runs the main loop of the game and calls into the DLL each frame so that it can perform game logic and processing. The loader periodically checks the compiled DLL to see if it's been updated, and if it has then the loader brings in the new one, initializes it by copying all of the game state from the old DLL to the new one, and then continues running the game using the new DLL. The result is that you can edit and recompile your code and see the effects immediately without restarting the game.

Where this starts to cause problems with the type maps is in copying the game state from the old DLL to the new one. The way this works is that the user writes a function called game_reload() that takes a reference to the old and the new engine and copies their managers and systems over:

#[no_mangle]
pub fn game_reload(old_engine: &Engine, new_engine: &mut Engine) {
    let old_player_manager = old_engine.scene().get_manager::<PlayerManager>();
    engine.scene_mut().register_manager(old_player_manager.clone());
}

While this winds up being a nasty bit of boilerplate it's necessary in order to recreate the trait objects for the new manager -- if I didn't do this then the ComponentManager objects would still point to the vtable in the old DLL, which at best would mean that some code wouldn't be properly updated, and would most likely lead to crashes.

Trying to retrieve the old engine's managers from the new DLL is where the problems happen. TypeId is only guaranteed to be consistent for a single compilation, the same type can have a different TypeId on different compiles even if it hasn't changed at all. This makes sense and doesn't cause any problems when I'm statically loading the game, but when I try to run it dynamically then I have to compare TypeIds from two different compilations of the game. The result is that I'll try to retrieve a manager to reload it and I'll hit a panic because no manager exists with that ID.

Using the Type Name

When it comes to uniquely identifying a type few things work as well as its name (after all, that how we identify it in code). Fortunately Rust has a compiler intrinsic type_name<T>() that can give us a &'static str representing the name of the type. Unfortunately the output of type_name() isn't necessarily formatted in a way that's consistent across crates and modules. As far as I can tell type_name() returns a static string with the fully qualified name of the type at the call site. So if I have a type gunship::transform::TransformManager then within the gunship crate type_name() returns "transform::TransformManager" and outside of the crate (i.e. from game code) it returns "gunship::transform::TransformManager". This is a nasty problem because if I add TransformManager to my type map from within gunship then try to access it from game code it mysteriously seems to not be in the map because the key is different.

My (very hacky) solution is to strip the leading portion of the name string that contains the full path and only leave the final token that represents the type's name (so "gunship::transform::TransformManager" and "transform::TransformManager" both get stripped down to just "TransformManager"). The result isn't technically correct because two types can collide if they have the same name even if they're from different modules or crates, but this is used primarily as a debug feature so I'm satisfied with the 90% solution for now. One nice part of this solution is that it doesn't take any extra memory allocation when stripping the string's prefix because you can statically take a sub-slice of a static string in Rust.

The Real Solution

As you might have guessed this is a Bad Solution™ because it is both unstable and not terribly performant (especially since this is a pretty hot path). So how do we handle this better? Well the ideal solution would be something that's exactly like TypeId but is more stable between compilations. If we dig a bit into the Rust source code we can find the hash_crate_independent() function which is what implements the type_id() intrinsic used by TypeId. If you look carefully through that big mass of code you'll notice that it includes the crate hash for user-defined types. This is the strict version hash of the crate which changes if there are any changes to the crate (see this issue for more details). What we want here is the same functionality but replacing the strict version hash for crates with a version-independent crate hash.

In theory we could achieve something to that effect using compiler plugins. As I understand it the current plugin API gives more or less unfettered access to the compiler internals, so we might just be able to write a compiler plugin that adds a type_id!() macro (or something similar) that does what we want. Unfortunately the plugin API is highly unstable at the moment (and it's exceptionally difficult to find good documentation for writing plugins) so even if it were possible I don't know that I would have the patience to actually put together a working implementation. What will likely happen is I'll stick with the nasty string-based method until run into a use case that breaks it, at that point I'll have the motivation to implement a better solution.

David LeGare

David is a game developer and systems programmer. He likes cats, he tolerates dogs, and he often remembers to dress himself before leaving the house.

Read More