Custom SBC Definitions Not Loading

TLDR: I figured out what would be needed to finish fleshing out the feature that allows mods to inject their own custom definitions into the game and allow for easier/more efficient cross-mod data sharing. It looks like y'all started down this road but were diverted due to what I can only imagine were more urgent priorities. I'd say that 99% of the work had already been completed though. It would be a shame to leave it unfinished. The changes that I applied to the decompiled source code to make this work have been spelled out line-by-line at the bottom of this post if that is at all helpful.

Hello,

I was recently investigating the potential for a mod framework the supports discrete patches for resolving many of the compatibility issues that can arise between mods. One of the most common compat problems that I've noticed deals with conflicts between sbc files. The definitions have to be completely overwritten if; for example, someone wants to make tweaks to planets in the game. This leads to a number of issues including those related to licensing, or game-breaking bugs when those same definitions are intentionally changed by Keen to name the most obvious. The latter is particularly irksome when the changes Keen makes aren't even relevant to the mod in question.

There's currently a solution called ModAdjuster that does try to solve this problem, but it isn't without flaws. The first is that it relies on users modifying a text file within the ModAdjuster folder structure to load compat xml files. Not only is this brittle and unfriendly to the laymen, but it ends up being a one-off solution for most people. They make the changes they need for their game and no one else in the community can take advantage of that work.

My idea was for a tool mod that ran last in the the mod list. It would search for custom patch definitions and then apply them accordingly. The patch definitions would be applied as discrete mods themselves. So if someone wanted to use two mods that are not normally compatible, they could just add the compat mods to their load order along with my tool mod and be done with it.

To that end, I discovered what appeared to be support within the game for deserializing custom definition objects into MyDefinitionManager by using the Definition element defined with an appropriate xsi:type attribute. The game currently registers mod classes decorated with the MyObjectBuilderDefinition attribute and inherit from MyObjectBuilder_Base. It also tries to load sbc files from mod Data directories.

However, I also discovered that the game was not registering classes decorated with the MyDefinitionType attribute and that inherit from MyDefinitionBase. This (seeming) oversight means that deserialization fails when the XmlSerializer encounters xsi:types that MyXmlSerializer is unfamiliar with. So, I set about making some tweaks to three classes in order to finish out the work that I think Keen intended to support at one time.

The first was simply to call MyDefinitionManagerBase.RegisterTypesFromAssembly(assembly) from Sandbox.Game.World.MyScriptManager (in Sandbox.dll) immediately after calling MyObjectBuilderSerializerKeen.RegisterFromAssembly(assembly). This registered my MyDefinitionBase implementation with the manager, and also setup the XmlSerializer accordingly. This is literally all that was required to get my mod framework to work as intended.

But it does leave the game in a dirty state that causes problems when leaving the game and then reloading. So I had to modify MyXmlSerializerManager, MyDefinitionManagerBase, and MyScriptManager to unregister the custom MyDefinitionBase implementations. Those details are spelled out in detail below. I then modified Sandbox.Game.World.MyScriptManager again to call MyDefinitionManagerBase.UnregisterTypesFromAssembly(assembly) immediately after MyObjectBuilderSerializerKeen.UnregisterFromAssembly(assembly).

I've since test a few different compatibility mods with complete success and without needing to copy any of the definitions from within the conflicting mods. And with the cleanup changes, I can load a save, leave, and start and entirely new game or load a different save without conflicts.

It is unfortunate that this required changes to the actual game engine itself. But I hope that alone isn't enough to push this out of contention given that the requisite changes have been provided. I believe that it is also worth mentioning that these changes would also resolve the issues below without any additional work on your end. The community would be able to handle these issues on their own.

https://support.keenswh.com/spaceengineers/pc/topic/24922-custom-component-support-for-mods

https://support.keenswh.com/spaceengineers/pc/topic/xpath-style-modding-api-patching

Thanks For your Consideration!

// VRage.dll

namespace VRage
{
	public class MyXmlSerializerManager
	{
		...
		
		public static void UnregisterSerializer(Type type)
		{
			if (MyXmlSerializerManager.m_serializersByType.ContainsKey(type))
			{
				MyXmlSerializerManager.UnregisterType(type, false);
			}
		}
		
		private static void UnregisterType(Type type, bool checkAttributes = true)
		{
			string text = null;
			if (checkAttributes)
			{
				object[] customAttributes = type.GetCustomAttributes(typeof(XmlTypeAttribute), false);
				if (customAttributes.Length != 0)
				{
					XmlTypeAttribute xmlTypeAttribute = (XmlTypeAttribute)customAttributes[0];
					text = type.Name;
					if (!string.IsNullOrEmpty(xmlTypeAttribute.TypeName))
					{
						text = xmlTypeAttribute.TypeName;
					}
				}
				else
				{
					using (HashSet<Type>.Enumerator enumerator = MyXmlSerializerManager.m_serializableBaseTypes.GetEnumerator())
					{
						while (enumerator.MoveNext())
						{
							if (enumerator.Current.IsAssignableFrom(type))
							{
								text = type.Name;
								break;
							}
						}
					}
				}
			}
			if (text == null)
			{
				text = type.Name;
			}
			MyXmlSerializerManager.m_serializersByType.Remove(type);
			MyXmlSerializerManager.m_serializersBySerializedName.Remove(text);
			MyXmlSerializerManager.m_serializedNameByType.Remove(type);
		}
	}
	
	...
}

// VRage.Game.dll

namespace VRage.Game
{
	public abstract class MyDefinitionManagerBase
	{
		...

		public static void UnregisterTypesFromAssembly(Assembly assembly)
		{
			if (assembly == null)
			{
				return;
			}
			if (!MyDefinitionManagerBase.m_registeredAssemblies.Contains(assembly))
			{
				return;
			}
			if (!MyDefinitionManagerBase.m_registered.Contains(assembly))
			{
				return;
			}
			MyDefinitionManagerBase.m_definitionFactory.UnregisterFromAssembly(assembly); // Would be more efficient if MyDefinitionManagerBase.m_definitionFactory.UnregisterDescriptor was public. We could then just call that within or loop below.
			foreach (Type type in assembly.GetTypes())
			{
				object[] customAttributes = type.GetCustomAttributes(typeof(MyDefinitionTypeAttribute), false);
				if (customAttributes.Length != 0)
				{
					if (!type.IsSubclassOf(typeof(MyDefinitionBase)) && type != typeof(MyDefinitionBase))
					{
						MyLog.Default.Error("Type {0} is not a definition.", new object[]
						{
							type.Name
						});
					}
					else
					{
						object[] array = customAttributes;
						for (int i = 0; i < array.Length; i++)
						{
							MyDefinitionTypeAttribute myDefinitionTypeAttribute = (MyDefinitionTypeAttribute)array[i];
							int postProcessorIndex = MyDefinitionManagerBase.m_postProcessors.FindIndex((MyDefinitionPostprocessor processor) => processor.GetType() == myDefinitionTypeAttribute.PostProcessor);
							if (postProcessorIndex >= 0)
							{
								MyDefinitionManagerBase.m_postProcessors.RemoveAt(postProcessorIndex);
							}
							MyDefinitionManagerBase.m_postprocessorsByType.Remove(myDefinitionTypeAttribute.ObjectBuilderType);
							MyXmlSerializerManager.UnregisterSerializer(myDefinitionTypeAttribute.ObjectBuilderType);
						}
						Type type2 = type;
						while (type2 != typeof(MyDefinitionBase))
						{
							type2 = type2.BaseType;
							HashSet<Type> hashSet;
							if (MyDefinitionManagerBase.m_childDefinitionMap.TryGetValue(type2, out hashSet))
							{
								hashSet.Remove(type);
							}
						}
					}
				}
			}
			MyDefinitionManagerBase.m_postProcessors.Sort(MyDefinitionPostprocessor.Comparer);
			MyDefinitionManagerBase.m_registeredAssemblies.Remove(assembly);
			MyDefinitionManagerBase.m_registered.Remove(assembly);
		}

		...        
        }
}