Spider Spawning Generates Phantom MissionTriggers entries in sandbox.sbc, cripples servers

Moleculor shared this bug 15 months ago
Need More Information

Current version: 1.196.016

Start an empty scenario with spiders enabled.


Spawn an Alien planet.


Stand on the Alien planet.


Save the game.


Look at sandbox.sbc. Find <MissionTriggers>

Under this you'll probably see two <item> listings, one for some random <stm> that I can't identify, and one for <stm> of zero.

Wait for a spider to spawn.


Save.

Look at sandbox.sbc again. Same location.

Under <MissionTriggers> you'll now find new <item> entries, each with an <stm> associated with your ClientID.

For *every* spider that spawns, a new <item> entry is generated.

So far as I can tell, no current method in the game clears these entries. Killing the spiders certainly doesn't. Saving and quitting doesn't.


----

What brought this to my attention was a modded server utilizing a couple dozen ish mods or so was starting to have 5 second freezes for everyone connected each time a player connected.


Examination of the 16MB (!!) sandbox.sbc discovered ~26,000+ <item>s under <MissionTriggers>, all structured the same way as you'll see in the above test. That's 318,000+ lines worth of data being tracked for all the spiders that had spawned during the life of the server.

Which had only been up for a day or three.


It's no wonder that auto-save was locking up the server a bit, or that players connecting to it were locking the entire server up for a short while.

Comments (2)

photo
1

Hello,


thanks for reporting this to use, we were aware that mission triggers sometimes appeared in the logs and we knew there was not a system to remove those, however we did not know it came from spiders.


I will try to reproduce this, meanwhile, can you send us the log from the server as well as the server save?


Thanks for your cooperation and have a nice day,


Keen QA

photo
2

I am one of the server admins from the server that Moleculor is talking about. We ran into these issues while building a server that mimics Splitsies popular Survival Impossible series. So we bumped up spider spawning significantly as part of those changes. What we found was after about a couple of days there were over 200k lines in the .sbc file dedicated to mission triggers taking almost 20 seconds for the object builder to add to a dictionary.

This process would cause the entire server to pause for that 10-20 seconds every time a client connected as well as more pauses on backup. I ran a dotTrace profile on the server which is how I ended up narrowing down this issue. I've attached the MissionTriggers below, the more clients that we have connected the faster these triggers are generated, we've seen upwards of 3,000 mission triggers generated in a single hour before. My bandaid for the server was a plugin that modifies the MySessionComponentMissionTriggers classes load method to drop all the mission triggers on restart and it seems to have resolved the major issues related to these. But as Moleculor mentioned we are starting to see tens of thousands of useless identities that belonged to spiders filling up the SPID faction

photo
1

Hi Keen QA, this morning I was working on something else and I found the source of this issue above.

private MyPlayer CreateNewPlayerInternal(MyNetworkClient steamClient, MyPlayer.PlayerId playerId, MyObjectBuilder_Player playerBuilder)
		{
			if (!this.m_playerIdentityIds.ContainsKey(playerId))
			{
				this.m_playerIdentityIds.TryAdd(playerId, playerBuilder.IdentityId);
				if (!this.m_identityPlayerIds.ContainsKey(playerBuilder.IdentityId))
				{
					this.m_identityPlayerIds.Add(playerBuilder.IdentityId, playerId);
				}
			}
			MyPlayer myPlayer = this.GetPlayerById(playerId);
			if (myPlayer == null)
			{
				myPlayer = new MyPlayer(steamClient, playerId);
				myPlayer.Init(playerBuilder);
				myPlayer.IdentityChanged += this.player_IdentityChanged;
				myPlayer.Controller.ControlledEntityChanged += this.controller_ControlledEntityChanged;
				this.AddPlayer(playerId, myPlayer);
				if (MyFakes.ENABLE_MISSION_TRIGGERS && MySessionComponentMissionTriggers.Static != null)
				{
					MySessionComponentMissionTriggers.Static.TryCreateFromDefault(playerId, false);
				}
			}
			return myPlayer;
		}
In CreateNewPlayerInternal, MyFakes.ENABLE_MISSION_TRIGGERS is checked and then a mission trigger is created. Since spiders are characters every single spider creates one of these triggers. Every player death and respawn also will create one of these triggers. I'm not sure if the bug is that MyFakes has mission triggers enabled to begin with or if there's something worse going on, but I hope that you will be able to resolve this issue

photo
1

And just in case you didn't notice (because I don't know how to reply in this system), there's another comment in here from me with most of the requested files you asked for.

photo
photo
1

It's not my server, so I don't have access to the server's save file and log.

I do have a single player version that the client generated when it took too long for the client/server to negotiate a connection.

That version is missing players' grids and such. It may actually be missing other info in the sandbox.sbc, too. Not sure.


I'll ask the admins and people who were working on the issue to see if they're willing to share and if they have any of the files still, now that they've 'fixed' the issue on their end (by writing something that just nukes all the MissionTriggers entries on restart).


They did say that even without all the mods and plugins (before they started nuking the MissionTriggers), the save-freezes still occurred, I think? Not sure if they tested the connection-freezes under those conditions.


When I asked earlier, they said something about having to use a profiler to see that the bucketload of MissionTriggers was causing "22 seconds worth of serialization in profiling".


BTW, I also noticed that there were about ~24k entries for spiders under the spider faction, as well? Don't know if that's healthy or not, but figured I'd mention it.

I've attached the single player variant of the world that I happen to have (generated by the bug linked above) that's missing data and I'll ask the server admin if they are willing/able to share 'the real stuff', if they have anything left to share.