Terminal inventory MainThreadFreeze when large connected inventories contain hundreds of item stacks

William Zinn shared this bug 7 hours ago
Submitted

Suggested titleTerminal inventory MainThreadFreeze when large connected inventories contain hundreds of item stacks — Avalonia InventoryItem UI realization

Game versionSpace Engineers 2 v2.4.0.95

System used for testing

  • Windows 11
  • Intel Core i7-1165G7
  • Approximately 64 GB RAM
  • NVIDIA GeForce RTX 5070 Ti 16 GB
  • .NET 9 runtime shipped with SE2

SummaryWe originally encountered this while debugging several Workshop mods and initially assumed one of our mods was responsible.

After a long isolation and instrumentation process, the evidence instead led into the vanilla/Keen terminal inventory UI.

On a sufficiently large connected inventory network, opening a cockpit, large Cargo750, or other terminal inventory can cause the main thread/UI to stall for several seconds. The VRAGE watchdog then reports MainThreadFreeze.

The affected save contains a large conveyor-connected inventory network with many inventories and many real item stacks.

The important finding is that the freeze appears to be caused by the client terminal UI attempting to create/style/layout hundreds of real InventoryItemModel controls at once.

A local presentation-only A/B diagnostic that limited only the number of item controls Avalonia was allowed to render eliminated the watchdog freezes without modifying any actual inventory contents.



Reproduction stepsThe issue is intermittent per individual terminal opening, but highly repeatable on the affected network.

  1. Load a world containing a large conveyor-connected inventory network.
  2. Have a large number of item stacks distributed throughout the connected inventories.
  3. Access the terminal from a cockpit or large Cargo750 container.
  4. Open the inventory interface.
  5. Close and reopen it several times and/or access another connected inventory.
  6. Large UI stalls can occur while the terminal inventory is being constructed.
  7. When the stall is long enough, the VRAGE watchdog reports MainThreadFreeze.

We reproduced this from:

  • Cockpit150
  • CargoContainer750
  • other connected production/storage blocks

Terminal search also becomes very slow on sufficiently large inventory datasets.



Expected resultOpening an inventory terminal should remain responsive regardless of how many connected inventories or real item stacks exist.

Inventory controls should ideally be virtualized/windowed so only items that need to be displayed are realized as Avalonia controls.

Actual resultOpening the terminal can stall the main/UI thread for multiple seconds.

Observed watchdog freezes during our uncapped testing included approximately:

  • 3.3 seconds
  • 4.2 seconds
  • 7.3 seconds
  • other earlier runs in the 5–13+ second range

An earlier dedicated capture produced repeatable freezes of approximately:

  • 12.6 seconds
  • 5.4 seconds
  • 11.6 seconds
  • 9.5 seconds

The render thread remained responsive during the captured main-thread freeze.



Why we investigated this so deeplyThis world was being used to test four Workshop mods:

  • Conveyor Tether
  • Grinder Mk4.1 / Salvage Tether
  • Sorter Purger
  • Sun Tracker Gyroscope

Because the terminal freezing appeared while testing inventory/conveyor-related behavior, we initially treated our mods as the likely cause.

We instrumented the relevant code paths so we could see exactly what each mod was doing before and during a freeze.

This included entry/exit tracing around:

  • conveyor traversal
  • inventory reachability
  • inventory transfers
  • topology refresh
  • sorter scheduling
  • tether processing
  • grinder inventory processing
  • gyroscope processing

Every observed mod operation completed. We did not find an unmatched mod operation surrounding the freeze.

Additional isolation included:

  • physically separating both sides of the custom sorter in the native conveyor graph;
  • leaving only a controlled scheduler bridge between the sides;
  • reproducing the freeze while the sorter moved zero items;
  • disabling the Grinder Mk4.1 global InventoryData observer;
  • disabling the Conveyor Tether global InventoryData observer;
  • retaining the actual save/network that reliably reproduces the problem.

The terminal freezes continued.

We therefore moved the investigation into the client UI.



Watchdog dump resultA managed dump captured while the application was actively frozen showed the main/UI thread working inside Avalonia UI construction and layout.

The significant stack path included:

Avalonia.Reactive.LightweightObservableBase.PublishNext

→ Avalonia.AvaloniaObject.RaisePropertyChanged

→ Avalonia.Layout.Layoutable.OnVisualParentChanged

→ Avalonia.Visual.SetVisualParent

→ Avalonia.Collections.AvaloniaList.InsertRange

→ Avalonia.Controls.Panel.ChildrenChanged

→ Avalonia.Collections.AvaloniaList.Add

→ Game2.Client!CompiledAvaloniaXaml.!AvaloniaResources+XamlClosure_1216+XamlClosure_1218.Build

→ Avalonia.Markup.Xaml.Templates.TemplateContent.Load

→ Avalonia.Markup.Xaml.Templates.DataTemplate.Build

→ Avalonia.Controls.Presenters.ContentPresenter.UpdateChild

→ Avalonia.Controls.Presenters.ContentPresenter.ApplyTemplate

→ layout/measure processing

→ Avalonia.Controls.WrapPanel.MeasureOverride

→ Avalonia.Layout.LayoutManager.ExecuteMeasurePass

→ Avalonia.Layout.LayoutManager.ExecuteLayoutPass

→ rendering/dispatcher callbacks

→ Keen.VRage.UI.AvaloniaInterface.Rendering.DispatcherImpl.DispatchAllCalls

→ Keen.VRage.UI.EngineComponents.UIEngineComponent.UIManagerTick

→ normal game update loop

This showed the main thread actively constructing and laying out UI controls rather than waiting on a conveyor operation or a managed lock.



Exact XAML template identifiedWe mapped the compiled XAML closure from the frozen stack.

The frozen method:

CompiledAvaloniaXaml.!AvaloniaResources+XamlClosure_1216+XamlClosure_1218.Build

maps to:

avares://Game2.Client/UI/TerminalScreen/Inventory/InventoryItem.axaml

Relevant compiled tokens observed in Game2.Client v2.4.0.95:

  • Parent XamlClosure_1216.Build: 0x06014D8A
  • Nested XamlClosure_1218.Build: 0x06014D8E

XamlClosure_1218 builds the realized inventory-item UI, including the item icon/name/amount/new-state controls.

Game2.Client.dll tested:

SHA256:

fdcd1c1eb01433be994c87893a88f720325bf93c1e797930d79859ed664bb9ba



InventoryGrid client-code findingWe decompiled the relevant client inventory controls from the installed v2.4.0.95 assemblies.

InventoryGrid.ItemsFilled() enumerates every real item in:

InventoryViewModel.InventoryItems

and yields every item to the UI.

Conceptually the code does:

foreach (InventoryItemModel inventoryItem in _inventory.InventoryItems) { numberOfItems++; yield return inventoryItem; }

If MaxStacks exists, it can additionally emit null placeholder cells up to that value.

Initially we suspected the empty placeholder behavior.

However, runtime instrumentation provided a more important result.



Hundreds of REAL InventoryItemModel entriesOn the affected connected inventory network, individual InventoryGrid instances were receiving very large numbers of real InventoryItemModel objects.

Observed examples included:

  • 290
  • 552
  • 452
  • 703
  • 741
  • 275
  • 393
  • 456
  • 235
  • 306
  • 308
  • 585
  • 683
  • 540
  • 657
  • 685
  • 651
  • 500

These were:

InventoryViewModel.InventoryItems.Count

They were not empty MaxStacks placeholder counts.

For these inventories, MaxStacks was frequently null.

This ruled out empty-slot padding as the primary cause of these particular freezes.

The client was actually attempting to handle hundreds of real inventory item models.



Inventory hierarchy / virtualization concernThe terminal inventory hierarchy also appears to use ordinary Avalonia ItemsControl instances for the connected inventory/container lists.

The mapped client inventory controls showed:

  • outer inventory container list using an ItemsControl;
  • each container using another ItemsControl for its inventories;
  • each InventoryGrid then providing its individual item models.

We did not find an inventory-path VirtualizingStackPanel in the mapped controls.

The frozen stack also directly showed a WrapPanel being measured while InventoryItem.axaml controls were being constructed.

This means a large connected network can potentially require Avalonia to realize a very large control tree at once.



Secondary rebuild amplifierWe also found that streamed inventory-container changes can cause the client inventory view-model lists to be rebuilt.

The relevant SetInventories(...) path clears/disposes the previous entries and repopulates the inventory list.

This appears capable of amplifying the issue because a large UI tree can be destroyed/recreated while streamed inventory state is still changing.

This also fits another visual symptom we observed before the root cause was known:

while viewing one side of the connected network, an inventory on the opposite side could visibly flicker/reappear while the terminal was updating.



Simulation inventory data did NOT show a duplication loopWe also checked the underlying inventory model update path.

InventoryModelImpl.Update() follows the actual InventoryComponent.Items collection by:

  • updating matching existing entries;
  • removing stale entries after the changed tail;
  • appending one InventoryItemModel for each remaining real stack.

We did not find evidence that the simulation model itself was accidentally generating an endless duplicate-item list.

The large counts therefore appear to represent real item-stack models being handed to the client UI.



Causal A/B diagnosticTo test whether the number of rendered inventory controls was actually responsible, we created a temporary local client-side diagnostic.

This diagnostic was NOT a Workshop mod and did NOT modify the save.

It did NOT modify:

  • server inventory contents;
  • InventoryModel.Items;
  • InventoryViewModel.InventoryItems;
  • inventory definitions;
  • MaxStacks;
  • conveyor behavior;
  • sorter behavior;
  • any Workshop mod content.

The diagnostic only intercepted the already-created client-side:

InventoryGrid -> PART_ItemsControl

and changed the inner Avalonia:

ItemsControl.ItemsSource

to a snapshot containing at most the first 60 existing real InventoryItemModel references.

The full underlying inventory remained unchanged.

Items after 60 were simply not rendered for the duration of this diagnostic.

We did not perform inventory transfers during this A/B test.



A/B resultThe display cap successfully activated.

It fired 280 times during the test.

Examples recorded by the diagnostic:

realItems=290 displayed=60 realItems=552 displayed=60 realItems=452 displayed=60 realItems=703 displayed=60 realItems=741 displayed=60 realItems=275 displayed=60 realItems=393 displayed=60 realItems=585 displayed=60 realItems=683 displayed=60 realItems=657 displayed=60 realItems=685 displayed=60 realItems=651 displayed=60 realItems=500 displayed=60

We then repeatedly opened the same problematic terminal interfaces on the same save, including:

  • Cockpit150 inventory
  • CargoContainer750
  • CargoContainer750 again
  • CargoContainer750 again
  • Gearforge250
  • Gearforge250 again

Result:ZERO MainThreadFreeze watchdog events occurred in the complete main game log during the capped run.

This is the strongest evidence we have.

The only deliberate variable changed was the number of real InventoryItemModel controls presented to Avalonia.

The underlying inventories remained intact.



ComparisonUnmodified item realizationHundreds of item models presented to Avalonia:

703, 741, 683, etc.

Result:

Repeated multi-second MainThreadFreeze watchdog events.

Presentation-only limitSame underlying inventory data.

Same world.

Same connected inventory network.

Same cockpit/Cargo750 access.

Only first 60 real item models rendered per problematic grid.

Result:

No MainThreadFreeze watchdog events.



Current interpretationThe evidence strongly indicates a client-side terminal inventory scalability problem.

The likely mechanism is:

  1. A large conveyor-connected inventory network exposes many inventories.
  2. Individual inventories can contain hundreds of real item-stack models.
  3. The terminal creates an InventoryItem.axaml control for each model.
  4. These controls are added/styled/measured in a non-virtualized or insufficiently virtualized Avalonia hierarchy.
  5. Streamed inventory changes can additionally cause large sections of this hierarchy to be rebuilt.
  6. UI construction/layout blocks the main thread long enough to trigger the watchdog.



Suggested areas for Keen to inspectThese are suggestions based on the observed stack/code paths rather than a claim about the intended architecture.

InventoryGridEspecially:

Keen.Game2.Client.UI.TerminalScreen.Inventory.InventoryGrid

and:

ItemsFilled()

Inventory item XAMLGame2.Client/UI/TerminalScreen/Inventory/InventoryItem.axaml

Especially the generated closures corresponding to:

XamlClosure_1216

and:

XamlClosure_1218

Connected inventory presentationInventoryContainer

InventoryList

InventoryListViewModel

and the SetInventories(...) clear/repopulate behavior.

Potential fixesPossible approaches may include:

  • virtualizing the actual inventory-item controls;
  • paging/windowing the displayed item model set;
  • realizing only items currently inside the visible scroll region;
  • avoiding complete connected-inventory UI rebuilds when streamed state changes;
  • incrementally reconciling inventory/container view models instead of clearing and recreating them.



About the mods used during testingFor full transparency, the affected save does have Workshop mods active.

We have not claimed that we reproduced the exact save/network in a completely mod-free world.

However, the purpose of this investigation was specifically to determine whether those mods were causing the freeze.

Relevant inventory/conveyor hooks were disabled or isolated one by one and the freeze persisted.

The eventual A/B diagnostic altered only the client-side Avalonia item presentation and eliminated the watchdog freezes while all underlying inventory data remained unchanged.

That is why we believe this is a Keen/client UI issue rather than a mod inventory-transfer issue.

We can provide additional isolation details/source instrumentation if required.



Diagnostic cleanupThe temporary 60-item rendering diagnostic was removed immediately after the A/B test.

It is not installed during normal gameplay and is not intended as a permanent fix.



Attached evidenceThe attached Core Evidence ZIP contains:

  • original watchdog freeze capture;
  • managed Avalonia stack capture;
  • uncapped run main game log;
  • uncapped inventory-model instrumentation;
  • successful 60-item display-cap A/B main game log;
  • display-cap instrumentation log;
  • decompiled client-code evidence excerpts;
  • technical evidence summary;
  • SHA256 manifest.

A larger full evidence archive is also available containing the complete XAML closure mapper and complete inventory control/model decompilation if needed.

Leave a Comment
 
Attach a file