A big project with potentially big performance improvements.

General discussion about Train Simulator, your thoughts, questions, news and views!

Moderator: Moderators

gptech
Very Active Forum Member
Posts: 19585
Joined: Fri Oct 10, 2008 5:48 pm
Location: Wakefield, West Yorkshire

Re: A big project with potentially big performance improvements.

Post by gptech »

Let's not forget that "Unable to write testdump...." is not what is causing the game to crash, but simply the game telling you that the only clue it could offer as to why it's crashes can't be left for you to read. The reason for the crash comes before any dump of the memory contents.
Others are having bother creating scenarios, but I suspect not on other routes so whilst you're on the right lines in focusing on the route the root cause really can't be totally any *failings/flaws* in the core of TS ---we wouldn't have any freeware routes if that was the cas.
A single *rogue* asset can bring the game to it's knees, are any amateur (as in not doing the job as a profession) assets used in the route?
User avatar
AndiS
Very Active Forum Member
Posts: 6207
Joined: Fri Sep 23, 2005 4:43 pm
Location: Jester's cell in ivory tower
Contact:

Re: A big project with potentially big performance improvements.

Post by AndiS »

Just as a footnote to this "Unable to write testdump....":

People, try LogMate. Yes, it sucks at times, but basically when it bothers you with too many messages, then there is something that wants to get resolved. I know there are things you can only shrug at, like when it tells me it does not find track ribbons and I see all the track is there and my checks cannot even find a reference to the ribbons that are listed. But other times it says "cannot find blueprint so-and-so" and getting that out of the way is a first step in the right direction.

In short, when you get "Unable to write testdump....", what does LogMate say?
secludedsfx
Well Established Forum Member
Posts: 635
Joined: Fri Sep 23, 2011 9:58 pm

Re: A big project with potentially big performance improvements.

Post by secludedsfx »

Tracks.bins are the main culprits when it comes to crashing really, if they're above 10MB the route will generally start to get more and more issues (pretty apparent with Western Mainlines as when I deleted half the track the route ran more stable).
PC Spec: i9 10900k, 32GB DDR4 3200 RAM, GTX 980ti, 1TB NVME SSD
User avatar
mikesimpson
Very Active Forum Member
Posts: 6361
Joined: Mon Dec 03, 2001 12:00 am
Location: Southern Hemisphere Penal Colonies
Contact:

Re: A big project with potentially big performance improvements.

Post by mikesimpson »

Hi Matthew,

Re your para "When I fired up the route I got a lot of visibly missing items, but it's important to add that it was my own scenario and this loaded with no errors and started as it should. So the AI could get properly lathes etc, but visually I had a lot of items missing that RW Tools is not reporting as missing. Below is a screenshot of what I currently have occuring so you can see for yourself."

Can you advise (or even email me direct) what type of missing items RW_Tools was not picking up? I am always glad to improve the program to try to improve its use.


Mike
Mike in OZ - Author of TS-Tools & Route-Riter.
http://www.agenetools.com
I'm not arguing (just explaining why I'm right).
deltic009
Very Active Forum Member
Posts: 4017
Joined: Fri Nov 27, 2009 1:06 am

Re: A big project with potentially big performance improvements.

Post by deltic009 »

secludedsfx wrote:Tracks.bins are the main culprits when it comes to crashing really, if they're above 10MB the route will generally start to get more and more issues (pretty apparent with Western Mainlines as when I deleted half the track the route ran more stable).
Hi, the Tracks.bin file for the SVR is 634Kb in size, so I don't think it's size is the issue.
mikesimpson wrote:Hi Matthew,

Re your para "When I fired up the route I got a lot of visibly missing items, but it's important to add that it was my own scenario and this loaded with no errors and started as it should. So the AI could get properly lathes etc, but visually I had a lot of items missing that RW Tools is not reporting as missing. Below is a screenshot of what I currently have occuring so you can see for yourself."

Can you advise (or even email me direct) what type of missing items RW_Tools was not picking up? I am always glad to improve the program to try to improve its use.


Mike
Hi Mike, I think but cannot yet be certain, that there are some nested file references in a lot of the blueprints. So, although I have moved the blueprints and told the various .bin files where to find them, something inside the blueprint is referenced via the method that isn't a single line for the full filepath.

To help clarify, these are the two types of nested references to other blueprints, shapes etc that we can have.

Code: Select all

<BlueprintID>Provider\Product\Folder1\Folder2\File.xml</BlueprintID>
For this type of reference it is perfectly fine to reference them from a different blueprint without needing to tick the specified Provider and Product in the tab - this is how a lot of child objects such as headboards are done to save adding ticks to lists of instructions for use.

The second type is more worrying and appears like this and is the one that gives issue.

Code: Select all

<Provider>Name</Provider>
<Product>Name</Product>
<BlueprintID>Folder 1\Folder2\File.xml</BlueprintID>
This means that unticking that provider will stop the nested item from showing. It is a shame that we can't just eridicate that method of making a nested reference inside a blueprint, but it means that I need to find them all manually and check that they get moved to the same folder location or at least one that is ticked.

I don't really know if any solution to this problem, unless you can cross reference the route properties file with the nested references in the blueprint and have RW Tools provide a list of things that won't show because the correct provider is not enabled.
Matthew Wilson, development team at Vulcan Productions

http://www.vulcanproductions.co.uk
https://www.facebook.com/VulcanFoundry/
deltic009
Very Active Forum Member
Posts: 4017
Joined: Fri Nov 27, 2009 1:06 am

Re: A big project with potentially big performance improvements.

Post by deltic009 »

Just adding a separate post for a slight update.

After what I'd learned yesterday I made a fresh start, new batch file and new clone of the route. Didn't touch the Tracks.bin file, started changing the filepaths in the scenery tiles only. I've got as far as providers starting with the characters 0-C. After doing this I took a look at the required asset providers again, and made a few deletions as appropriate, fired up my scenario (it's intense in terms of stock) and with the scenery quality dialled up to 66% the scenario loaded first time without issue.

I shall backup where I am at when I go home, and then endeavour to finish amendments to the scenery tiles completely. Having taken a quick nose at some lofted items I could see that their reference to the shape file appeared to be in the format not-preferred for ease of changing the location so there may be more work to do there, but even if I have to copy Geo's and textures, for only specific versions it will save a lot of space over the full Provider folders in their default locations.

It's going to be a long old haul I think, but perhaps there's a positive glimmer of hope emerging.

*NOTE* one thing I did spot is that the only Kuju folder used and enabled by the SVR is Kuju\RailSimulatorCore.
Matthew Wilson, development team at Vulcan Productions

http://www.vulcanproductions.co.uk
https://www.facebook.com/VulcanFoundry/
SkylineBoy
Well Established Forum Member
Posts: 825
Joined: Mon Jun 29, 2015 1:26 pm
Location: Crimdon

Re: A big project with potentially big performance improvements.

Post by SkylineBoy »

deltic009 wrote: *NOTE* one thing I did spot is that the only Kuju folder used and enabled by the SVR is Kuju\RailSimulatorCore.
Isn't that where all markers reside? Or am I off track here?
deltic009
Very Active Forum Member
Posts: 4017
Joined: Fri Nov 27, 2009 1:06 am

Re: A big project with potentially big performance improvements.

Post by deltic009 »

SkylineBoy wrote:
deltic009 wrote: *NOTE* one thing I did spot is that the only Kuju folder used and enabled by the SVR is Kuju\RailSimulatorCore.
Isn't that where all markers reside? Or am I off track here?
It may well be, my bringing it up was to point out that the behemoth Kuju\RailSimulator folder is not ticked at all for the route.
Matthew Wilson, development team at Vulcan Productions

http://www.vulcanproductions.co.uk
https://www.facebook.com/VulcanFoundry/
MisterWho
Established Forum Member
Posts: 477
Joined: Fri Sep 10, 2010 4:02 am

Re: A big project with potentially big performance improvements.

Post by MisterWho »

Perhaps I am barking up the completely wrong tree, but given that there are two short code segments above, would it not be possible to write a small utility to pre-process any .xml / .bin files that need to be changed for this task ?

If there are two styles for <BlueprintID> - one helpful, one not - then I am thinking that this utility could read each file that needs to be changed in turn, changing all the style two references to be style one references. To put it another way, to change all occurrences of -

Code: Select all

<Provider>ProviderName</Provider>
<Product>ProductName</Product>
<BlueprintID>Folder1\Folder2\File.xml</BlueprintID>
into occurrences of -

Code: Select all

<BlueprintID>ProviderName\ProductName\Folder1\Folder2\File.xml</BlueprintID>
Of course, this idea falls flat if there are any extra lines within a style two reference, or if the three lines are not always contiguous . . .

Just a thought !
deltic009
Very Active Forum Member
Posts: 4017
Joined: Fri Nov 27, 2009 1:06 am

Re: A big project with potentially big performance improvements.

Post by deltic009 »

MisterWho wrote:Perhaps I am barking up the completely wrong tree, but given that there are two short code segments above, would it not be possible to write a small utility to pre-process any .xml / .bin files that need to be changed for this task ?

If there are two styles for <BlueprintID> - one helpful, one not - then I am thinking that this utility could read each file that needs to be changed in turn, changing all the style two references to be style one references. To put it another way, to change all occurrences of -

Code: Select all

<Provider>ProviderName</Provider>
<Product>ProductName</Product>
<BlueprintID>Folder1\Folder2\File.xml</BlueprintID>
into occurrences of -

Code: Select all

<BlueprintID>ProviderName\ProductName\Folder1\Folder2\File.xml</BlueprintID>
Of course, this idea falls flat if there are any extra lines within a style two reference, or if the three lines are not always contiguous . . .

Just a thought !
It is an excellent thought, and a very sound one at that, but sadly although the format is identical, when replacing in RW Tools at least I have found that some blueprints have a different number of spaces between the Provider and Product line and I think in this case the Scenery Tiles, Loft and Road Tiles and then the Tracks.bin each have 1 of 3 different spacings for it.

First of all I don't mind doing it manually. But if we discover there is indeed a beneficial reason for what I'm doing that could be applied more widely in other freeware routes then perhaps it could be revisited.
Matthew Wilson, development team at Vulcan Productions

http://www.vulcanproductions.co.uk
https://www.facebook.com/VulcanFoundry/
gptech
Very Active Forum Member
Posts: 19585
Joined: Fri Oct 10, 2008 5:48 pm
Location: Wakefield, West Yorkshire

Re: A big project with potentially big performance improvements.

Post by gptech »

There's nothing new in making routes as self-contained as possible, it's how DTG work whether any assets are simple duplicates of those used in other routes or re-textured versions. No problem for DTG to do this as those assets are theirs.
Following the same strategy for freeware routes though does raise the potential problem of not all asset creators being open to their work being edited, or even just being installed in any location other than under their Provider/Product folders. To do the job *properly* would need all creators to give permissions for their assets to be used in such a way, and we all know some are more *protective* than others.

I believe the RailSimulatorCore P/P is enabled by default in all routes,
deltic009
Very Active Forum Member
Posts: 4017
Joined: Fri Nov 27, 2009 1:06 am

Re: A big project with potentially big performance improvements.

Post by deltic009 »

gptech wrote:There's nothing new in making routes as self-contained as possible, it's how DTG work whether any assets are simple duplicates of those used in other routes or re-textured versions. No problem for DTG to do this as those assets are theirs.
Following the same strategy for freeware routes though does raise the potential problem of not all asset creators being open to their work being edited, or even just being installed in any location other than under their Provider/Product folders. To do the job *properly* would need all creators to give permissions for their assets to be used in such a way, and we all know some are more *protective* than others.

I believe the RailSimulatorCore P/P is enabled by default in all routes,
Whilst I fully support creators rights to make reasonable restrictions I would really be surprised if they were actively unwilling to allow blueprint files to be permitted in location X, Y or Z. The original item would still reside there, this would just be a duplicate only.
Matthew Wilson, development team at Vulcan Productions

http://www.vulcanproductions.co.uk
https://www.facebook.com/VulcanFoundry/
gptech
Very Active Forum Member
Posts: 19585
Joined: Fri Oct 10, 2008 5:48 pm
Location: Wakefield, West Yorkshire

Re: A big project with potentially big performance improvements.

Post by gptech »

How do you define "reasonable restriction"?
If all you're doing is shifting a copy of the .bin to within the route's Asset folders it's only half a job----the game still has to flit about to reference and load those assets.
deltic009
Very Active Forum Member
Posts: 4017
Joined: Fri Nov 27, 2009 1:06 am

Re: A big project with potentially big performance improvements.

Post by deltic009 »

gptech wrote:How do you define "reasonable restriction"?
If all you're doing is shifting a copy of the .bin to within the route's Asset folders it's only half a job----the game still has to flit about to reference and load those assets.
It does, but it does so by not pre-loading EVERYTHING in that asset folder that isn't required - I wouldn't consider it half a job, it's trying to make the best or something better out of a bad situation without having to copy the shapes and textures and everything else that is very space intensive. At the moment my custom Product folder is a total of 6.37Mb on disk for over 1000 files.
Matthew Wilson, development team at Vulcan Productions

http://www.vulcanproductions.co.uk
https://www.facebook.com/VulcanFoundry/
gptech
Very Active Forum Member
Posts: 19585
Joined: Fri Oct 10, 2008 5:48 pm
Location: Wakefield, West Yorkshire

Re: A big project with potentially big performance improvements.

Post by gptech »

Assets not used in a route aren't pre-LOADED as such; pre-REFERENCED so they show as available, yes. Have you determined the RAM usage of the route without any scenarios in the World Editor, and compared that with the Scenery tiles removed? Has anybody investigated the possibility of a single *less than optimal* asset being at the heart of the issue?-----probably more likely than a fault in the TS core, which manages to run other routes with little bother.
Locked

Return to “[TS] General Discussion”