Complex scenarios crashing through lack of available memory?

[salopiangrowler]

Stick the railworks.exe in exceptions on your AV. Sometimes your AV will start scanning without you realising.

(For firefox users) Do not run Firefox unless you have a beefy machine. For some reason when firefox starts going funny it'll affect railworks and pretty much everything else you're trying to do.

[gptech]

Stick the railworks.exe in exceptions on your AV. Sometimes your AV will start scanning without you realising.

nearly...the on demand/on access routine in most AV apps scans all files when they're used---simply putting Railworks.exe into an exception list is not enough.

Ash, what AV do you use?--they all have a slightly different way of doing things; some use the term 'on demand scanning', others 'on access scanning'.
Preventing the AV from scanning the TS files as they're loaded into RAM (on access, on demand) is simply a case of adding the folder to a list--where that list is differs from app to app but it'll probably be an option tucked away in an advanced section of the options menu. This doesn't prevent the AV from scanning the files in any 'excluded from scanning on demand' folder when the AV runs a scheduled scan, or if you start a scan manually.

[ashgray]

I use Norton Security these days, but now that I know that it was the AV scans that were being referred to, I've checked and my entire Railworks folder is already excluded.

Ash

[peterfhayes]

I'll try and explain this one more time as posters are still talking about memry issues:

The Virtual Address Space and its relationship to Physical RAM, and why TS 2016 can crash

The Virtual Address Space (VAS) and Physical RAM have almost nothing in common, as long as there is enough physical RAM installed correctly so that data from the VAS can be used by the computer, ie the working set, then all is well. The VAS has nothing to with the “paging file” which is basically a hard drive extension of the physical RAM. The OS/VAS does communicate to the RAM via the by transferring instructions from the OS/VAS to the working set, but the size and speed of the RAM does not affect the VAS. The VAS is from the OS and RAM is hardware and the VAS and OS communicated with the RAM via the cpu.

Each time an application runs on a Windows (OS), Windows creates a new process and a new VAS for this process. Basically, this is so that if the application crashes, it doesn't crash Windows as well as it did in earlier versions of the OS.

I'll try to explain with a 32-bit app like TS2016* running in Windows (XP, Vista, Win7) the OS reserves 4GB of Virtual Address Space for the program to run in (this has absolutely nothing to do with how much RAM is installed). The reason for the VAS is if an app like TS2016 crashes, it does not bring down the whole windows OS as it would have done in earlier versions.

Now running TS2016 in a Windows OS 32-bit, Windows reserves 2GB VAS (aka kernel-mode VAS) for the system and 2GB for TS2016. However, if you have video card with say 512 MB VRAM you lose 512 MB off your 2GB VAS, leaving only 1.5GB VAS for TS2016 to operate in. To get around this in a 32-bit system only, the /3GB (userva=2560) switch is applied freeing up to 3GB VAS (NOT Physical RAM) to TS2016, but the video card will still pinch at least 512 MB. This is not without problems as the system has only got 1GB VAS to operate in and that can cause instability.

Fortunately in a 64-bit OS a 32-bit app like TS2016* has access to a full 4GB of VAS and the video card uses only a tiny, tiny amount of that. So now TS2016 has the 4GB VAS to operate in it becomes less prone to crashing and Out of Memory errors.

*TS2016 has a flag called ‘IMAGE_FILE_LARGE_ADDRESS_AWARE’ set so that it can use the full 4GB of VAS

You can have 1GB RAM or 16GB Physical RAM, plus a 4GB paging file installed but TS2016 will still only have 4GB VAS tops in a 64-bit OS.

So forget Physical RAM – it is not important as long as you have installed it properly and the correct amount for you OS version. It has very little impact on any crashes seen in TS2016.

The problem is that you only have 4GB VAS and as the program like TS2016 (especially in the editor) runs, the VAS becomes fragmented (VMMAP) and the contiguous space for TS 2016 to load into becomes less and less. Now once you reach a critical point (and even not having 1MB of contiguous space can cause this) the 32-bit app crashes. Nine times out of 10 the reliability monitor or event viewer will record a cx00000005 error the so called “memory mismatch”. Restarting TS2016 regularly will clear the VAS and make crashes less frequent.

TS2016 does NOT crash due to a RAM issue, unless it results in a BSOD – then something else is very wrong.

Once we get a 64-bit version of TS then all this will become part of history.
pH

[ashgray]

I'll try and explain this one more time as posters are still talking about memry issues

To be honest Pete, I wished I'd never raised the thread in the first place.

What you say may well be absolutely correct but it's way too techy for me (and I can handle some tech). I've pretty much understood most of the explanations above but prior to this exchange of knowledge, I'd no idea - I'd never have mentioned memory as being the "culprit" but those passing themselves off as experts on the Steam Community obviously have this all wrong.

Hey ho....

Ash

[peterfhayes]

Ash
I apologise for the geek speak - it is technical and the concept is difficult to grasp.
It took me 5 years to convince the FSX forums that VAS was the major issue in FSX crashes.
As I say once the 64-bit version reaches us this will be part of history.
regards
PeterH

[ashgray]

Not a problem Pete.

Anyway, the problems will likely remain for me - I've invested £thousands in TS and cannot justify starting out all over again with the 64 bit version.

Ash

[gptech]

I apologise for the geek speak - it is technical and the concept is difficult to grasp.

There's a nice, if rather old now, explanation of memory management under operating systems based on NTFS on Technet--not an easy read, about 120 pages
I'll see if I can track it down.

Point is, your explanation is brilliant in such a small space as a single forum post Peter.

[peterfhayes]

Gary
Thanks - I owe my explanation to Mark Russinovich the Microsoft guru who founded SYSINTERNALS - his videos and posts of memory management are the gold standard: https://blogs.technet.microsoft.com/mar ... al-memory/ and

This video is hosted by YouTube, a site which is outside of the editorial control of UKTrainSim and as such Parental Control is strongly advised.
To view the video, click on the PLAY button.
If you click anywhere else on this video, you will be taken to a YouTube page, where you may find yourself subjected to content that you feel is offensive.
If you feel THIS video is inappropriate, please report it to a moderator immediately for prompt action.
PeterH

[gptech]

The article(s) I have in mind may very well be also by Mr Russinovich--still haven't tracked it down and it does date back to when I was a TechNet subscriber (not individually, but as a part of a group when the grand a year, when shared, was very cheap for all the free software you'd get)
I have run across this though... http://www.anandtech.com/show/2272/2
and whilst it focuses on a different game does show how a shortage of VAS rather than physical RAM can be the cause of problems in many (most/all?) cases.

[ashgray]

So I guess the question becomes - how to maximise the available VAS on our PCs?

Ash

[Bradforth]

Hi all,
Interesting reading, some of it above my old head. But I can say that I don't get crashes in any scenario's even AP's GEML which are known for crashes, I don't even have to save and reload scenario's as suggested by AP.
My PC can be described as a mid range system, so am wondering if there is something else that we are all missing.
I am running Windows 7 the only thing that I can think of is that I have spent hour's researching what to turn off that is running in the background, I even turn off my AV when running TS.
I hope I am not asking for problems now I have written this post but people with much better systems seem to get this crash prob.
Without appearing cynical I make very little changes to TS like some users in the assumption that the people that write scenario's know much more than me.
Regards,
Pete.

[gptech]

So I guess the question becomes - how to maximise the available VAS on our PCs?

Ash

Basically, you can't, let Windows get on with it.
VAS is determined by the operating system so all we can do is keep that in good order.
Memory management is very complicated, https://msdn.microsoft.com/en-us/librar ... s.85).aspx might help to complicate it even more

I make very little changes to TS

Changes to TS aren't usually a problem Pete, unless they're very badly done of course, and I think most TS users who do tinker with the game do so with little bother. Problems start however when folk tinker with Windows a little *blindly*, there's still a tendancy to view anything by Microsoft as 'not being as good as this utility from xyz.com'

[ashgray]

Thanks Gary - although it doesn't help, it's helpful, if you know what I mean!

Ash

[peterfhayes]

You can't increase the VAS beyond 4GB in a 32-bit app, wghen we get TSUE4 - 64-bit if the correct flags are set then it will be able to utilize up to 8 TERABytes of VAS and you could in theory have the the editor running continuously for 6 months before you MIGHT see a crash.

Can you alter the VAS - Yes in a way you can! There is one very technical tweak for the VAS which prevents a portion of it becoming fragmented and maintains the contiguous space.
This started off as a tweak in windows 32-bit or Windows server but it does work with Win 64-bit, but your settings may have to vary for optimal effect.
Its called the "HeapDecommitFreeBlockThreshold this can preserve the contiguous space of the VAS and may prevent some crashes in TS2016.
https://support.microsoft.com/en-us/kb/315407

Therefore, virtual address fragmentation is lessened or even avoided.

Regards
pH