Quest For The Perfectly Clean Railworks Install

[wacampbell]

OK, I'll admit up front that this quest is somewhere between a technical challenge and an OCD behavior.

I have a small, separate railworks install that I use for the day-to-day running of my favorite route. I think its been described before how these separate installs are done - I won't go over it. It allows me to have an install that is protected from Steam updates and that contains just the routes I like and the appropriate rolling stock for them. It makes railworks load faster and its is easier to track and maintain a smaller set of assets. I keep things organized for good operation - the correct rolling stock, working sounds, standardized physics etc.

I have been spending time cleaning up this folder, ensuring that all the files are present and the links from one file to the next are properly set up. One thing I noticed is that there are a ton of files that are not actually used! I wrote some software to do a hunt and destroy on these surplus files. I thought the stats might interest you all:

BEFORE THE CLEANUPS

Assets = 5,152,386,872 bytes, 17,927 Files, 1,322 Folders

AFTER TRIMMING UNUSED FILES

Assets = 3,193,116,002 bytes, 9,797 Files, 1,003 Folders

Thats over 8000 unused files amounting to nearly 2 gigabytes of data. Most of these files were just extra assets not actually included in the routes I was running, but some were left over files in rolling stock folders etc that just didn't seem to link to anything. Of course everything still works with the unused files removed - I am pretty sure I haven't missed anything thats actually needed.

I was surprised at the quantity and feel quite good about the results of the cleanup.

So my next step was to search for duplicate files. For this I hacked a program together that hunted out duplicates not based on filename, but on actual content. If file A contained the same content as file B it would report it as a duplicate, regardless of its name or location.

UNNECESSARY DUPLICATES = 974,166,964 Bytes, 2,501 Files

Surprised again - almost a full gigabyte of duplicate files. There are duplicated bogie files, duplicated audio files etc, but many of these are .tgpcdx files where identical files are duplicated from one repaint folder to the next. I suspect since these identical files have different filenames, the graphics engine won't detect them as identical and will load multiple copies into the GPU wasting its finite capacity. I haven't started on how to clean this up.

My next step is to think up some good ways to remove these duplicates, sharing common assets when identical files are referenced. I'll report back if I make any progress in this.

Just an FYI for discussion....


Wayne

[Wikkus]

Not the first time I've looked at the contents of my Railworks folder and thought how hugely inefficient it is. A couple of iterations ago I thought my 50GB RW installation was a bit bloaty. Now, under TS2015 (and with yet more content), it's become morbidly obese

The problem is compounded, particularly with legacy (Kuju) content, when reskinners duplicate pretty much everything when representing a single locomotive, but I think also that RSC/DTG actually made things worse with their shift to the .ap file archive format along with "bundled" routes/stock, e.g. where a route comes with an item of stock that is then duplicated (albeit potentially with a different skin) in another route or stock pack.

I'd very much like to see my TS2015 installation go on a serious diet.

EDIT: Not to mention unwanted/needed .tgt and .cost files I've eradicated many thousands of.

Rik.

[smarty2]

Tell me! I regularly run Rosgars handy little destruction tool that resides in my assets folder, which also takes out xml files too, best not put that tool in the content folder then!!!

[august1929]

Very interesting Wayne,

I try and keep my mini installations as clean and trimmed as possible, but the folder structure of the sim makes it much more difficult than in MSTS days when you really could keep a tight control.

My main installation is large at 164GB. I am happy to go with a slow load into the sim for convenience but I am very aware that there is a lot of bloat!

Some form of cross referencing/sharing of assets would be excellent.

Rod

[deltic009]

Very interesting Wayne,

I try and keep my mini installations as clean and trimmed as possible, but the folder structure of the sim makes it much more difficult than in MSTS days when you really could keep a tight control.

My main installation is large at 164GB. I am happy to go with a slow load into the sim for convenience but I am very aware that there is a lot of bloat!

Some form of cross referencing/sharing of assets would be excellent.

Rod

Well, that was supposed to be the purpose of the reskin blueprint but alas it emerged broken and has never been fixed like so many things.

[Bradforth]

HI all,
I also have delete tool in my assets folder designed by gptech which gets rid of surplus garbage, he did state that is not an performance enhancing tool.
I don't agree, play a scenario that has micro stutters then run the cleaning tool and run the same scenario again after using the tool, I have definitely found that most of the micro stutters have disappeared. Give it A try and see if you have the same result.

Regards,
Pete.

[Wikkus]

HI all,
I also have delete tool in my assets folder designed by gptech which gets rid of surplus garbage

TBH, a "tool" (batch file, I'm guessing) isn't even necessary any more if you spend enough time as many of us do in Windows Explorer; I simply do a search with the following string in the Assets folder and then delete the results
*.pak OR *.tgt OR *.cost OR *.xml

Because Explorer retains a list of searches, I don't often have to retype it either.

EDIT: Failed to mention, you can create a shortcut to the Assets folder in Explorer which saves more time still.

Rik.

[gptech]

Yeah, just a .bat file and I really wouldn't claim to have designed it. If I remember rightly I just posted the 5 or 6 lines that comprise the .bat file I use in a thread some time back. I doubt it differs greatly from any of the other routines posted.

[wacampbell]

Yes, eliminating the .tgt, .cost, *.xml, .proxybins etc is the first step and just to be clear I had already done that to my BEFORE CLEANUPS stats above.

My cleanup went deeper and actually found almost 2 gigabytes of .tgpcdx, .geo and .bin files that were not used by any assets or routes in my install.

And BTW, my total asset folder in my mini-install is small at only 5 GB. So 2 GB of garbage is significant. I can only imagine what lies in Rod's 150 GB asset folder

Now if I can only figure out a good way to eliminate that extra 1 GB of unnecessary duplicate files.

[Wikkus]

Yeah, just a .bat file and I really wouldn't claim to have designed it. If I remember rightly I just posted the 5 or 6 lines that comprise the .bat file I use in a thread some time back. I doubt it differs greatly from any of the other routines posted.

It's all good

Different strokes for different folks, etc.


Rik.

[august1929]

And BTW, my total asset folder in my mini-install is small at only 5 GB. So 2 GB of garbage is significant. I can only imagine what lies in Rod's 150 GB asset folder

None of my minis get anywhere near that small - My current regular use one contains all 3 versions of Mark Howarth's fantastic Laneshaw Bridge, plus assets, stock etc to run the three eras and some additional stock that I might want to use some time - this one comes to 30 Gig, but I have thrown everything plus the kitchen sink in .

My main install is due for its regular pruning

Rod