An Open Rail Simulator?

Use this forum to discuss generalisations that apply to all / some / none of the present train simulators.

Moderator: Moderators

Re: An Open Rail Simulator?

Postby jkxx74 on Thu Jan 29, 2009 6:22 am

Note: I have not used an existing 3D engine I played around with a few,all were to limiting or to slow.Same thing with SDL the 3D part of these is chronically slow although the sound appears to be OK.
Note 2: Distant terrain is a MUST.... Very difficult to do along with the rest of the scenery..


Fortunately hardware has improved quite a bit in the past few years. In early 2006 I played around with Ogre3D a bit with the intention of maybe trying something train-sim related as it was just MSTS (for me) at that time and it had a horrible habit of bluescreening anything in its path. The speed definitely seemed at least adequate although having very little game design experience, the bulk of time would've been spent by me trying to figure things out.

OpenBVE though seems to have gotten things right, it easily gets 75 fps here at 1280x1024 with all filtering/AA settings on max. And I must say I'm impressed by the degree of realism, from the way fog seems to surround nearby objects, to lighting, to train movement, the appearance of track, and of course the incredibly accurate sounds as the train rounds curves and passes over points (both are very important to me).

Distant terrain is very important too.. I hope this is improved upon in RS with Mk3. RS already comes with good scripting abilities in the audio department so realistic sound is already possible with the right samples and scripting. Now we just need the dispatcher fixed and some cab sway :D

It's definitely very exciting to see the progress openBVE has been making though. Especially the vids posted here.

Cheers,

jkxx
User avatar
jkxx74
Been on the forums for a while
 
Posts: 272
Joined: Tue Jan 23, 2007 5:36 am
Location: Colorful Colorado

Re: An Open Rail Simulator?

Postby BadWhippet on Thu Jan 29, 2009 6:40 am

Having had trouble getting previous versions of BVE to install and operate, I hadn't really visited OpenBVE, but having just watched that video I am SPEECHLESS! Totally incredible work! Just... incredible! 8)

WANT! :D :multi:
SUE | i7 3820 @4.1Ghz | 16Gb DDR | nVidia 580 3Gb | SSD +6Gb/s Data | still only 14fps! :D
Image
Fabulous photos from Ali (desiro5), Mark Walker (almark) and David Hennessey
User avatar
BadWhippet
Been on the forums for a while
 
Posts: 202
Joined: Sat Apr 26, 2008 2:11 am
Location: East Anglia

Re: An Open Rail Simulator?

Postby rickloader on Thu Jan 29, 2009 1:01 pm

zlindku wrote:For about 18 months I have been playing around with the design of a trainsim. The object here is to see if it can be done rather than to produce a simulator although this MAY happen.I have done a "first draft" terrain engine but my main thrust has been vehicle "physics" and proper AI trains and timetabled operation. One of the main problems I have identified is processing power. The graphics can take every bit of power you can muster so as far as I am concerned the graphics will almost certainly need to be held back. I regard the graphics for MSTS to be quite good enough for starters. While such things as animated trees and grass look nice there are more important things to do.The real killer is the AI operations this is proving to require quite a head of steam.(what I am looking for is for the AI stuff to really exist and be controlled properly)........snip
The main purpose of this post is to say a decent sim requires a LARGE amount of steam to run and the people using it need to decide what they really want. You simply cannot have everything, there is not enough processing power.....snip

(hope you don`t mind me editing your post)
What an interesting post, and it gets to the heart of a of crucial issue.Oh! my vote is for the AI and timetables! (I think I`m becoming a bore about this)
I think a new sim with a heavy AI bias would be very suitable for an open sim because commercial sims need gorgeous graphics to sell. Just look at the rapture here over BVE! (does anyone know /care about BVE AI capability?)
So Zlindku keep at it, and if you can enlighten us further about AI, I, for one would love to know.
Best wishes Rick
User avatar
rickloader
Well Established Forum Member
 
Posts: 621
Joined: Mon Apr 11, 2005 5:11 pm
Location: Hampshire

Re: An Open Rail Simulator?

Postby zlindku on Thu Jan 29, 2009 3:30 pm

jkxx74 wrote:
Note: I have not used an existing 3D engine I played around with a few,all were to limiting or to slow.Same thing with SDL the 3D part of these is chronically slow although the sound appears to be OK.
Note 2: Distant terrain is a MUST.... Very difficult to do along with the rest of the scenery..


Fortunately hardware has improved quite a bit in the past few years. In early 2006 I played around with Ogre3D a bit.................. The speed definitely seemed at least adequate although having very little game design experience, the bulk of time would've been spent by me trying to figure things out.

OpenBVE though seems to have gotten things right, it easily gets 75 fps here at 1280x1024 with all filtering/AA settings on max. And I must say I'm impressed by the degree of realism, from the way fog seems to surround nearby objects, to lighting, to train movement, the appearance of track, and of course the incredibly accurate sounds as the train rounds curves and passes over points (both are very important to me).

Distant terrain is very important too.. I hope this is improved upon in RS with Mk3. RS already comes with good scripting abilities in the audio department so realistic sound is already possible with the right samples and scripting. Now we just need the dispatcher fixed and some cab sway :D

jkxx


The 3d requirements of a trainsim are really not that high. One of the most difficult things to do in games programing is collision detection for random collisions. This is not required in a train sim. All collisions (coupling cars together etc) in these are tightly controlled and relatively easy to allow for. Most 3D engines are designed for combat and racing sims, in these collisiion detection is critical.
The problem with frames rates is what is the level of detail and how far you are looking ahead and importantly how much traffic is the sim controlling. In theory in a 3D world the amount of detail goes up by the cube of the viewing distance. Experiments I have done indicates this is likley to be correct, this means it is very easy to bring a 3D engine to its knees by increasing the viewing distance. For this reason distant terrain is a real challenge, it has to be some kind of short cut you cannot do it by "brute force".
There is an old saying in simulation design "It does not have to be real it just has to look real" well worth remembering.
On the traffic side a sim may easily have to control anything up to 1000 separate vehicles and do the AI for a dozen or more trains,this takes significant amount of power.

The comment on cab sway brings up two random points. At this stage I am not considering "cab sway" at all. initial simulations of a vehicle running down a track shows that to be accurate this is very difficult if not impossible to do.There at least two causes of "cab sway" (variations in track and out of balance forces) neither of these are simple to do especially the track and psuedo sway is too easy to pick up as false.(In some of the more modern railways it is almost not there at all,the ballarat line for instance)
The second random point, I am not including manual firing, this is because in real life you cannot drive and fire at the same time unless possibly you are shunting.(this has been done in real railways)
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

Re: An Open Rail Simulator?

Postby paulz6 on Thu Jan 29, 2009 3:56 pm

zlindku wrote:On the traffic side a sim may easily have to control anything up to 1000 separate vehicles and do the AI for a dozen or more trains,this takes significant amount of power.


I'm surprised your already struggling with that number of vehicles to be honest. Are you sure your processors are actually processing and are not just appearing to be processing. Please don't feel insulted as I am sure what you are doing is very complex. Letting processors get on at what they do best is very difficult, especially when you go down the multi-threaded design route.

EDIT: Really don't feel insulted. I've worked on some large multi-processor machines where the software platform could struggle to perform 100,000's of trades a day. In this case the machines really weren't doing that much processing - a lot of processing was wasted on memory copying and thread synchronisation etc..
Last edited by paulz6 on Thu Jan 29, 2009 4:24 pm, edited 1 time in total.
User avatar
paulz6
Very Active Forum Member
 
Posts: 2246
Joined: Tue Oct 18, 2005 4:22 pm
Location: Disused Railway Lineside Shack

Re: An Open Rail Simulator?

Postby zlindku on Thu Jan 29, 2009 4:21 pm

paulz6 wrote:
zlindku wrote:On the traffic side a sim may easily have to control anything up to 1000 separate vehicles and do the AI for a dozen or more trains,this takes significant amount of power.


I'm surprised your already struggling with that number of vehicles to be honest. Are you sure your processors are actually processing and are not just appearing to be processing. Please don't feel insulted as I am sure what you are doing is very complex. Letting processors get on at what they do best is very difficult, especially when you go down the multi-threaded design route.


This number is just a guess at this stage I have not done any speed up work on the model as yet.A train of vehicles all effect on another. A good example to get your mind around this is a long string on a roller coaster ride, some of the vehicles are rolling down hill and some are being pulled up hill. In a decent mathematical model you have to work out what effect each car has on the next. This means having to mathematically traverse the string of vehicles a number of times, once for each vehicle in the train.A train is not one monolithic block.
Note: It may be quite on the cards that if an AI train is never to be taken over by the player to treat it as a monolithic block. I have a feeling that this would produce more trouble than it would solve.
One must always remember that you have every thing else to process in the sim. As for as I can see a lot of modern programmers never worry about execution speed and end up with a program that behaves like a snail on horse tranquilizers.
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

Re: An Open Rail Simulator?

Postby zlindku on Thu Jan 29, 2009 5:00 pm

rickloader wrote:(hope you don`t mind me editing your post)
What an interesting post, and it gets to the heart of a of crucial issue.Oh! my vote is for the AI and timetables! (I think I`m becoming a bore about this)
I think a new sim with a heavy AI bias would be very suitable for an open sim because commercial sims need gorgeous graphics to sell. Just look at the rapture here over BVE! (does anyone know /care about BVE AI capability?)
So Zlindku keep at it, and if you can enlighten us further about AI, I, for one would love to know.
Best wishes Rick


I am just starting on the AI, I completely agree with you comments though. Time tabled operation by the AI trains means good AI is a must. The obvious approach is to use a fuzzy logic (ie you grade the answer on a scale of say 1 to 10) rules based engine using various trigger inputs such track, loading stage, platform, length of carriage etc. ie what is known i believe as an expert system.
What i would like to be able to do is to say this train stops at these places and the system comes up with a valid time table. This means the AI must be able to drive the the route correctly and not necesarily in real time.
A large amount of the current work is research and working on algorithims, All the mathematics and routines are being done on paper first to see if they work. This is a technique I have found VERY successful in the past saving much programming time.
This is probably just as well at the moment as it is a very hot summer here and I have not been able to turn my PC on for more than about 1 hour.
To do a trainsim requires a good background in mechanics,physics,mathematics, thermodynamics won't go astray, a knowledge of electrics would be good and of course a decent background in railways and of course good research skills, programming comes last....full stop.
I do not regard myself as a programmer, I regard the trainsim as a research and development exercise.
Having a great deal of Fun though

Lindsay
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

Re: An Open Rail Simulator?

Postby paulz6 on Thu Jan 29, 2009 5:38 pm

Lindsay, good luck with your project. The level of detail you desire is going to require a lot of processing power. A little tip to squeeze that extra processing power is to consider how your data structures are arranged into memory pages. If you can keep groups of processing calculations onto a single memory page, the memory caching really lets the processors fly. Have you considered using multi-processing cores? I think for a real-time train sim it is going to be very difficult to get them working together effectively.

The way I have peen tackling this is to split the sim into two parts the graphics engine and the trainsim engine and run them on two pc's view a network connection.


This is going to be a very difficult nugget to crack. The main issue I see is getting all those detailed real-time calculations in the trainsim engine into the graphics engine satisfactorily. There is going to be latency between the trainsim engine calculations and the graphical display.
User avatar
paulz6
Very Active Forum Member
 
Posts: 2246
Joined: Tue Oct 18, 2005 4:22 pm
Location: Disused Railway Lineside Shack

Re: An Open Rail Simulator?

Postby zlindku on Fri Jan 30, 2009 10:19 pm

paulz6 wrote:Lindsay, good luck with your project. The level of detail you desire is going to require a lot of processing power. A little tip to squeeze that extra processing power is to consider how your data structures are arranged into memory pages. If you can keep groups of processing calculations onto a single memory page, the memory caching really lets the processors fly. Have you considered using multi-processing cores? I think for a real-time train sim it is going to be very difficult to get them working together effectively.

The way I have peen tackling this is to split the sim into two parts the graphics engine and the trainsim engine and run them on two pc's view a network connection.


This is going to be a very difficult nugget to crack. The main issue I see is getting all those detailed real-time calculations in the trainsim engine into the graphics engine satisfactorily. There is going to be latency between the trainsim engine calculations and the graphical display.


Thanks for you reply,

You said "The level of detail you desire........." This I believe is the corner stone of a lot of the complaints behind MSTS and RS (I do not like Trainz at all) and I identified as THE major problem with a decent trainsim within hours of starting this project. So much CPU is spent on the graphics there is little left over.

I am an old assembly language programmer actually, even now I still think in ways such as you have mentioned to enable the system to do as much as possible. In the graphics section the problem with SMP is that the amount of data flowing between the memory,CPU and GPU is massive and it greatly reduces the second CPU.s access to the bus.Note: this is an old and well known problem with SMP when a lot of data is flying around. For the trainsim itself though there as a very large number of independent items to model that do not require a large amount of data and require a fair amount of CPU and SMP may contribute well here. I am defintely going to give it a try.

ON the network model, I originally thought as you on this but I believe it is how you look at it. When you you are calculating and displaying something in the same program you still have to package the data in away the display section can use, for the network model this just becomes more formalized. I Linux there are a number of very good programs that work on the network model, not the least being freeciv.
Latency is something I am checking out, i believe it will be OK though. This will probably depend on how the control is done.
One of the many things that require research and work.
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

Re: An Open Rail Simulator?

Postby zlindku on Fri Jan 30, 2009 10:23 pm

Note to moderators and all others. Would it be worth while shifting this to a separate thread. I REALLY value the comments and discussions with others on this topic. This is a major reason why i placed the first post.
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

Re: An Open Rail Simulator?

Postby CaptainBazza on Mon Feb 02, 2009 1:21 am

I 'stickied' the thread, which can be reviewed at a later date, depending on how the thread develops.

Cheers Bazza (Mod's hat)
User avatar
CaptainBazza
Has a sign reading.. Its NOT the end of the world!
 
Posts: 17949
Joined: Tue May 13, 2003 10:21 am
Location: Land of the Long White Cloud.

Re: An Open Rail Simulator?

Postby rickloader on Mon Feb 02, 2009 11:49 am

zlindku wrote:
........Time tabled operation by the AI trains means good AI is a must. The obvious approach is to use a fuzzy logic (ie you grade the answer on a scale of say 1 to 10) rules based engine using various trigger inputs such track, loading stage, platform, length of carriage etc. ie what is known i believe as an expert system.
What i would like to be able to do is to say this train stops at these places and the system comes up with a valid time table. This means the AI must be able to drive the the route correctly and not necesarily in real time......... snip
and
........"The second random point, I am not including manual firing, this is because in real life you cannot drive and fire at the same time unless possibly you are shunting.(this has been done in real railways"...........( post sedited)
Lindsay

Taking time tables first, I don`t see the need for the Ai to generate the timetable. The user will either want to do this from an actual historical timetable or can easily drive the route noting passing times. In MSTS the activity writer is often frustrated by the AI generating the timetable. Surely the AI dispatchers` job is to generate paths and control traffic in response to the given timetable. A key part of this will be the priority level assigned to each train. (eg an express has priority over a goods train.) The dispatchermust be intelligent enough to modify paths and priorities even if the traffic deviates from the expected. eg a late running express may lose priority and be held in a loop for an ordinary train running to time. I think this sort of "weighted" priority would add much interest and realism.
Taking the second point, I think not including manual firing would be a pity. Part of the fun of driving steam is balancing the load and gradients against steam production.
I think the player should represent the complete train crew, driving firing and shunting. As shunter the player decides which cars are dropped and collected according to the trucks waiting and the traffic requirements at each stop on the route.
This is much more interesting than a rigidly scripted activity.
Rick
User avatar
rickloader
Well Established Forum Member
 
Posts: 621
Joined: Mon Apr 11, 2005 5:11 pm
Location: Hampshire

Re: An Open Rail Simulator?

Postby paulz6 on Mon Feb 02, 2009 8:00 pm

zlindku wrote:I am an old assembly language programmer actually, even now I still think in ways such as you have mentioned to enable the system to do as much as possible. In the graphics section the problem with SMP is that the amount of data flowing between the memory,CPU and GPU is massive and it greatly reduces the second CPU.s access to the bus.Note: this is an old and well known problem with SMP when a lot of data is flying around. For the trainsim itself though there as a very large number of independent items to model that do not require a large amount of data and require a fair amount of CPU and SMP may contribute well here. I am defintely going to give it a try.


I think there could well be a solution where the calculations can be done without pushing data around the system and keep the processors processing. Avoid a design where you have a shared memory data structure, protected by a thread synchronisation lock, where many threads work on local copies and write the results to the shared data structure. That leads to massive amounts of memory copying and cache-line ping-pong.
I would be looking at a physics engine that is designed in a way that it can be scaled over 1 to many threads. Assuming that your mathematical calculations work on a time sample basis, it would be wise to avoid context switching as the processor time slice could easily be greater than the granularity of time sample. It is best to keep the run queue empty for a fine solution. Providing the time sample is not to fine, it should be possible to communicate results between the necessary processes without incurring too much thread synchronisation lock contention.
For example, on a system with few CPU's, you could keep to one physics engine thread which would perform the calculations on every train and rail vehicle. On larger systems, multiple threads could work on their own dedicated set of trains and rail vehicles. Now where did I leave that 256 processor server!

zlindku wrote:ON the network model, I originally thought as you on this but I believe it is how you look at it. When you you are calculating and displaying something in the same program you still have to package the data in away the display section can use, for the network model this just becomes more formalized. I Linux there are a number of very good programs that work on the network model, not the least being freeciv.
Latency is something I am checking out, i believe it will be OK though. This will probably depend on how the control is done.
One of the many things that require research and work.


I don't think it is a good example to compare against freeciv. Freeciv is not a real-time system, and the latencies of the data returned from the server do not effect the game play. For a real-time system, there will probably have to be some time stamping of the data returned to the server, so that the client can replay the data smoothly regardless of potentially variable latencies. Otherwise, the visual playback is going to look very juddery.

I agree with the above post about the timetabling and operations.
User avatar
paulz6
Very Active Forum Member
 
Posts: 2246
Joined: Tue Oct 18, 2005 4:22 pm
Location: Disused Railway Lineside Shack

Re: An Open Rail Simulator?

Postby zlindku on Tue Feb 03, 2009 6:54 pm

rickloader wrote:Taking time tables first, I don`t see the need for the Ai to generate the timetable. The user will either want to do this from an actual historical timetable or can easily drive the route noting passing times. In MSTS the activity writer is often frustrated by the AI generating the timetable. Surely the AI dispatchers` job is to generate paths and control traffic in response to the given timetable. A key part of this will be the priority level assigned to each train. (eg an express has priority over a goods train.) The dispatchermust be intelligent enough to modify paths and priorities even if the traffic deviates from the expected. eg a late running express may lose priority and be held in a loop for an ordinary train running to time. I think this sort of "weighted" priority would add much interest and realism.
Taking the second point, I think not including manual firing would be a pity. Part of the fun of driving steam is balancing the load and gradients against steam production.
I think the player should represent the complete train crew, driving firing and shunting. As shunter the player decides which cars are dropped and collected according to the trucks waiting and the traffic requirements at each stop on the route.
This is much more interesting than a rigidly scripted activity.
Rick


The flexiblilty must be there for the AI to generate the timetable just in case data may not be on hand or someone (me :-) ) may wish to do it that way. I have not started working on the dispatcher yet but the comments are agreed with and have been noted.
The main problem I have with manual firing is that it is impossible to fire and drive a main line locomotive without there being very great risk of an accident. One of the main aims I have is for this to behave as much as possible like a real railway or tram system, What I would like to create is a true simulation not a train driving game.
I hope the "activity" system will be wwwwwwaaaaaaaaaayyyyyyyyy more flexible than is currently availible. As long as the player completes everything at should not matter which way it is done. No having to go down particular roads or stop exactly there and so on. I find the "activities" system in both MSTS and RS way to rigid.

"This is much more interesting than a rigidly scripted activity" ........ absolutely,yes, Amen brother...... ;-)
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

Re: An Open Rail Simulator?

Postby zlindku on Tue Feb 03, 2009 7:40 pm

paulz6 wrote:
I think there could well be a solution where the calculations can be done without pushing data around the system and keep the processors processing. Avoid a design where you have a shared memory data structure, protected by a thread synchronisation lock, where many threads work on local copies and write the results to the shared data structure. That leads to massive amounts of memory copying and cache-line ping-pong.
I would be looking at a physics engine that is designed in a way that it can be scaled over 1 to many threads. Assuming that your mathematical calculations work on a time sample basis, it would be wise to avoid context switching as the processor time slice could easily be greater than the granularity of time sample. It is best to keep the run queue empty for a fine solution. Providing the time sample is not to fine, it should be possible to communicate results between the necessary processes without incurring too much thread synchronisation lock contention.
For example, on a system with few CPU's, you could keep to one physics engine thread which would perform the calculations on every train and rail vehicle. On larger systems, multiple threads could work on their own dedicated set of trains and rail vehicles. Now where did I leave that 256 processor server!

I don't think it is a good example to compare against freeciv. Freeciv is not a real-time system, and the latencies of the data returned from the server do not effect the game play. For a real-time system, there will probably have to be some time stamping of the data returned to the server, so that the client can replay the data smoothly regardless of potentially variable latencies. Otherwise, the visual playback is going to look very juddery.

I agree with the above post about the timetabling and operations.


Good post could not have put it better myself. I have found most people programming do not have such a good grasp of the hardware as this post shows. The points made have already been put into consideration.

Freeciv was only used as an example for such a system. It is usual (as far as I am concerned) to put both a time stamp and a sequence number on the data going in both directions between the two systems to try and keep everything in order. This is an area I am testing at the moment. An issue of signifacants here is there is no reliable way in the current OpenGL to know when the GPU's have finished doing the current frame so a way has to be found to monitor what is going on to keep the frame rate smooth. Specially in something like this when the major data generation is being done outside the graphics display system.
zlindku
Been on the forums for a while
 
Posts: 226
Joined: Fri Sep 07, 2007 12:03 pm

PreviousNext

Return to Generic Simulation

Who is online

Users browsing this forum: No registered users and 2 guests