I suggest you group it as follows.
Reference manual. This is for the hardcore hackers. Don't say they are only a few because they are the nastiest of your customers.
For each of the (XML) files:
for each of its elements, describe which elements and attributes it can contain,
and for each attribute, spell out which values it can take.
If they are numeric (such as radius of a wheel), don't forget to tell us about permitted units and defaults (and default units). For symbolic values (such as type = carriage, freight ...) list all permitted values with their meaning (and the consequence of using them) plus all values which are reserved for future use. Should there be some numeric ranges of IDs such as in MSTS's track parts, make a plan how to assign these numbers and describe it here.
For each API available to Lua scripting (Signals, what else?) describe each function call (or whatever it is called in Lua) with all its parameters, as described above.
For each process that KRS models internally (physics of engines and wagons, weather, activities of AI drivers & passengers): Describe the process to the smallest detail, followed by a reference of all influence parameters. There is some necessary overlap with the description of the other parts of the reference manual here, but without describing both parameters or data structures on the one hand and internal calculations of KRS on the other hand, no-one will understand it all.
User Manual. Since normal persons will run away screaming if you present them with the reference manual only, you need a description of the tasks which can be done when modelling or configuring or adapting something in KRS. This is best organised in a task specific way. And it is a never ending story.
For starters, you need something like the tutorials available for MSTS. "How to build XYZ". Over the first year or so it should evolve to a collection of white papers or application notes, where external authors describe how they solved a specific problem. E.g., how to implement signalling system X from country Y in period Z. Such in-depth discussions are invaluable time savers for other developers, but it is clear that you (Kuju) cannot supply them yourselves. But you could try to coordinate writers, motivate them, and provide central discussion forums for each of the white papers.
Another complement would be write-ups of the most frequently asked questions, but not in the standard form, but prepared as dummy-proof tutorials which guide people through anything that many people do not understand, whatever that turns out to be – and irrespective of the fact that it may be described somewhere already. This would be created in a reactive way, as a complement to the proactively prepared core of the user manual.
Good practice recommendations. As Matt already mentioned, there should be rule for the soft factors of quality. It would be great to have tools supporting such an analysis, e.g., ShapeViewer shows the polygon count and distance for each LOD and some ratio of polygons and size of object. So to make this type of quality assurance work, we need rules plus tools. And of course everybody's understanding that deviations need not be crimes if well motivated (and only then

).
List of limitations. It is only wise to say frankly what will not work now. For each item, you have three options: Promise it for version 2 (only if you are sure about it, please); Propose a work-around (many things can be done via some extensive scripting, if that is possible); Just say: "sorry, this is beyond the scope" (fair enough for very exotic stuff).
List of special cases. Form a committee for the sick cases. Their task is to find all combinations of parameter values which could give a headache. For each of them, find out if the program crashes and make sure that it does not. Then describe whether such a case is allowed or not and what the consequences are. There is a wide range from not yet implemented feature to plain folly to configuration error.
Examples: 4th rail together with overhead wire; Narrow gauge track inside normal one – sharing one rail or not; Rack railways; Cable railways; horse pulled wagons; Turntables which cannot do a full rotation or slip switches. But also bad configurations such as mismatch of gauge, buffer position, couplers, connectors, speed limitations when composing a consist; Random engine designs; Plants on tracks (rigid or not, big or small).
Of course, items from the list of limitations need not be discussed here, instead the focus is on combinations of existing features which may cause problems. You could see this task force as theoretical beta-testers and indeed their ideas will be input to the QA department. The important point is that you make all these cases public, to show how KRS performs under extreme conditions. And it is always good to actively describing limitations instead of giving the impression that there is a sea of unexplained behaviour. Even more as many of these combinations will be very exotic. But it is nice to know that KRS does not crash under a certain subset of them.
As for the format, you certainly will use some form of hypertext, be it PDF, WinHelp or HTML. Otherwise, you cannot implement the many necessary cross-references efficiently.
Since the whole thing will grow over time (hopefully), a flexible scheme of downloads would be the best form of distribution. Maybe some first part comes as a complete package and the rest comes as supplement. In any case, the reader must have an easy way to determine which information he already has and which needs to be downloaded.
I strongly recommend that you have the documentation beta-tested, as much as the software itself. It costs much less than the software but many a program makes a bad impression because of its sloppy documentation.
And please do not translate the documentation before you have ascertained the assistance of a strong group of native speakers. I always read the English docu whenever I can get it because it is so difficult to figure out what people might have thought of when coming up with a certain term.