Do we need an activity quality ranking system?

[rdawes]

I believe that our hobby has evolved to the point where we can establish a rating system for activities. This could be accomplished by ranking an activity based on several criteria and the server would then apply a weighted score based on users input. Activities would be given a ranking from 5 green signals through to 5 red signals.

Here are the categories that I think should be ranked:

1) Packaging - An apk file, plus readme file, plus stock list, plus small jpg < 50k.
2) Instructions - Route and version, author's name and email, objective of activity, helpful hints and not all in upper case or center justified.
3) Stock List - Only non-default stock listed with file ID's or server links and instructions regarding any coupling or brake adjustments necessary. Fewer downloads = higher rating and no points for raw Activity Analysis listings.
4) Installation - Activity installs without any hitches and passes the Route-Riter/Route Control analysis tests.
5) Running - The activity runs flawlessly with good pop up instructions where necessary and acurate work orders or achieveable station times.

So for each category above we would assign up to 5 grades or stars by entering the activity file ID and then picking the ranking in each category. Based on the total scores, the server would assign icons against activities as follows:

5 green signals - Outstanding Activity (23 - 25 points)
4 greens - 1 red - Excellent Activity (20 - 22 points)
3 greens - 2 reds - Good Activity (16 - 19 points)
2 greens - 3 reds - Fair Activity (10 -15 points)
1 green - 4 reds - Poor Activity (5 - 9 points)
5 red signals - Not worth the bother (0 - 4 points)

Users would have to be matched according to their user ID so that they could only vote once and not be able to flood the ranking, something like the way eBay manages their feedback.

This would remove the frustration that new MSTS users experience when they have to jig stock and activities to get them to run and might encourge our activity authors to higher achievement. We could even have a gala awards night for the best!

Any opinions?

Bob

[kevarc]

Overall, a bad idea. Really the first three could be one. I don't know about you but very very seldom will I dl anything that does not have a readme. Or would I before I have come to understand the in and outs of MSTS would I install one without a good readme.

The amount of Dl's in the stock list trouble me very much. You want to ding someone because of the number of downloads required. While I agree that there should not be 30 dl's for a single activity, I refuse to use the default equipment supplied with MSTS. It is inappropriate for the routes and eras of my activities. If I feel that there are to many dl's for my activity I hold it until I have a pack of activities where the number of dl's per activity is less than 10, I try to get it down to 5 or 6 per activity.

Running - this is the ultimate in nonsubjective areas. What is good for one person is not for another. Some like pop-ups, some don't. I use them in mine, but only when I feel that they are the only way to get what I want done across to the end user. I also want the end user to do some thinking on his own, I don't want to hold their hand all the way through the activity. Afterall he\she is both the conductor and engineer, they are the ones who determine how switching should be done. I think a better thing would be for designers to state what the difficulty of the activity in the readme as most of the time we do not list it in the act file like we should.

I think the most needed thing is a workorder generator to pull the switch list from the act file and create a text file for it.

[suffolk]

I agree that some sort of standardisation needs to designed for activities, the amount available for download is now over 400, however I'm not sure as to what sort of "rating" would be efficient, after all "one mans meat is another mans poison" so to speak.

Taking the points that you've listed it is now common practise for all new activities to contain a full stock list with ID's, but I'm not sure about the size of the jpeg, because after all once downloaded it can be deleted.

By all means if an activity is for a route with more than one version the read me should state which version it is for, however problems could arise when a newer version is made some time after activities are uploaded, I don't really think it matters whether upper or lower case is used or what justification is used as long as the read me is clear in the instructions of any files that need to be changed (brake systems or couplings). and e-mail address should be incuded out of courtesy


As mentioned before stock list with ID's is now becoming standard, but some activities require lots of stock which will cause large downloads, a balance has to be drawn between prototypical realism and realistic downloads, people strive to make "their MSTS" as real as possible and with the quality of some of the downloads available (cabviews, sound packs etc) it really feels like you are in the cab sometimes, but I've seen posts on here saying use the same consist more than once for AI traffic, which to me is contradiction.

Not every body has got all the utilities such as route riter etc.

Again this comes down to personal preference, some people I would imagine like messages to keep them informed of what they are to do etc, but others (myself included) think that it stops the flow of the activity, if you're shunting at 5-10mph and your fingers are constantly on the keys it is not too much of an inconveniece, but if you're tanking along in a HST it can be a bit of a pain to suddenly come to a halt.

Please don't think that I'm totally dissagreeing with all you say because I'm not, I agree that some sort of grading system needs to be introduced, it's just a question of how do we go about it.

[rdawes]

All good points.

My issue with DL's is not to restrict them but use some common sense. I hate installing a bunch of loco's only to find them peppered about as loose consists. Also, I know that in reality you wouldn't have the same numbered loco appearing over and over but when you consider the load on your system to manage each different loco for the sake of accuracy I don't mind cutting some corners. The final bug is a stock list that doesn't identify the default items but is just a big AA list. Some author's create a their activities around a group of rolling stock and you don't need anything else once you've installed their list. That's what I'm getting at regarding quality.

Bob

[eurorail]

The Trenomania site http://www.trenomania.it does a total quality verifaction on some of the routes, engines etc posted on its site, and issues a "certificate".

This would seem to get around most of the issues. If the standards were issued, and volunteers checked that the activity or route met the standards then it could be given a TQM mark. I have never has a problem with routed downlaoded from them which have been quality checked.

This would be a system that would be easy to apply to UKtrainsim.

I have written and published some activites for French routes, and altougth they have been tested before hand by others, it is a nightmare with stock located on a number of dispersed sites, some only FTP, and I think with diferrent folder names for the stock so I get complaints all the time.

[autonerd]

A friend once said "Being a good welder is easy; being a great welder is extraordinarily difficult." It's sort of the same with writing activities. (Disclaimer: I don't consider myself "great".)

I don't like having to spend more time setting up an activity than I will playing it. Especially if, after all that, it doesn't work. I have written a handful of activities, only published a couple of them, but I go to great lengths to make sure that they work. Some of my personal guidelines...

1) Before I start writing, I uninstall MSTS, delete the Train Simulator directory, then reinstall just MSTS, the patches, the route for which I am writing, and the rolling stock I want to use. The "clean" install gets me closer to baselnie and guards against using outdated routes, stocks, or con files with equipment in a non-default directory.

2) I try to only use rolling stock that either has a self-installer or, at the very least, a subdirectory specified in the ZIP file.

3) I always make new consists (if I want to use an existing one, I use it as a template then save it under a new name). I also use a common prefix; for Tokyo British Invasion, every file name (cons, services, etc.) begins with "tbi_" (ie "tbi_path_player", "tbi_traffic", "tbi_intercity"). Makes it easier to back up, also easier if the player wants to sub his own consists.

4) Once the activity is complete and playtested, I package it, then back up ALL the associated files (services, traffic patern, consists, etc.). Then I uninstall MSTS, delete the directory, reinstall again, and try unpackaging the activity. If it doesn't work, I've got the files backed up and can try again. (It often doesn't; welcome to Microsoft.)

I figure this is the closest I can come to making my activities non-"system-dependent" (as in on mine).

Doing all of this takes for-friggin'-ever, which is why I don't publish all my activities (often I'm tired of 'em , but the hope is that others can enjoy them easily.

Another important thing - to me - is that if you're going to write activities, write what YOU like, not what people tell you. Me, I like watching trains go by, and I have an extremely short attention span, which is why I write passenger activities of 80 mins or less with lots of AI action. Then at least I know oNE person will enjoy the activity!!

[eurorail]

I haven't gone to such lengths, but I am beginning to think it is necessary.

The one thing not on your list is to have someone else check.

However, as I use multiple copes of MSTS, which I copy. I would have thought that you could just copy a clean installation rather than a complete reinstall.

I hate self-install exe files however as my start menu is now littered with menus to dis-install them.

As I said before you are lucky to have all the UK files on one website - I spent the whole of yesterday trying to sort someone’s problems with one of my French activities because he used the files from a different French, site which had different contents - and he had the patience to try and resolve the issues.

My main conclusion is that Activity writing is like teaching or nursing, hard work and vastly under appreciated.

[Neptune50006]

"one mans meat is another mans poison"

My exact thought when I read this. All a rating system will do is put people off uploading activities IMO.

[kevarc]

At T-S there is a fellow uploading quite a few activities, but his readmes and support are terrible. From posts there he does not answer email or respond to posts concerning his activities. The word soon got out that this was happening and it became strongly recommended that ppl not dl his activities. The forum telegraph is a pretty effective way to get the word out about an activity and\or designer. I know some of my early ones really sucked, but I tried to fix them as soon as errors became evident and answered posts and emails as soon as I could to help fix things. IMNSHO, I think the way the designer supports his activities and responds in a timely manner to problems is just as important as designing the activity. If the designer gets a bad rap by nonsupport of his activities, he might as well quit doing them.

One thing I have done is repackage many of the single cars into packages that follow certain themes. Yes, I have recieved permission from most, but some ppl are no longer around and for those I did a carefull review of the readme to make sure that it was ok to do so. I included the original readme for each car in the upload and I have added a list of the single zip file names that I used. This helps lower the number of DL's, but I also use these same dl's in more than one activity pack. I try to encourage other designers to use them to help limit the number of dl's in their activities.

[eurorail]

I have just been reading the Total quality management standards on http://www.trenomania.it . (in the TCQ section).

You can get a rough translation using the Google translation tool.

I think there is a lot of milage to the idea.

[pitleyfalley]

Im put off activity downloading, due to the fact that ive totally spannered MSTS before, with some dodgy activities. However some guidline to say it works would be a good idea.

Individual ratings wouldnt work IMHO due to peoples tastes. However something saying they were semi decent, would be great.

Chris

[rdawes]

That was my point.

You download and run an activity!

Then you access a ranking page which allows you to select a rank for the different parameters I specified in the original post. The server would average the overall scores and provide the Quality Level.

This way it is controlled by the players who have actually run the activity. Hopefully a good activity would receive more positive scores because users would want to help and encourage the author and poor activities would either receive bad scores or nothing.

Again, I hate spending 2 hours downloading files for a 20 minute activity. Or worse, spending two hours replacing the needed files with ones I have before running the activity which is what I usually do.

Bob

[rdawes]

Part Deux: I'll make my point.

I just downloaded a new London - Southampton activity.

Class 60 Rally Run[1].activity.zip

It needs (7) different Class 60's plus (2) 47's, a 66, a 67, (2) 90's and (2) HST's not counting the wagons.

The readme is full of garble and I think the author's name is there but it's not obvious. I expect that most of the loco's are loose consists but I'm not going to find out.

Bob

[ThinLizzy]

I think the ideas you started with, about what needs to be included with the download are a good thing. but the rating of peoples work would lead to trouble.
If this was overall, i think they would need to be a minimum quality setting, as stuff like Phat uk and Jolly's attempt at a CEP was disgracefull. If they had put some effort in and changed the right things, i think people wouldnt of minded, but to take a picture of a CEP front and paste it onto the front of a CIG and leave the rest is poor.
I mention this as its an example i am familar with, i am sure other people will have other examples, but we cant have a good bad . rating system.

[37413]

If all the activity writers read this thread and pays attention to what is being said by other members and they get there activitys tested by other members before uploading them to the site and providing they supply a good readme file with the proper ID Nos then there should be no need to have a control system. This also applys to me, so far I have uploaded 10 good activitys with very little problems to the members. I even get good private messages saying how good the activity was and how much they have enjoyed playing it.
Regards.
Alan.