Download Counts Now Fixed

[Darpor]

Taken from the front page. Hope the mods dont mind this topic but it has been discussed before about low download counts on here.

http://members.uktrainsim.com/index2.ph ... eport=1331

There is obviously something quite wrong with the way that the system currently increments the download counter as the figures do seem quite low considering the amount of activity on the site.

As well as increasing the counter for a file when it is downloaded, the download server also records a new entry in the database detailing who downloaded it, how big it was at the time and when (this is to support the "my downloads" screen that you'll find in the top left on the menu bar down the left hand side). Comparing the total downloads in this database with the counter gives in some cases a huge error.

As a temporary fix I have written a script to create a new secondary download counter based on that database. I will re-run the script on a nightly basis until I can put something more concrete in place. The old counters are still there (hidden away) and are still being updated, so no data has been lost in this change and if later investigation requires it I can revert back.

The end result is that almost all files on the site now have a bigger download count, I think one or two actually went down slightly (older files), and a few went up by tens of thousands - in a quick flick through the log it produced, I saw one that was 17,000 higher). In the main though, most went up quite a bit.

I will keep looking at this and put a more permanent solution in place.

In the mean time - do not waste your time refreshing screens to see download counters going up on a minutely basis - they won't, they will now only go up whenever this script re-runs (which will likely be on a nightly basis).

Matt.

Just poring over my downloads, it does make much more interesting reading to see how many people are downloading.

[davidhossack]

WOWZA

Just looking at my numbers. They are a little higher now.

I thought my Mk2s were unpopular and I reckoned they weren't worth the effort but now I see now that they add up to roughly 2500 downloads. Last I checked they didn't make 1000. And the NR HST-wow. Overall there is hardly anything under 500.

Thanks folks.

Big upload of stuff coming soon.

[bigvern]

Yes the figures are definitely a little more encouraging now, the previously quite pathetic tally for Tanigumi looks almost respectable - enough to make me at least consider doing another route later in the year.

[Acorncomputer]

The figures now look TOO good to be true

[NeutronIC]

Yes, that is the other risk and what I will do when I get the chance is to review the download server and see how likely wrong the download figures are in the new case. It seems to me far less likely...

Let me explain what I *think* has gone wrong with the older figures:

Basically it's a good old fashioned "race condition" in programming terms, must admit I wasn't expecting it as I thought i'd protected against it - but oh well, you win some....

What happens (I think, not reviewed code yet) is that it does the download and then when it records it, it updates the existing value - if multiple people complete at the same time the value goes a bit wonky since they all read the existing, add one, write back. What clued me in to it being a race condition is that people report the figure going DOWN as well as up, and since there is definitely no code in there to make the figure go down, it must be something odd along those lines.

In the download record table I simply write a new row for every download that is started, so there's no possibility of that kind of race condition.

Of course on the flip side, that table includes failed and repeated downloads too, so the figures are including all of those as well, however it seems far less wrong and more realistic.

I will try and get the code reviewed in the download server soon and put a more permanent solution in place but for now at least the figures look more realistic.

Matt.

[karma99]

Wow, up to 2377 for my Wagon Pack 1.. and less posts of thanks here than DT got for their new payware Wagon Pack.
Remind me again why I used to do this for free?

[davidhossack]

Thanks matt for your concise explanation.

When you say download started is that one entering the download queue, or is it when one reaches the top of the download queue?

If it is entering the queue there could be quite a miscount as, in my experience, I click download, see that I am 30th in the queue and then close the window. If others do this too there could be quite a miscount.

[growler37]

Hi
My West Somerset Railway is up to 1560 This shows just how popular freeware is,i think users are much more selective about payware these days,which can only be good,items will have to be that little bit more special to sell well.
Regards
Kevin

[RSderek]

Interesting numbers indeed...

Great stuff and well done to all that upload and share your content with others.
To those that use that content, it would not hurt to say thanks a little more often.

regards

Derek

[davveb]

Hee, hee, 1 or 10000, makes no odds to me.
Now if I got a dollar for every download, I might think differently
Of course, then you'd all be "customers" and I'd have to start being nice!

Dave B

[NeutronIC]

It's when the download actually starts, nothing is recorded relating to queue usage, so it should only be when bytes actually flow from the file to the user - if they abort it 2 bytes in, it's still recorded.

Just to echo what Derek says - never hurts to say thanks, there are some outstanding bits available and it's really generous of those that give it away, but so very very easy just to say thanks.

Matt.

[markpullinger]

Agreed Guys - I probably don't say thank you enough so big thanks guys!

[kevmt]

Wow, up to 2377 for my Wagon Pack 1.. and less posts of thanks here than DT got for their new payware Wagon Pack.
Remind me again why I used to do this for free?

Yes Pete, I had very similar thoughts when I totalled up my combined RS/RW locomotive downloads and it came to over 60,000!!

Kevin

[briyeo1950]

The problem with lack of feedback or people not just saying thanks is that there is no chance to do this at the point of download. Most sites I have used and uploaded reskins to usually have a star rating system and a comment box. You still wont get thanks from everyone but maybe it would be better.

[NeutronIC]

Another thing to keep in mind folks, is that people who use download accelerators like flashget, dap or getright, will perhaps download the file in say 5 different chunks - that will count as 5 different downloads.

I can pull out a stat that shows the number of unique users that have downloaded the file, which is probably a much more accurate and more useful stat - however it's a vastly more time consuming query to run on 14,000 files across hundreds of millions of download records (even with indexes it was taking about 1-2 minutes PER file...) I will work on making that stat available too - and make it a separate stat so you can see both. That way you'll see clearly how many individuals have downloaded the file, rather than just how many times the file has been downloaded.

I'm happy to add a feedback system on the files but previous discussions on the topic have agreed against the idea - but hey, let's re-open the discussion and see if times have changed. What do you think?

Matt.