Could use some DDS texture help please.
Moderator: Moderators
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Could use some DDS texture help please.
So I'm taking my first tentative steps into moving from the ACE texture format to the DDS texture format, and as always seems to be the case with this format, I can't for the life of me get it to work (it's a damn nightmare trying to use this in commercial applications because of all the variants and sub-formats).
Anyway - I can read/write the textures in Photoshop using the Nvidia plugins, or AMD plugins, and everything looks good in 3ds max but in the game, the textures come out with all manner of different artifacts.
Anything with an alpha channel (for specular for example) comes out with this awful grid on it.
Anything without an alpha channel comes out full of rainbow artifacts.
The top image shows how it looks with ACE textures, or with DDS textures in Max. The bottom image shows how it looks in the game. I'm using XRGB x.8.8.8.8 for the DDS format.
Some help would be appreciated. Thanks in advance.
Anyway - I can read/write the textures in Photoshop using the Nvidia plugins, or AMD plugins, and everything looks good in 3ds max but in the game, the textures come out with all manner of different artifacts.
Anything with an alpha channel (for specular for example) comes out with this awful grid on it.
Anything without an alpha channel comes out full of rainbow artifacts.
The top image shows how it looks with ACE textures, or with DDS textures in Max. The bottom image shows how it looks in the game. I'm using XRGB x.8.8.8.8 for the DDS format.
Some help would be appreciated. Thanks in advance.
Chris L
http://www.christrains.com
http://www.christrains.com
Re: Could use some DDS texture help please.
DDS export is broken, only DDS with alpha works correctly. You can't do anything, use ACE for textures without alpha.
Re: Could use some DDS texture help please.
I notice that completely coincidentally, Derek has posted up about this on his blog.
http://the-art-of-rws.blogspot.co.uk
http://the-art-of-rws.blogspot.co.uk
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
Ok I figured this out empirically yesterday. Anything saved in ARGB format works, those in XRGB format (no alpha) don't. There is a catch though. If you save XRGB format from the Nvidia photoshop plugins, those ones DO work. I can't see any difference in the resulting files, but something in the way the header is written must be different.malatzoc wrote:DDS export is broken, only DDS with alpha works correctly. You can't do anything, use ACE for textures without alpha.
Anyway, for the time being, I've put an alpha channel in all my "non alpha" textures and made it fully white. The RSC non-alpha shaders don't care about this extra channel - they don't look for it and don't use it, so now I can batch-convert textures using the ATI/AMD Compressonator tool and have them all work in the game.
Also a point to note : unlike a lot of games, Railworks doesn't need the normal maps to have any special treatment when saved to DDS.
Chris L
http://www.christrains.com
http://www.christrains.com
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
He popped that up there as a result of this topic - he just emailed me a link - nice one Derek - thanks !stuart666 wrote:I notice that completely coincidentally, Derek has posted up about this on his blog.
http://the-art-of-rws.blogspot.co.uk
Chris L
http://www.christrains.com
http://www.christrains.com
Re: Could use some DDS texture help please.
Just consumes twice as much disk space, and video ram... as textures with alpha are 2x bigger than the ones without alpha, even if it's not used.chrisell wrote: The RSC non-alpha shaders don't care about this extra channel - they don't look for it and don't use it, so now I can batch-convert textures using the ATI/AMD Compressonator tool and have them all work in the game.
chrisell wrote:Also a point to note : unlike a lot of games, Railworks doesn't need the normal maps to have any special treatment when saved to DDS.
Railworks is just like those games, but here DDS is an intermediate format like the ACE, it will be compressed by the TgPcDx exporter to the correct DDS format, which will be put into the TgPcDx container. That's why whole change to DDS doesn't makes any sense.
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
Hmm. That's true. Maybe I'll just stick with ACE all around until I'm forced to use DDS. Fortunately my version of 3ds max is so old I can still happily use ACE....malatzoc wrote:Just consumes twice as much disk space, and video ram... as textures with alpha are 2x bigger than the ones without alpha, even if it's not used.chrisell wrote: The RSC non-alpha shaders don't care about this extra channel - they don't look for it and don't use it, so now I can batch-convert textures using the ATI/AMD Compressonator tool and have them all work in the game.
Chris L
http://www.christrains.com
http://www.christrains.com
- Irishrailguy
- Very Active Forum Member
- Posts: 1338
- Joined: Sun Mar 21, 2010 8:58 pm
- Location: Dublin, Ireland
Re: Could use some DDS texture help please.
This is pretty coincidental, I've only learned about the whole DDS thing in the last week or so. I was told that DDS alpha channels allow for greyscale specular support, this seemed to be a solution to the issue I was having where I could only get a fully shiny spec or a fully invisible one. However on playing around with the .ace files today I think I notice that greyscale specular effects work with the _nm extension to the .ace file. So I'm wondering whether this is the case? (as I'm not too sure whether I am looking at a proper greyscale effect or not)
This whole DDS method is a bit odd to me, I'm told that using DDS as an intermediate format will produce better results, but I don't see any difference between my old .ace texture and the new .DDS one, the texture isn't any sharper nor does it look any better than the old .ace one. Yet I'm unsure as to whether I'm doing it correctly. How do others go from PS to TS2014 with DDS textures? I don't have RWTools so is there another way of getting it to work? I'm using the old RSBinTools but I don't know if that's just as good or not.
Thanks,
Kev
This whole DDS method is a bit odd to me, I'm told that using DDS as an intermediate format will produce better results, but I don't see any difference between my old .ace texture and the new .DDS one, the texture isn't any sharper nor does it look any better than the old .ace one. Yet I'm unsure as to whether I'm doing it correctly. How do others go from PS to TS2014 with DDS textures? I don't have RWTools so is there another way of getting it to work? I'm using the old RSBinTools but I don't know if that's just as good or not.
Thanks,
Kev
Re: Could use some DDS texture help please.
How specularity is handled is depends on the material used on that part of the 3d model. Some have constant specularity, some doesn't have specularity at all, some have alpha masked specularity. _nm is a totally different thing, _nm tells to the exporter that it should not compress the texture. It is only necessary for normal maps, where the compression caused artifacts. I don't know if still needed or not.Irishrailguy wrote:I was told that DDS alpha channels allow for greyscale specular support, this seemed to be a solution to the issue I was having where I could only get a fully shiny spec or a fully invisible one. However on playing around with the .ace files today I think I notice that greyscale specular effects work with the _nm extension to the .ace file. So I'm wondering whether this is the case? (as I'm not too sure whether I am looking at a proper greyscale effect or not)
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
It depends what internal format you save the DDS with and that's part of the problem here. In my years (decades?) of doing 3D CGI models commercially I've learned a lot about textures, and DDS is about the most obscure, effed-up texture format in existence. You'd expect as much given that Microsoft created it, but this is messed up even by their usually-low standards. So put on your geek-goggles and read on.Irishrailguy wrote:This whole DDS method is a bit odd to me, I'm told that using DDS as an intermediate format will produce better results, but I don't see any difference between my old .ace texture and the new .DDS one
DDS supports many different internal formats. The most common one is DXT1. The problem with that format is that it only uses 16 bits per pixel, meaning you're going to get terrible colour compression artifacts on anything that has a gradient in the colour.
To maintain the colour depth of your original texture, you need to save to DDS with the XRGB or ARGB formats, which use 24 or 32 bits per pixel. Those can be RGB, 888, ARGB_8888 or XRGB_8888.
So what does "16bpp" or "32bpp" mean? The easiest way to think of it is to split the number into 3. But 16 doesn't evenly divide by 3 so for a 16bpp image, you've got 5 bits of red, 6 of green and 5 of blue. (5+6+5=16). 5 bits means you have 2^5 possible values for the brightness of each component. So for the red and blue components there are 2^5 shades available, or 32 shades of red. Green has 64 shades available (2^6)
For a 24bpp image, each component can have 8 bits of each of red, green and blue. Meaning 2^8 shades, meaning 256.
For an image with an alpha channel, it adds one extra 8-bit channel for the alpha, making it a 32 bit image. 8 bits red + 8 bits blue + 8 bits green + 8 bits alpha = 32 bits.
More confusing still, most DDS plugins actually use the WRONG naming convention for DXT1, DXT3 and DXT5 formats, claiming they're 4 bits per pixel (implying 1 bit per component). They're not - they have 5 or 6 bits per component, meaning 16 bits per pixel. But conversely, the same plugins use the CORRECT naming convention for ARGB, RGB and XRGB formats - correctly calling them 24 bits per pixel and 32 bits per pixel.
The DXT1 format is further muddied by the inclusion of the "1 bit alpha" possibility. 1 bit means 2^1 shades - ie black or white. But you can't have a 17-bit image. 5-6-5-1 doesn't exist so DXT1 format "robs" a bit from the green channel, meaning you have 5-5-5-1 bits. Or 32 shades of red, 32 shades of green, 32 shades of blue and 2 shades for the alpha channel.
So how does this affect us in Train Simulator? Honestly I' wasn't really sure and that's why I asked the original question. It's not that I'm unfamiliar with DDS - it's that the use of it in the game seems like a really odd choice. It's like RSC threw out a hundred land mines and said "there you go - have fun!". The RSC tools look at the filename and if they see "_nm" on the end of the filename, it tells their texture compiler NOT to compress that particular image, meaning to keep it in 888 format - full colour resolution. Without that, all the game textures seem to get compressed to 565 format by default. I noticed this right on day 1 when I started building the NS trains because the slight colour gradient I was getting in my yellow trains with my dirt map on, was disappearing when in the game.
So if you save a DXT1 image (565) but give it the "_nm" filename, you're not gaining anything because you've already lost all the colour information when you saved the 565 DDS format.
Further confusing the issue for Train Simulator specifically is that some of the commercial tools for writing DDS files work, and some don't. THIS is where I came unglued and why I posted this topic originally. For example if you use the AMD Compressonator tool to batch-convert textures from a command prompt or batch file, the ones with alpha channels (8888) work perfectly in the game, but those without alpha channels (888) don't. But if you save the same textures, in the same format, using the Nvidia photoshop plugins, then both types of texture work in the game. Comparing the DDS files in photoshop shows no difference so I'm guessing something is being set in the DDS header through the AMD tools that RSC's converter doesn't like.
I discovered through empirical testing that if you save ALL the textures in full ARGB (8888) format, and simply make the alpha channel fully white for any texture where you don't really have transparency, then all the tools work all the time and the game ignores the extra alpha channel in the shaders that don't call for it. The downside is that doing this increases the amount of texture memory used in the game unnecessarily.
Compare all that nauseating complication in the previous 5 paragraphs to the .ACE format which was 8888 and just worked, and you have to ask why they bothered to adopt DDS in the first place.
So : the conclusion I came to (about the same time as Derek posted his blog entry) was this: if you use Photoshop, and the Nvidia plugins, save in the two formats Derek shows in his post, and the RSC tools work correctly from that point forwards. Save in any other format and it becomes a ..
This pic is from Derek's blog post showing those two save formats: This pic shows the issue with 16 bit compression. The left side shows the original texture, the right shows it after it's been into DXT1 format. In commercial systems we call this "tint banding" or "mock banding". This is what happens in Train Simulator if you don't use the "_nm" filename convention, or compress to DXT1 565 format.
Chris L
http://www.christrains.com
http://www.christrains.com
Re: Could use some DDS texture help please.
DXT is looks like that as there is no better solution. DXT is a fixed bitrate compression, so i can be read by single access from memory without performance penalty, just like the uncompressed, but as it is compressed, DXT consumes much less memory. There are better compressions, but all have variable bitrate, which have serious performance penalty to read. That's why DXT(aka S3TC) is used in 3d graphics. Nearly all engines use this, including RW/TS just RSC encrypts the DXT(and uncompressed) images to TgPcDx files, to make things more complicated (as there is no reason for it, as it can be easily decrypted since RS was released in 2007).
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
Actually DXT isn't fixed bitrate - it's done by the proprietary compression/decompression algorithms on the fly. The accuracy of those algorithms can be turned up or down depending on the speed vs. compression requirements.malatzoc wrote:DXT is looks like that as there is no better solution. DXT is a fixed bitrate compression, so i can be read by single access from memory without performance penalty, just like the uncompressed, but as it is compressed, DXT consumes much less memory. There are better compressions, but all have variable bitrate, which have serious performance penalty to read. That's why DXT(aka S3TC) is used in 3d graphics. Nearly all engines use this, including RW/TS just RSC encrypts the DXT(and uncompressed) images to TgPcDx files, to make things more complicated (as there is no reason for it, as it can be easily decrypted since RS was released in 2007).
I suppose the real differences between DXT1,3 and 5 are the amount of data they allow in the alpha channel. It's theoretically possible, if you scale back the compression formula, to create a .-looking DXT5 texture.
So it seems, then, like RSC are really only using DDS as an intermediate format where ACE would have continued to function just as well. Given that the ACE or DDS textures go through the same black box texture encoding routine, and the resulting TgPcDx files can be more or less compressed (the "_nm" in the filename). In other words, you might as well throw an alpha channel in all the images and save them as XRGB8888 because Railworks themselves do the "true" DDS algorithms only when they get converted to the encrypted DDS files.
Now I'm really confused. Why can't I just throw them TIF images or TGA images instead?
Chris L
http://www.christrains.com
http://www.christrains.com
- karma99
- Very Active Forum Member
- Posts: 2329
- Joined: Wed Oct 17, 2007 8:21 pm
- Location: Portsmouth, UK
Re: Could use some DDS texture help please.
Do they support mips? The main thing I've noticed with DDS and ace are the do. Is that why DDS is used for games, I couldn't find another more common format with mip support?chrisell wrote:Now I'm really confused. Why can't I just throw them TIF images or TGA images instead?
By the way thanks for writing all this, very interesting reading.
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
You can easily write a photoshop action to pre-mip the images and save them as a TIF (I've done it before).
In fact it could be argued that for lights and signal indicators (illuminated numbers), doing it in photoshop and then exporting the "pre-mipped" image will give you better results because you can hand-manipulate the lower mip levels to be more prominent.
All the tools / plugins / exporters have the ability to save "pre-mipped"
In fact it could be argued that for lights and signal indicators (illuminated numbers), doing it in photoshop and then exporting the "pre-mipped" image will give you better results because you can hand-manipulate the lower mip levels to be more prominent.
All the tools / plugins / exporters have the ability to save "pre-mipped"
Chris L
http://www.christrains.com
http://www.christrains.com
- chrisell
- Well Established Forum Member
- Posts: 533
- Joined: Mon May 06, 2002 12:00 am
- Location: England, Holland and America
- Contact:
Re: Could use some DDS texture help please.
So as best as I can work out, this is the process flow when using ACE textures (top) and DDS textures (bottom).
Looking at it like this you can see where I believe the redundancy is - the blue box in the bottom diagram.
The amount of compression used in the game is determined by the filename (does it have the "_nm" identifier?). What I've called the RSC Compiler spits out a resulting TgPcDx or pair of TgPcDx files depending on whether or not the filename indicates that the texture should be compressed.
When using the older .ACE format, this made sense as that file was what it was - a texture container and nothing more. But now we can determine our own compression level by writing DDS texture files directly, that extra 'compile' stage seems redundant as it takes the DDS file that you or I built, uncompresses it, looks at the filename and then decides if it needs to recompress it or not.
This my best guess anyway.
Looking at it like this you can see where I believe the redundancy is - the blue box in the bottom diagram.
The amount of compression used in the game is determined by the filename (does it have the "_nm" identifier?). What I've called the RSC Compiler spits out a resulting TgPcDx or pair of TgPcDx files depending on whether or not the filename indicates that the texture should be compressed.
When using the older .ACE format, this made sense as that file was what it was - a texture container and nothing more. But now we can determine our own compression level by writing DDS texture files directly, that extra 'compile' stage seems redundant as it takes the DDS file that you or I built, uncompresses it, looks at the filename and then decides if it needs to recompress it or not.
This my best guess anyway.
Chris L
http://www.christrains.com
http://www.christrains.com
