HOW TO: Reduce the number of Palettes

HOW TO: Reduce the number of Palettes; support/discussion/questions

Moderator: thunderchero

User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

first redownload the Lower palette files it has been updated.

Lower palette files

I am no longer getting "Texture Index 328 Not Found" error
brianmcfa wrote:One other thing I noticed that may or may not be related: When I replaced a ship, it sometimes didn't remove the old graphics files. For instance, if I replaced the vanilla Galor with any other ship I noticed that the palettes still went up. When I went into the textures section and scrolled down the file list, the old Galor's .gif files were still there and so was its palette. I would have to remove them manually to get rid of that palette.
it is not related, The only way to say it is adding ships with UE is for beginners. If you add ships manually you have much better control what gif that get replaced. when adding with UE it tries to replace files used by that model but if a new model only uses 1 texture it will replace the ***_a.gif and do nothing with the ***_b.gif and ***_c.gif. if a new model has more than 3 gifs it will use the 3 gifs assigned to the old model but will create a new gif prefix for the other gifs instead of using the unused gifs.

When I add ships I always have a list of used and unused gifs and use every possible gif before I started creating new gif prefixes. But this requires manual entering hex code for phasers and assigning textures to hobs and setting palette for each texture added.

Plus requires a lot of planning and renaming of file prefixes.

thunderchero
User avatar
brianmcfa
Lieutenant-Commander
Lieutenant-Commander
Posts: 134
Joined: Tue Aug 03, 2010 2:00 am
Location: Concrete, WA

Re: HOW TO: Reduce the number of Palettes

Post by brianmcfa »

That seemed to fix the problem. I have also developed a workaround to the stray palette issue. Instead of adding ships, I made all of the ships use the Calamarian Cloud (excluding the monsters). This left me with 9 ships in use. I then deleted all the vanilla ship files instead of replacing them and this took out the b and c .gif files as well. I am now down to 17 palettes and the game is playable but all ships look like a cloud. Now I'll start adding ships back and see what happens.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

That is a work around yes, but just be warned.

once your mod is created into an installer, all the vanilla gifs will still be present in the stbof.res. This is not a problem unless the user loads your mod with UE and uses the "register texture" button. (they should not need to do this if adding ship with UE only when a ship is added manually with new texture prefix.)

thunderchero
User avatar
brianmcfa
Lieutenant-Commander
Lieutenant-Commander
Posts: 134
Joined: Tue Aug 03, 2010 2:00 am
Location: Concrete, WA

Re: HOW TO: Reduce the number of Palettes

Post by brianmcfa »

thunderchero wrote:That is a work around yes, but just be warned.

once your mod is created into an installer, all the vanilla gifs will still be present in the stbof.res. This is not a problem unless the user loads your mod with UE and uses the "register texture" button. (they should not need to do this if adding ship with UE only when a ship is added manually with new texture prefix.)

thunderchero
Okay, I think that's a problem I can still move forward with. I may end up adding back some of the vanilla ships anyway if I need a type and don't have a permission for a redone model.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

brianmcfa wrote:I may end up adding back some of the vanilla ships anyway if I need a type and don't have a permission for a redone model.
who are you waiting on permission from?

thunderchero
User avatar
brianmcfa
Lieutenant-Commander
Lieutenant-Commander
Posts: 134
Joined: Tue Aug 03, 2010 2:00 am
Location: Concrete, WA

Re: HOW TO: Reduce the number of Palettes

Post by brianmcfa »

thunderchero wrote:
brianmcfa wrote:I may end up adding back some of the vanilla ships anyway if I need a type and don't have a permission for a redone model.
who are you waiting on permission from?

thunderchero
You've been the only one I've asked so far! :lol: Basically I'm still doing a lot of experimenting right now. I'm seeing what ships will work together and from what eras. Since you posted in another post that most requests for ship use is granted, I anticipated that there wouldn't be an issue and probably jumped the gun by adding so many in my first attempt. I'm going to be a little more methodical this time.
User avatar
Zweistein000
Ensign
Ensign
Posts: 21
Joined: Mon Apr 19, 2010 2:00 am

Re: HOW TO: Reduce the number of Palettes

Post by Zweistein000 »

After a bit (a lot) of googling I managed to do this with GIMP. You have to follow the exact same order and have to know how to work with the program, but it works.

EDIT: and don't tick trim the palette of the unused colours :P
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

Zweistein000 wrote: Mon Apr 24, 2017 8:42 am After a bit (a lot) of googling I managed to do this with GIMP. You have to follow the exact same order and have to know how to work with the program, but it works.

EDIT: and don't tick trim the palette of the unused colours :P
if you have some time, could you explain procedure you used with GIMP maybe some images and details?

thunderchero
User avatar
Zweistein000
Ensign
Ensign
Posts: 21
Joined: Mon Apr 19, 2010 2:00 am

Re: HOW TO: Reduce the number of Palettes

Post by Zweistein000 »

thunderchero wrote: Mon Apr 24, 2017 6:04 pm
if you have some time, could you explain procedure you used with GIMP maybe some images and details?

thunderchero
Right. Here's the procedure. Assuming we've fund the textures whose palettes we want to merge and have successfully extracted them from the stbof.res. Here I'm using cs__a.gif, cs__b.gif, cf_a.gif and cf_b.gif.

cs__a.gif
Image

cs__b.gif
Image

cf_a.gif
Image

cf_b.gif
Image

We realise that cs__b.gif has the widest spectrum of colours. There for we open cs__b.gif in GIMP first (Figure 1). After this we proceed to open another texture in GIMP. Since cs__a.gif already uses the same palette we open cf_a.gif (Fegure 2).

Figure 1:
Image

Figure 2:
Image

To convert the "cf_a.gif" to the colour palette of "cs__b.gif" we first click on the "image" drop-down menu and under "mode" select RGB (Figure 3). After we've done that we open up the same drop-down menu and under "mode" select "Indexed.." (Figure 4).

Figure 3:
Image

Figure 4:
Image

A menu appears looking like Figure 5. In there we first uncheck "remove unused colurs form colourmap". This is important otherwise the GIMP will remove any unused colours form the palette, meaning that the palette of "cf_a.gif" wil not be identical to the palette of "cs__b.gif". In other words we would have achieved nothing as we would have replaced the old palette with an entirely new (yet unused) one.

Figure 5:
Image

After we have unchecked "remove unused colurs form colourmap" we select "use costum palette" we clock on the button located left of the "web". A drop-down menu of re-existing palettes appears. We scroll up the menu unit we find "Colourmap of image #1" and select that as our palette (Figure 6).

Figure 6:
Image

After this we click convert. If we were successfulm, "cf_a.gif" is no using the colour palette of "cs__b.gif" (Figure 7).

Figure 7:
Image

The final step is to export the image form GIMP. To do this we simply click "file" in the top left corner and then "overwrite cf_a.gif" (Figure 8).

Figure 8:
Image

We can now close the GIMP window for "cf_a.gif". The same process must be repeated for "cf_b.gif", after which they need to be imported back into stbof.res and their colourmap must be changed to the same colourmap as "cs__b.gif" in ultimate editor itself (under "Edit->graphics->textures").

And if at this point you're asking yourself why GIMP, when photoshop is better, the answer is that GIMP is free software.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

Zweistein000 wrote: Thu Apr 27, 2017 11:30 am And if at this point you're asking yourself why GIMP, when photoshop is better, the answer is that GIMP is free software.
Free options is always a good thing to have, I just wish other programs had a free options.

Thanks for your tutorial.

thunderchero
User avatar
Mr Sol
Ensign
Ensign
Posts: 46
Joined: Mon Apr 11, 2016 5:44 pm

Re: HOW TO: Reduce the number of Palettes

Post by Mr Sol »

Hellows.

First of all I want to thank thunderchero for his ideas on textures's palette reduction, which I based myself and improved, and Zweistein000 for his GIMP hint.

I tried GIMP and besides it having the color converting tools I had a bad time with its selection tool, which I found to be counterproductive. Then I tried Krita, another free image editing tool, which dont have the color converting tool we need, but have a better selection tool. Then I found out it messed with the palett. Then I tried irfanview, my prefered image viewer, and it have the color convertion tool but the selection tool dodnt work as I need. I lost my patience and downloaded photoshop cs3.

Dont anderstand me wrong, Im not advocating piracy, but cs3 is almost 9 years old and Adobe dont make money from it anymore because its obsolete, so its not piracy to use oldware like this one. I strongly suggest it.

Anyway, lets get to the real thing:

1. FIrst of all I extract all the files from the stbof.res and give them a good look. I noticed that there are plenty of pictures with many color similarities then I decide which group of colors I will act upon. Since my mmod is about ships covered by the spacedock rules for ship building and space combat from the startrek rpg from last unicorn games, another dead rpg just to mention, I definitely wont mess with the ferengi ships, since there arent any oficial templates for them nor many info online around, unless you want to use noncanon stuff, and thats not the initial idea, so since ferengis arent being touched much, they are the perfect pick for this experiment. To find the startrek rpg suplement spacedock go to www.trekrpg.net.

So I select all the ferengi ships textures and put them in a different folder, along with they torp image.
pallete guide 000.jpg
pallete guide 000.jpg (81.45 KiB) Viewed 17906 times
2. Open a new document and make it big enough to display all the textures we will be throwing at it as I did in the pic. Let it be RGB and definitely 16bit color depth, so it can receive the 8bit color depth textures and adapt them, preserving the original color. If you want to be really perfectionist pick 32bit color depth, but its not really necessary since you wont notice the difference. 300DPI or 75DPI is irrelevant since we wont print anything.
pallete guide 0000.jpg
pallete guide 0000.jpg (46.37 KiB) Viewed 17906 times
3. Now you will be copying and pasting all textures into a big picture map, as I did. Notice that you need some space bettwin them or you will repent later. Select the pict in the order it shows in explorer, drag and drop on cs3. It will open the texture in a separate file. CTRL+A to select, CTRL+C to copy, CTRL+W to close the separate file, CTRL+V to paste it in the map we are making and align it with the move tool then hit the arrow keys on the keyboard to move it around so you have a space bettwin them. HENCE REPEAT for each texture till you have them all in the map.
pallete guide 001.jpg
pallete guide 001.jpg (64.44 KiB) Viewed 17906 times
4. Be ***SMART***! Now is the point to SAVE your work. Shit happens, you know, but you never know when they will happen, so save your work now.

5. Now comes the real interesting bit. Instead of having one palette for each ship as was suggested I can be more resourceful and make just one palette for ALL of them IF they share a lo t of colors. Isnt nice that all the ferengi texture is exactly like that? Hit the LAYER tab and down the list pick FLATTEN IMAGE. It will squish all the layers in just one. Thats exactly what we need. Now we go to the IMAGE tab, MODE then 8bits color depth, then again IMAGE, MODE then INDEXED COLOR. It will reduce the image to 8bit color depth, the right size for the palette we want, then will create the palette we want.
pallete guide 002.jpg
pallete guide 002.jpg (69.31 KiB) Viewed 17906 times
6. A box will show up. Pick PREVIOUS since its the color group you want to use: the original colors from the map you want to convert. Dither NONE. Hit OK. You're done with the color converting part.
pallete guide 003.jpg
pallete guide 003.jpg (63.33 KiB) Viewed 17906 times
7. You want to see the palette? You can do that and SAVE it, just in case! Hit IMAGE tab, MODE then COLOR TABLE.
pallete guide 004.jpg
pallete guide 004.jpg (83.96 KiB) Viewed 17906 times
8. Now comes another boring part. You will have to select each texture, one by one, and copy-paste it to a new file and save it. Remeber I said you would repent for not placing a space bettwin the textures? That's to facilitate you in selecting them. Use the SQUARE selection tool for that. Remember to UNCHECK the antialias option at the top of the CS3 window. Select the texture in the same order you placed it when you begun making the map. After you select, if you picked a part of the white background, use the MAGIC WAND selection tool. Uncheck antialias again. Select the SUBTRACT FROM SELECTION small icon, at the left side of the antialias option box. Click inside the white area to remove it from the selection and you will have ONLY the texture you want.

If you have any difficulties with selecting go to the EDIT tab, PREFERENCES, CURSORS and pick PRECISE for both and hit ok.
pallete guide 005.jpg
pallete guide 005.jpg (103.59 KiB) Viewed 17906 times
9. So the sequence is select the texture, CTRL+C to copy, CTRL+N for a new file, pick RGB color 8bit color depth so you dont mess with everything you did converting it back to 16bit color depth and throwing the palette away, hit OK, CTRL+V to paste it, then see step 10. Hence repeat for every texture till you're finished.
pallete guide 006.jpg
pallete guide 006.jpg (101.42 KiB) Viewed 17906 times
10. Convert the texture to INDEXED color again. When you create the new document it dont use the palette.
pallete guide 007.jpg
pallete guide 007.jpg (55.05 KiB) Viewed 17906 times
11. Now this is really important! Once you have one texture ready SAVE it over the original one in the folder you extracted them so you preserve its original name. This requires EXTREME attention so you dont mess with everything.

12. Now open UE and load stbof.res again if you closed it. Go to EDIT, OTHERS, FILES. Hit the ADD button, browse to the folder with the new textures and select them all and hit SAVE. They will all be added over the original ones.

13. Go to EDIT, GRAPHICS, TEXTURES. Browse in the small box to each of the textures you added. You will see something bizarre like the picture below. Its the right texture with the wrong palette.
pallete guide 008.jpg
pallete guide 008.jpg (66.69 KiB) Viewed 17906 times
14. Hit USE OWN PALETTE and it will resort to the palette we created when we worked the texture. Its part of the texture now and UE will resort to it from now one, fixing the colors. Hence repeat for all textures we added.

15. When we begun this with the original vanilla file it readed 120 palettes. Now that we did everything we click TRIM PALETTE LIST and it will fall from 120 to 109 palettes, opening space for adding more ships. Now check the final result and tell me if it isnt nice!
pallete guide 011.jpg
pallete guide 011.jpg (36.96 KiB) Viewed 17906 times
Hope you enjoy this little guide. Its almost the same thing thunderchero tells in the beginning but add up with making a bigger map. Hope it helps you.

Hugs!
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Increasing the trek.exe Palette Limit

Post by Spocks-cuddly-tribble »

Dr_Breen wrote: Sun Mar 22, 2009 10:55 amthe maximum numbers of palettes that trek.exe can use is 127. here's a small example how your ships look like when you use more than the maximum number of Palettes allowed:

Image

In vanilla botf 122 palettes are already in use, so after adding 5 custom ships to the game you will run into this problem.

The trek.exe limit can be increased by increasing below code pointers by +2*(number of extra palettes), but requires also preparing a new dynamic area in trek.exe (initialized with zero bytes).

Code: Select all

AUTO:00539C05                 mov     esi, 59FF30 offset dynamic data field length 104h 260 -> new length +2* (number of extra palettes)
AUTO:00539C55                 mov     esi, 59FE30 offset dynamic data field length 100h 256 -> new length +2* (number of extra palettes)

sub_538450
AUTO:00538454                 sub     esp, 108h
AUTO:00538497                 mov     [esp+100h], ebp
AUTO:005384A0                 mov     [esp+104h], ebp
AUTO:005384AE                 lea     eax, [esp+100h]
AUTO:005384BE                 mov     ecx, [esp+100h]
AUTO:005384CE                 mov     esi, [esp+104h]
AUTO:005384DC                 add     esp, 108h
AUTO:005384EE                 mov     [esp+100h], ebp
AUTO:00538503                 mov     [esp+104h], edi
AUTO:0053850C                 lea     eax, [esp+100h]
AUTO:00538573                 lea     eax, [esp+100h]
AUTO:00538596                 lea     eax, [esp+100h]
AUTO:005385BB                 lea     eax, [esp+100h]
AUTO:005385C9                 mov     ecx, [esp+100h]
AUTO:005385E6                 mov     [esp+100h], ebx
AUTO:005385ED                 mov     esi, [esp+104h]
AUTO:00538609                 mov     [esp+104h], edi
AUTO:00538612                 add     esp, 108h
AUTO:0053861D                 lea     eax, [esp+100h]
AUTO:0053863E                 lea     eax, [esp+100h]
AUTO:00538652                 lea     eax, [esp+100h]
AUTO:005386BC                 lea     eax, [esp+100h]
AUTO:005386C8                 lea     eax, [esp+100h]
AUTO:0053870B                 lea     eax, [esp+100h]
AUTO:00538754                 lea     eax, [esp+100h]
AUTO:00538762                 mov     esi, [esp+100h]
AUTO:0053877C                 mov     [esp+100h], edi
AUTO:00538783                 mov     ebp, [esp+104h]
AUTO:0053879A                 mov     dword ptr [esp+104h], 0FFFFFFFFh
AUTO:005387A7                 add     esp, 108h
sub_538CA0
AUTO:00538CA4                 sub     esp, 110h
AUTO:00538CAA                 mov     [esp+10Ch], eax
AUTO:00538CB8                 mov     eax, 130h   -> ???
AUTO:00538CCD                 mov     eax, [esp+110h]
AUTO:00538CD4                 lea     edx, [esp+104h]
AUTO:00538D4E                 lea     eax, [esp+100h]
AUTO:00538D78                 lea     eax, [esp+100h]
AUTO:00538D96                 lea     eax, [esp+100h]
AUTO:00538DB4                 lea     eax, [esp+100h]
AUTO:00538DDF                 mov     eax, [esp+10Ch]
AUTO:00538E07                 add     esp, 110h
AUTO:00538E3F                 add     esp, 110h

But we can try to use the fixed asm-address of the loaded combat.bin file (unused feature): :wink:

1. Overwrite combat.bin (stbof.res) with zero bytes (62500)

2. Set AUTO:00539C05 mov esi, 59FF30 -> 5B58A8 (adr loaded combat.bin now initialized with zero bytes)

3. Ignore AUTO:00539C55 mov esi, 59FE30 (redirecting 59FF30 redoubles code space).

4. Increase all listed code pointer by 100h.

5. Test up to 255 palettes. :smile:
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

Spocks-cuddly-tribble wrote: Sun Jan 09, 2022 4:49 pm My untested data here should increase max Palette Limit and limit for simultaneous palette/texture usage: viewtopic.php?p=54697#p54697
ironically I was just testing the increase max Palette Limit today. using combat.bin

and got crash on entry of combat when palette was above 127 (tested at 142?) did not crash when below at 119

from debug I think it is an issue in mpr565.dll ?

edit; from windows event log;
"Faulting module path: C:\botf\Vanilla\mpr565.dll"

note

here is my offset changes
<< 0x00139006 a8 58 5b 00
<< 0x00139056 a8 5a 5b 00

all other changes was
0* 01 00 00 -> 0* 02 00 00
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Re: HOW TO: Reduce the number of Palettes

Post by Spocks-cuddly-tribble »

thunderchero wrote: Sun Jan 09, 2022 5:06 pm"Faulting module path: C:\botf\Vanilla\mpr565.dll"
That's a pity. There was a question mark value I'm not sure about.

If the problem is MPR+ and/or some stack preparation codes in trek.exe for MPR+ then I'm not the best person to assist with this.

But you could try code changes with max 127 palettes and test if it helps with 'stack->bottom != NULL' crash in case it's caused by simultaneous textures used.

EDIT:
thunderchero wrote: Sun Jan 09, 2022 5:06 pm<< 0x00139056 a8 5a 5b 00
This looks wrong.

"3. Ignore AUTO:00539C55 mov esi, 59FE30 (redirecting 59FF30 redoubles code space)." -> 'Ignore' means don't change default code! #2 redoubled this!

59FF30 is now free usable for dynamic data 59FE30
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: HOW TO: Reduce the number of Palettes

Post by thunderchero »

Spocks-cuddly-tribble wrote: Mon Jan 10, 2022 10:54 am EDIT:
thunderchero wrote: Sun Jan 09, 2022 5:06 pm<< 0x00139056 a8 5a 5b 00
This looks wrong.

"3. Ignore AUTO:00539C55 mov esi, 59FE30 (redirecting 59FF30 redoubles code space)." -> 'Ignore' means don't change default code! #2 redoubled this!

59FF30 is now free usable for dynamic data 59FE30
I figured that out later, but the way I set it up should have still worked

since combat.bin is large enough I setup data;

AUTO:00539C55 mov esi, 59FE30 (changed to start of combat.bin + 00 02 00 00) a8 5a 5b 00

AUTO:00539C05 mov esi, 59FF30 (changed to start of combat.bin) a8 58 5b 00

that way both had double space in combat.bin area.

and I did reset your ? code back to vanilla and tested
Post Reply

Return to “HOW TO: Reduce the number of Palettes”