Posted: Fri May 04, 2018 6:18 pm Post subject:
Another "Map limit"?
I didn't wanted to bump this topic since it's been long time and this bug may require more research to clarify exact conditions, when it occurs.
I have been finishing my map (about 85-90% done) and I have noticed that huge chunk of terrain at bottom-left corner has been replaced with clear ground with height 0. My map size is 190x225. After that I have tried erasing some terrain I have recently added, and the replaced chunk become smaller.
So I suppose that there is some engine limit how much of clear tiles can be replaced, and if exceeded, game somehow cut extra tiles and generates clear ground in that area.
Bellow are in-game screenshots and FinalSun screenshot of map snapshot that has replaced terrain.
lol matze..
It's not a game limit that much i can tell you, rather just community members not bothering to investigate things fully
if FinalOmega is a thing still and AlexB still does work on that maybe he can find some solution, it could be simple but i highly doubt given how much of a mess the editor is being built on XCC code it will be. _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
Q45 also ran into this when creating the final mission of TI's Nod campaign. He couldn't find a solution, so he cut details from the rest of the map.
Yes, replacing tiles behind map border makes some room to use more details. Hope it will be enough to finish details on map.
tomsons26lv wrote:
lol matze.
It's not a game limit that much i can tell you, rather just community members not bothering to investigate things fully
if FinalOmega is a thing still and AlexB still does work on that maybe he can find some solution, it could be simple but i highly doubt given how much of a mess the editor is being built on XCC code it will be.
I am not sure how which technique uses Map Renderer to make preview of the map, but it doesn't show any glitch, same as FinalSun. So I suppose it is probably game.exe file.
MadHQ wrote:
This is the same limit I see in Ra2/YR. Your map width + height sum can not go above 400.
You are at 415, I am not surprised you are having problems.
I know that FS restricts using anything above 200 for both X,Y but didn't found anywhere that should cause any issues up to 416 for X+Y (only small glitch in map preview in FS). Also it was mentioned that engine should be capable to support X+Y up to 512. QUICK_EDIT
um, have you opened the saved map?
It sure makes no sense for the game to do this, it looks a lot more like a FA limitation
I'll test some things _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
um, have you opened the saved map?
It sure makes no sense for the game to do this, it looks a lot more like a FA limitation
Yes, same map when rendered with Map Renderer, looks fine. Once launched, glitch is there. Same if saved/loaded, or made snapshot.map using in-game hotkey. QUICK_EDIT
Right, not sure whats going on there i made a 200x200 map, filled it with water tiles, all is fine.
Also im not sure how you managed to get 190x225 if the limit in FA is 200x200 and it won't allow you to enter any higher values
Only thing i did see that stopped working properly is FA can't write the preview anymore _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
Right, not sure whats going on there i made a 200x200 map, filled it with water tiles, all is fine.
Also im not sure how you managed to get 190x225 if the limit in FA is 200x200 and it won't allow you to enter any higher values
Only thing i did see that stopped working properly is FA can't write the preview anymore
I started with 200x200 then opened map in text editor and changed
Size= into Size=0,0,190,225
When opened in FS again, it will add missing cells. QUICK_EDIT
Also it was mentioned that engine should be capable to support X+Y up to 512.
Where did you see this?
Anything bigger then sum 400 in Ra2/YR will cause the game to crash. So, I am not sure its a FA2 problem.
I am not sure what differences TS/FS has... I cannot expect much. _________________ MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.) QUICK_EDIT
Tired that, and checked the difference between a properly made 32x32 map and a 16x16 resized one, LocalSize is fucked then
Trippy things happen when its wrong
Tuc, could you open the map in a text editor and tell us the number of lines in the [IsoMapPack5] section?
I assume this section has some kind of limit.
When checking maps like Tiber, Archipelago and Africatorial, which are all well beyond 200x200, they all still managed to stay just below 10000 lines
I assume your map is beyond that and the game simply skips reading the last lines beyond 9999. Thus any information about these cells is set back to the default blank/clear tile.
@Tomson:
You can safely change the last parameter of Size=, as this makes the map become bigger to the south with blank/clear tiles. Only when changing the first 3 parameter, the map looses several tile information (though still the game fills them with blank/clear tiles).
For several of my big maps, i used the maximum that FinalSun can handle before it crashes,
detailed the map till the last tile in the south and also set the ingame playing field (blue border) to this last tile in the south.
Then when finished added +10 to the last parameter to make the map become 10 cells bigger to the south and give it this way a 10 cell wide border outside the playing field in the south to prevent any bugs with aircraft flying down there.
So far this worked perfectly fine. _________________ SHP Artist of Twisted Insurrection: Nod buildings
I don't believe there is a limit to the size of the IsoMapPack5 section in terms of lines. The game has hard coded limits of 40000 cells total however, which is 200x200.
tomsons26lv tells me there is also a check that no individual axis exceeds 512 so you could have 512x78 maps for example but not 513x77 maps even though both come under the total cell limit.
Going back to the original post, 190x225 is 42750 cells which exceeds the limit. Last edited by Blade on Fri May 04, 2018 10:30 pm; edited 1 time in total QUICK_EDIT
Tuc, could you open the map in a text editor and tell us the number of lines in the [IsoMapPack5] section?
I assume this section has some kind of limit.
When checking maps like Tiber, Archipelago and Africatorial, which are all well beyond 200x200, they all still managed to stay just below 10000 lines
It has 9759 lines for the version of map that has this glitch, and the version I have managed to get rid of it has 9742 (not the exact moment that the glitch was removed) QUICK_EDIT
LOL, you are right you can go above 400. It was so long ago, but I made a map the was 399x100 for YR.
Humm... I do not remember why, but there was a reason I stuck with not going over 200x200... But I do not remember why... _________________ MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.) QUICK_EDIT
I don't believe there is a limit to the size of the IsoMapPack5 section in terms of lines. The game has hard coded limits of 40000 cells total however, which is 200x200.
tomsons26lv tells me there is also a check that no individual axis exceeds 512 so you could have 512x78 maps for example but not 513x77 maps even though both come under the total cell limit.
Going back to the original post, 190x225 is 42750 cells which exceeds the limit.
Yes that is what I forgot, 40000 tiles was the limit.
Archipelago: 208x220 = 45760 tiles
Tiber: 208x214 = 44512 tiles
both maps work fine
If you mean however 40000 non-clear tiles, then this could be indeed the case, as my big DTA maps still use many clear tiles, and only about 50 to max 80% are non-clear terrain (cliffs, grass, water etc).
\Edit
just created a 200x210 DTA map and filled it with 1x1 platform tiles (non clear)
All the tiles were shown perfectly fine ingame. None were skipped or replaced with clear tiles. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Though crossing limit of W + H <= 512 or W * H < ~43500 causes glitches or crashes in FA2/FinalSun.
In-game, crossing W + H <= 512 either hangs or crashes the game, it can certainly go more than 400. Have tried 282x154 (43428), 352x124 (43648) plain maps in game and those work fine so 40000 cells is not the limit.
This one looks like another map limit for the game but it is not that how many clear tile is replaced. Just tried a 352x124 map fully filled with water tiles and had no problem.
@Tuc0: Is there any specific details/tiles that when removed reduces the effect? QUICK_EDIT
@Tuc0: Is there any specific details/tiles that when removed reduces the effect?
I also tried to replicate that with water map, but without success. So I will share some screens and previews, the map should be released soon anyways, unless that bug comes back. When the map will be released, I will surely drop all versions here.
Here are the previews:
(Ignore the right bottom area, just temporary workbench for buildings and tiles)
1: No Bug - started to build terrain in the center part of map
2: Bug appears - as the cliffs and ramps are build
3: Bug vanished after replacing water tiles behind map border.
4: Map screenshot, visible glitch in left-bottom minimap.
wow Tuc, that is one heck of a beautiful and big map.
ontopic: does that mean FinalSun is unable to write the 9758th line correct, while it manages to write all the others fine? This sounds very strange. _________________ SHP Artist of Twisted Insurrection: Nod buildings
This problem is indeed related to IsoMapPack5. Observe the bug in a past version of the N16 mission in TI:
Q45 averted this bug by simply replacing terrain details with clear tiles around the map.
Offtopic: @Tuc0: Are you using ice in that map? If so, how do you avoid the ice growth bug? Copy-pasting ice from WW maps?
I'm asking because I'm interested to know if anyone's yet figured out the exact chars/lines that FS writes incorrectly into IsoMapPack5, that cause the ice growth logic to break. _________________ QUICK_EDIT
wow Tuc, that is one heck of a beautiful and big map.
Thanks, hope I will be able to finish it soon
Lin Kuei Ominae wrote:
ontopic: does that mean FinalSun is unable to write the 9758th line correct, while it manages to write all the others fine? This sounds very strange.
But it looks like game engine issue, Map Renderer draws it correctly, like shown in FS.
Crimsonum wrote:
Q45 averted this bug by simply replacing terrain details with clear tiles around the map.
Yes, worked for me too, for now.
Crimsonum wrote:
Offtopic: @Tuc0: Are you using ice in that map? If so, how do you avoid the ice growth bug? Copy-pasting ice from WW maps?
I'm asking because I'm interested to know if anyone's yet figured out the exact chars/lines that FS writes incorrectly into IsoMapPack5, that cause the ice growth logic to break.
No, got same issue about ice growth. Plus it isn't quite AI friendly and leaves ugly tiles when melted. Also marble madness doesn't have those ice shores.
I am using just rough ground for passages. I also thought about making passage from small water streams passing through, but cannot be done without game mod.
ontopic: does that mean FinalSun is unable to write the 9758th line correct, while it manages to write all the others fine? This sounds very strange.
The problem is not in FinalSun but the game. When IsoMapPack5 size is greater than 9757 lines and 42 characters in 9758th line, the glitch in the bottom-left corner starts appearing. The glitch size increases with adding more lines. The game.exe would need to be hacked and fixed.
Removing the details by using clear tiles will reduce the IsoMapPack5 lines, that way the glitch can be avoided. When nearing such large IsoMapPack5, better to use smaller tiles for details rather than using like the large rough ground tile which spans over several cells. QUICK_EDIT
When nearing such large IsoMapPack5, better to use smaller tiles for details rather than using like the large rough ground tile which spans over several cells.
So for example 1 tile over 2x2 area takes longer line to write than 4 1x1 tiles? QUICK_EDIT
I haven't checked the format to do the math but when I tried to fill up the lines faster to test this problem in a map, I repeatedly used the largest celled tiles. The rate at which you can fill up the IsoMapPack5 lines is higher with a few of the 1st rough ground tile which has about 33 cells in a single tile when compared to say cliff tile or paved road tiles which has fewer cells to cover. QUICK_EDIT
I haven't checked the format to do the math but when I tried to fill up the lines faster to test this problem in a map, I repeatedly used the largest celled tiles. The rate at which you can fill up the IsoMapPack5 lines is higher with a few of the 1st rough ground tile which has about 33 cells in a single tile when compared to say cliff tile or paved road tiles which has fewer cells to cover.
Just tried small map: made 20x20 area filed with 2x2 water tiles. Then replaced that same area with only smaller water 1x1 tiles and surprisingly IsoMapPack5 was 9 lines shorter. Considering this, can I shorten the lines even with water tiles by implementing more 1x1 tiles into water pattern?
Something like this, it could also seem more natural instead of obvious square grid pattern.
EDIT: even overlapping same tile over itself shortens IsoMapPack5 lines.
OK, found another thing. If adjacent tiles are in the same repeating pattern, it takes shorter line to write, but it also depends in which direction are adjacent.
For example in 1st picture from NW to SE takes shorter line to write as the same tiles rotated by 90deg repeating in NE to SW direction in picture 2. The 3rd last picture do not have repeating pattern in one of these directions.
Code:
Filled area with clear ground ends with:
191=BQgkAEUA//8AAAAAABEAAA==
That finding is in expected lines. RLE compression is better when consecutive values are same. The coding would be like for every Y go from 1st X to last and then take next Y line, that would account for the direction. And Base64 encoding is used after RLE compression so that the file can be transmitted over the net without problems. QUICK_EDIT
Its not a RLE, its LZO, plain old http://www.oberhumer.com/opensource/lzo/
We(RA++ team) think even miniLZO can be used to read/write the LZO saved C&C data, not confirmed yet, yet to replace current code with minilzo
I guess a thought would be decode base64 and unpack the resulting binary, and check the actual data
It could be as follows
4 bytes coord
4 bytes tiletype
1 byte height
1 byte level
1 byte unknown
can't 100% confirm that as the TS code is a bit optimized, there seem to be some other entries written but im not sure if its written for every cell or after the cells are written, once the binary is decompressed you should see the pattern and see if it is that or not _________________ Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1
Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite QUICK_EDIT
Had just glanced at source code of zzattack's MapRenderer and Starkku's MapTool. Both use the same set of files for encoding. Both work fine with TS maps. Format5 -> miniLZO. That miniLZO would need some time to understand. The data structure used is like
short x
short y
int tileindex
byte subindex
byte level
byte udata // Set to 1 for ice growth, 0 to disable ice growth in TS snow maps
In any case, Tuc0's finding says that having same consecutive tiles gives better compression. Last edited by E1 Elite on Mon May 14, 2018 5:43 am; edited 1 time in total QUICK_EDIT
When IsoMapPack5 size is greater than 9757 lines and 42 characters in 9758th line, the glitch in the bottom-left corner starts appearing. The glitch size increases with adding more lines. The game.exe would need to be hacked and fixed.
You got that glitch exactly after 9757 line? Mine started in the middle of 9753.
I was curious what could happen if IsoMapPack5 exceeds during gameplay (deforming terrain, building pavement) and how it could affect saving and loading games.
I have made map version with longest lines, without glitch and it ended somewhere in the middle of 9753th line, with this:
Code:
9753=AJ7kSgjiAJ4BVQEAAAABABEAAA==
without glitch
9753=AOEAnuRKCOIAngFVAQAAAAEAEQAA
glitch appeared
So I started game, built some pavements and made saves and map snapshots during gameplay. But game loaded completely fine, also the map snapshot didn't have any glitch. It had over 9791 lines after pavements were built. QUICK_EDIT
Added the pavement and green to the map without glitch with lines 9757 in game and it was good within the game. Took snapshot and line went upto 9783, but playing that snapshot map showed the glitch. So the game is not able to handle it. QUICK_EDIT
Added the pavement and green to the map without glitch with lines 9757 in game and it was good within the game. Took snapshot and line went upto 9783, but playing that snapshot map showed the glitch. So the game is not able to handle it.
I was trying to point out that saving and loading does not suffer from this bug, it occurs only when .map file is used.
Anyway, I was able to finish all terrain, but lot of compromising was necessary. I needed to clear map border, rework 95% of water tiles - got around 70 lines from it. I also had to decrease map height by 1 which gave me another 30 lines. It felt like building "Jenga Tower", had to deconstruct and compromise already built foundation to reach desired height.
Now I can finally get to rules editing, triggers and playtesting. Map should be released in few days.
Thanks for clarifying about those IsoMapPack5 files, still wondering how could it differ when it occurs. I got it 3 lines sooner than 9757, probably could vary on map resolution, or width & height ratio. QUICK_EDIT
As the IsoMapPack5 are compressed, it could be the unpacked size of it. The map I tested, I cleared several times and reworked to get the glitch and it was at the same place. The unpacked good size was 958892 bytes or 87172 tiles (1 tile = 11 bytes), above that it was glitching.
If all 8 players make pavement then a buffer of about 200 lines would be safe. QUICK_EDIT
The uncompressed size remains constant as the number of cells in a map is same for a width and height. I was wrong in assuming that it could be different. The size difference in the IsoMapPack5 lines is because of the compression, this is where the game is getting limited.
Number of cells in a map = tiles in IsoMapPack5:
cells = ((width * 2) - 1) * height
cells = (width * height) + ((width -1) * height)
So a map with W = 20 and H = 18, total cells/tiles in that map is 702.
Edit:
Good news is that I was able to sort the tiles based on tileindex and then compressed it to make IsoMapPack5, which resulted in reduction of lines from 9783 to 8445 and works well in game. As the lines reduced, the glitch was gone. But editing again in FinalSun reverts it, so it has to be recompressed again. QUICK_EDIT
@Tuc0: Have modified Starkku's MapTool to better compress IsoMapPack5 based on tile sorting. Can take backup and copy the EXE and profile file to the MapTool folder. In the Profile file, can change the SortBy= and compare result based on map by map. QUICK_EDIT
@Tuc0: Have modified Starkku's MapTool to better compress IsoMapPack5 based on tile sorting. Can take backup and copy the EXE and profile file to the MapTool folder. In the Profile file, can change the SortBy= and compare result based on map by map.
Wow, nice job. It shortened lines quite nicely. SortBy=X did the best job, needed only about 92% of lines compared to original
Code:
Compressions for my map:
FinalSun 9724 lines
LEVEL 9290 lines
TILEINDEX 9085 lines
X 8901 lines
Interesting, so game can handle any of these compressions not just the one used in FS or game.exe itself. Tried running all these versions ingame and they all worked well, didn't noticed any errors.
So basically after finishing any big map in FS, needs to be compressed by MapTool and should be fine? Sounds awesome, thanks again E1 Elite. QUICK_EDIT
so under the line we can quite safely say
the problem is clearly in the routine that uncompresses the IsoMapPack.
The uncompressed data would have in all 4 sorting ways the same size, so the bug can't be there.
Very nice work. Using a different sorting of the tiles to minimize the base64 encoded data is pretty clever.
Tuc0 wrote:
So basically after finishing any big map in FS, needs to be compressed by MapTool and should be fine?
Only until that as well reaches the limit of the TS engine decompressor.
A well detailed 256x256 map would most likely have the bug even if the tiles got sorted by X before. _________________ SHP Artist of Twisted Insurrection: Nod buildings
So basically after finishing any big map in FS, needs to be compressed by MapTool and should be fine?
Only until that as well reaches the limit of the TS engine decompressor.
A well detailed 256x256 map would most likely have the bug even if the tiles got sorted by X before.
Of course, should have specified that.
Still nice to see this possible map expansion. With this compression, map dimensions can be expanded by another 4-5% for detailed maps. QUICK_EDIT
FinalSun reverts it to its own method for arranging the tiles, so when the map is finished, compress with the best sort method. Compression method is the same, just the tiles are arranged differently by FS or game snapshot and by these sortings.
An empty map of 190x225 itself takes 6567 lines. So all your detailing just took 2334 with best compression. You still got 850 lines more or 650/700 lines more with safety.
Blank maps don't get compressed further much with these sortings. A blank map of 256x256 will produce 10078 line in IsoMapPack5 which way exceeds 9750+ lines. 251x251 blank map will have 9688 lines. Unless some hack comes by and fixes game.exe to increase it. QUICK_EDIT
Updated the Starkku's MapTool for IsoMapPack5 compression. V2 has multi-level compression like sort by X then by LEVEL then by TILEINDEX. Also it includes option to remove Level 0 clear tiles which further reduces the IsoMapPack5 size.
Tested and works well with game, FA2/FinalSun and Map Renderer. Both sorting and level 0 clear tile removal is reverted by FinalSun so needs recompressing. QUICK_EDIT
Updated the Starkku's MapTool for IsoMapPack5 compression. V2 has multi-level compression like sort by X then by LEVEL then by TILEINDEX. Also it includes option to remove Level 0 clear tiles which further reduces the IsoMapPack5 size.
Tested and works well with game, FA2/FinalSun and Map Renderer. Both sorting and level 0 clear tile removal is reverted by FinalSun so needs recompressing.
Nice work, does game generate those missing tiles without any issues behind map borders? Very clever.
So if filled behind border area, it doesn't write any data for it, like map with smaller size? QUICK_EDIT
yes, you can delete the complete IsoMapPack section and the game will automatically create a blank map with only clear tiles.
Every tile, for which the game has no data, is automatically filled with a clear tile. _________________ SHP Artist of Twisted Insurrection: Nod buildings
Blue border that is [Map] LocalSize= is not affected. Tiles beyond blue border are created by the game/Finalsun when not present. Total cells is based on Map red border that is [Map] Size=, which is used for IsoMapPack5. You have already extended a map of 200 size to 225, FinalSun extends it, same with game and its map snapshot, those extra cells get tiles. QUICK_EDIT
Needs testing, to check whether this fixes the ice bug in TS snow maps. Modified Starkku's MapTool with a profile file.
FinalSun was always setting UData in snow maps to 0, whereas TS vanilla snow maps have set UData value same as the Level field.
Test on snow map made by FinalSun and having ice at height level above 0 and at 0 level also.
If this fixes it, then unknown data's purpose will be known and will update this fix on the tool's page.
Edit: New version available at Starkku's MapTool page. So attachment removed. Last edited by E1 Elite on Sun May 13, 2018 9:23 pm; edited 1 time in total QUICK_EDIT
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
Another "Map limit"?
