OK most of you won't know what TM2 is, but it's a second generation terrain Tile Matrix for EAW, which allows for things like beaches and cliffs. It was written by Will Gee, who created the original TM for EAW, which we still use today.

I'm not sure where we got to with this project, but Will did send some test files, which he thought would work, but I don't think they were ever tested.

Is there anyone out there who would like to have a look at this now?

It's one of the last major changes to the game, which with 512 tiles make the terrain look far more realistic.

I posted a bit on this topic again at the Coders Forum in the old thread about it.

The tests showed the TM2 does work at least to break the 64 tile barrier.
The key point my readme to wit:


It just needs time for someone to get back to this issue in the code.

This is the pic ingame proving that the barrier was broken, since the 64th tile (center 4 tiles of pic) in the 4 orientations rendered:


...Pleas note the proper place for the technical details is the Coders forum as the matter requires programming and posting of code to understand

Hi Roy.

Thanks for the reply, but as there is no code work to do, and the project was at a dormant stage, so I felt this was the place to try and get this project going again.

At the Coders forum, no new people can get involved, so this is exactly the right place to discuss this project.

I'd forgotten you'd tried it and made that post, but like several ideas, which were on the go at the time of the release of 1.28E, once the EXE was released, we all wanted to take a break from coding projects. Well, a lot of water has passed under a lot of bridges, so maybe it's time to bounce a few ideas around again?

Yes of course, np.
I should say that theorectically greater than 64 tiles should work. It's never been done by any of us, and Will did not
leave any more details about any tests he did, other than to say he checked the code and it should be working.
My test used 64 tiles, which works because that .exe reads the EAW.tm2 (and .tt2) which has 2 bytes data for every tile location,
in theory this could allow for some 65,535 (256 x 256 - 1) (or FF FF in hex) combinations, e.g. possible tiles.
(correction.. some 16,000 + tiles since you have to divide by 4 for each orientation or direction)
That doesn't necessarily mean it is to use a huge number of tiles in a terrain yet, or the .exe would be able to load them all
in memory.
But it is highly likely in theory it should work to use more than 64 tiles, because only so many tiles show in visual range at any given time.
This seems to me it should work already without memory issues (at least with the 256 x 256 tiles) or minor ones,
since from my understanding only so many tiles in visual range are loaded into graphic memory at any one time.
When I looked at the code (which is extensive and monotonous - tons of lines related to the whole tile loading graphics
video code relations) that was the impression I got at least.

To test more than 64 tiles we would need an .exe coded for the tile names of the new tiles (tile = a terrain file) so the .exe can use them.
Then code a .tm2 with the tile number, etc.
If sucess then when one is at say at position x,y the 65th tile shows, etc. If that test works then on to more to see if the matrix is working, no memory issues, etc.

Then there are more things to address, like relation to the .tt2 (EAW_TTD.dat), and whatever else.
Then also see if it works with 512 x 512 tiles, etc.

Copy of a post made at coders a minute or two ago



Jel

So, I made a tool that reads the eaw.tm and writes a eaw.tm2 file.
I put in the option that every water tile (tile2 in all four orientations) could be recoded as tile 64 in the corresponding orientation.
I made more modifications to the exe, including increasing the tile numbers from 64 to 65 defining tile 64 (the 65th counting from 0) as "Jelly1.ter".
I copied the BNALCTY.TER and LRALCTY.TER from a winter terrain into the root folder and renamed them as BNJelly1.TER and LRJelly1.TER.

Then I flew a mission using he exe I had made.





Bingo! A wintery city where the sea should be

64 tiles are working - there's a lot more to do, expanding from 64, removing a hard coded list of tilenames, and fixing eaw_ttd.dat.


Jel

The exe gives an error message if the eaw.tm2 file is missing



Jel

Hi Tony.

So, we are on the verge of having a much more diverse terrain, but does TM2 allow you to fold a tile more than just by the 4 corners?

TM2 was supposed to make it possible to create cliffs, rounded hills and beaches, but I don't know if Will got that far.

I do not know how far Will got, but I just downloaded Ralf's code for his fix_08, added Will's code and compiled a new exe.
I had ETO_01 loaded, Planeset 2, plus the test eaw.tm2 and my "Jelly1" tile(#64)in the root folder:





The second screenie shows that it is working correctly.

Jel

An explanation for members who wonder what this is all about

The eaw world that we play on is a rectangle of 640x320 tiles, with the layout stored in a file named "eaw.tm".
This file contains 640*320 bytes, one byte per tile. The value of a byte runs between 0 and 255.

If the byte is 0 then the tile to be used is "BNField8.ter" or "LRField8.ter" depending on resolution, and the tile faces North.
If the byte is 1 then the tile to be used is "BNGrass.ter" or "LRGrass.ter" depending on resolution, and the tile faces North.
If the byte is 2 then the tile to be used is "BNWater.ter" or "LRWater.ter" depending on resolution, and the tile faces North.

However, in order to join tiles to make a proper tilemap each tile needs to be able to face East, South and West also. On the south of England many coastal tiles face South, but on the coast of France many need to face North. Others need to face East or West to create the curves in the coastline.
The original programmers achieved this by adding 64, 128 or 192 to the north facing value.
So tiles coded as 0-63 all face north, but a code of 64 is 0+64 so a "BNField8.ter" or "LRField8.ter" facing East.
A tile coded 200 means 200/64 which is 3 remainder 8 which identifies it as a West facing "BNField5.ter" or "LRField5.ter" depending on resolution.
So, because there is only a single byte of data, and the mathematical system used for orientation, we can only have a maximum of 64 types of tile.

Will Gee's work on the eaw.TM2 system allows four bytes of data, the first three being the tile number and the last being a 0, 64, 128, or 192 for the orientation.
This gives a maximum of 1099511627776 possible tiles - a slight increase on the previous max of 64

The eaw.exe that I am working on includes Will Gee's code and reads the "eaw.TM2" file and not the "eaw.tm" file. It has been able to read and display the 65th tile.
So the 64 tile limit is truly broken

Jel

Hi Tony.

1,099,511,627,776 possible tiles should be enough to please most people.

And drive poor Shawn into an early grave!

What is theoretically possible is the entire EAW world done on individual tiles- only 204800 of them LOL

Jel

It is actually easier in many ways to make a terrain now because you can just make a large image and cut it up, we don't have the edge problems since each tile will only need to face up to its immediate neighbours in a single point in the map, so photoreal terrains are a distinct possibility... only one problem now....anyone got any 1940's satellite images of europe ?

The main question is going to be the the effects on framerates and memory management issues I would think...

this development doesn't address the problems of only being able to give 4 height values to the corners of the tiles though, so no cliffs, or steep river banks etc... yet...

The HM file is going to be a lot bigger too I suppose, since there are now a lot more tiles to give height values to... does that still work OK ?

cheers,


Pobs

Hi Tony.

I'd be happy if we can have some coastal tiles town/city, chalk cliff, and beach tiles, canal tiles, better rivers, and a few more field tiles of various types to break up the countryside.

Maybe 150 tile to choice from would be more than enough.

Do you know how this will work with the ttd.dat yet?

AFAIK the current height map with EAW.TM2 is the same as the old one, unless I missed something.

Jel

Perhaps it's time to drop Will an email?

Sorry Jell,

my mistake, of course the hm is related to the number of tiles in the matrix not the number of types of tiles, so it won't be affected.....


cheers,


Pobs

Further news.

Today I hardcoded the exe to 100 max, naming the extras as "Tile64.ter" to "Tile99.ter".
I renamed "BNJelly1.ter" and "LRJelly1.ter" as "BNTile64.ter" and "LRTile64.ter".

When I ran the exe it gave a CTD because it could not find ""BNTile65.ter". This tile was not used in the EAW.TM2 but the exe looked for it anyway.
The band aid solution that I made was to make a program that reads a selected file and makes copies named "BNTile65.ter" - "BNTile99.ter" and "LRTile64.ter" - "LRTile99.ter".
When I referenced a folder containing these in a "Dir.set" file it worked fine.
The file I chose to copy was one of the small dummy tpc files used with bitmapped multiskins. It is only 6 kilobytes in size.
It is becoming even more important that we make this system flexible, so that the maximum number of tiles is not hard coded, but counted in the routine which reads the names.

Jel

Guys.

I've been in contact with Will, and he's very pleased we have got TM2 working.

He's going to look into making a new HM2, and get back to us with some ideas.

The easy way would be to reduce the scale of the existing HM and enlarge the file size to cover the same area, but I pointed out we would have to chop up all of our tiles to fit the new scale. So that idea is a bit of a non goer.

So, we need to wait a bit for a better solution.

Progress report

I have not yet been able to get the exe to read and use the tilenames from an external file, but I have got it reading an external file "NumTiles.cnt" which contains the number of tiles.
These screenies show that it is working:

In the first two the silly EAW.TM2 was loaded which has the reference to water tile (#2) replaced by a snowy city tile (#64) in all four orientations.
For the first one the "NumTiles.cnt" file contained a value of 64, so the game is limited to the 64 tiles from #0 to #63, and tile #64 is not displayed.
Hence, apart from the coastal tiles the sea is black.


The difference here is that the "NumTiles.cnt" file has been edited to a value of 65, so this time tile #64 is displayed, and there is snowy city texture where the sea should be.


In the final one the "NumTiles.cnt" file remained with the value of 65, but a correct "EAW.TM2" was loaded, and the sea appears as it should.
Tile #64 would have been displayed, but it was not referenced in the "EAW.TM2" file.


Jel

A correction

Roy is absolutely correct in that there is a comment in the code that the 64th tile could only be displayed in three orientations.
However, the comment is factually incorrect. The tiles are numbered from zero, so the 64th tile is #63 when it is North
Add 64 and it is #127 facing East
Add another 64 and it is #191 facing South
Add the last 64 and it is #255 facing West

Somebody was counting from 1

Jel

Hi Guys.

I've had a reply from Will, which I have copied below:



Now the question is, which way do we want to go? I would have thought cutting each tile into 4 or 16 sub tiles would be more than enough to produce a good lever of detail.

The Bump map would give us a highly detailed terrain, but a Bump map would need to be made for each tile.
HM http://en.wikipedia.org/wiki/Bump_mapping

A large Height map similar to the one we use now to create our current HM would be the easier to create, but as Will points out the file size will be very large.
http://en.wikipedia.org/wiki/Heightmap

Comments please.

"I would have thought cutting each tile into 4 or 16 sub tiles would be more than enough to produce a good lever of detail."

does this mean with 4 sub tiles 64 original tiles that there would be 256 tiles to make?....or 16 sub tiles of the 64 original 1024 tiles?

Yes, Shawn.

That is an idea I thought would not be good, because we could not use any old terrains, with that type of HM.

Hi Guys,

OK can I clarify what we are talking about here. As I understand it, at the moment we can only define the height of each corner of the 4Km tiles in the hm file. Because of that restriction we can only raise the whole edge of the tile, although in certain circumstance the tile will also be split across its diagonals to fit the corner elevation readings, ie on pointy hill tops or ridgelines that run diagonally across the tilemap.

What we want is to define heights at a number of points along each tile edge so the tile can be 'sculpted' to represent cliffs or steep gully's within a 4 km tile, but still linking at the edges with all the surrounding tiles (of course)....

We are not talking about rescaling the tiles themselves or repainting any terrains are we ? the basic 4Km tile will still be the mainstay of the terrain won't it ?... or are we redefining a 'tile' to be the distance between any 4 points within the old tile ?.... requiring us to reproduce all the old terrains anew...

It seems to me that we want to change the way the hm handles the existing tiles rather than re-defining the whole system, if we are going to do that we might as well go for it and create a mesh based terrain system !!

I dont think bump mapping will be any use for creating river valleys and cliffs since it doesn't actually change the heights of the tile it just adds detail to the painted surface, so we could paint on a cliff but flying over it we would find it wasn't there...still bump mapping would allow us to add a lot more detailing to our terrains perhaps, so that would be nice


cheers,


Pobs

edit : just checked, a typical hm file at the moment for SAW for instance is about 400Kb. just to get an idea of the possible scale of the file if we went for 16 points per tile instead of the current 4, I assume it would be 4 times as big...

Hi Pobs.

At the moment you can create a HM file from a graphic I think?

How difficult would it be to create an HM with a more detailed grid?

What I'm thinking of is have a base grid with our normal HM layout to tell the exe where to place the tile, and then a sub grid splitting the tile into 16 sub elements, a bit like piggybacking, which both could be created from the same graphic. As there is no Normals or RS to worry about, bending the same texture as you like should then be possible.

It sounds good, but a nightmare to do probably.

Hi John,

yes the hm is just a raw file so you can make it from a greyscale raw image, 660 x 330 dimensions then rename it to hm and it will work as a height file. As such the size is irrelevent for creating the hm itself, we would just make the image to the new size and each pixel would still represent a single height reading, so no problem there really, I am sure Jelly could adapt one of his editors to work with the file so we could have a GUI to see what we were setting up. If we could actually load the tile map and then have the editor show us the 3d view so we could raise or lower areas around features such as airfields, towns and rivers it would work fine. We already have a feature like that in one of the existing map editors but it never worked very well for me.


cheers,


Pobs

Back to the TM2 stuff.
At Coders I have just posted a description of the hurdle that I have encountered in getting the exe to use data it has read from an external file containing the tilenames.
In it I have the describe the precise help that I need in order to get over this hurdle.
I would have posted it here, but we cannot post code.
Had I been able to then a reader of this forum might have been able to provide a solution.

Jel

Great work Jel on the TM2 issue and the corrections.

Nice Col. G. you are in contact with Wil and he is interested.
I would only caution about getting other terrain isssues working, like hm2 or whatnot,
and not ask more for now, as programmers have limited time and each time before when we asked
too much we lost their interest understandably.
First we need the TM2 fully working and the related issues to do with that.
The TM2 is far more important I hope everyone understands this.

in other words a bird in the hand worth 2 in bush

Hi Roy.

Will is very happy we have got his basic change working, so basically TM2 is OK.

I sent him a few screens of our new 24bit World, just to show him just how far things have gone in the past year, or so.

HD2 is really the next step, and Will has been working on a few ideas and he's happy to help us further.

As posted at coders a few minutes ago I found my own solution and the new exe is running well using data from four external files
A complete set of code, support files and other stuff has been put into a 7-zip self extractor and uploaded to Will's folder at Coders.

Here's that 65th tile again with otherwise stock terrain




Jel