[TerrainEngine] Voxel Terrain : Smooth,Cubic,2D,Hexagonal Infinite Procedural Terrain

Wow, holy smokes, awesome work Thomas! Thanks also for sharing the script right here in the forum. Those terrains are pretty nice looking with a lot of detail. Are those procedural/noise based or a converted terrain? Regardless, nice improvement on the water, will be updating to Aquas water promptly!

 

Those terrains in the last post look blocky (you can spot the voxels even if the are smooth voxels). Is there a way to get them as smooth looking as Unity terrain?

 

He must did some tests on something, probably used a Cube to dig terrain around. The new Octree LOD system from TE make terrain very smooth, as close as Unity Terrain, but much less polygon counts.

 

That's because he was modifying the terrain with a block like shape. It started smooth and was dug out.

 

It is not that.
I just notice when generating terrain it looks really smooth, but when he converts from Unity terrain to TerrainEngine, it seems to lose some quality.

 

Indeed frpmta. Original unity terrain size is 1024 *1024 *1024.
I've a lot increased the resolution so artefacts appear. It will never appear on a procedural terrain or on bigger unity terrain converted.


And as you can see in post. There are lakes at 2 differents level. A river example is coming soon with a curved system river between them with provedural flowmap using generated heightmap as water direction, water will also avoid obstacles like rocks or trees and add waterfall if elevation is to different between 2 points of river curve.

 

I will upload a new Reflection script soon.
I don"t know who made the old one i posted before but the performance is horrible.

Old Reflection : (40-45 fps 2560*1080) (and a lot of memory allocation) (2048 Reflection TextureSize)


New Reflection : (65-90 fps 2560*1080) (0 garbage). (2048 Reflection TextureSize)


Stop using value.ToString() +"," + value2.ToString(). etc etc
use StringBuilder class
Stop using new Vector4() new vector3() etc etc all time
STOP USING LOCAL FUNCTION, unity doesn't have inline functions system
Reuse all variables, stop using GetComponent(typeof(Skybox)), don't store ONE HASHTABLE per script, and so much problems on the old script ....

And WHY WHY WHY :

Code (CSharp):

1.     //<summary>
2.     //Calculates reflection matrix around the given plane
3.     //</summary>
4.     private static void CalculateReflectionMatrix (ref Matrix4x4 reflectionMat, Vector4 plane)
5.     {
6.         reflectionMat.m00 = (1F - 2F*plane[0]*plane[0]);
7.         reflectionMat.m01 = (   - 2F*plane[0]*plane[1]);
8.         reflectionMat.m02 = (   - 2F*plane[0]*plane[2]);
9.         reflectionMat.m03 = (   - 2F*plane[3]*plane[0]);
10.         reflectionMat.m10 = (   - 2F*plane[1]*plane[0]);
11.         reflectionMat.m11 = (1F - 2F*plane[1]*plane[1]);
12.         reflectionMat.m12 = (   - 2F*plane[1]*plane[2]);
13.         reflectionMat.m13 = (   - 2F*plane[3]*plane[1]);
14.         reflectionMat.m20 = (   - 2F*plane[2]*plane[0]);
15.         reflectionMat.m21 = (   - 2F*plane[2]*plane[1]);
16.         reflectionMat.m22 = (1F - 2F*plane[2]*plane[2]);
17.         reflectionMat.m23 = (   - 2F*plane[3]*plane[2]);
18.         reflectionMat.m30 = 0F;
19.         reflectionMat.m31 = 0F;
20.         reflectionMat.m32 = 0F;
21.         reflectionMat.m33 = 1F;
22.     }

Seriously .... inside a OnWillRenderObject() function ...

Vector3 (plane[0] or plane[1] or plane[2])
============

Code (CSharp):

1. public float this[int index]
2. {
3. get
4. {
5. if (index == 0) { return x; }
6. if (index == 1) { return y; }
7. if (index == 2) { return z; }
8.  
9. return 0;
10. }
11. set
12. {
13. if (index == 0) { x = value; }
14. if (index == 1) { y = value; }
15. if (index == 2) { z = value; }
16. }
17. }

TerrainEngine goal for next years :
Procedural Voxel Terrain (volcanoes,rivers,realtime fluid flow, biomes,etc)
Procedural Voxel Planets (Space -> Ground -> Space : 0 loadtime)
Procedural Fully explorable Cities (Buildings, roads,sidewalks, Vehicles, Pedestrians on destructible Terrain,etc)
Procedural Universes ( Realistic physics, persistant universe and multiplayer )

 

Thanks for the tips I'm always interested on optimization practice. You should share more tips like this, very useful.

 

Stupid question: does this engine come with a normal, old school style, height map rendering engine? If so, is it faster than the Unity one?

 

Not a stupid question at all. I can roughly answer you that. TE can convert a 3D voxel-based terrain to a 2D height-based terrain, (with its own LOD support - not sure, I have not tested it yet). So you got Unity terrain as usual , and it can work with the TE random scattering systems. Btw, Unity 2D terrain is the fastest one, but only in some less ms.

 

Thanks I understand. I never wrote a terrain renderer, so I am not expert, but still I feel like Unity 2D terrain engine creates more draw calls than needed which its patched LOD system.

 

This looks great. I have a few questions, thanks for any info you can give.

1. How does the voxel modification compare to Voxel Farm? Does it support boolean operations with arbitrary shapes? Could everything in this video (Landmark, using Voxel Farm) be done using TerrainEngine, specifically the building ramps and cutting holes with shapes.

2. Do the buildings made using the Construction Kit exist in the same format as the rest of the terrain? Is the engine suitable for building large, complex structures with the voxel system?

 

2: construction kit is a separate processor, like a mini voxel engine able to merge
cubes, ramps, any shape you configure into an optimized mesh with all hidden faces removed. It's integrated into the terrain multithreaded chunks management system, the lifecycle of structures is in sync ( streaming new ones, pooling, saving ... ). The structures are saved in their own binary files sitting next to the terrain. There is an API to have the constuction affect the terrain, like digging holes when placing cubes.

Imho TE's source code is worth 3 years of full time dev by a pro developer. In my country that's more than $ 300000

 

Anyone else receive an email claiming to have bought the intellectual rights to the author of TerrainEngine. They say they can sell it to you for (closed source) $99.00 dollars from Terrainengine.io.

Sounds scammy (too good to be true). Maybe be aware of this and if a statement could be made here by the real author.

 

I'm dyox and I've already said I'm the author and I'm the person who sent emails.
I'm now working with Junglee games.

 

Okay. Thanks for quick reply. I just thought it weird when I get a reply to an email from months ago and then nothing in this forum about the sale of TerrainEngine (this was pretty much the only place to find information about TerraineEngine).

Good luck with the future of TerrainEngine.

 

Seconded on the good luck. In case you where wondering what that swishing sound was. . . That was me throwing cash at the screen. Already bought a closed source license and I do not think I am alone in that.

 

You won't be disappointed when playing with TE. One side I'm glad TE getting more users, one side I want to keep that secret sauce ><; to as little people know about it as possible, weird haha

 

For us, the biggest hurdle to overcome with TE is the lack of tutorials and clear concise information on how to set everything up. Once a new complete demo scene comes out for V10 that will probably help things out a lot.

 

This is now available for $99? Where from?

 

https://www.terrainengine.io/

but the $99 offer has run out now I think.

Phil

 

Its now $199.00 (still good price). terrainengine.io

 

V10.9 : New Multi-threaded Trees LOD with Fading/Batching : 0 GC impact, fully interactive and working with network.
New grass/details/gameobjects/fluids generation system
New Tree/Grass shader, new terrain shader, new Cell system....
And more... (Rivers, Caves, Roads,etc...)
WIP V11 :

 

++ New noises modules : Volcano, Geyser + New way to add custom data ( geyser can contains fluids and particles,etc...)
++ Fastest loadtime : 30s -> 8seconds for custom demo scene. (from 90 to 20seconds on low config)

Voxel Size : 0.5m, memory : 700mo : 6km3 with fluids/grass/trees

 

Wow, that amazing to look at :0

 

Roads? That sounds very interesting - any screenshots or more details on this? I'm interested in creating a large world - about 32km x 32km. Roughly how much memory would a scene like this use? Is it compatible with assets like World Streamer, at least for game objects placed in the world (like buildings)?

What do you use for trees, if not SpeedTree? Standard Unity trees? Custom meshes? Does your tree/grass shader support wind zones?

Also, does Unity physics (like WheelColliders in particular) work OK on the voxel terrain?

Sorry for all the questions I'm considering moving from standard Unity terrains, but it seems like a big jump. And I'm gutted I missed the $99 price deal .

 

Looks great, but aren't voxel engines generally memory hogs and poor performance compared to standard Unity terrain? Or is this an exception?

 

Someone with the engine, or Dyox, would be better to speak to that. But, from what has been posted by users it is only slightly behind std unity terrain in performance. Memory usage is probably somewhat higher, but the figures that have been posted in thread seem fairly reasonable all things considered.

It is my supposition that a lot of care has been taken to optimize code, memory alignment and GC in Terrain Engine versus Unity terrain, which I think is much less optimized. End result is a much smaller than expected delta. I am hoping that the ability to efficiently scale large quantities of objects (trees, grass, etc) in TE will ideally wash that out, or maybe even end up ahead. Defintly eager to find out.

 

So, on their slackchat I asked a bit about their optimizations in relation to objects. They have custom LOD handling for normal and speed trees. The claim was 60fps with 600k trees - presumably on a modern PC with a burly vid card. But still, standard Unity Terrain w/Speed Tree would struggle at number far, far lower than that as far as I know.

Bottom line is it looks like we will pay a bit up front for the Voxel overhead, but reap some pretty impressive rewards for large crowded scenes.

 

Hi
I want to make the race track there is a cave.
Banked curve roads and loop roads also.
Can i make When the road system is completed?
And, This asset can be handmade straight gentle slope?
For example, such as the slope of the 10 ‰.

 

You mean 700 mb memory? If so then it's totally useless. For example, the entire Wii U memory available is only 1GB and some of that obviously taken up by Unity and OS. Maybe on a monster PC it'd be OK.

 

Have you seen some of the big games using this type of tech? It's not for casual, older simple games, nor for casual gamers (or kids on a kids console like Wii). This is serious tech for a bit more of a serious genre of games. For example, Everquest Next was a great idea with a bad implementation behind it. However, the tech requirements were very similar to what is needed for Terrain Engine, since it used similar technology. You use this tech because your game NEEDS it, it's almost like a gimmick like Minecraft. You don't use a voxel terrain to replace Unity terrain, that's overkill and misuse of the tech. You use a voxel terrain because your game is going to be built around the idea, like deformable terrain for a Battlefield like game where you have huge destructible environments, or in our case you want Minecraft like functionality with a Skyrim like look. I know a studio that is making a construction game that utilizes the deformable terrain. 7 days to die is another game that uses voxel terrain, has huge overhead, but has sold millions of copies just like Everquest Next. It's a genre/niche type thing, and will work on consoles but only next gen power consoles, not a child's toy like the Wii U. Plenty of money to be made on that console so not knocking the console, just its limited horsepower. Personally, we put our minimum spec needed for our game, which is on Steam, as needing 4 mb of ram minimum. Any modern day gamer who is in any way serious about PC gaming will have at least that if they have bought a machine in the last 4 years. This isn't a power PC, that's like a budget gamer PC to require minimum of 4 gigs of ram. Look at the minimum specs for Battlefield 1 right now - that's modern gaming and its selling in the millions of copies, which means there is plenty of market there for games in need of medium spec machines.

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=battlefield 1 requirements

 

I would add that you can scale back the world size from the 6km ^ 3 and/or I would imagine reducing the .5 meter resolution to say 1 as well. All of that will doubtless reduce that memory footprint quite a bit. Napkin math of 6*6 = 36 and 2 * 2 = 4 giving a theoretical 9x reduction for just switching to a 2km view and fogging the rest (I ignore 3rd dimension as that will most likely already be largely optimized out in engine detecting empty/all the same chunks).

And, what Wii games have up to 6km viewable with 100,000s of objects on screen regardless of tech used? Not sure, but I doubt many if any do that. Hmm, and I believe there is also a 2DTeraria like mode which is again is very likely much lighter weight. Or, no doubt the mine craft non-smoothed mode reduces complexity, but not memory, as well.

So, yes Voxels are not "free" we still need the 2D rendering that our graphics card wants, and also still have to store the chunked 3d data to represent the 3D volume of those objects. No amount of optimization can completely eliminate the delta between well optimized traditional height map approach vs Voxels. So, what do we get here in return? By the tin...

1. Dynamically destructable terrain and objects, networked
2. Fully working caves, overhangs and whatever. No need for custom models or workarounds
3. Fluid dynamics
4. 3D multi threaded path finding capable of responding to dynamic changes in terrain / objects
5. Some advanced profiling features along with object management
6. Procedural terrain with biomes. Apparently seamless too.
7. Ability to really detail your terrains with LOTS of objects beyond what Unity will currently will do with acceptable performance. Fixes for Unity LOD speedtree object scaling issues. Say good bye to 65k object max on a terrain. etc.
8. Potential for procedural roads, cities, galaxies, planets. I say potential because it is not here yet AFAIK, just deving/planned.

That in my mind is very far from useless - if, of course, it delivers. Perfect magic bullet? No of course not. We have
1. More memory consumption.
2. How fast can one move? There is an overhead to loading and generation of voxels, so at some point such as in a fast jet flight sim there are likely to be issues.
3. While terrain is smoothed. Actual objects may not be, although you are free to use traditional objects/models. I have seen folks working on 'voxel normal' technologies to address this, but probably not here yet.
4. No construction system like the system seen in Landmark/Voxel Farm.

Up in the air?
1. For a PC type devices the memory footprint does not really bother me. But what about DX/GL hardware needs though? That is a whole other form of potential limitation depending on fall backs and the 'costs' of those fall backs
2. And specifically, would it work acceptably on Intel's later integrated graphics? Love it or hate it, that is a big chunk of PCs and casual players.
3. Some say the engine pre-buy out and update has a big learning curve. Apparently they are working on that for the release, but 'delivered' and 'working on' are admittedly two different things. So I slot this as neutral until I see it.

 

^Well said Fenris2. It definitely matters what your target audience is. The folks who make Battlefield obviously don't care about the sector of casual gamers with Intel integrated graphics. But EA has other studios that do make casual games for those casual types of gamers. A game that is going to use Terrain Engine definitely needs to consider this. Are you going to shoot yourself in the foot by trying to cover too many basses in regards to market and target audience, or are you making a niche market game with a targeted audience(which also can be a shot in the foot if you don't understand your audience or fail to reach them)?

 

Great post fenris.

To add to that, I would also say what is the actual use case? Do you just need overhang or destruction is important?

Anyway there is a bunch of decision that can be made to reduce the impact of a given voxel terrain:

- we primarily need difference between air and solid, that 1bit, which mean a 32x32x32 chunk can be store in just a 32² array. Visible voxel are one who need extra different data (other voxel are hidden). You can therefore separate carving from differentiating in the perlin generation. Some optimization do a search (generally djikstra), from the camera position, within the frustrum, that is culled by subfrustrum created by visual blocker (full voxel). Basically every solid voxel revealed by the search can run the extra differentiation, that lower the number of voxel from surface to only visible.

- we can pack data better by having full chunk marked as full or empty (with a 1bit bool) and discard the 32² array, and use compression to store.Most games really need surface data, so we can generally concatenate empty chunk together into a sort of heightmap until caves or surface chunk. It allow to have bigger range of height (especially in static terrain).

- we need LOD and that's what Dual contouring is great at
- You can tessellate and perturb voxel surface to have extra detail even at low voxel resolution. 2m cube (also close to what heightmap use) is great for gameplay (enforce good metric) purpose and avoid the hi frequency artifact that pain physics and navigation, it's also 32 768 less data than .25 m resolution. Surface perturbation do the trick of adding the missing details, so is scattered decoration.

- Voxel are at integer interval, so you can discard the voxel data, after mesh generation, for a just in time generation after modification, and quering the "perlin" (or other local generation) just for the few revealed voxel. That's basically as easy as hashing the returned raycasted point with (x / gridsize), removing the vertex in the mesh within that and regenerating the missing surface by querying the relevant voxel position. Generating and storing rich cubic data is generally the main performance hit in voxel generation.

I haven't got terragen yet (soon, I need to validate some step before), but I'm sure Dyox have already implemented some of these ideas.

 

Hi,

-I'm not using this system. Difference between air/solid is not usefull if you don"t store data as octree, i'm storing data as chunk, and air/solid can have a volume used previously. Example : You dig on stone : < 0.5 volume, it's air, but it's still hard to add/remove stone, so i need to store type/volume for each voxels and each processor use the type at given X/Y/Z value. But I have a system to remove full empty data from memory

-I'm not using Dual Contouring. Everquest next and any other games using it are too slow to be usable in realtime and on any platforms (FPS, MOBA, Etc), i'm using Transvoxel, Targerting all platforms (working well on VR, specially with the new LOD/Batching system)

-Yes, tessellation is simple and possible, i'm already providing shaders for it.
BUT, only for high-end garphic card, not specially for AMD GC. AMD + Tesselation = slowdown. Specialy with splatmap support even when combining textures.
And 2m3 is too high for real usable terrain, we want dig,create,etc... 2m ... too big scale (even if it's better for the GC/Physics/Memory)
I'm actually using 0.5, and planing to scale down to 0.3m with same terrain size. Garbage Collector for the moment is my limit, but i'm working on a lot of new clever systems, without using any classes or pooling.

-I'm not using any external tool, i know terragen, but exporting heightmap, or working with DLL is not my choice for TerrainEngine, i want TerrainEngine working on all platforms, and DLL is not an option. I'm working closely with customers to improve the procedural editor, the unity terrain importer etc.

Also i will not talk about the WI U ... or any system that have less than 2Go of ram. GTX 970,less than Quad core or under Galaxy S6.
It's a voxel solution, working on all platform, any heightmap based terrain can be used for low spec. ( Except if you want a fog at 75m, 0 shadow and a simple tiled shader)

 

Oops. Not working on all platforms after all.

I honestly don't really care, but don't talk out of both sides of your mouth.

 

970GTX is a minimum or a type-o? Because that is some pretty serious hardware and way above a galaxy s6 which is adreno based yes?

 

We have a 650ti in our low end testing machine. I have this posted as our minimum spec card on our Steam page for system requirements. It's not the best (running barebones TE it probably has no issues), our game has a lot more logic on top of TerrainEngine, which is what kills our performance, but with a 650ti, very old slow i5 and 8 gigs of ram we can maintain about 24fps(and with HUGE terrain with 5 biomes). On the other hand, on our Occulus machine we have a gtx 970, newer fast i5, 8 gigs of ram and in our game I get 55-65 fps with settings on high. Running Terrain Engine on its own on my 970, with just epic size terrain and a couple trees I get a good 135-150 fps. The average mid range gamers card of 760, 860, 960 all will work well as well. Of course everyone's mileage will vary depending on how much more logic is running in their game on top of TE.

 

Yeah, and you could not talk out of your mouth at all on this thread, if you want to start getting rude. Apparently you have no idea about this technology and what the purpose of a voxel terrain would be used for, why are you bothering to comment on the thread? Your obviously not developing anything on this level so you don't need this technology and don't need to leave snarky comments.

 

I wasn't being rude. Just want him to say what he means. Don't say, for example, that you want it working on all platforms and then in the same post say you won't even talk about the Wii U. You read rudeness in my post where there wasn't any. I came to this thread looking for a potential alternate to the Unity Terrain system.

 

I said it works on all platforms.
And I said : I will not talk about WI U. I never said it doesn't work on Wi u or any other platforms. I just don't want to talk about that because it's a non sense to use a voxel terrain with all the limitations it come with, on this low spec hardware. You're free to use something smaller for voxels.
TerrainEngine have fluids,lod,batching,threading etc, it's a complete terrain system, and for just creating holes with a draw distance of 50 meters on a Wi u it's completely useless to use such a big voxel solution.

Terrainengine is not an alternative to unity terrain. It's a new terrain system that provide differents functions and allow you to create the game you want.
If your game don't need a voxel terrain, dont use terrainengine, unity terrain will always be faster than any voxel terrain.

And I've said gtx 970, because is the first gc available for vr. TerrainEngine works on very low spec, it always depend of what you draw and what density of objects you've set. Just the terrain take 50 batch and 200k vertices. .. add grass,details,trees,fluids,post effects,shadow, etc and target your minimum spec by reducing the complexity of each system.

 

Sorry, I was talking about VR for this GC. terrainengine works on very low spec. You can tweak settings to target your fps like on all games. Low medium normal ultra etc...

 

Well from experience I have tried with very crude system, I don't believe that, it depend on how you design your game It all depend on your exact need and your ability as an artist (also because I use perlin displacement on the vertices to remove straight line and spawn visual break, so the visual are designed around the terrace look).

But I'm reacting because is this implying that voxel resolution is fixed and cannot be changed?

Also I want to address a flaws I forgot to mention in my post:
- 1bit maps is great if you don't have gradient base voxel (ie air/solid is define by a percent), which mean you will have some more fiddly with the other optimization (frustrum visibility search), and you still need to store a float anyway.
- It works with non smooth terrain best (ie it has visible terrace) I bet most people want their smooth terrain anyway

Just a precision about tessellation. I didn't meant shader necessarily. Just that the construction mesh part in the LUT being more densely divided (not in real time), then having their vertex displaced (generally along the normal) with the hi frequency noise data saved by having big voxel (low resolution) do the part.

Which bring me to the next point, just for anyone interested. I'm a game designer first, the way I approach level is through design metrics, in my experience metrics lower than 2m block step (ie a human size) create a lot of gameplay problem, especially navigation and physics. Now that does not mean that inside the block, the module will fill the entire space, you can have 50cm fence inside the block step.

The problem come from having too much high frequency noise, it's a beginner problem to have hi frequency terrain that mess movement and create opportunity to get stuck everywhere! Even when the editor allow for smaller step, space tend to be organized around that scale or bigger (4m), rarely lower than 1m. Below that threshold it's decoration and props to create visual clutter that don't create gameplay clutter.

In fact go back and look at your favorite game and you will see they are quite spacious in step, and the best organic one are just great at hidding it with great visual design (but the principle is the same). It also allow to have nice natural ramp (if you have vehicle),etc ... But the real kicker is that it's cubic data, you will jump from 700mo to 10mo (ie 64 times less data as 2^3= 4^3 x0.5) for the same volume WITHOUT any optimization at all, just the vanilla service of terrain engine, that's why I care that I can change the resolution!

It's a design concern, 0.5 is truly overkill in most case to fluid design.

 

So I noticed that on your website (terrainengine.io), it says that your product is for sale. Are you still doing the student discount referenced on your old thread? If so, does it apply to the closed source version. If those two are a thing, since I'm a high-school student and a wanna-be indie dev, could I get a discount? If so, how much will be taken off the $199 price point?

I'm looking forward to using your product soon. FYI, I currently have around $80 saved up, and could earn more shortly (and might be able to pool it in with some game-dev friends)....

 

V11 progress :
-80 to 25 seconds of loadtime : Ultra Mode
-Asteroids, Voxels Objects, new shaders
-Caves,Rivers,Floating islands
-Realtime fluids with LOD
-and more...

@TheModdersDen : I'm not in junglee team. I don"t have any information about them. I'm only working on TE, updates and support.

 

@dyox - have you had a chance to come up with a working V10 demo to show how to use all of the features of the engine? I know you were working on a new demo scene a while back.

Thanks

 

20 days later and still no response?

 

@HolyFireGames check out Voxel Farm, so far from what I have tested with the demos and the $20 studio version seems like a great voxel tool, ive shopped around for many voxel tools with unity and Voxel Farm seems like probably the best bet with actual demos

 

@dyox Can you please respond to my PM or email? Im very interested in your terrain engine.

@Reiika Would you know if Voxel Farms can create terrain like CubeWorld?