Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

WorldComposer a tool to create real World AAA quality terrain

Discussion in 'Assets and Asset Store' started by eagle555, Dec 6, 2013.

  1. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,492
    While working on the offset I found a work around, sharing just in case other people want to make something big too.

    When building the terrain use texture max size (I used 128) before hitting "create terrain". The source is still whatever size you capture it in (4096 in my case) so you can check and swap texture size using unity texture set up by selecting the texture of the tile.

    So to visualize region you create prefab of them with the lower rez map, then erase all current terrains in memory, drag the region prefab back and swap lower rez setting to high rez.

    Notice that unity editor can "break" sometimes, for example it didn't display texture past a certain distance for me, even in lower rez. You will need to restart unity to reset the behavior.

    This will come handy when I can access to zoom 19 at 0.25 pixel per meter texture :D and I have to deal with 19gb of data (instead of 6gb at zoom 17).
     
  2. Xtense

    Xtense

    Joined:
    Feb 18, 2015
    Posts:
    34
    Hey Nathaniel, imported as you asked into a fresh new Unity 5 RC3 project, redownloaded just in case even and... Well, oh dear.



    Here's the full log just in case: http://pastebin.com/ShaPwZ3C
     
  3. Xtense

    Xtense

    Joined:
    Feb 18, 2015
    Posts:
    34
    Sigh, I'm very stupid apparently - Unity was determined to keep the old pre-patch files and they conflicted with each other. Removed them manually and everything is working as intended.

    Thank you Nathaniel! Everything seems fine now :)
     
  4. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello,

    a). You can use image offset to get heightmap and image alligned correctly. I write about in the WC manual. You can do it on 1 tile of an area, and when its correct, export the whole area.
    b). c) You have a good point. I will add your request to the WC feature list. I could look into making that are region can be created like an area, and then it could be divided into multiple areas. At the moment I'm working on a big TerrainComposer update. Then after that I want to build in Google maps in WC, then after that I might have time to look into this.

    Nathaniel
     
  5. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello,

    Yes this is possible, thanks for sharing. Actually if a texture is imported in Unity it's on default on 1k. WC puts it on the size you export in the 'Create Terrain' tab, it can be changed manually, and if you disable auto, it won't apply the image import settings. So you can already do this before you click the 'Create Terrain' button, and don't need to do the rest of the procedures you are doing. You can change the image import size any time, even if the terrains are in the Scene. Just change max size and click 'Apply' button.

    Nathaniel

    ImageImportSettings.jpg
     
  6. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,492
    Thanks for replying.

    Yeah that's exactly what I was referring to, :) But yeah I notice after that the heightmap data is trivial, there is no need to break it into region UNLESS you try to see a bunch of big rez textures at the same time.



    Whats' the format of the raw image, I guess it's a list of raw 8bit value laid into RGB pattern? I still can't find a soft that allow me to open them :(



    A last thing, Maybe terrain composer do it I have yet to tried it. It's optional and only a convenience.

    Will there be a possibility to generate LOD?

    I explain right now, due to low resolution, it's rather easy to keep the base heightmap all in memory (assuming no conversion into hi rez). However texture that's another problem, especially for large terrain. It would be cool if there was a tool to do merging of texture.

    Let me explain, let say you have capture the max possible texture captured size (4k), but you are using smaller terrain patch at player's horizon , let say the ratio is 1k texture for 256m² terrain patch. For smaller patches you need to break the texture into appropriate size.

    But for bigger patch you need merging texture, for exemple the relation is 1:1 at 1024 m² patches for 4k textures, with 2048m² you would need 8k textures which is not supported hence the need to merge 4 x4k textures and reduce them to one 4k, and so one for bigger patch (16 x 4k textures merging into 4k for 4096m² patches).

    For my current project the whole island would fit into a 65k patches for a big 4K textures, which mean the biggest patch would be 32k (assuming the player is always on the island).

    However this is an edge case (maximum texture resolution). Ideally It would fit a custom curve that break down patches and textures according to hand given (or automated) number. I'm working on simple algorithm that would give the necessary LOD resolutions for far objects based on their vertical screen coverage depending on screen resolution (so smaller rez give less details).

    I don't know if that make sense.

    I'll had to make it anyway, so it will be a plus for other customer!

    I'm satisfied with the product, I'have not yet rate it on the asset store but 5 stars incoming ;)
     
  7. zombee81

    zombee81

    Joined:
    May 15, 2013
    Posts:
    11
    Could anyone tell me what is the size of the landscape each image represents? (preferably in Meters). Thanks in advance.
     
  8. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,492
    So I used a concurrent product to get access to early goodle map and it only highlight how much WC is the best solution, even though the other one is having multiple provider (bing and SRTM for elevation ; ARC GIS, Google, map quest, nokia, virtual earth, Open streemap, (an option for entrring custom provider) for texture map).

    I have been ban from map quest already for accessing too much ... I have dl zoom 17 map at 128 for each other, google is still the best. I have trouble going beyond zoom 17 with higher map setting and many provider (like google) kick you if you try to access many data at conce (so no map beyond 128 at zoom 17 roughly 1gb of data, less for mapquest).

    That's something to consider in the implementation to allow the spread of the capture in multiple session, giving the relevant info to do so (timer, data estimation, etc ...).

    That's not mandatory that would be really cool! It would be the ultimate package.
     
  9. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,492
    What do you mean?
    with maximum heightmap zoom, arc 1 give 25m per pixel, arc 3 is 90m
    zoom 19 is 0.25m per pixel for texture
    now you can conclude, base on the size of each map and their zoom level the real life size. For example a 4096 texture at zoom 19 give 1024m
     
  10. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello,

    As for the format of the raw images. I write about it on page 9 in the WorldComposer manual. Raw means there's no information inside the image file about the resolutions used. So raw consists of a large strip 8 bit RGB values. So the file size is resolution*resolution*3 bytes. You can open them in photoshop:

    upload_2015-3-1_11-45-8.png

    As for handling textures if you want to do streaming and have a the terrain the same resolution (so not using the multi resolution terrain), it is most convenient to use Amplify Texture 2 for texture streaming, as texture operations during runtime are too expensive:
    Amplify Texture 2 doesn't have a 32k limit (Unity 5 doesn't have this limit, but your scene will still start to hick, as you can see in the video). I tested amplify with 32K WC image and the results and detail area very impressive.



    Nathaniel
     
  11. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,492
    I have already mention I can't use photoshop ;) However I can code a it so ...

    Amplify is compatible with low end mobile (gpu mali 400, 512mb ram)? They says nothing about that or web plugin, plus it state for unity pro ... I'm not there yet. And they are expensive too.

    I plan to convert all terrains to mesh anyway, terrain are already low resolution, the cost being the texture.

    However The LOD is not mandatory, just a convenience. Another convenience would be to find a way to hook mappity such as I can tile their data within the whole large terrain, they only have a radius of 5km using the center of the whole area. Also optional, I don't think there will be much request for that beyond mine.
     
    Last edited: Mar 1, 2015
  12. Earthsnake

    Earthsnake

    Joined:
    Feb 25, 2014
    Posts:
    30
    Hi Nathaniel, I've been using your method on getting real world landscape into terrain composer based on this video tutorial:


    But my result in terrain composer was like this: black-landscape.jpg

    I've followed your steps, what's missing here? : (

    Thanks
     
  13. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    You can ask amplify about mobile compatibility and map-ity for the feature...Mobile has a texture limit compared to PC do to GPU memory. Mobile can do max 2k textures and it's best to do benchmarks with test scenes before you start designing.

    Nathaniel
     
  14. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello,

    I see that you have RTP in your project. You first need to setup RTP Lod manager as Tom explains in his tutorial video on the RTP Asset Store page:
    https://www.assetstore.unity3d.com/en/#!/content/5664

    Nathaniel
     
  15. Xtense

    Xtense

    Joined:
    Feb 18, 2015
    Posts:
    34
    Hello again,

    I've been thinking a little about the project I'm doing and I'm starting to think there may be some space for optimizations. I know you suggested Amplify Texture 2 to stream textures as one possible upgrade and Dynamic Terrain Loading Kit as another - the latter will indeed be used to handle the massive amounts of terrain.

    Currently, my workflow depends a lot on being able to fit together tiles, so I devised something like this:

    1. Divide Europe into 8 vertical region stripes, all starting on the same latitude, so the tiles both have the same size between the stripes.
    2. Export the heightmap only once at level 10 for each stripe to have a basis for the terrain without murdering the API key ( :) )
    3. Export the images twice: once with image zoom 12/resolution 512, the second time with image zoom 14/resolution 2048 - this way the tile size remains the same and I can substitute the textures for tiles I need in higher resolution with not a lot of work. (it would be incredibly beneficial if I could mark tiles as "Do Not Export" here - both to avoid exporting water unnecessarily and to choose which tiles I need to have in a higher resolution)
    4. Generate the terrain once for each stripe separately, then prepare particular tiles as prefabs for the Dynamic Loading Kit, where they will be ultimately merged into one complete dynamically loaded map.

    My stripe setup looks like this at the moment:



    Honestly I'm a little bit scared of the data I'll have to push around for this to work, but, well, sacrifices have to be made ;) .

    If you have any ideas how I could optimize my setup, I would gladly hear them.

    [EDIT]

    One more thing: is there maybe a way to increase the density of the terrain's heightmap points after it was generated? I will most probably need to carve lakes and riverbeds manually where applicable and at this resolution (129x129 per tile) it's not precise enough. I tried to re-generate the terrains with a higher resolution, but predictably achieved the minecraft look. Smoothing helps of course and, if pressed, I will generate single required tiles and smooth them, but I was hoping for a more automated solution.

    Best regards,
    Xtense
     
    Last edited: Mar 4, 2015
  16. jalapen0

    jalapen0

    Joined:
    Feb 20, 2013
    Posts:
    136
    Hello, I grabbed Unity 5.0 yesterday (along with half million other people apparently) and I'm having problems getting Worldcomposer and terraincomposer to compile. I've removed my old asset folder, uninstalled 4.6, installed 5, started a new project, download WC and TC and import and I get over 160 errors in the console. All kinds of "Ambiguous reference" to many many classes. I can post a screenshot if needed. I figure this has to be a common problem.
     
  17. khos85

    khos85

    Joined:
    Jul 21, 2013
    Posts:
    541

    Yeah, I get the same issues with the new unity 5 ga release..Hopefully Nathanial can provide some guidance for this, I'd like to update my existing game to v5 :)
    Do the errors mean there are duplicate scripts? if so where are they duplicated too?
     
  18. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello,

    TerrainComposer and WorldComposer work fine in Unity5. You have to delete the Global_classes 1.js script from TerrainComposer/Scripts folder. WC/TC share this file and U5 makes a clone instead of replacing it.

    Today or tomorrow I will release an update, that TC/WC run out of the box in U5 without needing the api-converter.

    Nathaniel
     
    jalapen0 likes this.
  19. MarchOfPenguin

    MarchOfPenguin

    Joined:
    Mar 4, 2015
    Posts:
    3
    Hi like some help on this if i should get it or not?

    I am using this to create heightmaps of the real world to use it in games that has limit to how big the map can be. Would I need to buy such an expensive software for this or would it be better to use other simple option for the heightmaps? Like google earth?

    What is the advantage of getting WC if I am just wanting grey heightmaps???

    Also do I need unity to run the program? Or is it a standalone program? It says on the page that I need unity?
     
  20. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Extracting the heightmap required my to do quite some programming, so it's not that simple. WorldComposer can indeed be used for this. It extract 16 bit grey scale raw images, it's the format Unity terrain can directly import.

    WorldComposer can only run in the Unity Editor.

    Nathaniel
     
  21. khos85

    khos85

    Joined:
    Jul 21, 2013
    Posts:
    541
    Excellent news, many thanks as always Nathaniel!
     
  22. khos85

    khos85

    Joined:
    Jul 21, 2013
    Posts:
    541
    Just curious, do you perhaps know why Unity 5 does this cloning of the script? Seems a bit unnecessary and will potentially affect many older/existing projects..
     
  23. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    It's because there's no meta file information inside the packages, as in U5 it's no longer checked on script name but on meta-id. Yesterday I submitted both TC/WC with meta files, that have this issue solved. Also they work directly in U5 without the need of the api-updater. If you want to try them out already you can send me your invoice id by email and I will send the download links.

    Nathaniel
     
  24. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello Everyone,

    The new TerrainComposer/WorldComposer is now on the Asset Store and these work out of the box in Unity5.

    Nathaniel
     
  25. DCrosby

    DCrosby

    Joined:
    Jan 13, 2013
    Posts:
    86

    I'm having some issues with the Combined Raw Image Editor (To Remove Shadows), I've left it running overnight for a 8x8 tile Section of Yosemite Valley, and it doesn't want to finish. Is this normal ? I've tried other regions and find the same issue. The Progress bar moves to the right but at a snails pace, and it's at this point going to take 10+x longer than terrain generation to get the shadows out. Using Unity 5, hence the reply to it working.

    I take that back, it never completes.
     
    Last edited: Mar 14, 2015
  26. sylvain

    sylvain

    Joined:
    Feb 19, 2013
    Posts:
    46
    it's Sunday, and may be I am not really awaken, but I missed something on heighmaps values.

    In WC (latest version, 1.3) and Unity 4.6, I create a simple 1x1 area, in sea (Mediterranean Sea). Height center is at 48m (???). When exporting to heightmap, I compute min/max values. I got 1077 / 11
     

    Attached Files:

  27. sylvain

    sylvain

    Joined:
    Feb 19, 2013
    Posts:
    46
    Sorry, I posted before the end. I got 1077/1157... (meters ?). There is a small rock on the upper-left sided corner, and it supposed to be 20 or 30 meters above sea level.

    How can I get real values ? I missed something ?

    BR
     
  28. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Can you try it by disabling 'Save New', does it work then?

    Nathaniel
     
  29. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello, WC adds 100 meters because Bing sometimes return negative values around the beaches and sea levels, this would then always result in a flat surface as 16 bit raw heightmap values need to be in the 0-65535 range. How do you compute the values? WC uses this to convert meters into the raw heightmap value:
    rawHeight = ((height+100)/9000)*65535.0;

    Nathaniel
     
  30. VoidGames

    VoidGames

    Joined:
    May 17, 2014
    Posts:
    22
    Hello,

    A couple months ago you had told me that you would create a tutorial on how to use the shadow remove tool on water islands and such...

    can you please finish that..
     
  31. DCrosby

    DCrosby

    Joined:
    Jan 13, 2013
    Posts:
    86
    I can try that tonight!
    Quick question, how long should say a 8x8 terrain take to remove shadows @ 2K or 4K Tiles, I've tried a 6x6 @ 2K and it took overnight and reported 10% done, I left it running while at work, and we'll see when I get home, what it is up to...
     
  32. DCrosby

    DCrosby

    Joined:
    Jan 13, 2013
    Posts:
    86
    No did not work much better, it is making progress but not by much and I don't really know what it's computationally fighting...
     
  33. DCrosby

    DCrosby

    Joined:
    Jan 13, 2013
    Posts:
    86


    This is the final time for the conversion, I left it for 3 days.
     
  34. OnePxl

    OnePxl

    Joined:
    Aug 6, 2012
    Posts:
    307
    o_O;
     
  35. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    I was working on the TerrainComposer and WorldComposer updates that needed to be ready for Unity5. Now I'm working on SnazzyGrid which also needs to be updated for Unity5. Then later this week I will have time to make the tutorial. It's nothing fancy though. The reason why shadow removal takes too much time, because the water pixels are seen as shadow. All you need to do is cut the ocean out in photoshop and replace it by an ocean bottom texture, I think a satellite image from a desert works very well for that. Then you let the shadow removal tool run over that.

    Nathaniel
     
    John-G and OnePxl like this.
  36. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Wow, it should indeed not run that long, what I can see that is wrong is the second color rule, it probably most % of the image pixels into the shadow range. You have to delete that one. The shadow algorithm works fast and good (much faster as photoshop's content aware fill) as long as there's not too many % of shadow pixels in the image. That's why for example it takes too long on islands, and the ocean needs to be replaced, as ocean around it is seen as shadow. Also the color range you choose for shadow removal, the second color is quite to the white and this will also take 2-4x longer. A good (default) grey tint is 57.

    Nathaniel
     
    John-G and OnePxl like this.
  37. DCrosby

    DCrosby

    Joined:
    Jan 13, 2013
    Posts:
    86
    Well it Finished, and the textures look cool, but can you go over in a PDF or a Video, how you integrate World Composer, with Terrain Composer to smooth out the jaggies in the terrain, and then move it over to RTP 3 and generate some of the amazing landscapes you see in the demo videos ? I'm getting to the terrain fairly easily, but I'm having some issues getting up close and seeing what's depicted in the Island demo or others. Also if you're using standard packages or packages of plants or other brush to use with Grass and Trees, that are non-standard. I see you have plenty of Here's how to use Terrain Composer, and World Composer, and RTP videos but I can't really follow the path of what you get from each package that you then don't have to make yourself, and what other issues (smoothing etc..) it presents when you get your data from somewhere else other than a perfect height-map etc... I hope that makes sense.



    Like here, all the terrain is really "Shiny" and the heightmap needs some smoothing, and then I need to get into RTP, and see if I can't make it pretty up-close.
     
    Last edited: Mar 19, 2015
  38. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello,

    As for blockery terrain I explain that in WC's trouble shooting:
    http://www.terraincomposer.com/troubleshooting/
    So this can be corrected in WorldComposer by not changing the heightmap resolution in the 'Create Terrain' tab. If smoothing is still needed you can do it in WC with the 'Smooth' button and TC has it as well in Quick Tools -> Smooth.

    I created a pdf how we made the 24 hour demo using WC/TC and RTP:
    http://www.terraincomposer.com/demos/

    Since TC/WC generate standard Unity terrain, and RTP works on standard Unity terrain, it works the same if you would create the terrains manually from the Unity. So to learn this you need to learn RTP (tutorial video's and documentation).

    Nathaniel
     
  39. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
  40. khos85

    khos85

    Joined:
    Jul 21, 2013
    Posts:
    541
    Hi Nathanial, this looks really useful ! do you need terrains added already or will this tool create terrains into the horizon? Is there a demo or video to show how to use it perhaops, I'd like to take a look at that, or other media you have available.
     
  41. Becoming

    Becoming

    Joined:
    May 19, 2013
    Posts:
    781
    http://forum.unity3d.com/threads/horizon-on-make-your-horizons-look-real.259733/
    This is the forum thread of Horizon[ON], i dont have a video of the setup yet but you can find a lot of information there, the documentation and also a demo that shows it in action. If you still have questions let me know :)
     
  42. khos85

    khos85

    Joined:
    Jul 21, 2013
    Posts:
    541
    Am I the 1st person to play the cool Balloon popping game/comp? What a cool idea :)
    upload_2015-3-20_23-11-30.png

    Wrong thread probably...
     
  43. Becoming

    Becoming

    Joined:
    May 19, 2013
    Posts:
    781
    Hehe, no you are not the first one but you posted in the wrong thread :D see my reply above...
    There have not been many players yet but there are already some pretty impressive scores ;)
     
  44. khos85

    khos85

    Joined:
    Jul 21, 2013
    Posts:
    541
    I just saw that, wow!
     
  45. camel82106

    camel82106

    Joined:
    Jul 2, 2013
    Posts:
    304
    Hello,
    I'm evaluating ArcGis. So I have used ArcMap to interpolate contours to raster.
    Than I have converfed raster to ascii:
    Conversion Tools/From Raster/Raster to ASCII


    Than I was wanting to convert it with WorldCompser to 16bit raw heightmap.
    But I get this result (I open it as 1channel 16bits IBM PC, but still tried other combinations with no good result):


    I'm I doing something wrong?
    Thanks
    Peter
     
  46. eagle555

    eagle555

    Joined:
    Aug 28, 2011
    Posts:
    2,705
    Hello Peter,

    Can you upload the ascii file and send me the link? (Nathaniel_Doldersum@hotmail.com). I will take a look then...

    Nathaniel
     
  47. camel82106

    camel82106

    Joined:
    Jul 2, 2013
    Posts:
    304
    Thanks, You have it in email.

    Nice day
    Peter
     
  48. ibyte

    ibyte

    Joined:
    Aug 14, 2009
    Posts:
    1,047
    Hi Nathaniel, would you happen to know why non square height maps generated by WC don't seem to load properly in to World Machine?
     
  49. DarkRides

    DarkRides

    Joined:
    Mar 6, 2013
    Posts:
    22
    How are you making more overlapping area topography line areas can update the location of the relationship between real time location
     
  50. szlangtue

    szlangtue

    Joined:
    Mar 27, 2015
    Posts:
    9
    to eagle555:
    I'm Chinese,For many years engaged in the development of GIS.I very bullish on WC & TC.The TC and WC demo videos are prohibited in the Chinese.I had to use a network proxy to watch these videos.
    I am a Unity3D beginners.I download TerrainComposer.unitypackage and WorldComposer.unitypackage(version 0.9
    ).First,go to the TC or WC(double click file .unitypackage), TC or WC menu show in Unity3D,but I can't see the map, just like you in those demo videos in such.
    And now,go to the TC or WC(double click file .unitypackage), TC or WC menu can not see in Unity3D.when I import .unitypackage,show error:

    Assets/TerrainComposer/Scripts/MouseLook_tc.cs(65,21): error CS0619: `UnityEngine.Component.rigidbody' is obsolete: `Property rigidbody has been deprecated. Use GetComponent<Rigidbody>() instead. (UnityUpgradable)'

    The version of the Unity3D is the professional version 5.0, a crack version.I does not pay for TC & WC.

    PLS help me.