Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

After playing minecraft...

Discussion in 'General Discussion' started by jc_lvngstn, Oct 8, 2010.

Thread Status:
Not open for further replies.
  1. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Oh okay i thought you still use 6 lists and add that through your Faces function that you call.

    Nevermind then. Well then i really dont get whats the difference, the only thing that comes to my mind is the function that looks for the voxel id in the big array.

    Do you use a simliar approach to beezir, storing the voxel type in a dictionary? He uses it's coordinations of the chunk as key, and then goes through a 16x16x16 int array (i will change that to byte too for memory reasons) and delivers the voxel type. Sounds good to me. Maybe dictionaries are slow? I actually think they are fast but i don't really know. Hm.

    The function that delivers the coordinates is this:

    Code (csharp):
    1.  
    2. public int[,,] getChunk(int x, int y, int z)
    3.     {
    4.         IntCoords coords = new IntCoords(x, y, z);
    5.         if (coords.Equals(m_lastCoords))
    6.             return m_lastData;
    7.  
    8.         m_lastCoords = coords;
    9.  
    10.         if (m_data.ContainsKey(coords))
    11.         {
    12.             m_lastData = m_data[coords];
    13.             return m_lastData;
    14.         }
    15.  
    16.         IntCoords chunkSize = TerrainBrain.chunkSize;
    17.  
    18.         m_lastData = new int[chunkSize.x, chunkSize.y, chunkSize.z];
    19.  
    20.         TerrainBrain tb = TerrainBrain.Instance();
    21.  
    22.         for (int t = 0; t < chunkSize.x; t++)
    23.         {
    24.             for (int u = 0; u < chunkSize.y; u++)
    25.             {
    26.                 for (int v = 0; v < chunkSize.z; v++)
    27.                 {
    28.                     Vector3 loc = new Vector3((float)(x+t), (float)(y+u), (float)(z+v)) / TerrainBrain.noiseMultiplier;
    29.  
    30.                     m_lastData[t, u, v] = tb.getDensity(loc);
    31.                 }
    32.             }
    33.         }
    34.  
    35.         m_data[coords] = m_lastData;
    36.         return m_lastData;
    37.     }
    38.  
    Maybe there is the slowdown, just don't know how i could test this right now. Maybe i'll just let it return the same blocktype for every field to see if it's faster than going through the list.
     
  2. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Hm it doesn't change much, still get like down to 20 or less. Weeeird.
    I'm out of ideas now :/
     
  3. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    I don't have a dictionary, if it's solid it's 1, if air it's zero. I never got around to different materials. That might make sense as to where the slowdown is but it doesn't speed up when you set one blocktype? That is weird.

    Well, I'm off to bed, need to meet a prof tomorrow morning. Should have webplayers up tomorrow afternoon. I'll see about adding more block types to see if that slows it down if I get a chance in the next day or two.
     
    Last edited: Oct 13, 2010
  4. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Doesn't seem so, i just let it return 1 or 0, depending on what size it is at. So it won't have to check the whole array but it didn't change much.
     
  5. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    FPS is much higher in the windows standalone than the web player.

    Controls:

    WASD to walk, space to jump. Right click to dig. Q to toggle cursor hide and lock. If you fall off the edge or dig through the bottom hit E. F toggles the headlamp.

    I get about 60 FPS on the webplayer, between 80 and 1000 fps on the standalone (averages at about 150~200) and each takes about 25 seconds on the nose today. The FPS numbers don't seem to change too much but the load times seem to depend on the time of day and the weather...

    World is static (beyond digging), going to see about generating more as you wander tonight.


    Edit: Almost forgot. Chunks are 16x16x128 and the world is made of 16x16x1 chunks. I might post variants later for comparison but right now I want to get my vertices to triangles ratio down.
     
    Last edited: Oct 13, 2010
  6. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Thanks!
    Yes the FPS are good.
    But i would love to see the dynamic loading, cause that is whats lagging for me.

    What algorithm do you use for that terrain? It looks really nice and interesting.
     
  7. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    I ported the libnoise Perlin code over to C#.

    To use it I generate three sets of noise:
    1. The first one is the base 2D heightfield (I need to scale and translate it up, it tends to dig deep and not go that high)
    2. The second is a multiplier that smooths it out a bit, helps to create some flatter areas. This process needs tweaking, I think I can get a similar effect without generating this second map.
    3. Then I generate the caves using 3d noise and setting any cell where Abs(density(x,y,z)) >= 0.9 to air. This carves out the caverns.

    I zoom the noise different amounts for each pass, seems to create decent looking terrains.

    I think I'll work on getting new terrain to come up as you go, then I'll optimize the mesh builder.

    To generate a single 16x16x128 chunk (including full init) seems to take between 0.05 and 0.09 seconds. Hopefully reusing a chunk will take less time so there isn't a 1 second lag when generating a new edge. I'll hopefully upload something this evening
     
    Last edited: Oct 13, 2010
  8. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Ok cool thanks, i will check that out and see if i can implement it to my project. I liked the structure of your terrain.

    Yes generating a single chunk seems to be fast for me too. But when i have the view range high (100 - 200), it just lets kills the FPS when loading the chunk.
    I guess i will just live with it for now and keep going adding new game features to it.
     
  9. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    Just got inf terrain working and for me it's the noise that's killer. I have some ideas but not sure how well they'll work. Might try going to simplex to test again.

    Minecraft does hiccup when it makes new sections, I think Notch has us at a disadvantage because he can do the generation asynchronously if he wants. I'm not sure how to pull that off with Unity. I'm going to try coarser grained noise, hopefully that won't kill my terrains.

    Edit: Hrrrrmmmmm, it seems to be about 60/40 for noise/mesh gen... When creating new meshes you're stuck, you have to go through the whole array.

    Could coroutines be useful? I've never used them before. I think we need another thread to prep everything for us.
     
    Last edited: Oct 14, 2010
  10. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Hehe welcome to my problem!

    I used coroutines too but they slowed it down even more, dunno why.
    Multithreading would be cool but i don't know much about it, won't people with older machines have lags then?

    But as far as i know this won't work with unity anyway.

    Maybe it would be better to use 16x16x16 chunks (or maybe 32x32x32?) and then only spawn them if you look into the same direction + if they are close enough. Then you could ignore all the layers below you and just create the smaller pieces infront of you.

    EDIT:
    Actually i've just tried minecrafter again. I haven't noticed this before but i get lags there too when new chunks are generating. They are not as bad as mine, but it's lagging nonetheless.

    Also even with the very far fog settings, i still see the chunks pop up.

    Another thing i've noticed, if he really i using 16x16x128 (and that's what i've read on his blog once), and the world height is really only 128 (im pretty sure it is, i once build a tower on water level, it was 64 blocks height) ,then he doesn't create a chunk completely in one loop.

    I just saw a smaller mountain generating, but first his lower half got generated, then his upper half.

    Maybe he is not looping the complete 16x16x128 voxels per chunk in one frame, but rather split it up into maybe 4 frames. I guess that's something i should try.
     
    Last edited: Oct 14, 2010
  11. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    lol, if only we had Unity MPI or OpenCL or something....

    That might be true about the older machines, I guess the problem is that gamers don't all have redoncufast rigs anymore.

    The problem with layering it is if you have a high cliffs but I think you're on to something with improved culling. If there was a way to only generate chunks inside the frustum then this would probably be faster.

    Heh, just tried coroutines and they 'sort of' fix the problem. Instead of a long pause it stutters. I wonder if there's a way to balance out how often new chunks are made with walking speed...

    Edit: I think I found a balance... I'll post a webplayer after a bit of testing.
     
    Last edited: Oct 14, 2010
  12. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    I actually found a good balance now, i get down to like 32 fps when new chunks are generated.

    I wait a frame after every X count when generating the chunk (so all Y and Z for 1 x row gets created in one frame, then it waits a frame) and i also wait a frame after telling one chunk to generate so they won't start generating at the same time.

    Enabling the simplex noise kills the FPS again though hehe.

    Can't wait for your webplayer. Is that with the noise or without?
     
  13. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    With heightfield, multiplier, and 3d cave noise. Without the cave noise it goes really fast, with it stutters. I just realized that I have a bug in the north/south infinite terrain code. Give me a few then I'll post a webplayer with and without caves.
     
  14. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    Caves

    No caves

    Chugs hard with cave noise.

    Lol, I've just seen the effects of race conditions. If you keep going while it's processing the new chunks you can sometimes force it to abandon a chunk which causes a square pillar or hole.
     
    Last edited: Oct 14, 2010
  15. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Nice!

    Yes the cave version is killing the fps too when walking around, especially when walking diagonal.

    But the no caves version is fast, i get over 50 fps almost all the time. Sometimes it goes down to 30 but still great!

    I guess it's my algorithm that slows things down, hm i will have to check that again.

    The hole thing happens to me too sometimes hehe, it's because i only check the front row, not the rows inbetween. I got some nice looking pillars in your example lol.
     
  16. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    Well, starting a new game in minecraft first generates a smaller, immediate area. When you actually get in the game, it still has to draw the whole thing. You can sit there a while and watch it generate it into the distance.
    It also seems to put priority into chunks in your view frustum.

    Regarding threading, from what I understand you can thread all you want in Unity, as long as it's not calling Unity functions. Seems like the terrain density functions would be perfect for this, generating more terrain into the queue to draw. Once you get the whole area generated, the terrain code is "done" generating terrain, though I suppose it could go ahead and draw the line of chunks just outside your area, in anticipation of you moving forward.

    (Edit) I read some more on the threading, and it looks like coroutines run on the same thread as the Unity engine, which maybe why there's not much of an improvement in performance. I need to confirm this tho.

    But, I'm a little bit behind you guys. I'm still working on getting terrain generation going on my end, drawing the meshes, etc. You may have this licked by the time I get there.
     
    Last edited: Oct 14, 2010
  17. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    Thanks Kiyaku.

    Gah, I swapped out the 3d Perlin for Simplex again and it won't load the level. Everything I've read on simplex says it's faster but multiple octaves seem to kill performance for me. I also tried decreasing the octaves of the cave noise from six to three to one, not much of an improvement speedwise, huge drop in visual quality. One octave of simplex is fast but looks weird, big empty spherical hollows every now and then, like bubbles or something.

    The way I generate new chunks is really naive so I'm thinking that there might be a way to improve performance (at least for diagonal movement) if I don't completely do both edges. It'll be harder to keep track of things because I won't have a square terrain anymore but if it works...

    I think you might be onto somethere there jc, I'd have to check minecraft to be sure but I think that it might generate a chunks outside the viewing range when you load it up, then it starts to stutter when you get outside of the loaded 'safe zone.' Not sure though, just a feeling, I'll need to play it a bit to be sure.

    Heh, if a coroutine can start a coroutine then I can do that on the cave noise... but then there're even more threads and the delay would mean I'd probably get the pillar/hole thing happening more because of movement speed. Hrmph. I think I'm fried for the night.

    I'm going to do some reading on 3D coherent noise tomorrow, there might be a new algorithm or a lesser known one that is fast but gives decent results.

    If we could integrate OpenCL code this would be cake. Night guys.
     
  18. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    I'm having some difficulty getting the noise functions to create meaningful data...for one, the code for the noise generator doesn't appear to use meaningful variable names...so I'm not only unsure of what range to expect as a return value, but also not sure how to 'adjust' phases and such in the noise.

    Anyone have any documentation on this?

    (EDIT) I switched back to perlin, and decided to just generate it like a heightmap first. Just got to get a few visibility bugs addressed...
     
    Last edited: Oct 14, 2010
  19. andeeeee

    andeeeee

    Joined:
    Jul 19, 2005
    Posts:
    8,768
    The algorithmic complexity of simplex versus classic Perlin noise won't make much difference for three dimensional noise. Classic noise is O(2 ^ n) while simplex noise is O(n ^ 2), so simplex is actually more "complex" in 3D (ie, 2 ^ 3 = 8 for classic, 3 ^ 2 = 9 for simplex). Your classic noise code might actually be faster purely because it's an older technique and more optimisation tricks may be in widespread use for it than for simplex noise.
     
  20. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    Yes, Perlin seems to be working well for me now.
    There are so many things I want to do with this...something that would be helpful for me is the ability to tweak different values (frequency, octaves, etc) and visually see the impact "on the fly" to my terrain generation.
    for now, I'm excited that I can generate a nice chunk of terrain and handle visibility.

    And then there's the caves.
    And then resource distribution.
     
  21. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
  22. beezir

    beezir

    Joined:
    Apr 3, 2009
    Posts:
    133
    Ah, I just skimmed over the "faster for higher dimensions" and didn't pay attention to the actual complexity. If you want 4d or higher Simplex, you'll start to see an advantage over perlin. Probably better for things like clouds with a time-based component.
     
  23. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    @Taintspore, thanks for the links!.

    Also, I'm curious about the performance issues you have with caves caverns. I'm generating a 32x32x32 chunk of blocks using perlin, with a mesh, and it's taking about 0.03 seconds. Is there something in particular about the cave stuff that is a lot slower? I just got my perlin landscape going this morning, so I have yet to put in caves and am curious.
     
  24. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    No problem, I found simplex demystified had a good, easy to follow explanation of what was going on.

    Generating just the heightfield is quick but it only needs to compute XxZ cells. The caves slow it down because it needs to compute every cell at or below the terrain level so on average about XxZxY/2 cells. In my case that means it goes from 16*16 for the height map to 16*16*64 for cave noise per chunk. I would like to have a 32x32 chunk world but that means that (I'll use your time of 0.03s) 0.3*(64 *32) = 61.44 seconds to generate the new terrain as you move forward. That's approximate but it gives an idea.

    If another thread could be used to just do noise then I think the problem would go away, or at least be diminished somewhat.
     
  25. beezir

    beezir

    Joined:
    Apr 3, 2009
    Posts:
    133
    I haven't played with it, but does Unity allow multi-threading as long as you don't use any unity-specific calls? If so, you might be able to use some sort of lock-free queue that could be checked per-frame to see if data is ready to be used.
     
  26. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    I've looked around the forums and unity answers and I found out that unity isn't thread safe so no API calls but it from what I understood you could thread all you liked as long as you keep the thread safety in mind. That sounds like it would work, but I have limited knowledge on multithreading so I'm not sure. I could generate noise easily with a cluster though...
     
    Last edited: Oct 14, 2010
  27. beezir

    beezir

    Joined:
    Apr 3, 2009
    Posts:
    133
    On second thought, lock-free might not work well here since it needs a CAS instruction like Interlocked.Exchange which I'd guess is either non-portable or non-accessible in Unity's mono - I'm not at my development computer at the moment to check. But basic multi-threading could still be done pretty easily as long as you make sure the secondary thread is only doing noise processing.
     
  28. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    The mesh generator is quick enough to use with coroutines (or even without, a dig op forces a regen without a hit the fps even without a coroutine), so all the other thread would really need to do is the noise calculation.

    Other than knowing that another thread is needed/useful I'm out of my comfort zone. Looks like I've got some reading to do.

    I'm also going to experiment with 3d value noise. If it's much faster without too much of a quality hit, it might make a decent replacement for 3d perlin. I'm not getting my hopes up but you never know.


    Unrelated quickie: anyone from this thread going to Unite?
     
    Last edited: Oct 14, 2010
  29. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
  30. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    Just tried coherent value noise and it didn't go so well, slower than perlin. Looks like it'll come down to threading.
     
  31. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    I started adding multithreading to read out the simplex noise infos. So far it's quite good but i didn't optimize it yet at all. Right now i call it for every chunk, i guess i can change it so i can call it for all chunks everytime i move into a new chunk, but we will see.

    My frames keep over 60 now with that method. I still have a problem placing the right mesh and collision right now but that has nothing to do with the multithreading, just a problem with converting my positions ^^.

    I guess i will investigate that a bit more and see how fast i can make it.
     
  32. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    I should have something similar in this weekend. The approach I would like to take is upon launch, I would queue up the chunks closest to the player, and move outward from there. The initial chunks that are queued up will just be the immediate visible area to the player. Then, once the world is player area is loaded, as the player moves and new chunks need to be drawn, the old ones would be dropped and the new ones would be queued up.
    Another thread would be responsible for reading the chunk processing queue, generating the block data and the chunk mesh, and flagging the finished chunk as "done". I'm interested to see how multiple threads help with the chunk processing, like having multiple threads each process chunks, or even try something fancy like having multiple threads process a single chunk at a time.

    The system has to be able to generate two (for possible diagonal movement) rows or columns of chunks faster than the player can cross a chunk. My perlin and mesh creating code takes about .03 seconds, even if adding caves/whatever bumped that up to .1 seconds per chunk, that would mean I would have 3.2 seconds to generate two rows/columns. I don't plan on implementing rocket boots for the player, so that should be fine :)
    All theory, of course...I still haven't decided how "deep" my world is going to be.
     
  33. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    On a side note...should we move this discussion to the collaboration forum?
     
  34. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    @Kiayaku
    That's awesome! Does the web player allow for multiple threads?

    @JC
    That sons like a good idea, do we start a new one or do we ask a mod to move it.


    Since I'm new to multithreading, where should I start? Know any sites that have good info for noobs?

    Edit: found the mono threading beginner guide, looks simple enough, looks similar to using MPI on a cluster. I'll have a go at it today.
     
    Last edited: Oct 15, 2010
  35. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    Don't forget, if you are using unity 3 you have support for .net 3.5 and below. Hopefully, I'll have more time to work with the threading stuff this weekend, if this nasty cold will back off :(
     
  36. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    @Taintspore,
    haven't tired webplayer yet, will do that!

    @jc_lvngstn,
    why start a thread in collaboration? I think this fits quite well into gossip as it's not one project we are starting, but several smaller ones lol.
    And basically just talking about how we can create it. A Game Design/Theory category on the forum could be nice.

    I'm new to threading too. I just create a new thread with "Thead newThread = new Thread(new Threading(MyThreadMethod));" and then start it using "newThread.Start();".
    the first time i did a 256x16x128x16 calculation inside a thread i was stunned how it spit out "start thread" and "end thread" in the same second lol.

    BTW if the thread is done, you have to create it again using the newThread = new Thread(...)" stuff, before calling Start() on it again.
    I think i will change that so i only have one function and can use pause or a bool to repeat it.
     
  37. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    I'd suggest looking at the ThreadPool class. With Unity 3, you should be able to do something like this:

    private readonly TerrainChunk[,,] m_Map = new TerrainChunk[WidthInChunks,HeightInChunks,DepthInChunks];

    private Queue<TerrainChunk> m_TerrainChunksToBeProcessed = new Queue<TerrainChunk>();

    public void BeginProcessing()
    {
    ThreadPool.QueueUserWorkItem(state =>
    {
    while (true)
    {
    if (m_TerrainChunksToBeProcessed.Count > 0)
    {
    TerrainChunk chunk = m_TerrainChunksToBeProcessed.Dequeue();
    GenerateTerrainForChunk(chunk);
    }

    }
    });

    m_TerrainChunksToBeProcessed.Enqueue(m_Map[0, 0, 0]);
    }

    private void GenerateTerrainForChunk(TerrainChunk chunk)
    {
    // Create terrain and mesh for the chunk, activate it, etc
    }

    the only difference is, you would have to use a thread safe queue, I believe an example of one was in one of the links I posted on the previous page.

    Anyway, the above should create a seperate thread whole sole purpose is to constantly monitor the queue, and if chunks to be processed are added it immediately generates the terrain for them.
    Hopefully sometime this weekend I can actually give this a shot.

    Oh, and as far as taking it to the collaboration forum..I dunno, it seems like that is where people would go to discuss unity projects, coding algorithms, etc. Though we aren't working on the same project...we sort of are, we just have our own.

    But, I do think a game design/theory discussion forum would be awesome.
     
    Last edited: Oct 15, 2010
  38. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    So this would allow to call Unity API functions as well, cause its a thread safe queue?
    That would be nice!
     
  39. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    Actually...I think we can't call the Unity functions from other threads, but I haven't confirmed that. So, the mesh generation would have to happen back on the main thread if this is the case.
    But I don't see why terrain generation and visibility couldn't happen in other threads.
     
  40. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    According to the ms docs the queue is thread safe.

    I'm still confused about how much of .NET mono implements. I've seen something about "everything but the foundation classes." I'm not too familiar with .NET yet so I'm not sure what that means. Most of my .net exposure is actually mono.
     
  41. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    According to this if you set the start delegate to be from an object instance that already exists then they should share variables right? So couldn't you just start a thread for each chunk as you go and the each new thread would update the corresponding chunk's data? I know starting a tonne of threads isn't a good idea, but I'm curious. Can't get it to work though but I'm not sure I'm doing any of it right as it is :p


    Edit: does "no unity functions" include stuff in Mathf?
     
    Last edited: Oct 16, 2010
  42. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    Dunno about Mathf but you could still use System.Math if you need to.

    I just created 2 theads now.

    Everytime i move around and need new chunks, i add them to my queue (i had that system before from beezir already).

    Then a while checks if there are new entries in that queue, if so, it starts the first thread that fills the array with the new simplex noise infos. This is pretty fast. After that, it calls the second thread that generates the lists for vertices, uvs and faces for the chunk.
    If it's done, it will call the chunks regenerateMesh method and delivers the new created vertices etc list to. So all it needs to do is apply those lists to the mesh.

    When i start the game, it takes a few seconds to generate the array with the simplex noise. But because its a second thread, there is no FPS drop at all. So you can just put in the "loading bar" at this moment. After that, when it generates the 16x16x1 chunks, there is no FPS drop either.

    I just get a fps drop when i walk around and add new chunks. Still have to figure out why cause it uses the same technique at the start where it runs fast.

    Another bug i have is that when i just switch to a new row, and it has to generate the chunks, and i remove a block from another chunk in the same moment, you need to wait for the new row to be completed before you see the block disappearing. So i guess i have to add some kind of priority, or a 3rd and 4th thread extra for removing/placing blocks.

    EDIT:
    As i really don't know much about threading, is there some kind of limit for threads? Like how many thread should you call max. at the same time?
    I always thought that one thread runs on one CPU so you would need at least dual core to use that but i guess that's wrong lol.
    I should read more about that topic.
     
    Last edited: Oct 16, 2010
  43. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    Here's what I have so far. It's in Unity 3.0.

    http://www.filefactory.com/file/b3f8fgd/n/MineClone.unitypackage

    Let me know if you have any issues downloading it, I just setup an account on filefactory, seemed like a good file hosting site.
    If it runs (hah), you'll notice that the chunks have gaps in them. I haven't put in the border block/visibility checking in yet.

    But, I do have some threading going on. The world.cs file is the starting point, it has a NumberOfChunkProcessors or something that declares how many terrain processors you want to use. Each terrain chunk processor fires up a thread that monitors its own queue.

    The processor polls its work queue for chunks to process. It generates the terrain using perlin noise, and also uses visiblity tests to generate the vertex, color, and uv lists and such, for later drawing. I played around with changing the number of processors, if I had 20 running I noticed some slowdown. Your mileage may vary :) I think ideally, the game would initially assign about 10-15 terrain processors and after the initial load, remove all but one or two.

    the Update function in world.cs actually pulls the finished terrain chunks and instantiates the prefabs.
    It's pretty basic right now.

    I may try using the .net queue, but it has to be public and static...so there will only be one instance, for all of the threads to pull from, which should be fine.
    My approach in general was to create seperate processors, with their own queues and threads, and the world.cs file just submits the terrain chunks. Honestly...I still expected it to run faster than it does. Tomorrow maybe I can do some optimizing.
    Any suggestions/comments/curses welcome :)
     
  44. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    @Kiyaku

    From Threading in C#
    When I tried threading it earlier tonight I got an error that IIRC said "gc error: too many threads"

    My non threaded code doesn't use a queue so I think I have a fair amount of rewriting to do. Well, maybe not, maybe just my terrain manager.

    If a cpu has hyperthreading it's using a technique called Simultaneous Multithreading. Basically, a cpu can only do a one operation of type x (say a memory op, floating point, fixed point, etc) per tick but with simultaneous multithreading it can do two. That way two threads can be run at once even if they want to do the same kind of operation.

    Not sure about amd but if you have a HT intel chip and you look at your resource manager you'll see double the amount of cores than you actually have. I have a quadcore but windows shows me eight cores.

    I wonder why it slows down? With coroutines (no cave noise) sometimes it would start to chug repeatedly on new chunks after running for a while. Maybe it's related to starting new threads?

    @JC

    Awesome thanks man! I'm having issues getting results back from threads so having a working example should help out a tonne. I'll play with it and let you know how it goes.

    Unrelated: What do you guys use for source control?

    Edit:
    @JC

    Download worked fine. With stock code/settings I get about 8.5 fps until it's done, then it takes off and has a nice (+150) frame rate. After looking at your implementation I don't think I have that much work to do to get a queue in there, not sure why I figured it would be such a pain anyway (I blame it on continuously passing out on the couch at weird times every night this week).
     
    Last edited: Oct 16, 2010
  45. CaptainKiyaku

    CaptainKiyaku

    Joined:
    Feb 8, 2009
    Posts:
    324
    @JC,
    great im downloading your package right now to try it out and then comment on it later.

    @taintspore,
    That's actually quite a huge number of threads you can have, thanks a lot for that info!
    The profiler showed me that the "UpdateBody" coroutine, that goes through my queue, slows down the engine while generating new rows. Sometimes its just hard to see in the profiler what exactly is causing it. It also shows huge spikes of Physics calls for MeshColliders, maybe that's the hickup for me? I don't know.
    I just can't see why the beginning is fast (when it places the starting area) but after that its slow when generating rows. Maybe its because at the beginning i first fill the blocktype array completely before starting to generate the chunks. Later on i do it after each other per chunk, like add chunk -> fill array of blocktypes with new chunk -> generate mesh -> add chunk -> fill array of blocktypes -> generate mesh -> etc.

    I will play around with it a bit.

    But yeah adding Queues is pretty easy and i just heard about them when i got Beezirs code so i guess ill use them more often now, they are really nice.

    EDIT:
    I forgot to answer your source control question, i use TortoiseSVN with my own server.
     
    Last edited: Oct 16, 2010
  46. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    Hope the sample is useful...even though I'm still in the early stages.
    Right now, I'm not using any source control...just backups

    I'm going to try to get more work done this weekend. My immediate goals are:
    1. Remove the gaps at chunk boundaries
    2. Improve performance in terrain generation overall

    I was being conservative with the initial values for the number of threads...I bumped it up to 50. It gets done faster...but slows down the framerate considerably.

    I think I'm going to have to change my design a little...I think I need to generate the terrain all in one pass, then do the visibility checks in another.
     
    Last edited: Oct 16, 2010
  47. beezir

    beezir

    Joined:
    Apr 3, 2009
    Posts:
    133
    If all the threads stay busy all the time, you generally want one per CPU (possibly two for Intel Hyperthreading). If they're not busy all the time, it's usually fine to have several dozen without much issue. At some point the thread switching and locks involved hurt performance more than they help.
     
  48. Taintspore

    Taintspore

    Joined:
    Sep 29, 2009
    Posts:
    185
    @JC

    I managed to partially integrate your threading method with my code. I've got some (read: tonnes) issues with world to terrain to chunk coordinate spaces now but those are definitely fixable. When I manage to get it working right I'll post a player.

    Edit:

    Basic threading done. Thanks a tonne JC, your example code was really helpful. I haven't hooked up the code that handles neighbours (ie: face visibility on chunk edges) and the mesh generator is still procedural. If you fall through the terrain when it's generating at the beginning just hit e and you should end up above it again.

    The infinite terrain is still procedurally created but I'm going to hook that up to the threading right away here. Now that the basic framework is in place hopefully it'll go more smoothly.

    Linky

    Swapped out the player for one that seems to go a bit quicker.


    Update:

    I stuck the mesh generation into the other thread. When the mesh lists go into the queue (inside the chunk) they're full but when the chunk is picked up by the original thread the lists are empty. Not sure why, I'll keep poking around.
     
    Last edited: Oct 17, 2010
  49. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    That's awesome, Taintspore. I had hope to make more progress..sadly, my cold prevented me from doing anything useful today.

    I have decided to generate all of the terrain first, with threads processing the chunks, and only then move on to visibilty. Before, I had threads trying to do both at the same time...and it complicated some things, and I've decided it isn't even necessary.

    I do think I'll have a cleaner design, also.
     
  50. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,508
    By the way...I think there are some basics when it comes to mesh creation that I don't understand.

    For each side of the cube that I create, I provide a shade value. The top side gets full brightness, the sides get less, and the bottom the least. I provide then when creating the colors list for the mesh.
    But this doesn't appear to be happening. Could anyone provide any insight or suggestions on why this is? I'll also try to find more reading info on this.

    Also, I've made more process. I redesigned my threading so that terrains are generated by threads, then when all terrains are generated mesh creation and visibility is handled by threads. I think it's a much better design.

    The terrain and mesh generation all happen pretty much in a second or less. It's the prefab instantiation part that is taking the other 14 seconds or so. I can knock off a few seconds by removing the mesh optimization, which I don't seem to need anyway.

    One other thing...I noticed I was getting gaps in my mesh creation, some chunks weren't being processed correctly. I switch back to the threadsafe queue, and the problem went away. Just something to keep in mind.
     
    Last edited: Oct 17, 2010
Thread Status:
Not open for further replies.