[RELEASED] SECTR STREAM: Seamless Scene Streaming

Cool. I'll put it on my backlog and try to get to it sooner rather than later.

PS - all licenses for editor extensions in the asset store are licensed as one seat per user, not per team, so you should technically buy one seat for each user on the project. I'm not saying this to call you out, just to point out a common misconception. Also, other licensees may think differently, but I'm totally fine for folks to buy one license early in dev and then buy the rest if/when their game actually is funded or ships or otherwise starts generating revenue.

 

Great to hear that you'll be adding the feature. Regarding the licensing, I understand about one seat per user. It's just that currently our project has 3 people working on it, with me solely in-charge of the level building aspect, thus the need of 1 license at this moment (we did acquire 3 uScript licences, however). Should we involve more people down the road, I'll definitely purchase additional licenses.

 

Sounds very interesting, especially with the Madness Sale going on ;-)

I would be especially interested in using this for large (multi-) terrains. From the asset's description I understand that you're doing something in this regard, but couldn't figure out exactly what, therefore

1) Is your mentioned Terrain Sectorizer script already in the current release, or will that come out with V1.1?
2) Does it work by splitting one single terrain into chunks, or does it also set up sectors and portals for multi-terrains (e.g. a grid created with TerrainComposer)?
3) To avoid jitterung etc. I will probably have to regularly reset the camera/player to the origin, instead shifting the terrains and everything else around them. Would that interfere with Sectr Stream?

Thanks!

 

Thanks for the interest!

1) The current release has the Terrain tool in it. I just took it out of beta.

2) It will make Sectors and Portals for a single terrain and can split that terrain if you like. If you split the terrain, it will attach the edges (like how Terrain composer works). That tool doesn't currently know how to sectorize terrain grids, but I'll add that to my backlog. Shouldn't be hard.

3) This shift won't affect the most interesting parts of SECTR stream. However, the exported data is in world space and would either need to be shifted post-load (you can do this in a callback) or you'd want to make some tweaks to the core export and load logic. It's a pretty straightforward change, but you might want to email support@makecodenow.com once you get deeper in, especially if you run into issues.

 

FYI that SECTR 1.0.8 is live now. It fixes the STREAM import bug introduced in 1.0.7.

 

SECTR 1.0.9 is out! Major improvements include better terrain sectorization and streaming (including Terrain Composer support) and a pretty big optimization to streaming performance.

SECTR 1.0.9 Release Notes
CORE
* Terrain window supports automatic Sectorization of Terrain Composer created terrain grids.
* Fix crash splitting terrains with trees in the Terrain window.
* Sectors no longer include Particle Systems in their bounds computation as they are too dynamic to be reliable. Also fixes issues where sometimes Sector would expand to include the origin due to a particle returning an invalid Bounds.
* Custom inspector class updated to more perfectly match behavior of default inspector, including support serializable, private, non-hide-in-inspector variables.
* Updated documentation with more information about advanced bounds logic and the new portal determined membership mode.


STREAM
* Major optimization of Chunk loading, reducing hitches by spreading out CPU heavy work over several frames. Requires a re-export of Chunks to see benefits of optimization.
* Support for streaming terrains made in Terrain Composer.
* Add Region Loader to menus and documentation. Ideal for open world streaming.
* Hibernator component can hide renderers during hibernation.
* Fixed bug in Stream window import/export where components on the Sector were not copied correctly.
* Fixed bug in Stream window export for 1x1 special case.

 

Hello. Sectr Stream looks like a great solution, might not be suitable for my current project but sill, thanks for making it available.

I have a question. My game uses terrains made by terrain composer and lots of gameObjects for building blocks and characters. Both blocks and characters can be moved, modifeid, created and destroyed by player or npc interactions. They will be moving from one chunk to another for sure when surrounded chunks are active. Are those changes in chunks handled(and saved) by SectrStream or do you have to make them global gameObject(as described in the manual) ? Basically, SectrStream only takes care of loading static stuff(terrain, decoration, static buildings etc) in chunks?

 

Good questions! SECTR STREAM only handles the saving and loading of the static parts of the scene. However, it has Events that you can hook into that will be called when a Sector is loaded or unloaded. You could use these callbacks to save off whatever state you want and to restore that state after the load. Or, as you say, you could make those dynamic objects global, possibly using the included Hibernator component to shut them down when their part of the world is unloaded.

 

Alright, thank you for clearing that up. Keep up the good work

 

SECTR 1.0.10 is out. Includes bug fixes as well as a preview of an exciting new Chunk Proxy Mesh feature. For folks doing open world games, check out the new Chunk Inspector and the docs in the Stream Manual and lmk what you think!

SECTR 1.0.10 Release Notes
CORE
* Added documentation about working with Terrain Window to CORE Manual.
* Fix bug where missing null check in Member could cause crashes on some platforms (like iOS).
* Fix bug where results from Portal’s viewport Sector selection/swap GUI would be erased in play mode when performed on a Portal in a Prefab when the user pressed Play.


STREAM
* Beta implementation of Chunk Proxy, which is a mesh that will be displayed when the Chunk is unloaded. Useful for some open world games. Includes an Inspector Tool that will create custom Proxy meshes out of a selection of Chunk members.
* Modified Stream Window exporter to center exported chunks at origin and to adjust them on import. This makes it possible for multiple Chunks to reference the same scene file.

 

How does the region loader work. It looks like it adds a box-like collider around the player. So when this box touches a terrain portal, it begins to load the objects in the portal? And, if he walks back and it leaves the terrain portal it just touched, would it start unloading it? Is that how it works?

Or is there a middle area that after it loads it unloads after moving back a certain distance? I've been testing on some devices, I have a few spots that lag pretty good and I am wondering if it is because I am going in and out of these terrain portals or if its something else. If it does load like just touching like a trigger, then at a two or even a four terrain intersection, it could potentially start loading up three terrains, and then unload them just by taking a few steps back.

Would you be able to clarify how the region load works?

 

Region Loader is simple. It just loads all of the Sectors that overlap the bounds of the Region Loader (and which also pass a Layer test). You're right that it can be jarring if you move back and forth right over a boundary. I'll add some hysteresis so that it isn't as sensitive to that kind of thing.

PS - apologies for a slow reply. Wife and I had a baby a couple of weeks ago. Things have been hectic

 

Yes. That buffer area would be very beneficial. I threw a debug.log on OnEnable() on a script in an area to be loaded. And sometimes just walking into the new Sectr area will trigger the log two or three times.

Looking forward to it thanks!

 

Just an FYI that this is done and will be in the next update, which should be out this week. I'm working on one bug that's new to 4.5 and then I'll be uploading everything.

 

SECTR 1.0.11 is now out. Major fixes include seams between terrains when streaming or splitting, the requested unload buffer for Region Loader, and some UI polish in the Stream Window.

SECTR 1.0.11 Release Notes
CORE
* Sectors created by the Terrain window are now parented above the Terrain rather than being added to the same object. Fixes issues with certain 3rd party scripts expecting to be on the same object as the terrain.
* Fixed seams appearing between terrains when using the split feature of the Terrain Window.
* Guard against null lights in the baked only lights table.
* Fixed PlayMaker integration compile issues in Unity 4.5.


STREAM
* Added UnloadBuffer to Region Loader so that there is some hysteresis between load and unload (i.e. less popping when moving small amounts back and forth).
* Added Import/Export/Revert buttons to Chunk component inspector (for convenience).
* Added Revert All button to Stream window.
* Fixed seams when streaming terrain sectors created in Terrain Window.
* Fixed PlayMaker integration compile issues in Unity 4.5.

 

Awesome. Just updated, but haven't added it to my project yet. I've heavily used the prior version. Are there any gotchas or anything that might break any setups in the prior version? Should I have all the Sectrs imported before I update?

 

Everything should be fully backwards compatible. The only major change for former users is that Culler (in VIS) is now deprecated, but even that is backwards compatible. Lmk if you have any issues!

 

Small bugfix update just released.

SECTR 1.0.11a Release Notes
STREAM
* Fix null deref on shutdown bug in SECTR_Chunk.

 

Hi MakeCodeNow, I have watched the videos and read the manuals for this and it looks like a superb asset. I am wondering how I could use this in my project and would appreciate some input from you if you have the time.

The game I am working on is a modern urban environment with city streets and a surrounding forest landscape (using a terrain). The majority of the city buildings feature multi level interiors, therefore it is a combination of an indoor and outdoor environment. An important part of the game is being able to move from exterior to building interior (and vice versa) via any doors or windows, similar to how chase breakers worked in AC3:

(Window)

(Door)

Therefore it will be common to move from the exterior to the interior back and forth. Another aspect is the fact that when within the interior it will be possible to see outside through the open windows/doors, and of course when outside it will be possible to see through an open door/window into a building. Because of this I am wondering how I could use SECTR Stream or SECTR Viz (or both) in my particular project. For example if the player is standing inside a building and their camera is angled to see through an open window they should be able to see the other buildings outside and the forest landscape in the distance, not have these objects culled or unloaded. I suppose it could work by setting only interior assets to use Stream, and have all exterior assets use an LOD and occlusion culling system.

So I would appreciate your thoughts on whether this system would be beneficial in my particular case. It looks like a great asset so i look forward to your advice.

 

I think something like SECTR would help a lot in this kind of game. It's very much what it was designed for.

The basic idea would be to use a grid of sectors for your terrain. You can use these for your city or you can create one per block of neighborhood or the like. Lastly, you want to have one or more sectors for each interior. You create portals at windows and doorways to link the building interiors with the sectors outside.

From there, you can use streaming, occlusion culling, LOD, or a combination. SECTR supports all of them, and the right mix depends on your game. At minimum though you probably want to have interiors stream in when you get close and stream out when you move away from the building.

 

Interesting, thank you for answering my questions, if you don't mind I have a few more:

1) When creating Portals is it only possible to do so using the built-in drawing tool, or can for example a planar mesh be used to define the shape of a portal. I ask because in my case there are many windows and doors that would need to be "portalised" and so I would need an efficient way to do this, such as making 1 portal and then duplicating and adjusting it.

2) Is it possible to use Asset Bundles with Sectr Stream & Viz? For example if I would like to have a minimal build, which would then stream in content from an external source, could this streamed content be controlled with Sectr Stream and Viz?
2.1) Going on from the previous question, is there a built in way to handle the caching of external data, so that for example when content is unloaded so that it doesn't have to be restreamed again from an external source, but simply gets loaded from the cache.

3) If there some sort of fail safe in Sectr Stream for situations when a particular sector(s) takes a long time to load? For example if a player is entering a room, what stops them from falling through the floor before the room (and its floor) is loaded?

 

Good questions!

1) Any planar, convex mesh will work for portals. The included drawing tool is just a convenience.

2) VIS doesn't care about asset bundles, it just operates on whatever is loaded. STREAM loads from scene files, which should work if those scenes were downloaded in asset bundles created by Unitys make bundle from scene tech.

STREAM does not include the tech to download/cache/version asset bundles. Maybe some day, but there is a lot of work for something that would work on all games, and it would be pro only. If you build it, though, STREAM should be fine as long as the chunk scene is there when its time to be loaded.

3) No, there's no fail safe as there is no generally correct solution for that problem. The right way to hide latency is very game dependent, so I leave it up to you. Note that you can easily check to see if a Sector is loaded, so that is easy at least.

 

Thanks for that, your answers are really helping me understand how this works.

For (3) I was more focussed on the loading of colliders rather than controlling/minimising the visual pop of when assets take longer to load. For example take a simple building whose exterior and interior are loaded separately, the interior only loads when the player is close enough to the building or enters the door. Say the player walks through the building's exterior doorway and now finds himself/herself inside the building; while the interior loads the player can theoretically roam around anywhere inside the building. However what if when the interior loads the player happens to be positioned exactly where a staircase is supposed to be, wouldn't the player now be stuck inside the staircase's collider? How would this work exactly? The only solution I can think of is to have colliders and meshes controlled separately, and only allow passage to a sector if its colliders have been loaded. Since colliders should be much faster to load then entire meshes with textures etc this would theoretically improve the performance as well and not require loading screens for this.

4) I have another question regarding the terrain splitting feature. I have read the manual but I would like a few more details on how it operates. For example, when an existing terrain with splats, trees etc is split using Sectr Core, does each part physically become a separate terrain (eg. 1 terrain split in half becomes 2 terrains), or is it only "split" for the purposes of Sectr Stream loading? Further, for objects positioned right on the divide line where the terrain is split do these objects get deleted or how does the system know which terrain part the objects should go on. This is important for me because if for example I have a detailed forest, and I decide to split its terrain I wouldn't want all of the objects, plants, trees etc positioned on the terrain join to be deleted.

I look forward to your response.

 

3) Generally the best solution is to ensure the player can't get into unloaded areas. Don't allow the door to open or window to break until the data on the other side is loaded. STREAM includes a door component that does exactly that. If you can't make that gratantee, then you need to get extra clever, separating collision into its own sectors or keeping some of it global or whatever. Again, there is no universally correct approach.

4) It is split into different terrains, much like what you would get if you used terrain composer (which I also support). Objects that straddle generated terrain sectors are left global so that the user can decide which Sector (if any) they should go into. They certainly will never be deleted as SECTR tries very hard to be nondestructive.

 

This sounds great, once I purchase Unity Pro I will definitely be purchasing this!

 

Awesome!

 

Sectr is causing my app to crash when loading scenes.

Explaining my setup;

2kmx2km map, most of the map isn't a sector and gets loaded in beginning. There are 4 sectors in corners of the map. Sectors have buildings, roads and some other props.

Player's vehicle has a regional loader.

Everything seems to work fine. When level begins, if player is in one of those corners it loads that sectr very well. And if player moves, it destroys/loads those sectors perfectly fine.

But, only sometimes, when restartlevel or nextlevel is clicked, and it loads same level with application.loadlevel(1), it crashes game with out of memory error ( I can see this in visual studio).

Note that, next level also opens same scene, but modifies somethings that it changes level with spawning player at a different location and some other things.

I am having this problem since 3 weeks, and just found that if I import all chunks and disable regional loader, it doesn't crash anytime.

There is definitely a really bad glitch with sector. I can guess there may be some problem with unloading chunks or something when loading same level.

I need help with this as soon as possible. Note that I am building for windows phone and 512mb devices, there is 180mb commited memory limit. And right now I've set up my game so that it uses less than 180mb with or without using sectr. I've Imported every chunk in the scene and it still has less than 180mb usage. But if enable sectr and export scenes, it crashes with this error:

Spoiler: Error

Unloading 21 unused Assets to reduce memory usage. Loaded Objects now: 44359. Operation took 513.875244 ms.

System memory in use: 51.5 MB.

The thread 0xcd4 has exited with code 259 (0x103).
The thread 0x4b8 has exited with code 259 (0x103).
The thread 0xeb4 has exited with code 259 (0x103).
Could not allocate memory: System out of memory!
Trying to allocate: 502476B with 4 alignment. MemoryLabel: Serialization
Allocation happend at: Line:334 in C:\BuildAgent\work\d63dfc6385190b60\Runtime/Utilities/dynamic_array.h
Memory overview


[ ALLOC_DEFAULT ] used: 33482288B | peak: 34284908B | reserved: 37163921B

[ ALLOC_GAMEOBJECT ] used: 18374996B | peak: 18563951B | reserved: 21665560B

[ ALLOC_GFX ] used: 21451373B | peak: 21451373B | reserved: 21558806B

[ ALLOC_PROFILER ] used: 767592B | peak: 767592B | reserved: 2827863B

Could not allocate memory: System out of memory!
Trying to allocate: 502476B with 4 alignment. MemoryLabel: Serialization
Allocation happend at: Line:334 in C:\BuildAgent\work\d63dfc6385190b60\Runtime/Utilities/dynamic_array.h
Memory overview


[ ALLOC_DEFAULT ] used: 33482288B | peak: 34284908B | reserved: 37163921B

[ ALLOC_GAMEOBJECT ] used: 18374996B | peak: 18563951B | reserved: 21665560B

[ ALLOC_GFX ] used: 21451373B | peak: 21451373B | reserved: 21558806B

[ ALLOC_PROFILER ] used: 767592B | peak: 767592B | reserved: 2827863B


(Filename: C:/BuildAgent/work/d63dfc6385190b60/Runtime/Allocator/MemoryManager.cpp Line: 909)


The program '[3724] TaskHost.exe' has exited with code -2147483645 (0x80000003).

 

Thanks for the info. There's clearly a bug here, either in SECTR or the underlying Unity Async code. I'll try to help you get to the bottom of this as best I can.

My first guess is that there is a bug in Unity that happens if you call LoadLevel while there are some in process LoadLevelAdditiveAsync calls. If so, that would explain why the bug is only sometimes. If so, that's a Unity bug but we can try a workaround.

The workaround is when you want to load a new level, first disable the region loader. This will start unloading everything thats loaded. Next, wait a bit. Start by waiting a crazy amount of time, like 10 seconds. Then do you your load level. If that works, you can probably wait much less time, like 1 second.

Give that a shot and lmk what happens.

 

Well I am working with unity free. I don't think I have anything being loaded async.

Should i still do that test you've suggested?

 

Hmmm. That's strange indeed. Let's give it a shot anyway if only to see if the unload before load helps at all.

 

Well it just crashed with sectr disabled. It might not have any connections with sectr at all. But it was crashing more often with sectr enabled. It could have reduced memory cost and make it crash less often. I will test it and post here.

 

Keep me posted. The crash looked like it might be a fragmentation issue as it died making a pretty big alloc. I bet SECTR just makes it happen soon/more often because it makes more allocations so the problem surfaces sooner.

You might try to run a GC before level load.

 

I actually just built an executable and had my game crash with out of memory errors while running SECTR Stream. Interesting considering the scene with all the objects loaded + more actually never crashes out of memory, only when I split up the terrain into Sectors and start trying to stream them does it run out of memory. This happens whether its the first time playing the level or not (can't get to a second time because it crashes so soon)

 

What platform and what Unity version? I have many clients using STREAM in production games. Also, can you attach the crash dump?

 

I'll try to upload it tonight. I did some further testing last night after that post and I think it may be related to the # of terrain tiles I had it slice my terrain to, and potentially it increasing the amount of foliage visible on each tile afterwards.

I decreased the amount of tiles and also decreased the terrain grass/detail object density for each terrain slice after slicing and it seems to have behaved ok after that

I do have a question regarding AI/moving objects that are children of a Sector:

I have AI's that could potentially wander from one Sector into another if they are chasing the player after the Sector they are in has been streamed into the scene. Am I correct in assuming that if I just attach a Member component to their root GameObject that if they happen to wander into a new Sector, they will not be unloaded from the scene if their original Sector is unloaded?

IE: Guy is part of Sector1 in original scene. He somehow chases player into Sector2 as player runs away from guy, does Guy get unloaded when Sector1 is unloaded or Sector2?

 

Excessive foliage or trees could cause issues. If you were using the included Terrain window that should preserve the original amounts but if you used something else or increased after the split that could do it.

On the AI question, if he's a child of the Sector he'll unload with it no matter where he is. If you unparented him he won't. If he should be SECTR aware (for culling or his own behaviors) then give him a Member.

 

Yeah I think what happened is that Unity terrains let you paint over the max visible grass/tree amount which is determined by your resolution settings for the terrain. Once you hit the limit, you can continue "painting" things onto the terrain but they are invisible, so you can "overpaint" details that are never visible. So when SECTR split my terrain into tiles, it kept the terrain resolution, but since the terrains were now smaller and had the same resolution, it started showing the trees/grass that had been overpainted but wasn't showing before due to max resolution amounts.

I decreased my resolution on each tile drastically and that seems to have stopped the memory crash at least last night when I did one test. I did get a funky error on my console about #if Editor flags that I had never seen before, so I'll try to track that down and see where it came from

 

I've made script that enables sectr regional loader after 0.1 seconds, and, when "next level" or "restart level" or "main menu" buttons are hit, it destroys regional loader and waits 0.3 seconds, and after waiting it loads scene.

I didn't test much, but it didn't crash yet.

Did you see this blog post: http://blogs.unity3d.com/2014/08/04/multi-scene-editing/

Very nice features and it would make lot easier to modify sectors. I hope sectr will support this features.

And a suggestion to anyone that's willing to use streaming. "Combine Everything" Reduces hitching in streaming nicely. I had an harbor with lot's of containers, I've combined those containers and I noticed the hitching in streaming is reduced.

 

Glad that script seems to be working. Lmk how things go as you test more.

I hadn't seen that blog post, but I knew that Unity was planning on building some stuff like that. I think it'll be really awesome when it's done. I'll definitely keep an eye out for the API release and support the parts of it that make sense.

Combining stuff to reduce stream hitching is good advice. There is per-object overhead in Unity that you eliminate when you combine objects.

 

By combining everything, what do you mean exactly? Combining meshes, that sort of thing? Because I'm actually looking into reducing hitching while streaming, so I'd love to get any tips from people who have tackled that already.

I'm guessing in part it may also have something to do with my Awake/Start calls on the objects getting loaded, since my game is pretty script heavy due to its sandbox nature.

 

Not everything, but combine where it makes sense. If you can combine a stack of 10 crates as individual objects, then making them be one object will help (provided they don't need to bounce around).

Regarding lots of Awake/Start/OnEnable calls, those definitely add up. Anything you can do to make them lazy/on demand will really help with the hitching.

 

Hey folks just saw that this is on sale. I just left a review based on my experience with the asset. Honestly if anyone reading this thinks it may EVER come in handy you really should get it. It's well documented and as you can see from this thread, pretty well supported. Code is in C# and all included. What more can you ask for?

Got 5 stars from me. I'm sure it will prove even more useful as I learn the in's and out's of how to make it fit well into my project but at the current sale price it's really a no brainer

 

You are as a prince amongst men

Thanks for the good review!

 

I seldom write glowing reviews but I definitely think this asset was worth every penny We need more people supporting developers who do good work so I'm happy to spread the word!

 

SECTR COMPLETE is on sale now for 50% off as part of the UAS August Madness sale. Also, SECTR 1.1.0 is out, just in time for the sale.

SECTR 1.1.0 Release Notes
CORE
* Added new Quick Start panel including live information on news, sales, and updates as well as quick links to useful videos, docs, and support.
* Make sure to never double-add Sectors for Portal Determined Members.
* Removed legacy Unity 3.5 code, added initial compatibility code for Unity 5.

STREAM
* Add safety checks in Chunk to prevent shutdown/exit errors in Proxy.
* Rework Import/Export/Revert inspector button logic to avoid modifying the scene during Inspector Update, which apparently Unity dislikes.

 

Exciting news! SECTR is a technical achievement finalist in the 2014 Unity Awards! If you like SECTR, please take a moment to vote for it!

 

I've a problem using Sectr Stream with Static Batching. I'm using the Neighbour Loader with Max Depth: 1 with a total of 6 sectors. It seems that only mark as static the objects that are inside of sector where the Neighbour Loader is standing in. If i increase the max depth to 2, keeps doing the same thing but it consider one more sector for static batching, and leave the last sector without static batching.
Is this a bug?
Is there any way to have static batching for all sectors loaded?

Thanks!

 

Hi! Unfortunately the specifics of static batching are all internal to Unity and I can't do much to control them. Maybe there is a Unity bug where the last scene to get LoadLevelAdditive is excluded from static batching or something. One thing you can try to do is open the scene chunks files and make sure the settings are ok and then resave them. It's possible that will "jiggle the handle" and fix the issue that's the problem.

 

I made a simple test copying the same sectors into a new scenes file without Sectr chunk ref, i loaded all this new scenes from a clean scene with just one script that load all this sectors every 30s using Application.LoadLevelAdditiveAsync and it works fine, all the sectors loaded are correctly marked as static and batched by Unity. Also i re-create all the sectors and re-exported back again, and i didn't work in fact it was worse, because now none of the sectors loaded are considered as static for batching.
Do you still think this may be a Unity bug?

 

Thanks for the detailed info. Sounds like a SECTR bug. I'll try to dig in this weekend or early next week.