Organic model to voxel generator including SkinnedMeshes!

Hello everyone

I decided a few days ago to make a editor tool that generates a voxel representation of any organic model. Im glad to say it has turned out rather well so far in that I have it working for mesh filter or any skinnedMesh models

Before any further explanations here is a web player preview :

http://www.shaderbytes.co.za/ian/voxel_generator/

Controls are :

click hold drag left mouse button to rotate models on x and y axis

click hold drag middle mouse button to translate the camera on the xy plane

mouse wheel to translate camera on z axis.

here is a screen shot of what you will see in the web player :




The tool generates a completely new character with all armatures and animations as per the source model. When testing a few different kind of models I realized that there is not always sufficient vertex data in the source to properly generate the new voxel mesh that will deform properly

I then wrote a subdivision utility (first phase only , no position smoothing) that subdivides the data of the original mesh and iterpolates the bone weights similar to how it is done in a regular 3d application when all edges have maximum creasing.. I tested my results vs a subdivision done in blender to compare.

here is a screen shot to show you the results of this process in unity :



on the left is a simple 6 face 12 tri cube that has 2 bones to deform the mesh

on the right is the subdivided mesh (3 iterations ) showing how all the new vertices bone weights have been interpolated.

Using this data I then find the best suited vertex to transfer data to the voxel mesh, redo the bindposes and there it is If you have looked at the webplayer you will see 3 examples 20 ,30 ,40 unit generations. Have a look at the lowest 20 unit generation and you will see that ever there the deformations are behaving well.

Going forward I still want to iron out a few edge case situations in the face generation of the voxel mesh so its as optimal as can be and I will post some more examples soon.

When its finished I plan to put it up for sale on the asset store for $5 or $10 to try and make a little cash

 

This looks pretty awesome :O

 

Very cool! You know this could be a really nice compliment to our Cubiquity voxel engine. Cubiquity could be used for modeling the environment, while your system could create characters which naturally fit in to the visual style.

 

Very cool. How long does it take to calculate everything? I worked on an Organic model to voxel generator for a side project but the calculations took a few seconds so it couldn't be used in real-time. I basically did a raycast in four directions for each voxel and stored the all the enter and exit points into a collection. Then I went through the data and processed it to figure out which areas were solid, worked pretty well but I didn't store any color values.

-Dane

 

Thanks Dan

David I have seen your thread and video of the tank blasting up the environment .. looks great

Dane The calculations are very heavy indeed as im not only building the voxels mesh but also selectively subdividing the data of the original mesh recursively to meet a threshold and then interpolating all the bone weights and selecting the top 4 weights from those again. From there each vertex in each voxel finds the closest vertex from the subdivided data within a set bounds to be weighted best. Then duplicating the entire bone structure and calculating new bind poses.

Also I am using editor only functionality to assign the animation clips from the original to the new model.

That been said.. Since it does all happen in the editor all the work is already done so at runtime you can implement switching between the organic model and the voxel model without any computations required.

That would make a neat game mechanic of sorts i guess but I was aiming more for people who do voxel game development as a hobby or side project/stint and have a library of organic models stored up from all their other development endeavours which they can now use to quickly make a character to use in there project at the click of a button.

 

Hey thats mighty no 9!

Anyway, really cool. This is a pretty awesome thing. I wanna see more

 

Snowconesolid +100 XP for you been the first to identify the character used in my example

I built him to do a #MightySalute for the kickstarter campaign. Here have a look at the webplayer i sent , it has the schematic drawing in the background better lighting and cube map reflections and the salute animation button

https://googledrive.com/host/0B5GQag5iS_vzT3h6QU16Zl9jRlk/index.html

I will post more examples for the O2V tool soon.

 

That's just wonderful. Can't wait for it to make it to the Asset Store.

 

thats awesome man! Really great Mighty no 9 model

 

Nice. With that, you could make a nice "3d" version of retro 8-bit games.

 

Any release date in mind?

 

Hi Mementos

I will put it up real soon perhaps in the next few days up to a week. Im not sure how long it will take from when I submit it to when it gets approved by unity. But I dont think it takes that long there after.

I have added another feature now which calculates the AO as part of the build process. The AO values are then stored in the alpha channel of the vertices color. There is a property in the shader to adjust the strength of the AO overlay effect.

This is new and different in that in the first example above I used an external tool to generate the AO.

I will post another demo scene up by tonight or tomorrow to display the above mentioned additions and there control with different models again( a parrot and a dog perhaps ).

After that there is only one more item I want to add which is to do full edge splits and a vertex color surface shader so its as complete as can be for release. The edge splitting is required for proper hard edges on surface shaders and will increase the vertices count by up to 5 times tri count will remain the same. This option will be exposed in the inspector so it can be toggled according to what is required.

 

UPDATE 2

Hello again

As mentioned here is a demo of the AO baked into vertex colors as part of the generation process using a different model.

http://www.shaderbytes.co.za/ian/voxel_generator/sample_2/

image of what you will see in the webplayer :



Use the slider to adjust the AO value from nothing to a maximum of 2. I have it set to 0.5 as default which is very subtle but effective. It really makes a difference , Set it to 0 and rotate around and you will see it is impossible to see the contours because its purely vertexlit.

You can click the animation button to cycle through a run,walk,jump,idle animation set and use the slider to adjust the speed.

--------------

I currently have 3 methods I used to determine the voxel generation itself and have found some varying results between the 3. The best method was via using the subdivision data but It also had a problem of been to accurate and some voxels are not desired.

With this I have now decided to include an additional human intervention step in the process to fine tune the voxels before the rest of the process continues. This intervention step can be toggled on or off in the inspector.

What this does is creates a game object with every voxel as an independent object/mesh. You can then make edits to this game objects children by deleting voxels and changing the color etc..

Once you are happy then you can complete the build which will then build up new voxel data based on this game object and from there it will complete the rest of the process.

I had an idea on how to factor in edge case problems where the limbs get joined because the left/right voxels are adjacent in the grid. I will define a divider logic that will be tested against to prevent the joining. I wont try explain it all here it would be best that I make a demo video that will explain the usage when im done.

I have edge splitting already built now as part of the human intervention step so this can be carried across to the final build it you which to have it ( for hard edges with correct normals in surface shaders )

 

UPDATE 3

Hi Everyone

Before all the text - here is a web player with another demonstration :

http://www.shaderbytes.co.za/o2v_beta/test_2/

here is a screen shot of what to expect in the webplayer :



This demonstration is definitely worth checking out as the tool was tested on a fantastic model with a huge variety of quality animations built by the good people at 3DRT . They also sell this product in the Unity asset store here Warbots Micromarines


I have been working day and night adding features and increasing production quality.

The main new addition is the human intervention step in the generation process. What this does is builds a debug version of the object where every voxel can be selected and you can change its color or exclude it from the final build or change the expanded region it uses to bind bone weights and lastly to force a dividing logic between any adjacent voxels.

This example was achieved with this process.

I first thought the Auto generation was always about 95% good and only required minor tweaks Until this last test where the combination of shapes and default pose did not generate close to such a ratio .. I had to do quit a bit of tweaking in the debug stage , mostly excluding unwanted voxels and changing many of the colors and then several dividers by the head area

For colors this texture had plenty of fine detail in it , so the current method used for automated color selection from the uv map did not produce good results here , nearly every voxel was a different color. Currently I select the subdivision vertex closest to the center of the voxel. I will have to upgrade this process to something more advanced.


IMPORTANT ...
Its evident now more than ever that :

a. The automation is not clever enough on its own so tweaking is generally going to always be required.

b. It will not work in all cases of all models , Im now not sure how to express this limitation in the commercial release?

All the data is serialized properly to disc and you can save and close unity without losing your progress. From this debug rig you can generate the final model and if something is still not as you want you can delete the model and make more edits to the debug rig and generate again etc..

The final model is now serialized to disc as a prefab with the mesh as sub asset , just like a regular imported model.

I wrote 3 shaders for the models that can all render the AO over vertex colors . Unlit , vertex lit and the last one is a modified version of the builtin vertexlit which can render the AO to a certain degree. This is only for a fallback. The first two should be fine.

You can now opt to have edge splitting for every face and the last feature was to add a semi-smoothing option

I have it so it can counter any bizarre armature and skin mesh rotations like those that come from blender etc.. so far so good in this regard , tested a few variations here


Other limitations that will be present in first release:

I dont split mesh generations above 65k into sub meshes yet .. it will fail if exceeding this limit set in unity for a single mesh
I dont support models with sub meshes yet
I dont support any GameObject hierarchy with multiple models nested
It uses vertex colors based on the original uv map. So even if you switch to a regular texture shader and add the original texture .. it would look the same as the vertex color shader because all stored uv coords point to a same pixel value per voxel .. in other words no detailed texture mapping on any faces.

Wow that was a mouthful , any feedback on the voxel model made in this demo please feel free to let me know

 

Hi there! =D Amazing idea!
Would you like to test it with UMA?
http://forum.unity3d.com/threads/153689-procedural-character-generation

Cheers!
Fernando R.

 

Hey Fernando thanks.

My tool might just blow up trying to test it on your awesome generator

Jokes aside , I have read bits and pieces on your thread in the past but cant remember if you have any sub mesh in your characters?

I do not handle sub meshes yet ( or multiple materials for that matter )

but this is one of the limitations I do think I need to resolve to increase usability going forward.

Also once my model is generated it does not use uv maps anymore its vertex coloured so this is surely a contention for a mismatch ?

I will have to go read up again on your tool to get a grips of what could possibly be used for interesting results with mine

cheers

 

Hey =)
Female avatars on example scene have an extra material with opacity for eyelashes, but of course you can consider not including those (just one line of code on UMACrowd), so all avatars would be one material/mesh.
Looks like we could get amazing results =DDD
Default shaders uses a diffuse and a normal map+Spec+Gloss atlases, so I take you could just access diffuse atlas and bring the color values to vertices.

Cheers!

 

If you want to try this out, please send me an PM, and I can add you at skype, so I can help you with initial setup.

 

pm sent thanks Fernando

 

I made a few demo videos which display how to use the tool. This is the first time I have ever made instructional videos so please bare with my novice errors. I also could not do high bitrate videos as my computer does not have enough grunt , I have a 3 year old onboard chipset.

There are 5 short videos, the first 4 videos display how I used/improved the tool to generate the voxel marine model posted above and the last video discusses the two different algorithms and other options.

The marine model was a great test because it is an example of a problematic model which requires some tweaking to get good results. Take note that not all models would require this level of editing.

I made a mistake in video 3 and had to do one process twice ( separating the head ) but did not want to record the video again and these videos can only be used in the WIP stage anyway. I will have to do new videos without the marine model when the tool becomes commercial.

Part 1:

Part2:

Part3:

Part4:

Part5:

 

Hello again ..

Here is a preview of the new updates to the O2V tool. The previously experimental post smoothing option is now a full feature and is included as a component which can be controlled in the inspector or via code at runtime.

I also made the changes to the layout of the debug window

 

This is pure awesomeness And also, the only 3D that makes sense to me, along with low-poly

 

Looks awesome and something we look forward to play around for gunjitsugame.com.

 

Thanks guys

BFGames gunjitsugame looks great The tool is almost ready for submission , the last thing I want to implement is handling submesh objects and then I think its ready for release.

 

Great! Send me a PM when its submitted

 

Great news everybody.

I have built in support for submesh's now and while I was busy I also added two other features.

1. If the original model had no texture assigned to the material , the material color will be used as the color for the vertex colors
2. If it does have a texture , the color of the material is multiplied with the vertex color sampled from the uv map.

So I took a break from the code and designed a logo for the tool yesterday and today quickly made some poster art for the asset store.

I need to just make some documentation and check through all the requirements for asset store submission and then it should be all sorted

here is the poster art (and logo)..

 

looks so awesome!

 

Hi Shaderbytes,

it looks very good; i'm grabbing this as soon as it releases.
I hope it does very well for you on the store!

El

 

Thanks @Ellandar

UPDATES :

Im about past half way with making the PDF documentation I think .. I got distracted with feature creep!

So I have had it in the back of my mind to add some sort of vertex reduction method at some point , even if it was something I updated after releasing the tool.

For vertices on the same axis plane , not part of a hard edge, the main issue with this was subtle differences in vertex colors , you can't merge two vertices with different colors without also knowing which color is then the best to use. I am already reducing subtle color variations somewhat via creating additional up-sampled textures from the original textures mipmaps. This produces a slight bit of blur to the texture so sampling is better but its is not enough on its own. There are still to many subtle differences.

The needed solution was a color threshold algorithm to reduce similar colors. Im glad to say I have spend the entire day evolving from a color threshold noob into having a good (getting better) working solution. I had a very naive approach at first and soon realized it was going to be more tricky than I first anticipated.

My final result now is using hue channels that have several intensity channels themselves. True grayscale pixels had to be handled separately as they do not have a hue value. I also added in a validation for almost grayscale pixels.. these pixels will be made grayscale so they can also be reduced in the handling of grayscale pixels.

This is a feature all to its own then but was needed to be able to get better vertex reductions when I do eventually implement that.

Here is a sample of the results :

Item on the left without optimization - item on the right after optimization.

 

This is a superb new feature. Getting better and awesomer

 

Hello everybody

Best news ever! I could not resist going the full monty for first release and I implemented the mentioned vertex reduction algorithm after I finished the color threshold feature. I had to rewrite some patches ( read : over a 1000 lines ) of code here and there to facilitate this as best possible.

Reductions are in the range of 40% - 60% ! Its about as optimized as could possibly be. While I was on roll I saw room to improve the AO algorithm as well. The previous version was actually written before I did edge splits and had some nasty side effects of having AO bleeding around edges because of duplicated data.

Thankfully the new under laying code structure for vertex optimization was also the clue to how handle calculating the AO properly - no more data sharing errors

Two other color manipulation features were added , tinting and grayscaling + various other enhancements.

ok So No more feature creep now .. Going to finish the documentation off and submit soon I hope.

Here is a video to demonstrate what is discussed above :

 

Here is a web player sample of the dog styled as a cyborg . I excluded many voxels to give it this kind of feel, but zoom up with the mouse wheel and notice that you can see everywhere inside him and everything is connected properly. I think its an interesting type of character transformation via the tool.

cyborg dog only:
http://www.shaderbytes.co.za/o2v_beta/test_7/

original model included:
http://www.shaderbytes.co.za/o2v_beta/demo_8/


Image sample of the webplayer :

 

It's amazing, keep it up, and good luck!

KirbyRawr

 

This is absolutely great, one of the best clicks of the night. I thought about you leaving the voxels as seperate game objects and a game happened in my head

Even if it's all pregenerated seems like something id love to put into something im doing for someone, accuracy no big deal, leaving as seperate game objects as a feature possibly? I don't know if it's undoable or just too much effort, i'm quite ignorant to these things, but for say, an enthusiast/hobbyist who just likes fooling around, so they dont care about having to make things for computers with less than 8 cores or a high end gamer card, can thread the process or put it to a gpu? Cant really give you extra money for this but you'll get my money anyways (when i have money) but if it had a few bits and bobs that made it feasible at runtime and so on, theres a game, or an entire aesthetic there. A pal will be extremely pleased

 

Wow this looks frickin awesome!!! I'm going to buy it instantly!

 

I'm still here! .. this Time I really really think I've reached the point of finishing the documentation up and submitting. So what have I been busy with ..

Well Two awesome things have happened making the system rock solid :

Firstly I implemented SAT ( separating axis theorem ) as the means to detect voxel creation

http://en.wikipedia.org/wiki/Hyperplane_separation_theorem

This works 100% as proper mathematics like this should .. but it introduced some problems with some other code that relied on the previous algorithms data , namely the semi smoothed voxel feature. I could have just removed the feature but I liked it too much and secondly - I realized that the minimum bounds penetration step in the smoothing feature which actually removes voxels not needed would be a very useful on its own for the regular builds.

So I bit the bullet and did some research on how to do ray-polygon intersection maths and settled on this algorithm :

Ray - Polygon Intersection

http://www.cs.virginia.edu/~gfx/Cou...t MinimumStorage RayTriangle Intersection.pdf

Between the PDF and some old XNA code I found online I managed to get it working 100% as well.

This is the raw maths of ray tracing , not Unity raytracing - no colliders required

Now every voxel has exact data for every polygon intersection, both cube to triangle and triangle to cube.

here is a picture of one voxel showing some debug lines pointing to all intersections of itself



So now I have exposed the voxel elimination process to the debug rig for regular builds as well. It uses a two phase approach which first validates a threshold square of each axis and then a threshold of the cubed area ( same settings as the internal semi-smooth math ) but it does not delete anything at this point , it just marks them for exclusion.. I will post a video demonstration soon.

There are other big changes to the system namely I do not serialize all mesh data and ScriptableObjects to disc anymore .. serializing to the scene was sufficient and much faster. Changed the data structure for each voxel to be a MonoBehaviour not a ScriptableObject. The unity Undo system is implemented where possible but Its slows down the tools use when editing many voxels at once so I have included a flag in the inspector to disable The code using the undo system whenever you like.

All good then , I hope to submit soon , price will still be $15 as mentioned before so please support it if you can

 

Great news, i can't wait when I try it

 

Wonderful! This will be hugely useful for many people I am sure! Its on my shopping list!

 

I can't wait to but it! Let us know ASAP when it's available

 

Oooo SNAP!! I love the voxel looks. I really loved Voxatron btw!

I want to turn everything into voxels with this!

 

It looked fantastic when you began it, but now it looks even more fantastic, wow! *_*

 

Thanks Gentlemen wont be long now

Ok last video to display the latest mentioned features and another new feature added in as of today to round it off well.

The video will demonstrate :

1. Minimum penetration exclusion calculation exposed in inspector.

2. (new feature) Voxel duplication to fill up any gaps should you need to. Note this is not a means to create whole models from scratch like other voxel creation tools. This is a converter tool and so creation/duplication is based from existing voxels created in the conversion only.

I know the videos quality is not enough to view full screen and it may not be evident to see exactly what im babbling about as I demonstrate.

I will try do some new videos at my friends place as he has a strong enough machine after I finish and submit.

 

Any updates?

 

Hey skoandi

Yes should submit very very soon now

I have finished the documentation PDF and images for the asset store and have downloaded the packaging tool to get going..

What happened then was I found that I could support mecanim characters if I just changed some of the code slightly so I rather made that change now then have to update the tool soon after release.

Here is a video showing the tool working on a model from the Unity mecanim example package.

I made the video quality better but the frame rate took a slight hit which you can notice when I run the scene to test the character but atleast you can choose to view it in 480p or 720p HD now

 

Mecanim example 2 :

 

Are you still working in the o2v? I am interested in the shader, mine doesn't do any batching and even 100 voxels start to feel and lag :S

 

it's just a vertex shader rgb for color and has some ao values in the alpha channel. Batching is built into Unity engine. Also i dont leave my editing rig which is made of 1000's of voxels as is .. it builds a final model from that into one mesh. It is rather complicated the process of getting the colors from the original textures , it samples and upscales a low mip map level to achieve this and reduce complex dense variant pixels. The editing rig provides some color manipulation as well. When the final model is built another texture with all the used colors is created. it uv maps to a singe point in each color quadrant. so normal or other textures cant paint the surface, it is very custom. Anyway this is a large project and to build get 80% seems fast .. the last 20% will take months and loads of code to complete. i think this project had about 15k lines of code and it was really still a toy.

proper voxel engines are out there, go and study those, they use chunks and binary trees to create a dynamic scene. For example:

http://minddesk.com/learn/article.php?id=47

This rabbit hole is deep and requires months of development and research.

Most scenes only create and process the voxels of the area you are in / chunks. so iit seems like everything is dynamic but the rest outside the chunk is actually one model / models not actual individual voxels / gameobjects etc ..