AI system project

Hello,

I am starting an AI system for Unity. I don't know yet exactly to what extent I can sophisticate it, but it is not a commercial project and there are no deadlines.
I already started a while ago a basic vision system and I think I am going to start with improving it.

I will post news of the development when I can, and it would be great to have your feedback on what you would like to see.

Minsc.

 

very interesting , sounds great ill be keeping an eye on this thread , i like the idea of the ai using opacity to see through stuff. great work keep it up!

 

Thanks a lot dudester (sorry to be so late i didn't review this thread since -.-'), it has been a very long time I didn't touch this project in main part due to time constraints but I will try to be busy about it a bit more now. So I thought about it and there are two main approaches that I see for continuing this:

The first is to continue using raycasts tricks like above. This may at best allow to detect an object by raycasting its "skeleton". For exemple, you put in a human model prefab colliders to raycast at the limbs, the bot raycast each, depending on which are visible just as determined in the prototype above, a certain form is detected, the bot would have a database of "skeletons" for humans, tables, dragons whatever and a filter search would be possible to determine depending on the rotations (including possible to recognise a crouched human, which may be very computationally expensive to work out combinations!)/sizes of the limbs and the limbs detected what the stuff would be. However there is a major problem: raycasts are Unity API and not threadsafe. Also a workaround should be found for lightning, for confusion between say a crouched hobbit with a cape on and a rock, so it still is a lot of tricks.

The second approach, which is closer to natural vision, is to directly use computer vision, just like what is done with google images. The only thing constrained to unity api is the rendering of the textures from a camera view (here the bot camera), the other stuff can be done outside unity API and be for sure fully optimizable.

I will do both approaches, the second may give better results eventually but I think it is much more complicated so it may take a while

 

Hi there,

The project is progressing, and I opted for the second approach: images from the bot camera which I control in this video are recognised by using a (deep) neural network on the cloud (Clarifai) . I filter the tags in the response to keep only (for now) those that are about general categories (e.g house,man,woman,door) and display the tag with the highest likeliness value among those who are kept. The average time is around 2 seconds between the stimuli (keydown for screenshot) and the filtered response. I found that surprisingly good considering that I am very far from the datacenter on which the computation occurs (7000 km) and that the overall process can be optimized, I think the delay between the image recognition request and the result can easily descend to a half second.

art assets used in this video are from Mixamo, Unity, Dean Hunt, and Proportional Studios

 

little update: running inference on a local machine using a deep neural network lasted only 0.4 seconds at maximum, while using cpu only (i7 3k series), which is around five times better in average than using a web service as above. On a gpu it will likely be orders of magnitude faster though I don't have access to a linux instance yet (most ML frameworks only work well on linux).

 

Hi @Minsc , I'm also trying to make Clarifai work with Unity but I haven't been successful. I wanted to ask if you can help me how to use Clarifai with Unity3d.
Thanks