What is conscious AI?

If only I had actually gone on with that cruel social experiment and minor cash grab

On the topic of ai - you'd see all of it's memories and experiences in a directory with the name you choose to give it. Everything it learns about you through interaction will be stored there somewhere. And everything it reflects on will be quantified and compared to other experiences, to have meaning.

You and I don't exactly rate things that happen with numbers, but we can compare experiences as better or worse than others. To some degree, it's quantified. Why should it be any less real for a digital life?

 

Sometimes.....

 

But what happened next??!!!

 

For me A* or normal edge detection is not intelligence, because the human designed what to do and how. So it did not learn and cannot learn in the sense I use learning.

For me stuff like deep neural nets (DNN) are better examples. And of course Watson.

The following for me is some form of intelligence (DNN based), it learns to play atari games from just getting pixels and the score. From that it learns how to play. It works for different games, it learns what to do from just that (Google bought these guys for 500M).
Awesome video, watch it!!!

Edge detection that is "intelligence" is when the edge detection is created by learning. E.g. if you feed a convolutional DNN just pictures, it learns edge detectors on the lowest layers. On the next layer it e.g. learns parts of faces if you feed only faces. On the next layer how to combine these parts to complete faces. Google learned object recognition from feeding a DNN millions of images without telling it what they are. It learns to differentiate stuff in images. Then you give it examples where you tell it what it is so it can then tell you if it sees that.

Here an example of the learned layers to construct faces (go to 31:22):

Earlier approaches in speech or image recognition used human built feature detectors (e.g. like edge detectors) and then fed the extracted info to some form of machine learning. But these approaches have no chance against purely DNN based ones in today's competitions and benchmarks (of those I know off).

The DNN learned base features seem to be just better than what a human can engineer.

Note: cool is that you can use some kind of these DNN to output a "new car design" e.g. it can output a new car because it has some kind of concept of what a car looks like (for it).

Recent Developments in Deep Learning (actually more short history of DNNs):

Edit: Disclaimer: built some DNNs because too lazy to teach the computer stuff if it can learn it itself.

Edit2: But yes, consciousness is far away. Except if Skynet already brought it back.

 

This blows my mind. I am a sceptic though. Did the computer really learn to play those games by itself? Or was there a helping hand.. hmmmm... seems to good to be true.

 

I assume google tested their claims before paying 500M$ for them. Just the pixels and the score, that's all the computer gets to learn the games.

There are multiple open source attempts at reimplementing it. But when I last checked they were not at the same quality, so they did not dominate the games so clearly. But it is to be expected that the open source guys need some time, because DeepMind was created by some awesome guys (Google paid to get these guys not for the tech, because how they did it is more or less known).

 

Well, they also bought a "quantum computer" which nobody has proved is actually quantum. Google has a lot of money to waste. But I hope it's true! Thanks for posting the videos! I love this kind of stuff! \/

 

I assume Google knew that it could be that it is not really quantum. But these things still solve certain things very fast with their annealing technique, even if it is not quantum. It does just not scale like quantum annealing would. And quantum annealing is anyway far away from "real" quantum computing.
So I assume they bought a D-Wave to have a cool toy, not because they were sure it is quantum. And it still is a cool toy even I wouldn't buy one.

 

We seem to be throwing definitions all over the place (myself included) for what is intelligence & artificial intelligence. When citing something as intelligent (artificially or otherwise), please cite your standard! thank you

 

I use this (Oxford Dictionary): intelligence: A person or being with the ability to acquire and apply knowledge and skills.

That for me means it can learn. So the solution/algorithm was not hard coded by a human.
This of course is not black and white but grey with many different shades, e.g. the DNN algorithms are too hard coded. But these algorithms implement learning and not the specific solution to your problem.
These can be used to learn many different things, from image recognition or playing atari games to driving trucks.

 

That AI learning how to play games by using feedback (score) is excellent. Basically the same way we learn (if we are not complaining about lack of tutorials). It is very interesting stuff but I don't consider this to be AI as we would have in games. These Learning Machine algos are interesting in and of themselves and cool stuff to blow huge sums of cash on. For a game AI does not need to be a learning machine but to represent intelligence. Where intelligence here includes one or more of memory, logic, planning and problem solving. Logic is a set of rules that provide the overall strategy. Problem solving can be performed by these rules. More complex rules may rely on memory and planning. In this way it (a computer controlled opponent in a game) simulates behaviors of intelligence. Where I see artificial as being akin to simulates.

 

So is a machine intelligent if you place an object into the world and ask the little ai to find the machine, and then it forms a path to that machine? It wouldn't pre-compute anything, it would navigate with A* and save the path it found dynamically so if you asked it again later, it would respond "I think it is here", just as we would, but then look for it again upon failure (also as we would) and then update the path to that object.

The only flaw in your logic there is that no algorithms can be hard coded. Well what about humans? We have hard coded algorithms in our brain for learning new ones. Instinct or otherwise, we do have hard coded algorithms from the start.

 

Yes, what I talk about is normally not used in games (but there are some). Because the initial question was about consciousness I went to that more machine learning heavy interpretation. For me this is the only way "real" consciousness could be achieved some day.

If we talk more about game specific AIs, I think one that creates emergent behaviour can look conscious in that it creates behaviour that looks meaningful and which was not explicitly hard coded in.

I think I covered that in my previous post. Something is always hard coded. The point was that not the solution was hard coded but the learning, in a way that it can adapt and learn different things. As said that is grey with many shades.

 

@thxfoo oh, that's clearer, sorry.

That grey area will fade eventually. Maybe even in the summer of 2015

 

i think a thing youre missing is that computers are just coded 1s and 0s .. i think for it to be conscious, it would have to be... somehow different than this..

this quote you wrote tomnnn certainly points out the problem

it would require to be an instantiated copy... since the computer systems are really just glorified abacus beads

but iam a strong believer in supernatural? and spiritual mind abilities, and iam unable to accept that "living things are just machines"


about the "tinkerer" thing, yeah, it seems the "spirit" (for lack of better word) of organisms always want to improve upon their designs, they make iterations that experiment in different ways, evolve,

lol i had more to say but i should be sleeping... ... LOL what does our "computer" do while we sleep?

 

Didn't make my life any less full of wonder to accept this. In fact, it makes interesting possibilities for the future where robots are considered living things!

It switches from the conscious system of inputs to the subconscious system of inputs and paralyzes the body so you don't hurt yourself while responding to the subconscious inputs. The frontal lobes become less active, thus insane & impossible things happen because our 'fact checker' is running in power save mode

I discussed this with a game dev teacher a bit. By this logic, are you stating that a clone of a person would not be a person? The end result of that very long discussion is that there are too many variables to speculate on. There was a guy who survived losing a large chunk of his brain after a pipe went through his head. What if instead of that piece dying, a portion of his brain was instead moved to a computer / robot body? Can moving the brain piece by piece preserve a stream of consciousness?

That's what I was getting at with the bit-by-bit part with transmitting consciousness of the very cruel virtual pet.

I'm sure going very far into this with my PhD research will upset many people who would prefer to believe there is something special about us

 

yeah i was thinking about that too... arent whatever brain cells do, migrate to other cells and the original cells die?? ..
i dont know much about biology, for some reason i actually consciously decided to avoid learning biology, (ignored it, got F in HS) because of some idea its necromancy or something (i value really old ideals sometimes LOL .. not religious, philosophical)

I hold some belief that human Will is a very powerful force that can even, if you were to "program" mechanically the brain to function in some way, I believe the human Will can overcome it ... i guess thats just faith in a belief though..
the book "secret of the ages" hit me in the most profound way.. lol

i think the actual things that make up human conciousness are long to be known, chiefly due to the faults in scientific method... some results simply Cannot be replicated with any consistency.

i KNOW almost doubtlessly that i have performed telepathy one time, only time i tried, it seems almost beyond acceptable to be a coincidence... loong, kinda embarrassing story

 

Brain cells and nerve cells don't regenerate when they die. The right side of my face has been partially paralyzed for over 10 years and partially cross wired too - hence I can rub my finger under my right lower eyelid and my upper lip on the right side is in pain.

Recovery from strokes and other nerve and brain cell affecting maladies generally involves training other brain cells and nerve cells to take over the dead cells functions and that is often dependent on physical access by other cells to regions nearby enough and similar enough in function for that to be effective.

Example is paralysis of the legs due to spinal injury. Typically motor leg function is never recovered yet the nearby nerves that operate the muscles in the blood vessels blood supply of the legs continues to function.

I remember reading in the 70s of a Japanese researcher actually growing nerves in the lab in an issue of NatGeo but haven't read anything on the front since although with stem cells they are probably getting closer.

With regards to transfer of memory - no, that is gone. What those cells 'knew' has to be relearned by other cells. It's not a simple transfer of 'bits'.

 

yeah... i did a quick look up last night about it and found that neurons dont regenerate... but yeah it was a quick look up..

but yeah... I strongly think, for AI to have actual consciousness, computers would have to be redesigned altogether, 1s and 0s .. 5v and 0v..."glorified abacus beads" is not sufficient

 

Actually the best explanations for dreaming come from machine learning, and some forms of it are even used in some learning algorithms.
One explanation is that dreaming is "unlearning", it helps differentiate memories that are near to each other. If you create a Neural Net to store memories by energy minima, 2 nearby minima can merge into one (fake memory). Dreaming means setting the system (or brain) into a random state and unlearn that state. By that you can differentiate memories near that state better. That is also why you can only remember the last dream before waking up: the brain is in erasing mode and not in storing mode.
Some programmer art for that (A+B is a fake memory; memories are stored as minima; memory A and B are so close that the combined minima A+B hides A and B):

A computer can represent floats or integers (by combining bits). But the brain is 0 and 1 (neuron on or off), and neurons fire at random intervals. For both of these there are mathematical explanations from machine learning why it is better to use these restrictions (nature could easily work analog (with "floats") and with timed neurons, but it does not). Ask for details if you wonder why.

There are multiple million dollar prizes offered by some billionaires to show any supernatural ability under scientific conditions, nobody ever won. Even it would be possible to win by (very much) luck.

A brain injury can completely change your personality. So it cannot come from the soul/spirit. As with god or supernatural stuff, you can never disprove it, but there is not the slightest evidence that something like that exists.

 

I've started looking at this online AI course from MIT it looks interesting. Probably quite basic though.

 

What do you mean by 'the brain is 0 and 1 (neuron on or off)'? Are you saying a single neuron can only represent a boolean? Also, what do you mean by 'neurons fire at random intervals'?

 

Their are a lots of states that are chemical dependent and chemical interdependent and chemical saturation dependent. It's definitely not on or off.

 

I know, that's why I got curious if that's what thxfoo actually meant. Perhaps that wasn't too clear from my post.

 

A neuron sends a pulse to another neuron. That pluses in the network are the data processing.
These are
1) binary: the next neuron does not react to the power of the pulse, only to the existence of the pulse (in that it must be above a threshold, biology shows that)
2) the point in time a neuron pulses is not synchronized to others, it is kind of random

Why 2) is better (so having random pulse intervals) is easy to explain (but needs some math/Ai terms):
It is the same as using dropout when training a DNN with dropout (you turn off neurons with p=0.5 for each training sample). That way you train 2^h models at once (h=number of neurons in that layer). That means your network will not overfit because you average 2^h models at once at evaluation time (you use all neurons later but at half power), with that neurons do much less co-adapt (so do not depend too much on what neighbours do, because the specific neighbour is not always there).
Not overfitting is not only important for DNNs, but also for the brain, because you have much more parameters ("weights" of neuron connections) than experiences in your whole life (aka "training samples" in AI).

Edit:
2) also explains why you get better results if you think a little: more and more models for that problem are evaluated (e.g. once the solution when neurons 1,3,6 fire and once that for 3,5,7).
Answer the following without thinking, say the first thing that comes to your mind: Ready? Go:

Spoiler

What do cows drink?

A little thinking will get a better result.

 

Pulses randomly with binary values and you can read this? You're right I must not be thinking to imagine that some random stream of binary neural pulses is actually language.

 

That's why I write "kind of random". I mean that it does not matter if it fires now or a little later. The neurons don't have to work in sync to achieve a result. Because there are "multiple models", so combinations of neurons, that solve a problem. If 1 and 6 do not fire but 3,5,7 in the above example you get a solution for your problem (and that they fire depends on the input they get, so nothing like a random stream). If you wait till 1,3,6 fired too you get 2 solution that get averaged to get a better solution and so on.

This stuff works to achieve learning and problem solving, artificial deep neural nets are state of the art for many machine learning problems. You often use exactly these things there (except the effect of the random interval thing is achieved differently, you randomly omit neurons on training and use all synchronized at evaluation; the brain maybe does not do that for speed of reaction and to save energy costs).

Edit: What do you mean with "read this"? You don't have to read your brain, you are your brain.

Edit2: Watch the first video I posted above about the AI that plays Atari games from getting pixels and the score. That's the kind of thing you can achieve with such ideas (it learns enough what the pixels mean to play well).

 

The course I described was a 'basic' course. The first half of the class was logic, the second half of the class was image recognition and path finding.

There's also the factors of unreliable transmission and signal leaks

 

Thats the easiest way to explain how ANN's work.

I have like 5 different versions of these hanging on my wall from back when I was programming a massive ANN for the state military where I live.

 

I think neural networks are over-hyped. When it comes down to it. It's just curve fitting. If you have 5 inputs and two outputs you are just trying to find 5D curves f(x,y,w,u,v) and g(x,y,w,u,v) that best fit the data. There are more ways to do this than neural networks. Plus the backwards propagation in neural networks is a trick that doesn't come from anything in the brain. But just because these networks "look a bit like neurons" then people think they solve all AI problems.

But there is more to intelligence than curve fitting. For example, neural networks have a finite number of outputs, contrasting to long term memory which seems to be able to store almost limitless amounts of information and learn new things. And the brain is self directed based on goal seeking. Which has more in common with a chess playing algorithm than a neural network.

I think we should try to copy the behaviour of the brain rather than the biology.

 

Yes, but most other ways are less powerful than ANNs. E.g. SVMs (support vector machines), while their kernels may be non-linear, the way you combine them is linear. That means ANNs can solve problems for which you would have problems with SVMs. In short, ANNs are mathematically more powerful than SVMs and most other machine learning constructs. Also, look at most competitions and benchmarks, atm DNNs rule them.

You can do other stuff in ANNs. E.g. energy based minima, no back propagation there. One could even argue that that probably is what the brain does.

I would say an ANN has more in common with the basics of the brain than a chess playing algorithm. But that maybe depends on perspective.

Many ideas in modern ANNs you can also find in the brain. But the trick is to just do what the brain does if it makes your ANNs better.

 

@yoonitee you're forgetting the biggest part that nobody has.
A super computer strong enough lol.

Now if we had computers with molecular memory (that they are developing now) which would be equivalent to like almost a terabyte of ram for one chip (at least this was 4 years ago when Microsoft announced they were developing it).
Or at least, attempting lol.

If we had computers fast enough, then sure, yeah they could learn as fast as a human brain, again, if they are fast enough.

Processors have a big thing to do with it as well, obviously..
But once molecular technology comes out, or if it already it, once it's cheaper to make to be used for a purpose without a high debt/income ratio. No company is gonna spend 500,000,000 on a chip even if it is the fastest, it's not something that can be used and make sense to use as you'll be in debt for decades trying to pay it off lol.

The NSA (National Security Agency) here in the U.S. has a Quantum Computer, it is able to within miliseconds look at an image and figure out what everything is in the image. But that's all of course here say, they aren't going to tell anyone what it can really do.

One thing I can say is, the U.S. Military has satellites in space that once they know your shadow, the satellite can pick up your shadow in a crowd of people and know exactly who you are, because even though there are over a billion people on earth, it's scientifically proven we all walk a different distint way from anyone else on earth.

Whether it's how faster we walk, how far our feet spread apart from each other, how far our legs go out when we walk, how long it takes for a complete sequence of walking, the satellite can figure that out, it's some crazy crap, but that's another ANN.

 

Can see a lot of talk on Neural Networks here.

It is important to remember that NN's are supervised learning, meaning that they learn based on a dataset of inputs (first layer) and related outputs (output layer). After training them they should be able to make the "right decision" based on some input given, however that decision is based on the dataset.

So lets say the data set is based on recording myself play. Then it will learn to play like me which is cool but it will NOT adapt to play in different manners than how i am playing. This is a big limit for NN's overall, also deep neural networks. Multi layered (deep) neural networks just helps solving complex non-linear problems.

Some approaches are taken to adapt NN's such as Neuroevolution of Augmenting Topologies (NEAT) where you use genetic algorithms to change the NN toplogy and hence you can return a fitness for different networks which in reality would make it adapt. The big problem is that NN's takes a LOOOOONG time to train. Realtime training of NN's is not really well done today.

Reinforcement learning is for me really interesting as it learn from "experience" and only needs information of previous states but not states that precedes it, hence its better at naturally adapting to its environment. That's why there is a lot of interest of algorithms like Q-learning in robotics because they can learn to solve problems with no "previous knowledge". They just need a goal and rewards for doing the right things.

 

@BFGames actually, you can make them unsupervised learning lol.

I've built them, so I know, but it still always needs a given set of data to calculate with, but after that, it will either predict with good results or not, and if not hopefully gets better overtime.

 

To me the difference between AI and humans is consciousness, and ultimately, desires.

But maybe there is no difference. When I think about it, its just shades of grey. We're just organic machines. There is however, a line, we place between intelligent conscious entities, and artificial intelligence. But from reading this topic, everyone draws that line in a different place, which just confirms that its all shades of grey. Both machines and humans are heading towards the same end, we just have completely different origins, and we're just "a few" more years ahead of machines. Machines move at a higher speed though.

Humans can be a bit arrogant, and the only way an AI is considered intelligent, is if it behaves as a human. Including our flaws, emotions, and other vices. Not just our qualities.

I was interested in real artificial intelligence and neural networks sometime ago for a couple days, when I was wondering what would it take to build a real helper bot, that would help around the house, be used as a bodyguard, friend you can consult, and oher stuff, but I gave up once I learned that right now the most intelligent machine is pretty much as intelligent as a human toddler, and because its so hard to find anything I can understand. I learn best through practice and I coulnd't find anything to practice.

You should definitely make that! I would so play it!

 

Aye that is true. But they still base their learning on a data-set. Of cause over the course of time that will become "non-relevant" but it often will take a lot time to learn and adapt, hence it runs into the same problem as NEAT. Reinforcement learning don't need a data set to begin learning.

I don't see NN's as a great way for fast adapting AI in the near future, for any type of games. But for offline learning and data-mining they can be pretty great.

Working on real-time MCTS agents in a 4 player multiplayer scenario for my master at the moment. This video shows the first basic test where the agents estimate what action to take based on own position, moves available, other agents position and ammo available. Even in such a basic scenario it adapts pretty well so far which i find very interesting. The video is after only a weeks work on it so its by no way perfect yet. Watch it from 1:40.

The problem is that we need to run at least 300 simulations in order to get them perform well. At the moment we run 600 simulations in 0.4 second, and then update the agents action taking every ½ second. We can thread each agent so thats all good.
But it would not work for a huge amount of AI.

 

Look at the first video in my post with 3 videos above. The learning can be online. This AI learns to play any Atari game. It starts playing bad and then gets better over time. It collects experience like a human would.

In your MCTS you don't have to learn the rules of the game because you code them in. So with a ANN you need some time to learn the rules first. But that means you have to code them for each game with MCTS. So you could not just give it another game like shown in the video I link.

Edit: Atari playing AI video repost

But yes for standard games AI probably only rarely used. Also it would crush most humans as game AI and it would probably be hard to fine tune that to be fun.

 

In my case you are right. It is based only on our game, and rewards and simulations are set for that particular game because thats what we are researching (we don't research for best performance, but human like behavior in a case scenario).

However if you build it abstract enough you can use it for whats called general game playing agents which is also showcased from the Neural Network video. There is a lot of research on MCTS and general game playing agents now, so it is very much possible.

And yes their agent can learn different games, but:
1. it is very slow. In the video it took 4-6 hours to play well in a simple 2d game, thats NOT what i define as well performing online learning. Its not useful for fast adapting AI (yet!).
2. They still need individual output performance evaluation for each game and they still need that in regards to possible actions - so its not as "overall" general as it seems.
3. How would they adapt their approach into 3D games with the depth of the z axis, or more complex games where high dimensional NN's is probably needed. Problem is high dimensional networks learns at a slower rate hence the problem consists.

I am not saying that it is not impressive or useful in other cases (self driving cars anyone?!?!?). It very much is. But i just don't see NN's of any kind becoming fast enough to adapt well in real-time games in any near future. I think other approaches are more interesting for now.

 

1) Only to learn the game initially. After some hours it is a master at the game. No more basic learning needed. And for many games it could probably adapt to a human really fast if it already masters the game. Except for games it is already that good that it does not even have to adapt to the human.
I would compare this learning phase to the phase you implement your MCTS interface for the new game by hand. With that perspective some hours are very fast.

2) But they just need the score, one single number extracted from memory. Implementing that interface takes 5 minutes. Also allowed actions are just buttons on the gamepad. If you are lazy you just allow all buttons all the time, it figures out which buttons do nothing fast (but probably remove the pause button, otherwise it learns to pause the game if it would lose).

 

Also MCTS is basically trying everything (or a lot) and then take the best move, similar to what chess computers do. That is not what I would consider "real" intelligence that maybe could one day achieve consciousness.
MCTS is more akin to brute forcing with some tricks.
Also my DNN poker bots beat my MCTS poker bots. But that probably means my MCTS implementation s#cks. I'm sure you could help me with that. Looking for a job?
Edit: yes I know I should do Counterfactual Regret Minimization (CRM) instead, but my DNN ones are good enough atm (NL6max).

 

Neural nets are indeed slow to learn. I naively built one too allow enemies to adapt to player strategies. It adapted alright, after the player used the same strategy 500 times in a row. Then a simple strategy change threw the whole thing back to square one. A neural net simply cannot learn fast enough to be viable as a learning opponent for a single player video game.

Where I do neural nets as having potential is using cloud computing. An AAA game can expect thousands of plays every day (hour, minute...). By collecting this data and training the net on everyone's play strategy learning can be done pretty quickly. I remember reading an article about using cloud computing being used this way a little while ago. So it wouldn't surprise me if someone is already working on or has already deployed something like this.

The question of consciousness is a moot point. Machines will not be considered conscious until the day when a machine can actively articulate the concept of consciousness. Humanity in general will not acknowledge machine consciousness or machine rights until machines actively demand that acknowledgement. Probably through some violent means. If you don't believe me, look at your history books. How many times have 'superior' groups given rights to an 'inferior' one without being compelled too? Fortunately at the rate technology is progressing we won't see this in our lifetimes.

 

MCTS learn's from experience. It is a statistic tree search so it does "learn" not to go down the wrong path from failed experience. In the same manner that a baby animal learns to move from experience and not take the wrong moves after trying multiple times.
So if the world around it changes it can gain new experiences and converge towards a new optimal estimation of a choice.
For me that is some sort of conscious.

However i do agree that if we are talking about getting to a perfect human like state of conscious then NN's got more potential in that regard, but think of dimensions we would need in the NN to get even close to that... a simple ant got 250K neurons in its brain
Also the topology WILL need to change to truly adapt, and then we need techniques like NEAT for NN's which makes it an even harder problem.

Hence for game purposes i am still more interested in stuff like MCTS, Temporal Difference learning/Q-learning.

 

There is a big difference using MCTS vs Mini-max which i guess you are talking about when we are looking a chess computers.

In mini-max you know all the possible proceeding states, you have a full domain knowledge, and you use this to optimize the player choices.

In MCTS and stuff like Q-learning, you only know what previous states (markov property) that led to the state you are exploring in a compressed form, not states that precedes it. You do random play-outs of a game and keep experience for how well it performed at that state. This means that the same state might give good and bad results, but from 100-1000's of trial's you get a good statistical estimate of what the best action might be, but it might not be the optimal option.

There is a big difference there. Thats why mini-max is really bad at high branching games and MCTS is not.

 

That's what I mean with "brute forcing with some tricks", you use sampling results to control sampling frequency of child nodes.
My MCTS poker bot uses all think time it can get to evaluate as many possible futures as possible. Kind of brute forcing.
I think it should win against my DNN bots, but MCTS can be hard to implement, e.g. how to know what the opponent is doing with what probability? You often end up adding an ANN or something similar just for that.
And also picking the right abstractions for complex games is hard, because otherwise the tree gets much too large.
So for me getting a reasonably good player was much easier with DNNs. I'm sure you can beat it with a MCTS player, but that would be some work.

Edit: I'm not sure what you mean by "experience"? In my MCTS poker bot I would start each hand with blank memory (except the opponent model which is external from the MCTS). Saving old trees? There are too many, you never meet the same one again. Or is my MCTS implementation broken and I understood something wrong?

I agree with that. Each tool has its use cases. And some are more specialized than others or easier or harder to use, sometimes also depending on use case.

 

Normally you start from scratch, so the idea is that you gain enough experience every time you run it! That is still "learning" just in a very short period of time. Do your NN save old weights and performance of those? A bit of the same problem.

However there are some very interesting techniques you can use in MCTS. As an example you can use something called roulette wheel selection. With roulette wheel selection you choose a possible action from A from a given state S, but the actions are weighted, such that you have a bigger chance of choosing action A over B. The weights can be modified at runtime if they perform bad or well, or you can use a genetic algorithm to change them. A basic MCTS implementation is, well, basic, but the idea behind it can be tuned a lot.
I mean, in a basic MCTS simulation you do a random playout till you reach an end state, but as we are going for an estimate that is not necessary.... but most people do that because thats what textbooks tell them to do.

 

I train them offline and then they know how to play. I can train different ones for different opponents. Online learning would work, so to get better against the current table, but was never needed for me.

You would lose a lot of money in real money poker games if you would do the textbook thing, you need an opponent model that knows what the opponent does with what frequencies. And for that you need another machine learning technique like ANNs.
Except you can assume the opponent plays like you, then you can "train your MCTS against itself" offline and save the game tree with the game theoretic unbeatable play. So online you just look up actions in that tree. So we end up doing Counterfactual Regret Minimization, but that is not nearly feasible for NL6max (and also perfect play is not possible there). But for heads up limit holdem exactly this was solved this month, so the unbeatable HU limit bot exists now.

 

Your offline trained NN is based on the certain dataset you use to train it. So yes you could train new ones based on different players... but that's not really adapting realtime.

 

I think one aspect which I don't know if it has been solved before is the ability for an AI to classify it's own behaviour somehow, in a hierarchical way, and then use this classification in planning.

For example, the AI could learn to move various limbs to different positions then store these.
Next up it could combine these limb movements into a walking motion.
Then to do something more complicated like go to the shops it would have to combine these into a sequence of "walk", "turn", "run","walk", etc.
Next, to do something like do a job, it could combine these higher order processes into a sequence of "go to the shop", "buy some staples", "do tax return".

Not only this but you need to somehow not only learn sequences but algorithms because the walk to the shops can be different every time.

Maybe "deep learning" neural networks can do this as they combine small features into bigger features etc. But I'm not sure how that would work with learning these processes. Also, we don't know how many of these physical processes are innate (a giraffe can walk as soon as it's born, while a human can't). The more that's all ready pre-programmed by evolution, the less we have to actually learn.

My guess is that we have basic limb movements pre-programmed in, so-to-speak, when we're born and we learn to combine them in different ways. Perhaps things such as "grab" and "move hand to mouth" and "eat" are pre-programmed in. Questions: is the ability to understand numbers built in? Maybe only so far as one, two, many. What about logic? Or are these things learnt?

Also, some people think that these higher order processes might require us to think with language. I'm not sure that's true though because crows can solve quite complicated problems yet can't talk (we don't think!)

It's strange how after so many years of AI research we still don't even know the basics!

 

Well the human brain consists of 86.000.000.000 neurons. Its pretty hard to grasp how it all works

In regards to learning how to walk, run, turn and so on, thats actually what you often use Q-learning for in robots. You give them limbs and a reward for doing things,e.g getting forward in a linear direction at highest possible speed. They will explore how to use the limbs and see what works best through experience. They will in time converge towards running without knowing anything about the term walking, running and so on.
If you they also have information of how much power they have left, then they can "learn" to use different methods based on their "stamina". So in some regards it learns to do stuff without having a term defined for it.

Here is a video on reinforcement learning and a robot example (See example at 4:50):

 

At the end of the day we too follow "rules" so what is intelligence?

Is it the simple matter of following basic to complex understandings and logic to accomplish a predetermined goal?

If so, than even the Pac-Man ghosts of the 1980's had Artificial Intelligence.

In my own opinion,backed by experience and programming knowledge; artificial intelligence, in video games, comes from developing specific logical reasoning and behaviors to your game objects to assess and respond to the game environment and logical problems in a manner best suited to your game type.

If your making a zombie survival game do your NPC just blindly wander along patrol patterns ignoring the zombie horde until they are first attacked? Do your NPC ignore localized gun shots, footsteps, or other audio cue that something or someone is nearby? Do your NPC fight to the death at every battle no matter if they have 100% health or 1% or are out numbered 10-1?

Creating a truly artificial intelligent enemy (depending on what the game object is) requires a large amount of planning and development on the logic and behavior of the game objects.

Logic is the properties the unit has when it is spawned into the game. These can include things like arrays to store information on the scene environment, other objects in the scene, as well as specific settings on what it is and what it can do and what it has to work with. This allows it to "learn" from the scene as it encounters other objects or problems. It will add info to one or more arrays that it will rely on to determine what behavior it will follow when it encounters a problem that requires a solution. Most of these properties are set when the object spawns, others are there for storing future information to assist in the objects behavior when needed.

Behavior is the methods the unit will use to determine what it will do based on the logic it currently has. The behavior is a collection of methods that are called in the co-routine of the object when it is first spawned into the game. These include things like movement, combat, requesting help (read: spawning additional enemies), finding another path, choosing who to attack and with what weapon, deciding if they must retreat, find an exit, find a companion, investigate noises, etc. while they are active.

So a 30,000 foot view of this is a game object (called player) has an array called _hostiles. This array contains a list of object tags, so when ever the player "sees" another game object it will check that objects tag against the current list of hostile tags to determine if it will attack the object or not. Also another bit of logic could be based on the object type, so even if it is not in its hostile list, its object type would than be used to determine what the player will do which may not be attack but avoid, approach, assist, heal, etc.

For example say the player object A comes across another player object B that is currently dead, so Object Condition Type of player object B is Corpse. The player will now check its behavior for finding something dead.

Player object A will continue to default to its next logical behavior based on the current properties until it finds a behavior it will follow until something causes it to begin this logical behavior process again.

Depending on the game type this could be several things like patrol local area, alert other player units, seek out closest player unit, as well possibly altering its properties. Now the general alertness goes up so now the investigate noises chance goes up, the listen range goes up and they may now move in a cautious speed and may now avoid going into specific areas without backup.