The gBrain Project

[Lyinginbedmon]

Well since the genes decide fundamentally how the neurons react to different stimulus, you could train a few neurons for one task and then slot that gene string into another similarly-composed neuron set. Traditional neural networks aren't capable of this because the weights are adjusted manually, genetic algorithms in this manner allow us to "swap neurons" between brains.

Still, I doubt this is possible in a practical sense. You've simply got too many variables to account for in a drone. Neural networks work better in theoretical environments with more constant and direct cause/effect scenarios. Your going to have a hard time getting enough training data and training the network in any reasonable amount of time.

I think it's possible. You neglect that Garry's Mod is a simulated environment, and there's only so many things that a drone can find out about it's environment actually within Wiremod (Rangers, water detectors, etc.), and if you take a look back at the loop shown in post #3, it has the capacity to account for a lot of variables from it's environment. The trickiest part really is telling the genes to only look at certain variables in the input or at least variables X to Y in the array.

[Lyinginbedmon]

Obviously the simplest way to get the neuron groups define is to have a sequence inbetween the neuron data that the code can break apart and discern.

For example, it might be in the pattern of "//Pop+Val1-Val2+2.4.3.5.20+3.5.1.6.16+etc.//" for defining one neuron grouping.
At the beginning of a run, the groups are divided into a string array by Grps=Genes:explode("//").
Then, it looks at the first grouping in that array by Grp=Grps[G,string]:explode("+") (Where G is a persisted variable for looping through old school), and derives the cap value for the number of neurons to run through with Pop=Grp[1,string]:toNumber() and the values to use with a quick loop from Inputs[Val1,number] to Inputs[Val2,number] (All inputs need to be broken down to numbers, especially ones from the initial sensory inputs) to create the initial Vals array. To summarise:

• Grp=Grps[G,string]:explode("+")
  • The data for the group in question, G is persisted
• Pop=Grp[1,string]:toNumber()
  • The number of neurons, persisted for the loop between neurons
• Vals=(Array from loop between entry Val1 to Val2 in Inputs)
  • The initial Vals array from the Inputs numerical array
• Neur=Grp:remove(1) Neur:remove(1)
  • The neuron data array used for the neuron loop

Although, if the Pop and Vals data is stored with the neurons, it means we need to take the player-set arrangement and insert it into the neuron gene data, so perhaps better to hold it in a separate array, with just "Pop+Val1-Val2" in each entry and then have the neurons be one big clump, with Neurons the string and then Neur being the array resulting from Neurons:explode("+"), just removing the neuron data as it goes through until the chip runs out of groups or neurons.

[N00bDud3]

This is awesome. I hope that you are able to do this, because anything is possible... wait.. if anything is possible, then it has to be possible for something to be impossible, but if anything is possible, the task that is impossible is possible *head asplodes*

[SystemsLock]

I think it's possible. You neglect that Garry's Mod is a simulated environment, and there's only so many things that a drone can find out about it's environment actually within Wiremod (Rangers, water detectors, etc.), and if you take a look back at the loop shown in post #3, it has the capacity to account for a lot of variables from it's environment. The trickiest part really is telling the genes to only look at certain variables in the input or at least variables X to Y in the array.

How do you even plan to train this thing? You will need far too much training data then you can provide by manually interacting with it. Concepts such as long term goal setting are complex problems in the actual scientific field. Your going to have to start with the small scale and work your way up.

[Lyinginbedmon]

What I'm trying to think about at the moment is the method by which the neurons alter and change to move closer to and away from certain outcomes according to the good/bad impulse.

Firstly, the cause must be located. Considering the Intuition side works with basic definitions, that makes things rather simple. We have a variable holding the most recent deciding neuron group, as it can be assumed the most recent impulse was caused by the most recent action undertaken (Recalling that the impulse is only generated by intuition-responsible actions, not the Pattern side which tracks it's own performance and accounts for it personally).

The hard part though is figuring out how to have the algorithm remodel the neurons involved in that group. Perhaps in stages? So we have an array with the most recent element of a given neuron group to be altered, based on the breakapart of the group with a brief loop exploding it by both "+" and "." (Thereby revealing the individual elements of each neuron in a large bundle). Whenever the impulse is received for that group, the algorithm goes to the next one in line and alters it accordingly. Switches from good to bad or vice versa reset the algorithm's position in that group and it starts over again (So a good impulse isn't hindered by the workings of a bad impulse and such).

So, for example, let's say we have 7 neurons in a group, and they make the most recent Intuition decision and receive a Bad impulse.
This group only just started misbehaving, so the algorithm starts with the first neuron, #7, and lowers the weight of it's first connection.
Unfortunately, another Bad impulse is received, so it lowers the weight on neuron #7's second connection.
Things aren't going too well for this group, and #7's activation value gets raised.
Neuron #7 doesn't have any more connections, so the algorithm shifts down to neuron #6, with the fifth element.

Probably best if the algorithm has a duplicated table from it's gene string actually, just exploded by ".". That way it can discern what elements are connections and what elements are activation values (Because the latter will always be a string with "+" in the middle, rather than a string-ified number). This allows it to differentiate whether to lower or raise the value.

Of course, once the algorithm runs out of elements to chase in the group, it goes back to the start again.

How do you even plan to train this thing? You will need far too much training data then you can provide by manually interacting with it. Concepts such as long term goal setting are complex problems in the actual scientific field. Your going to have to start with the small scale and work your way up.

Well "learning" depends on the neuron modification. Let's assume the method above is the one used and take a fairly simple homeostatic example.

We have a drone that has it's motion in a state of requiring an engine to run at a reasonably constant speed of 25 mph. The gBrain attached to it is assigned to give one group of neurons power over the throttle of that engine, and another group of neurons power of the brake. Both groups are told to pay attention to the speed and the delta of the speed of that engine.

• The engine reaches 25 mph with delta +1 and all is well, but then keeps going and the engine passes 25. Now, it is 25+ and still $1. At this point, the group controlling the throttle starts receiving a Bad impulse.
• Eventually, the throttle group backs down and the brake group pulls in, turning the delta to -1, and gradually the speed goes down, because past 25 mph a negative delta receives a Good impulse.
• However, once again the speed goes out of bounds and hits 19 mph. So now the brake group gets a Bad impulse again. But, on the other hand, whenever the throttle group decides to shoot, it gets a Good impulse.
• After some time passes, the brake is calibrated to only activate when the speed exceeds 25 mph regardless of it's existing delta, and the throttle learns to trigger in similar fashion when the speed drops below 25.

Discerning what is a good and bad response is part of the sensory perception of the drone, which must be plugged in from an external source to the gBrain as a simple Good/Bad impulse which the central core acts upon whenever the Intuition side has caused the action. (So you would in this instance need a secondary simple E2 also monitoring the speed and delta attached to the core)

The neurons trigger actions, but they don't figure out what those actions are. That part is in the Pattern side, which figures out what to do to walk and to reach objects and such where simply letting the neurons do so wouldn't be appropriate (This is where certainty comes into the equation, so the central core knows which to decide from).

Pattern is the side that deals more with long-term processes, because it doesn't evaluate singular actions as good or bad, but instead evaluates an overall string of events according to their success against a specific target. Since we haven't looked at programming that yet we can't really be certain how it'll work but we can see how it will perform within the duality of the gBrain with Intuition.

[SystemsLock]

Ok, but I still don't see how your training it. As I'm sure you know, you simply can't tell a network that it's been good or bad and expect it to shape itself up. I believe you mentioned in your first post you were using an evolving neural network. So the good and bad inputs affect the networks fitness and its likelihood of reproduction. This works but not necessarily in live action. Your going to need at hard wire a training set (or training environment) for it to play in and learn before its pushed out into the real world.

The method you just mentioned however makes no sense for a practical standpoint. It sounds like your trying to do backpropagation, while avoiding it. Backpropagation may work in your situation but you'll have to look into it. However with that your going to really need training sets and it can often take thousands of runs before the network has stabilized.

Oh yeah and I don't quite see why you need so many miniature networks. One large network is commonly how these things are built. The network will formulate its own groupings on its own.

[Lyinginbedmon]

Would you care to explain how backpropagation usually works then? :huh:

And the grouping neurons instead of using all neurons for all problems is because we are for the sake of ease of coding using loops through neurons rather than predefined architectures and smaller networks on average learn faster than larger networks simply because there is less to change.

[Caples]

I think that something so outrageously ambitious should start with something that is already being done. Supercomputers have been made with PS3s and 360s that are designed to mimic the construction and function of a very basic brain. I think it would be best to begin with something on this scale before trying to created a "learning" AI. And, by the way, nobody has created a "learning" AI yet even with supercomputers.

[Schilcote]

And, by the way, nobody has created a "learning" AI yet even with supercomputers.

Wrong. There's been learning AIs for a long time. Video games have a lot of them.

Here's one.

[jesse1412]

black and white had an excellent feature, a creature which would learn from doing something, succeeding, failing, being punished or being treated