>
!"#$%&'()*+,./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\]^_`a
!"#$%&'()*+,./01Root EntrydO)+a3[Current User&SummaryInformation(PowerPoint Document()DocumentSummaryInformation85(
/00DTimes New RomanDArial" DCourier New10DWingdingsR
`.
@n?" dd@ @@``\T _
c$@dʚ;2Nʚ;g4OdOdv0ppp@<4!d!d`
0L><4dddd`
0L>H
` ` ̙33` 333MMM` ff3333f` f` f` 3>?" dd@,?" dd@ " @ ` n?" dd@ @@``PR @ ` `p>>f(
6j
T Click to edit Master title style!
!
0< j
RClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level!
S
0X V
>*
0 vV
@*
0$ !V
@*H
0h ? ̙33 Default DesignB (
V
B[
EA, neural networks & fuzzy systems
Michael J. Watts
http://mike.watts.net.nz$______#g,`eee`e`e@ Hh8<
c$ ? ̙33z0(
0T
vLecture Outline_@ Hh8
0CTX@8___PPT9
0
Advantages of fuzzy systems
Problems with fuzzy systems
Applying EA to fuzzy systems
Problems with ANN
Applying EA to neural networks8
" _8
" _8
" _8
" _8
" _ @ Hh8<
c$ ? ̙33@4(
0T
6Advantages of Fuzzy Systems_@ Hh8
0CTX@8___PPT9
.Comprehensibility
Parsimony
Modularity
Explainability
Uncertainty
Parallelism
RobustH8
" _
8
" _8
" _8
" _8
" _8
" _8
" _
@ Hh8<
c$ ? ̙33RP(
0T
<Disadvantages of Fuzzy Systems_@ Hh8R
0CTX@8___PPT9
Defining the rules
where do the rules come from?
major problem with rulebased systems
need to get enough rules to be accurate
rules need to be expressive
comprehensibility
rules need to be accurate
mustn t use too many rules8
" _8oP_&8oP_(8oP_8oP_8_8oP_8oP_ %'@ Hh8<
c$ ? ̙332*`(
0T
<Disadvantages of Fuzzy Systems_@ Hh8
0CTX@8___PPT9
Optimisation
a change in the MF can require a change in the rules
a change in the rules can require a change in the MF
each parameter / choice effects the others
multiparameter optimisation problem
8
" _58oP_58oP_+8oP_%8oP_ 44*%@ Hh8<
c$ ? ̙33pJ(
0T
8Applying EA to Fuzzy Systems_@ Hh8
0CTX@8___PPT9
BDMany of the problems with fuzzy systems are combinatorial in nature
MF parameters
MF / rule interdependencies
EA are well suited to solving combinatorial problemsD8
" _8oP_8oP_58
" _C
5 @ Hh8<
c$ ? ̙33 <(
0T
8Applying EA to Fuzzy Systems_@ Hh8
0CTX@8___PPT9
42There are three main ways in which evolutionary algorithms have been applied to fuzzy systems
optimising MF
optimising rules
optimising the entire system^8
" _8oP_8oP_8oP_]
@ Hh8<
c$ ? ̙33~.&$(
$
$ 0T
$Optimisation of MF_@ Hh8
$ 0CTX@8___PPT9
Selection of MF parameters
Use a fixed number of MF
Fixed number of rules
EA selects e.g. centre and width of MF8
" _8
" _8
" _'8
" _ ' @ Hh8<
$c$ ? ̙33ld((
(
( 0T
$Optimisation of MF_@ Hh8
( 0CTX@8___PPT9
Optimisation of existing MF
Evolve deltas for the centres / widths of the MF
Fixed number of MF
Initial parameters determined a priori8
" _18
" _8
" _'8
" _ 0 @ Hh8<
(c$ ? ̙33@,(
,
, 0T
$Optimisation of MF_@ Hh8X
, 0CTX@8___PPT9
nProblems with this approach
Fixed number of MF
Optimising MF without optimising rules
Must have the optimal number of rules beforehand
how do you know this if the MF aren t optimised?8
" _8
" _'8
" _18
" _18oP_ & 0 1@ Hh8<
,c$ ? ̙330f(
0
0 0T
*Optimisation of Rules_@ Hh8,
0 0CTX@8___PPT9
lBFixed number of MF
Fixed number of rules
EA selects which MF is active for each input and output for each rule
May vary number of antecedents
null entries for MF8
" _8
" _F8
" _8
" _8oP_ E @ Hh8<
0c$ ? ̙33j4(
4
4 0T
*Optimisation of Rules_@ Hh8
4 0CTX@8___PPT9
*Problems with this approach
Fixed number of MF
Fixed parameters of MF
are the MF optimal?
Fixed number of rules
are there enough?
are there too many?T8
" _8
" _8
" _8oP_8
" _8oP_8oP_ @ Hh8<
4c$ ? ̙33h8(
8
8 0T
:Optimisation of Fuzzy Systems_@ Hh8j
8 0CTX@8___PPT9
Evolve both MF and rules simultaneously
Obviates problems with interdependency of MF and rules
Many methods in use(8
" _78
" _8
" _' 6 @ Hh8<
8c$ ? ̙33j<(
<
< 0T
:Optimisation of Fuzzy Systems_@ Hh8l
< 0CTX@8___PPT9
NOne rule for each combination of input MF
EA evolves
parameters of input MF
output MF to activate
Evolving a rule matrix
Still problems with fixed number of rules / MF *8
" _8
" _8oP_8oP_8
" _/8
" _)
/ @ Hh8<
<c$ ? ̙334@(
@
@ 0T
:Optimisation of Fuzzy Systems_@ Hh86
@ 0CTX@8___PPT9
vPUse a messy GA
Evolve the number of rules
Evolution will retain only the necessary rules
Rules will be minimal length
Rule encoding consists of a list of MF parameters8
" _8
" _/8
" _8
" _28
" _ . 2 @ Hh8<
@c$ ? ̙33t$D(
D
D 0T
z"Problems with ANN_@ Hh8
D 0CTX@8___PPT9
Choosing the number of hidden layers
how many are enough?
Choosing the number of hidden nodes
how many are enough?
As number of connections approaches the number of training examples, generalisation decreases%8
" _8oP_$8
" _8oP_^8
" _$ # ^ @ Hh8<
Dc$ ? ̙338 H(
H
H 0T
z"Problems with ANN_@ Hh8R
H 0CTX@8___PPT9
Initialisation of weights
Random initialisation can cause problems with training
start in a bad spot8
" _78
" _8oP_ 6 @ Hh8<
Hc$ ? ̙33D<0L(
L
L 0T
nEA and ANNs_@ Hh8
L 0CTX@8___PPT9
Many aspects of using ANNs can be approached by EA
Topology selection
number of hidden layers
number of nodes in hidden layers38
" _8
" _8oP_!8oP_2 !@ Hh8<
Lc$ ? ̙33@Pb(
P
P 0T
nEA and ANNs_@ Hh8<
P 0CTX@8___PPT9
Connection weights
initial weight values for backpropagation training
EA based training8
" _38oP_8oP_ 2@ Hh8<
Pc$ ? ̙33>6PT(
T
T 0T
8ANN Topology Selection by EA_@ Hh8
T 0CTX@8___PPT9
Hidden layers
Arena, 1993. Treats each chromosome as a 2D matrix
Each cell of the matrix indicates the presence or absence of a neuron8
" _38
" _F8
" _
2 F @ Hh8<
Tc$ ? ̙33"`Xn(
X
X 0T
8ANN Topology Selection by EA_@ Hh8&
X 0CTX@8___PPT9
f<Schiffman, Joost and Werner, 1993
Chromosome determines number of nodes present
Also indicate connectivity of the nodes
Problems
initialisation of the weights"8
" _.8
" _(8
" _ 8
" _8oP_!  ' @ Hh8<
Xc$ ? ̙33H@p\(
\
\ 0T
0Initial Weight Selection_@ Hh8
\ 0CTX@8___PPT9
Performance of backprop influenced by initial weights of network
Belew, McInerney and Schraudolph, 1991
Used a GA to select the initial values of the network
Fitness determined by how quickly the network trains and how well it solves the problemA8
" _'8oP_68
" _X8
" _@ &5 X @ Hh8<
\c$ ? ̙33`P(
`
` 0T
8Selecting Control Parameters_@ Hh8
` 0CTX@8___PPT9
HChoi and Bluff, 1995
Used a GA to select for an MLP
number of hidden nodes
learning rate
momentum
training epochs(8
" _8
" _8oP_8oP_ 8oP_8oP_
@ Hh8<
`c$ ? ̙33Bd(
d
d 0T
8Selecting Control Parameters_@ Hh8F
d 0CTX@8___PPT9
Watts, Major and Tate, 2002
Used a GA to select for an MLP
input features
hidden neurons
learning rate
momentum
Epochs\8
" _8
" _8oP_8oP_8oP_ 8oP_8oP_
@ Hh8<
dc$ ? ̙33^h(
h
h 0T
nEA Training_@ Hh8
h 0CTX@8___PPT9
EA used to select values of connection weights
Fitness determined as the inverse of the error over the training set
EA will seek to minimise error
Initial weights of network encoded into an individual in the initial EA population/8
" _E8
" _8oP_S8
" _. D S @ Hh8<
hc$ ? ̙33d\l(
l
l 0T
nEA Training_@ Hh8
l 0CTX@8___PPT9
GA can be used
permutation problem
EP has proven to be useful for this
Blondie 24
ES is not commonly used to train ANN8
" _8oP_$8
" _8oP_%8
" _ #
% @ Hh8<
lc$ ? ̙33LDp(
p
p 0T
$Other Applications_@ Hh8
p 0CTX@8___PPT9
.Hugo de Garis fully self connected networks
Each neuron in the network is connected to every other neuron, as well as itself
Can only be trained by GA8
" _Q8
" _8
" _, P @ Hh8<
pc$ ? ̙33`t(
t
t 0T
$Other Applications_@ Hh8x
t 0CTX@8___PPT9
Learning algorithms can also be evolved
rather than using an existing learning rule, the EA evolved one
Not widely used(8
" _@8oP_8
" _' ? @ Hh8<
tc$ ? ̙33`x(
x
x 0T
fSummary_@ Hh8
x 0CTX@8___PPT9
EA are capable of optimising several aspects of fuzzy systems
Best to use an EA to evolve the entire system
optimisation of components ignores problems with interdependencies between these components
Many of the problems associated with ANN can be addressed with EA
Topology selection, parameter selection and training are the most common>8
" _.8
" _\8oP_B8
" _I8
" _=  [A I @ Hh8<
xc$ ? ̙33 d(

 0T
lReferences_@ Hh8@
 0>TX@8___PPT9
P. Arena, R. Caponetto L. Fortuna and M.G. Xibilia. M.L.P. Optimal Topology via Genetic Algorithms. In: Artificial Neural Nets and Genetic Algorithms pg670674
B. Choi and K. Bluff . Genetic Optimisation of Control Parameters of a Neural Network, In: Proceedings of ANNES'95 pg174177, 1995
R.K. Belew, J. McInerney and N.N. Schraudolph. Evolving Networks: Using the Genetic Algorithm with Connectionist Learning. In: Artificial Life III pg511547
W. Schiffmann M. Joost and R.Werner Application of Genetic Algorithms to the Construction of Topologies for Multilayer Perceptrons. In: Artificial Neural Nets and Genetic Algorithms pg 675682@8
" _8
" _8
" _8
" _h[ @ Hh8<
c$ ? ̙33r=+O$) .
3]8!=
iBGLmQV\aesjn'sQx}Gi
u ݡ
t_GoogleOh+'0`h
PowerPoint PresentationGoogleGoogle1@0BwU@,p&XWQet.5RylQ]J*2tVcJj2xI#
g3dIoj@o+Iij(.:BJB>8$\$,a#x,X<@V3s//5CVp
'R~q$FFby/gnLzU^^CUC$kMicrosoft PowerPoint՜.+,0
lCustom)Fonts UsedDesign Template
Slide Titles#ArialCourier NewTimes New Roman
WingdingsDefault DesignSlide 1Lecture OutlineAdvantages of Fuzzy SystemsDisadvantages of Fuzzy SystemsDisadvantages of Fuzzy SystemsApplying EA to Fuzzy SystemsApplying EA to Fuzzy SystemsOptimisation of MFOptimisation of MFOptimisation of MFOptimisation of RulesOptimisation of RulesOptimisation of Fuzzy SystemsOptimisation of Fuzzy SystemsOptimisation of Fuzzy SystemsProblems with ANNProblems with ANNEA and ANNsEA and ANNsANN Topology Selection by EAANN Topology Selection by EAInitial Weight SelectionSelecting Control ParametersSelecting Control ParametersEA TrainingEA TrainingOther ApplicationsOther ApplicationsSummaryReferences