**Abstract**

The main objective of this paper is to introduce a new intelligent optimization technique that uses a predictioncorrection

strategy supported by a recurrent neural network for finding a near optimal solution of a given

objective function. Recently there have been attempts for using artificial neural networks (ANNs) in optimization

problems and some types of ANNs such as Hopfield network and Boltzmann machine have been applied in

combinatorial optimization problems. However, ANNs cannot optimize continuous functions and discrete

problems should be mapped into the neural networks architecture. To overcome these shortages, we introduce a

new procedure for stochastic optimization by a recurrent artificial neural network. The introduced neurooptimizer

(NO) starts with an initial solution and adjusts its weights by a new heuristic and unsupervised rule to

compute the best solution. Therefore, in each iteration, NO generates a new solution to reach the optimal or

near optimal solutions. For comparison and detailed description, the introduced NO is compared to genetic

algorithm and particle swarm optimization methods. Then, the proposed method is used to design the optimal

controller parameters for a five bar linkage manipulator robot. The important characteristics of NO are:

convergence to optimal or near optimal solutions, escaping from local minima, less function evaluation, high

convergence rate and easy to implement.

strategy supported by a recurrent neural network for finding a near optimal solution of a given

objective function. Recently there have been attempts for using artificial neural networks (ANNs) in optimization

problems and some types of ANNs such as Hopfield network and Boltzmann machine have been applied in

combinatorial optimization problems. However, ANNs cannot optimize continuous functions and discrete

problems should be mapped into the neural networks architecture. To overcome these shortages, we introduce a

new procedure for stochastic optimization by a recurrent artificial neural network. The introduced neurooptimizer

(NO) starts with an initial solution and adjusts its weights by a new heuristic and unsupervised rule to

compute the best solution. Therefore, in each iteration, NO generates a new solution to reach the optimal or

near optimal solutions. For comparison and detailed description, the introduced NO is compared to genetic

algorithm and particle swarm optimization methods. Then, the proposed method is used to design the optimal

controller parameters for a five bar linkage manipulator robot. The important characteristics of NO are:

convergence to optimal or near optimal solutions, escaping from local minima, less function evaluation, high

convergence rate and easy to implement.

**Keywords**

[1] J. Hopfield, and D. Tank, Neural computation

of decisions in optimization problems,

Biological Cybernetics, Vol. 52, 1985, pp. 141-

152.

[2] G.E. Hinton, and T.J. Sejnowsky, Optimal

perceptual inference, Proceedings of the IEEE

Conference on Computer Vision and Pattern

Recognition, Washigton, 1983, pp. 448-453.

[3] D. Amit, H. Gutfreund, and H. Sompolinsky,

Spin-Glass models of neural networks,

Physical Review Letters A 32, 1985, pp. 1007-

1018.

[4] Y. Akiyama, A. Yamashita, M. Kajiura, and H.

Aiso, Combinatorial optimization with

gaussian machines, Proceedings IEEE

International Joint Conference on Neural

Networks 1, 1989, pp. 533–540.

[5] T. Kohonen, Self-Organized formation of

topologically correct feature maps, Biological

Cybernetics 43, 1982, pp. 59–69.

[6] A.H. Gee, and R. W. Prager, Limitations of

neural networks for solving traveling salesman

problems, IEEE Trans. Neural Networks, vol.

6, 1995, pp. 280–282.

[7] M. Goldstein, Self-Organizing feature maps for

the multiple traveling salesman problem

(MTSP), Proceedings IEEE International

Conference on Neural Networks, Paris, 1990,

pp. 258–261.

[8] Y. P. S. Foo, and Y. Takefuji, Stochastic neural

networks for job-shop scheduling: parts 1 and

2, Proceedings of the IEEE International

Conference on Neural Networks 2, 1988, pp.

275–290.

[9] Y.P. S. Foo, and Y. Takefuji, Integer Linear

programming neural networks for job shop

scheduling, Proceedings of the IEEE

International Conference on Neural Networks

2, 1988, pp. 341–348.

[10] J.S. Lai, S.Y. Kuo, and I.Y. Chen, Neural

networks for optimization problems in graph

theory, Proceedings IEEE International

Symposium on Circuits and Systems 6, 1994,

pp. 269–272.

[11] D.E. Van Den Bout, and T.K. Miller, Graph

partitioning using annealed neural networks,

IEEE Transactions on Neural Networks 1,

1990, pp. 192–203.

[12] S. Vaithyanathan, H. Ogmen, and J. IGNIZIO,

Generalized boltzmann machines for

multidimensional knapsack problems,

Intelligent Engineering Systems Through

Artificial Neural Networks 4, ASME Press,

New York, 1994, pp. 1079–1084.

[13] A. Yamamoto, M. Ohta, H. Ueda, A. Ogihara,

and K. Fukunaga, Asymmetric neural network

and its application to knapsack problem, IEICE

Transactions Fundamentals E78-A, 1995, pp.

300–305.

[14] K. Urahama, and H. Nishiyuki, Neural

algorithms for placement problems,

Proceedings International Joint Conference on

Neural Networks 3, Nagoya, 1993, pp. 2421–

2424.

[15] K.E. Nygard, P. Jueli, and N. Kadaba, Neural

networks for selecting vehicle routing

heuristics, ORSA Journal of Computing 2,

1990, pp. 353–364.

[16] A.I. Vakhutinsky, and B. L. Golden, Solving

vehicle routing problems using elastic nets,

Proceedings IEEE International Conference on

Neural Networks 7, 1994, pp. 4535–4540.

[17] L. Fang, W. H. Wilson, and T. Li, Mean-Field

annealing neural net for quadratic assignment,

Proceedings International Conference on

Neural Networks, Paris, 1990, pp. 282–286.

[18] G.A. Tagliarini, and E. W. Page, Solving

constraint satisfaction problems with neural

networks, Proceedings IEEE International

Conference on Neural Networks 3, 1987, pp.

741–747.

[19] M. Kajiura, Y. Akiyama, and Y. Anzai, Solving

large scale puzzles with neural networks,

Proceedings Tools for AI Conference, Fairfax,

1990, pp. 562–569.

[20] N. Funabiki and Y. Takefuji, A neural network

parallel algorithm for channel assignment

problems in cellular radio networks, IEEE

Trans. Veh. Technol., vol. 41, Nov. 1992, pp.

430–437.

[21] K. Smith, and M. Palaniswami, Static and

of decisions in optimization problems,

Biological Cybernetics, Vol. 52, 1985, pp. 141-

152.

[2] G.E. Hinton, and T.J. Sejnowsky, Optimal

perceptual inference, Proceedings of the IEEE

Conference on Computer Vision and Pattern

Recognition, Washigton, 1983, pp. 448-453.

[3] D. Amit, H. Gutfreund, and H. Sompolinsky,

Spin-Glass models of neural networks,

Physical Review Letters A 32, 1985, pp. 1007-

1018.

[4] Y. Akiyama, A. Yamashita, M. Kajiura, and H.

Aiso, Combinatorial optimization with

gaussian machines, Proceedings IEEE

International Joint Conference on Neural

Networks 1, 1989, pp. 533–540.

[5] T. Kohonen, Self-Organized formation of

topologically correct feature maps, Biological

Cybernetics 43, 1982, pp. 59–69.

[6] A.H. Gee, and R. W. Prager, Limitations of

neural networks for solving traveling salesman

problems, IEEE Trans. Neural Networks, vol.

6, 1995, pp. 280–282.

[7] M. Goldstein, Self-Organizing feature maps for

the multiple traveling salesman problem

(MTSP), Proceedings IEEE International

Conference on Neural Networks, Paris, 1990,

pp. 258–261.

[8] Y. P. S. Foo, and Y. Takefuji, Stochastic neural

networks for job-shop scheduling: parts 1 and

2, Proceedings of the IEEE International

Conference on Neural Networks 2, 1988, pp.

275–290.

[9] Y.P. S. Foo, and Y. Takefuji, Integer Linear

programming neural networks for job shop

scheduling, Proceedings of the IEEE

International Conference on Neural Networks

2, 1988, pp. 341–348.

[10] J.S. Lai, S.Y. Kuo, and I.Y. Chen, Neural

networks for optimization problems in graph

theory, Proceedings IEEE International

Symposium on Circuits and Systems 6, 1994,

pp. 269–272.

[11] D.E. Van Den Bout, and T.K. Miller, Graph

partitioning using annealed neural networks,

IEEE Transactions on Neural Networks 1,

1990, pp. 192–203.

[12] S. Vaithyanathan, H. Ogmen, and J. IGNIZIO,

Generalized boltzmann machines for

multidimensional knapsack problems,

Intelligent Engineering Systems Through

Artificial Neural Networks 4, ASME Press,

New York, 1994, pp. 1079–1084.

[13] A. Yamamoto, M. Ohta, H. Ueda, A. Ogihara,

and K. Fukunaga, Asymmetric neural network

and its application to knapsack problem, IEICE

Transactions Fundamentals E78-A, 1995, pp.

300–305.

[14] K. Urahama, and H. Nishiyuki, Neural

algorithms for placement problems,

Proceedings International Joint Conference on

Neural Networks 3, Nagoya, 1993, pp. 2421–

2424.

[15] K.E. Nygard, P. Jueli, and N. Kadaba, Neural

networks for selecting vehicle routing

heuristics, ORSA Journal of Computing 2,

1990, pp. 353–364.

[16] A.I. Vakhutinsky, and B. L. Golden, Solving

vehicle routing problems using elastic nets,

Proceedings IEEE International Conference on

Neural Networks 7, 1994, pp. 4535–4540.

[17] L. Fang, W. H. Wilson, and T. Li, Mean-Field

annealing neural net for quadratic assignment,

Proceedings International Conference on

Neural Networks, Paris, 1990, pp. 282–286.

[18] G.A. Tagliarini, and E. W. Page, Solving

constraint satisfaction problems with neural

networks, Proceedings IEEE International

Conference on Neural Networks 3, 1987, pp.

741–747.

[19] M. Kajiura, Y. Akiyama, and Y. Anzai, Solving

large scale puzzles with neural networks,

Proceedings Tools for AI Conference, Fairfax,

1990, pp. 562–569.

[20] N. Funabiki and Y. Takefuji, A neural network

parallel algorithm for channel assignment

problems in cellular radio networks, IEEE

Trans. Veh. Technol., vol. 41, Nov. 1992, pp.

430–437.

[21] K. Smith, and M. Palaniswami, Static and

dynamic channel assignment using neural

networks, IEEE Journal on Selected Areas in

Communications 15, 1997, pp. 238–249.

[22] T. Bultan and C. Aykanat, Circuit partitioning

using parallel mean field annealing algorithms,

Proceedings 3rd IEEE Symposium on Parallel

and Distributed Processing, 1991, pp. 534–541.

[23] U. Halici, Artificial neural networks, EE 543

Lecture Notes, Middle East Technical

University, Ankara, Turkey, 2004.

[24] J.H. Holland, Adaptation in natural and

artificial systems, University of Michigan

Press, Ann Arbor, MI, Internal Report, 1975.

[25] Y. Shi, and R. Eberhart, A modified particle

swarm optimizer, Proceedings of the IEEE

international conference on evolutionary

computation, Piscataway, NJ: IEEE Press;

1998, pp. 69–73.

[26] J.G. Ziegler and N.B. Nichols, “Optimum

settlings for automatic controllers,” Trans. On

ASME., vol. 64, pp. 759-768, 1942.

[27] Z.L. Gaing, “A Particle Swarm Optimization

Approach for Optimum Design of PID

controller in AVR system,” IEEE Transactions

on Energy Conversion, vol. 9, no. 2, pp. 384-

391, 2003.

[28] Z.Y. Zhao, M. Tomizuka, and S. Isaka, “Fuzzy

gain scheduling of PID controllers,” IEEE

Trans. System, Man, and Cybernetics, vol. 23,

no. 5, pp. 1392-1398, 1993.

[29] S.Y. Chu, C.C. Teng, “Tuning of PID

controllers based on gain and phase margin

specifications using fuzzy neural network,”

Fuzzy Sets and Systems, vol. 101, no. 1, pp.

21-30, 1999.

[30] G. Zhou and J.D. Birdwell, “Fuzzy logic-based

PID autotuner design using simulated

annealing,” Proceedings of the IEEE/IFAC

Joint Symposium on Computer-Aided Control

System Design, pp. 67 – 72, 1994.

[31] R.A. Krohling and J.P. Rey, “Design of optimal

disturbance rejection PID controllers using

genetic algorithm,” IEEE Trans. Evol.

Comput., vol. 5, pp. 78–82, 2001.

[32] D.H. Kim, “Tuning of a PID controller using a

artificial immune network model and local

fuzzy set,” Proceedings of the Joint 9th IFSA

World Congress and 20th NAFIPS

International Conference, vol. 5, pp. 2698 –

2703, 2001.

[33] Y.T. Hsiao, C.L. Chuang, and C.C. Chien, “Ant

colony optimization for designing of PID

controllers,” Proceedings of the 2004 IEEE

Conference on Control Applications/

International Symposium on Intelligent

Control/International Symposium on Computer

Aided Control Systems Design, Taipei, Taiwan,

2004.

[34] D. Wang and M. Vidyasagar, “Modeling of a

five-bar-Linkage Manipulator with One

Flexible Link,” in Proc. IEEE Int. Symp,

subject, Turkey, pp. 21–26, 1988.

[35] D. Wang, J.P. Huissoon and K. Luscott, “A

teaching robot for demonstrating robot control

strategies,” manufacturing research corporation

of Ontario, 1993.

networks, IEEE Journal on Selected Areas in

Communications 15, 1997, pp. 238–249.

[22] T. Bultan and C. Aykanat, Circuit partitioning

using parallel mean field annealing algorithms,

Proceedings 3rd IEEE Symposium on Parallel

and Distributed Processing, 1991, pp. 534–541.

[23] U. Halici, Artificial neural networks, EE 543

Lecture Notes, Middle East Technical

University, Ankara, Turkey, 2004.

[24] J.H. Holland, Adaptation in natural and

artificial systems, University of Michigan

Press, Ann Arbor, MI, Internal Report, 1975.

[25] Y. Shi, and R. Eberhart, A modified particle

swarm optimizer, Proceedings of the IEEE

international conference on evolutionary

computation, Piscataway, NJ: IEEE Press;

1998, pp. 69–73.

[26] J.G. Ziegler and N.B. Nichols, “Optimum

settlings for automatic controllers,” Trans. On

ASME., vol. 64, pp. 759-768, 1942.

[27] Z.L. Gaing, “A Particle Swarm Optimization

Approach for Optimum Design of PID

controller in AVR system,” IEEE Transactions

on Energy Conversion, vol. 9, no. 2, pp. 384-

391, 2003.

[28] Z.Y. Zhao, M. Tomizuka, and S. Isaka, “Fuzzy

gain scheduling of PID controllers,” IEEE

Trans. System, Man, and Cybernetics, vol. 23,

no. 5, pp. 1392-1398, 1993.

[29] S.Y. Chu, C.C. Teng, “Tuning of PID

controllers based on gain and phase margin

specifications using fuzzy neural network,”

Fuzzy Sets and Systems, vol. 101, no. 1, pp.

21-30, 1999.

[30] G. Zhou and J.D. Birdwell, “Fuzzy logic-based

PID autotuner design using simulated

annealing,” Proceedings of the IEEE/IFAC

Joint Symposium on Computer-Aided Control

System Design, pp. 67 – 72, 1994.

[31] R.A. Krohling and J.P. Rey, “Design of optimal

disturbance rejection PID controllers using

genetic algorithm,” IEEE Trans. Evol.

Comput., vol. 5, pp. 78–82, 2001.

[32] D.H. Kim, “Tuning of a PID controller using a

artificial immune network model and local

fuzzy set,” Proceedings of the Joint 9th IFSA

World Congress and 20th NAFIPS

International Conference, vol. 5, pp. 2698 –

2703, 2001.

[33] Y.T. Hsiao, C.L. Chuang, and C.C. Chien, “Ant

colony optimization for designing of PID

controllers,” Proceedings of the 2004 IEEE

Conference on Control Applications/

International Symposium on Intelligent

Control/International Symposium on Computer

Aided Control Systems Design, Taipei, Taiwan,

2004.

[34] D. Wang and M. Vidyasagar, “Modeling of a

five-bar-Linkage Manipulator with One

Flexible Link,” in Proc. IEEE Int. Symp,

subject, Turkey, pp. 21–26, 1988.

[35] D. Wang, J.P. Huissoon and K. Luscott, “A

teaching robot for demonstrating robot control

strategies,” manufacturing research corporation

of Ontario, 1993.