Design of ensemble neural network using entropy theory

Publication year: 2011
Source: Advances in Engineering Software, In Press, Corrected Proof, Available online 14 June 2011

Zhiye, Zhao , Yun, Zhang

Ensemble neural networks (ENNs) are commonly used neural networks in many engineering applications due to their better generalization properties. An ENN usually includes several component networks in its structure, and each component network commonly uses a single feed-forward network trained with the back-propagation learning rule. As the neural network architecture has a significant influence on its generalization ability, it is crucial to develop a proper algorithm to determine the ENN architecture. In this paper, an ENN, which combines the component networks using the entropy theory, is proposed. The entropy-based ENN searches the best structure of each component network first, and…

 Highlights: ► We apply the entropy to combine the component neural networks. ► The entropy reduces the over-fitting of the component networks. ► The entropy-based weights improve the overall performance of the ENN. ► The Newton’s method is used for the optimization problem. ► The ENN accuracy is verified by analytical and practical examples.