QA199 : Presentsaneuralnetworkmodelforsolvingtheregressionproblem
Thesis > Central Library of Shahrood University > Mathematical Sciences > MSc > 2013
Authors:
Abstarct: SUPPORTVECTORMACHINES(SVMs),motivatedbystatisticallearningtheory, have recently received considerable attention in the field of machine learning [1]–[17]. Their foundation has been developed by Vapnik and has obtained popularity [1], [2]. The main feature of SVMs is that they use the structural risk minimizationratherthantheempiricalriskminimization. Unlikethewell-known multilxayer perceptron (MLP) and radial-basis function (RBF) networks, training anSVMisequivalenttosolvingalinearlyconstrainedconvexquadraticprogramming problem. Since the MLP and RBF networks rely on the minimization of a nonlinear error function which may be nonconvex, the local minima problem in trainingcanbeavoidedbyusingtheSVMapproach. SVMswerefirstdeveloped to solve the classification problem. Since SVMs have robust properties against noise, SVMs have also been applied to the domain of regression problems [4]. baxsedonnumericaloptimizationmethods,severalalgorithmsandtheirimprovementsforsupportvectorclassification(SVC)andsupportvectorregression(SVR) have been proposed [9]–[15]. In many engineering and military applications, the demands on real-time data processing is often needed, such as classification in complexelectromagneticenvironments,recognitioninmedicaldiagnosticsradar objectrecognitioninstrongbackgroundclutter,etc. [16]. Therefore,paralleland distributed approaches to training SVMs are necessary and desirable [17]. It is wellknownthatthereal-timeprocessingabilityofneuralnetworksisoneoftheir mostimportantadvantages[18]. Inrecentyears,neuralnetworkapproacheshave demonstrated their great promise for optimization [19]–[23]. Reported results of numerous investigations have shown many advantages over the traditional optimization algorithms, especially in real-time applications [24]–[27]. A two-laxyers neural network has been applied to SVC [26], [27]. The neural network is suitableforanaloghardwareimplementationandgivesagoodsolutionofSVC.This paper further proposes a one-laxyer neural network for SVC and SVR learning. The advantage of the proposed neural network is twofold. Unlike the existing twolaxyers neural network, the proposed neural network has a low complexity for implementation. Thetheoreticalanalysisshowsthattheproposedneuralnetwork can converge exponentially to the optimal solution of SVM learning. Moreover, the rate of the exponential convergence can be made arbitrarily large by simply turning up a network parameter. Three illustrative examples shows the superior performanceoftheproposedneuralnetworkforSVMlearning.
Keywords: