The usage of perceptron, feed and deep feed forward artiﬁcial neural networks on the spectroscopy data: astrophysical & fusion plasmas

. Artiﬁcial neural networks are gaining a momentum for solving complex problems in all sorts of data analysis and classiﬁcation matters. As such, idea of determining their usability on complex plasma came up. The choice for the input data for the analysis is a set of stellar spectral data. It consists of complex composition plasma under vast variety of conditions, dependent on type of star, measured with calibrated standardized procedures and equipment. The results of the analysis has shown that even a simple type of perceptron artiﬁcial neural network could lead to results of acceptable quality for the analysis of spectra of complex composition. The analyzed ANNs performed good on a limited data set. The results can be interpreted as a ﬁgure of merit for further development of complex neural networks in various applications e


Introduction
The usage of machine learning algorithms is a growing field of research (D'Isanto et al., 2016;Baron, 2019;Kates-Harbeck et al., 2019). Since the computer power is constantly growing its usage is often found in a wide variety of applications: from determination of objects on a photograph all the way to expert systems capable to determine adequate states and predicted outcomes of complex systems; from difficult-to-maintain machines states and prediction of conditions, up to the assistance in human health monitoring.
Even the specific fields of spectroscopy rely deeply on artificial neural networks, as is the case for instance with medical spectroscopy application Wang et al. (2015), or for instance agricultural application Basile et al. (2022); Longin et al. (2019). The material recognition in extraterrestrial spectroscopic probing is also a very difficult task, since the limitations of the mass and resolving power of the onboard instruments are a very difficult limiting factor (Koujelev et al., 2010;Bornstein et al., 2005).
Usage of the artificial neural networks (ANN) fell into focus of our interest because of flexibility of their application, as well as a variety of complex problems that they have already solved.
All of the mentioned has been a factor for applying neural networks to the decision process of determining a stellar spectral type as an example of application on astrophysical data (Albert et al., 2020). Artificial neural networks are often used in astrophysics (e.g. for the integral field spectral analysis of galaxies in Hampton et al. 2017). There is an expectation of development of further focus on convolutional neural networks application on spectroscopic data (Castorena et al., 2021). Also, even more complex predictions based on back-propagation in neural networks as well as complex artificial neural networks structures in spectroscopic usage are known (Li et al., 2017). In order to have insight of applicability of the ANN usage we have limited our research on simplest case as a figure of merit.
Few random spectral curves from database Pickles (1998) are presented here in results. Entire database set consists of spectra for 12 types of stars, spectral type O normal; B normal; A normal; F normal; F Metal rich; F metal weak; G normal; G Metal rich; G metal weak; K normal; K Metal rich; K metal weak; and M normal. Our aim was to create test case as a method of determining a quality of specific ANN in various machine learning analysis, from stellar and fusion spectra analysis, material analysis, up to extremely specific cases as enhancing a low resolution instrument performance for specific applications (Marinković et al., 2019;Albert et al., 2020).

ANN basics and principles
The usage of systems related to the functions of neural networks has been in focus of investigation since mid-1940McCulloch & Pitts (1943, but the real usage has evolved with the application of modern day digital computers, which enabled construction of networks of enlarged complexity. One of the simplest neural networks, that could be seen more as a test case of validity of operation of artificial intelligence systems, is perceptron (Rosenblatt, 1958). The prediction as well as sensitivity of the training data set is in favor of more complex networks. It is the primary goal of our investigation, along with their application on spectral data sets and measurements.
The choice for the dataset was made on open access data files for the 131 stellar spectra published by Pickles (1998) (available at accompanying reference appended to the bibliographical entry, as seen in May 2022). The results are promising and further research on the field is expected. The quality of the trained artificial neural network prediction is related to the data set as well as its structure. An effort of applying it on a large scale dataset or database should be carried out.
The problem of finding out a category of data subset is an inherent problem for any sort of machine learning and as such for the artificial neural networks also. The artificial neural network is a system of mathematical functions trying to resemble a simplified animal brain. The network consists of artificial neurons. A neuron is described as a function that adopts output value based on its input values and bias value by the means of reaction function. The simplest neuron concept could be seen on a Figure 1. The neuron determines its output state as an output of activation function based on a weighted sum of input values and a bias value itself, and could be described by equation where f act is a activation function, x i and w i are the i-th input value as well as adequate input weight. The neuron reaction on external stimulus is strongly dependent on its reaction function. In order to determine the neuron behavior on a micro scale, the reaction function as well as the method of adopting the weight values plays a determining role. Four most common reaction functions are shown on Figure 2. A topology of the neural network as well as the learning method are determining the global reaction of the neural network. For the goal of usability analysis the simplest ANN topologies, perceptron, is chosen. The Feed Forward and Deep Feed Forward topologies are based upon fully connected dense layers of neurons, see Figure 3. The two specific layers, input and output, have the dimensionality of the input data and output states consequently and are the only limiting factors of the network. When there is more than one hidden layer, the neural network is considered to be the deep one.

Results and discussion
In Figure 4 several random spectral curves from database Pickles (1998) are presented. Entire dataset consists of spectra for 12 types of stars, spectral type O normal; B normal; A normal; F normal; F Metal rich; F metal weak; G normal; G Metal rich; G metal weak; K normal; K Metal rich; K metal weak; and M normal. Each epoch of the dataset was divided into 70% for training set and 30% for the test set.
As a test bench for the application of the ANN to the selection set of perceptron, Feed Forward and Deep Feed Forward networks are used. As a reaction function ReLU (rectified linear unit) was used, and the input data was normalized to unit using standardization F metal rich K metal weak K metal rich M normal Figure 4. Several sample spectra. Spectra i.e. data is taken from Pickles (1998).
No additional data preparation was imposed. The investigated neural network topologies were let to train on the set of data for 200 epochs. Input layer consisted of 4771 input values of available data, output layer consisted of 12 types of stars, spectral type O normal; B normal; A normal; F normal; F Metal rich; F metal weak; G normal; G Metal rich; G metal weak; K normal; K Metal rich; K metal weak; and M normal. The hidden layers consisted of 5000 neurons in first, 1000 in second and 512 neurons in third layer. They were included consequently in order to compare ANN behavior, see Figure 5.
It is obvious, by the analysis of calculated data presented in Figure 5, that the deeper ANNs are capable to learn faster and have better predictions after smaller epochs of learning. This capability is a winning solution in the case of complex spectra. The ANN could fall into pseudo stable states and produce a non-minimal error. Such falls into local minimum state could be avoided by several advanced methods one of which is providing an algorithm for forgetting of the learned state, e.g. algorithm that disturbs a learned state after each application.
Also, there are probably better methods for the input dataset preparation, from pure mathematical procedures up to convolutional ANN (CNN) incorporation. It is proven that, even in its simplest forms, ANN could be used for such tasks. It is to expect that there are better ANN topologies for such a task, and this is a field for further investigation.
From the above it is obvious that even in its crudest form artificial neural networks are capable to successfully deal with the spectra classification. It is confirmed that this case could be used as a figure of merit for the further development of ANN and machine learning applications in general.

Conclusions and future possibilities
The results are promising and the further research on the field is expected. The first goal of analysis of a single set of complex spectral data recorded under similar conditions is achieved with reasonably good prediction. Concerning minute differences in comparison to each other it is considerable result for the basic ANN structure.
Since the quality of the trained artificial neural network prediction is related to its structure as well as the dataset quality and volume, an effort on a largescale database collection should be carried out. One of the first steps should be inclusion of pre-trained convolutional ANN for the purpose of input data pre-processing before entering of selector ANN.
Commercial packages as well as some specific open-source solutions for the analysis of the spectra with the help of predefined ANN exist. Their application is usually very specific and does not allow the opportunity to fit the best ANN nor to perform unique mathematical procedures during input data preparation that could be best suited for the sought purpose. This possibility is the winning factor in each specific case. Such approach should enable systems for more specialized problem solutions, from stellar and fusion spectra analysis up to more specific expert systems related to technical solutions. The further development in both ANN structures as well as data preparation should be carried out with the specified problem in mind.