{"id":37429,"date":"2021-09-29T11:38:22","date_gmt":"2021-09-29T08:38:22","guid":{"rendered":"https:\/\/forklog.com\/en\/?p=37429"},"modified":"2025-08-29T16:57:02","modified_gmt":"2025-08-29T13:57:02","slug":"what-is-a-neural-network","status":"publish","type":"post","link":"https:\/\/forklog.com\/en\/what-is-a-neural-network\/","title":{"rendered":"What is a neural network?"},"content":{"rendered":"<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>What is a neural network?<\/strong><\/h2>\n<p>An artificial neural network is a mathematical model inspired by the biological neural networks that make up the brains of living beings. Such systems learn to perform tasks by studying examples rather than being explicitly programmed for a specific application.<\/p>\n<p>They deliver state-of-the-art performance in fields such as speech and image recognition, working with unstructured data like recorded audio and photographs.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>How do neural networks differ from AI and ML?<\/strong><\/h2>\n<p>Artificial intelligence is a broad field of computer science focused on creating intelligent machines capable of performing cognitive tasks.<\/p>\n<p>Machine learning, a subfield of AI, solves problems not by hard-coding rules but by finding patterns in data after training an algorithm on many examples.<\/p>\n<p>Neural networks are a subset of machine learning. As noted above, they make predictions from unstructured data.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>What is a neural network made of?<\/strong><\/h2>\n<p>Like its biological counterpart, an artificial neural network consists of neurons and synapses.<\/p>\n<p>A neuron is a unit that receives information and performs computations on it. It is the simplest structural element of any neural network. Neurons are typically arranged in layers that together form the network.<\/p>\n<p>Most neurons operate in broadly similar ways, though some variants serve specific functions.<\/p>\n<p>Core types of neurons:<\/p>\n<ul class=\"wp-block-list\">\n<li>input \u2014 the layer of neurons that receives information;<\/li>\n<li>hidden \u2014 one or more layers that process information;<\/li>\n<li>output \u2014 the layer that represents the result of the computation.<\/li>\n<\/ul>\n<p>A synapse is a connection that links the output of one neuron to the input of another. Signals passing through it can be amplified or attenuated.<\/p>\n<p>A synapse has a parameter called a weight\u2014a coefficient that scales the information transmitted between neurons.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"517\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_1-min-1024x517.png\" alt=\"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u0430\u044f \u0441\u0435\u0442\u044c?\" class=\"wp-image-151132\" srcset=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_1-min-1024x517.png 1024w, https:\/\/forklog.com\/wp-content\/uploads\/NN_1-min-300x152.png 300w, https:\/\/forklog.com\/wp-content\/uploads\/NN_1-min-768x388.png 768w, https:\/\/forklog.com\/wp-content\/uploads\/NN_1-min.png 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<p>Activation functions play a crucial role in a network\u2019s architecture. As in living brains, they determine which signals pass through neurons and which do not.<\/p>\n<p>For example, when you grasp a hot kettle, nerve endings in your fingers relay information to neurons in the brain, where an activation function decides whether to pull your hand away from the heat or keep transmitting signals.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>How does a neural network work?<\/strong><\/h2>\n<p>Information enters the input layer, then flows via synapses to the next layer. Each synapse has its own weight, and any neuron in a subsequent layer may have multiple inputs. The signal propagates until it reaches the final output.<\/p>\n<p>Consider handwritten-digit recognition: the algorithm must cope with great variation in how data are represented. Each digit from 0 to 9 can be written in many ways; the size and exact shape of each symbol vary by writer and circumstance.<\/p>\n<p>The input layer receives values representing the pixels of an image of a digit. The output layer, in turn, predicts which symbol is shown.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"517\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_2-min-1024x517.png\" alt=\"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u0430\u044f \u0441\u0435\u0442\u044c?\" class=\"wp-image-151134\" srcset=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_2-min-1024x517.png 1024w, https:\/\/forklog.com\/wp-content\/uploads\/NN_2-min-300x152.png 300w, https:\/\/forklog.com\/wp-content\/uploads\/NN_2-min-768x388.png 768w, https:\/\/forklog.com\/wp-content\/uploads\/NN_2-min.png 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<p>The circles in the diagram are neurons, organised into vertically stacked, interconnected layers.<\/p>\n<p>The links are coloured to indicate the importance of connections between neurons. Red links strengthen the value as it moves between layers, increasing the chance of activating the recipient neuron.<\/p>\n<p>Activated neurons are shaded red. In \u201cHidden layer 1\u201d, they indicate that the image contains a particular combination of pixels resembling the horizontal stroke at the top of a handwritten 3 or 7.<\/p>\n<p>Thus Hidden layer 1 can detect characteristic lines and curves that ultimately combine into a complete handwritten figure.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>How does a neural network learn?<\/strong><\/h2>\n<p>During training, the model learns which connections matter for accurate predictions. At each step it uses a mathematical function to gauge how close its latest prediction was to the expected result.<\/p>\n<p>This function produces error values the system uses to calculate how to update the weights attached to each link, with the aim of improving accuracy.<\/p>\n<p>Over many training cycles, with occasional manual tuning of parameters, the network generates ever more accurate predictions until performance plateaus. At that point\u2014for example, when handwritten digits are recognised with accuracy above 95%\u2014the network can be considered trained.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>What is a dataset?<\/strong><\/h2>\n<p>A dataset is a collection of homogeneous data used to train neural networks. To train a face-recognition algorithm, for instance, it must be shown many photographs of people. The more data, the more accurate the model.<\/p>\n<p>Datasets typically come in three types:<\/p>\n<ul class=\"wp-block-list\">\n<li>training \u2014 used to fit the network;<\/li>\n<li>test \u2014 used to assess accuracy;<\/li>\n<li>validation \u2014 an independent set for a final evaluation of the algorithm\u2019s accuracy.<\/li>\n<\/ul>\n<p>Data can be of any format: tables, photos, video, audio and more. In supervised learning the data are often labelled with specialised software. Yet inaccuracies in datasets can lead to errors in the resulting models.<\/p>\n<p>In April 2021 researchers at the Massachusetts Institute of Technology found that popular datasets contain many mistakes. In widely used benchmark sets, for example, a mushroom might be labelled as a spoon, a frog as a cat, and a high note by Ariana Grande in an audio file marked as a whistle.<\/p>\n<p>Another MIT study showed that careless work by contractors on Amazon Mechanical Turk hampers the development of text-generation systems. They are paid per item labelled, so they tend to work quickly with little regard for accuracy.<\/p>\n<p>Researchers therefore urge developers to practise data \u201chygiene\u201d.<\/p>\n<p>In reinforcement learning, data do not need labelling, as an agent must discover patterns in an environment and is rewarded when it achieves a goal.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>Where are neural networks used?<\/strong><\/h2>\n<p>Neural networks are used for many tasks: recognising and generating images, speech and language, and\u2014combined with reinforcement learning\u2014for games, from board games such as Go to video games like Dota 2 and Quake III.<\/p>\n<p>Such systems underpin many online services. Amazon uses them to understand speech for its Alexa voice assistant; Microsoft uses them for real-time translation in the browser.<\/p>\n<p>Every Google search query invokes several machine-learning systems to parse the language and personalise results.<\/p>\n<p>Beyond the consumer web, they are spreading across industries, including:<\/p>\n<ul class=\"wp-block-list\">\n<li>computer vision for autonomous cars, drones and delivery robots;<\/li>\n<li>speech recognition and synthesis, and language for chatbots and service robots;<\/li>\n<li>face identification in video-surveillance systems;<\/li>\n<li>assisting radiologists in spotting tumours on X-rays;<\/li>\n<li>helping researchers identify genetic sequences linked to disease and molecules that could improve drugs;<\/li>\n<li>predictive maintenance of infrastructure by analysing data from internet-of-things sensors, and much else.<\/li>\n<\/ul>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>What are the challenges or drawbacks of neural networks?<\/strong><\/h2>\n<p>A major drawback is the volume of data required for training. Datasets can be vast: not long ago Facebook said it used one billion images to achieve record performance in image recognition.<\/p>\n<p>Because of dataset size and the number of training cycles, powerful and costly hardware\u2014typically with high-performance GPUs\u2014is often needed. Whether you build your own system or rent cloud capacity, training incurs significant cost.<\/p>\n<p>Another challenge is dataset noise. As noted, people make mistakes when creating datasets, which can affect the final result.<\/p>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>What types of neural network exist?<\/strong><\/h2>\n<p>Roughly 30 types of neural network are in use, suited to different tasks. Convolutional neural networks (CNNs) are common in computer vision, while recurrent neural networks (RNNs) are used for language.<\/p>\n<p>Each has its quirks. In CNNs, early layers specialise in extracting features from an image, which are then passed to a standard neural network to classify objects.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"517\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_CNN-min2-1024x517.png\" alt=\"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u0430\u044f \u0441\u0435\u0442\u044c?\" class=\"wp-image-151138\" srcset=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_CNN-min2-1024x517.png 1024w, https:\/\/forklog.com\/wp-content\/uploads\/NN_CNN-min2-300x152.png 300w, https:\/\/forklog.com\/wp-content\/uploads\/NN_CNN-min2-768x388.png 768w, https:\/\/forklog.com\/wp-content\/uploads\/NN_CNN-min2.png 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<p>RNNs differ in that neurons receive information not only from the previous layer but also from a recurrent connection to themselves. This lets the network learn the sequence of inputs.<\/p>\n<p>Their difficulty lies in the so-called vanishing-gradient problem: the network quickly forgets information over time. Although this affects the weights rather than the neurons\u2019 states, information accumulates in those states.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"517\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_RNN-min-1024x517.png\" alt=\"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u0430\u044f \u0441\u0435\u0442\u044c?\" class=\"wp-image-151139\" srcset=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_RNN-min-1024x517.png 1024w, https:\/\/forklog.com\/wp-content\/uploads\/NN_RNN-min-300x152.png 300w, https:\/\/forklog.com\/wp-content\/uploads\/NN_RNN-min-768x388.png 768w, https:\/\/forklog.com\/wp-content\/uploads\/NN_RNN-min.png 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<p>Generative adversarial networks (GANs) comprise two networks: a generator that creates content and a discriminator that evaluates it.<\/p>\n<p>The discriminator receives either training data or data produced by the generator. Its success at guessing the source contributes to the error signal.<\/p>\n<p>Thus a contest emerges: the generator learns to fool the discriminator, which in turn learns to detect the fraud. Training is difficult, as each network must be trained and balanced against the other.<\/p>\n<p>Typical applications include photo stylisation, deepfakes, audio generation and more.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"517\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_GAN-min2-1024x517.png\" alt=\"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043d\u0435\u0439\u0440\u043e\u043d\u043d\u0430\u044f \u0441\u0435\u0442\u044c?\" class=\"wp-image-151142\" srcset=\"https:\/\/forklog.com\/wp-content\/uploads\/NN_GAN-min2-1024x517.png 1024w, https:\/\/forklog.com\/wp-content\/uploads\/NN_GAN-min2-300x152.png 300w, https:\/\/forklog.com\/wp-content\/uploads\/NN_GAN-min2-768x388.png 768w, https:\/\/forklog.com\/wp-content\/uploads\/NN_GAN-min2.png 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n<div class=\"wp-block-text-wrappers-cards single_card\">\n<h2 class=\"card_label\"><strong>Will neural networks lead to artificial general intelligence?<\/strong><\/h2>\n<p>Today, neural networks are used for narrow, specialised tasks\u2014what is known as weak AI.<\/p>\n<p>No models yet qualify as artificial general intelligence, capable of tackling as broad a range of tasks, with comparable understanding, as a human. When such systems will arrive is unknown: some forecasts put them within the next decade; others, not for 1,000 years.<\/p>\n<\/div>\n<p>Follow ForkLog news on Telegram:<a href=\"https:\/\/t.me\/forklogAI\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">\u00a0ForkLog AI<\/a> \u2014 all the news from the world of AI!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>An artificial neural network is a mathematical model inspired by the biological neural networks that make up the brains of living beings. Such systems learn to perform tasks by studying examples rather than being explicitly programmed for a specific application.<\/p>\n","protected":false},"author":1,"featured_media":37430,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"select":"1","news_style_id":"1","cryptorium_level":"2","_short_excerpt_text":"","creation_source":"","_metatest_mainpost_news_update":false,"footnotes":""},"categories":[2113],"tags":[2130,438],"class_list":["post-37429","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cryptorium","tag-101-artificial-intelligence","tag-artificial-intelligence"],"aioseo_notices":[],"amp_enabled":true,"views":"38","promo_type":"1","layout_type":"1","short_excerpt":"","is_update":"","_links":{"self":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts\/37429","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/comments?post=37429"}],"version-history":[{"count":1,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts\/37429\/revisions"}],"predecessor-version":[{"id":37431,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts\/37429\/revisions\/37431"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/media\/37430"}],"wp:attachment":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/media?parent=37429"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/categories?post=37429"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/tags?post=37429"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}