-
Search and Optimization
Many problems in artificial intelligence can theoretically be solved by intelligent searching among many possible solutions:[1] Reasoning can be reduced to performing a search. For example, a logical proof can be viewed as a search for a path from premises to conclusions, each step being an application of an inference rule.[2] Planning algorithms search through trees of goals and sub-goals, attempting to find a path to the end goal in a process called means-end analysis.[3] Robotic algorithms for moving limbs and grasping objects use local searches in configuration space.[4] Many machine learning algorithms use optimization-based search algorithms.
Simple and exhaustive searches[5] seldom suffice to solve most real problems: the search space (number of places to be searched) quickly grows to astronomical numbers. The result is a search that is too slow or never completes. The solution to many problems is the use of “heuristics” or “rules of thumb”; Prioritize decisions in favor of those most likely to achieve the goal, and do it in fewer steps. In some search methodologies, heuristics can also be used to completely eliminate certain choices that are unlikely to lead to a goal (known as “search tree removal”).The heuristic gives the program a “best guess”; for the path where the solution is located. [6][7] The heuristic limits the search for solutions to a smaller sample.[8]
In the 1990s, a completely different type of research, based on the mathematical theory of optimization, gained prominence. For many problems, it is possible to start the search with some kind of guess and then gradually refine the guess until further refinements can be made. These algorithms can be represented as climbing blind hills: we start our search at a random point in the landscape, then we shift our guesses over jumps or ladders until we reach the top.Other optimization algorithms are Simulated Annealing, Beam Finding and Random Optimization.
Evolutionary computing uses a form of optimization research. For example, they could start with a population of organisms (guess it), then let them mutate and recombine, selecting only the fittest to survive each generation (guess how sophisticated they are). Classic evolutionary algorithms include genetic algorithms, gene expression programming, and genetic programming. Alternatively, distributed research processes can be coordinated using swarm intelligence algorithms.Two popular swarm algorithms used in research are particle swarm optimization (inspired by a flock of birds) and ant colony optimization (inspired by ant tracks).
-
Logic
Logic[16][17] is used to represent knowledge and solve problems, but it can be applied to other problems as well. For example, the Satplan algorithm uses logic for planning[18] and inductive logic programming is a learning method.[19]
Various forms of logic are used in AI research.Propositional logic[20] includes truth functions like “or”; and not”. First order logic [17][21] adds quantifiers and predicates and can express facts about objects, their properties and their relationships to one another. Fuzzy set theory assigns a “true degree” (between 0 and 1 ) overly vague statements such as “Alice is old” (or rich, tall, or hungry) that are linguistically too imprecise to be fully true or false. If you are close to your destination station and moving fast, increase the Press on the brakes of the train.” These vague
rules can then be numerically incorporated into the legitimacy of the combination of fuzzy logic-based reasoning.[22][23][24]
Predefined logics, non-monotonic logics and constraints [25] are forms of logic developed to facilitate the predefined reasoning and qualification problem. Several logic extensions have been developed to support specific areas of knowledge, such as B.: descriptive logic; situation count, event count, and charter count (to represent events and time); causal account; Belief Review;[29] and Modal Logic.[30] The logic was also designed to model contradictory or inconsistent statements that occur in multi-agent systems, such as B. Paraconsistency logic.
-
Probabilistic Methods for Uncertain Reasoning
Many artificial intelligence problems (in thinking, planning, learning, perception, and robotics) require the agent to operate with incomplete or uncertain information.Artificial intelligence researchers have developed a number of powerful tools to solve these problems using methods from probability theory and economics.[17][31]
Bayesian networks[32] are a very general tool that can be used for a variety of problems: reasoning (using the Bayesian inference algorithm),[33] learning (using the expectation maximization algorithm),[34 ][35] planning ( using decision networks)[36] and perception (using dynamic Bayesian networks).[37] Probabilistic algorithms can also be used to filter, predict, smooth, and explain data flows, and help perceptual systems to analyze processes over time (e.g. hidden Markov models or Kalman filters).[37]
Compared to symbolic logic, formal Bayesian reasoning is computationally intensive. For inference to work, most observations must be conditionally independent of each other. Complicated graphs involving diamonds or other “loops” (undirected cycles) may require a sophisticated method such as a Markov chain Monte Carlo that takes a team of random walkers in a Bayesian network and attempts to find convergence to derive conditional probabilities evaluate. Bayesian networks are used on Xbox Live to rank and match players; Victories and defeats are “trials”; what is the level of the player. AdSense uses a Bayesian network of over 300 million edges to determine which of
ads to serve.[38]:Chapter 6
A key concept in economics is “utility”: a measure of how much something is worth to an intelligent agent. Precise mathematical tools have been developed to analyze how an actor can make decisions and plan using decision theory, decision analysis[39] and information theory of value[40]. These tools include models such as Markov decision processes,[41] dynamic decision networks,[37] game theory, and mechanism design.[42]
-
Classifiers and statistical learning methods
The simplest AI applications can be divided into two types: classifiers (“if shiny, then diamond”) and controllers (“if shiny, then raise”).However, controllers also classify conditions before they act, so classification is a core part of many AI systems. Classifiers are functions that use pattern matching to find the best match. They can be set by example, which makes them very attractive for use in AI. These examples are known as observations or models. In supervised learning, each model belongs to a specific predefined class.Class is a decision you must make. All observations together with their class designations form a data set. When a new observation is received, it is ranked based on previous experience.[43]
The classifier can be trained in several ways; There are many statistical and machine learning approaches. The decision tree[44] is probably the most widely used machine learning algorithm.[38]:88 Other widely used classifiers are neural networks, k-nearest-neighbor algorithms, kernel methods such as support vector machine (SVM), Gaussian mixture model, [50] and Bayesian naive classifier, which is very popular. [51][52] The performance of a classifier largely depends on the properties of the data to be classified, such as B. the size of the data set, the distribution of the samples in classes, the dimensionality and the noise level. Model-based classifiers work well when the assumed model matches the actual data very well. Alternatively, if no matching model is available and accuracy (rather than speed or scalability) is the only concern, the
wisdom is that discriminatory classifiers (especially SVM) tend to be more accurate than model-driven classifiers as “naive ‘Bayes'”. ” to the most practical datasets.[53][54]
5.Artificial Neural Networks
A neural network is a group of interconnected nodes, similar to the large neural network in the human brain. https://handwiki.org/wiki/index.php?curid=2018324
Neural networks were inspired by the architecture of neurons in the human brain.A single “neuron” N accepts input from other neurons, each of which, when fired (or “fired”), emits a weighted “voice”; for or against having Neuron N fire on its own. Training requires an algorithm to adjust these weights based on the training data; A simple algorithm (called “pull together, connect”) involves increasing the mass between two connected neurons, since firing one neuron triggers the successful firing of the other. The neural network creates “concepts”; arranged in a sub-network of shared neurons [55] that tend to fire together; a concept meaning “leg”; can be connected to the
subnet, meaning “foot”; containing the sound “foot”. Neurons have a continuous firing spectrum; In addition, the neurons cannot process the input linearly, instead of weighting direct votes. Modern neural networks can learn both continuous functions and, surprisingly, numerical Boolean operations.Neural networks’ early successes included stock market predictions and (1995) a largely self-driving car. Chapter 4 In 2010, advances in deep learning neural networks brought AI to mainstream awareness and contributed to a massive increase in corporate spending on AI; For example, AI-related mergers and acquisitions were more than 25 times higher in 2017 than in 2015. [57] [58]
The study of learning-free artificial neural networks[45] began a decade before the field of Walter Pitts and Warren McCullouch . Frank Rosenblatt invented the Perceptron, a single-layer learning network similar to the old concept of linear regression. Early pioneers also include Alexey Grigorevich Ivakhnenko, Teuvo Kohonen, Stephen Grossberg, Kunihiko Fukushima, Christoph von der Malsburg, David Willshaw, Shun-Ichi Amari, Bernard Widrow, John Hopfield, Eduardo R.Caianello and others.
The main categories of networks are acyclic or feedforward neural networks (where the signal travels in only one direction) and repetitive neural networks (which allow for feedback and short-term memory of previous input events). Some of the more common predictive networks include perceptrons, multilayer perceptrons, and radial core networks. Neural networks can be applied to an intelligent controller (for robotics) or a learning problem using techniques such as Hebbian (“pull together, connect”) learning, GMDH, and competitive learning.
Neural networks are nowadays often trained with the backpropagation algorithm, which has existed since 1970 as a mode of reverse automatic differentiation, published by Seppo Linnainmaa [61] [62] and introduced to neural networks by Paul Webbos.[63][64][65]
Hierarchical temporal memory is an approach that models some of the structural and algorithmic properties of the neocortex.
In summary, most neural networks use some sort of gradient descent in addition to a homegrown neural topology. However, some research groups such as Uber argue that simple neuroevolution to mutate new neural network topologies and weights could compete with sophisticated gradient approaches. One benefit of neuroevolution is that it’s less likely to hit “dead ends.”
-
Deep Feedback Neural Networks
Deep learning is based on the use of artificial neural networks that have multiple layers of neurons between the inputs and outputs of the network. Deep learning has dramatically improved program performance in many key sub-domains of artificial intelligence, including computer vision, speech recognition, natural language processing, and more.
According to a study[71], the term “deep learning” was introduced to the machine learning community by Rina Dechter in 1986 and gained popularity after Igor Aizenberg and his colleagues introduced it to artificial neural networks in 2000. The earlier deep learning function networks were developed by Alexei Grigorevich Ivanenko and W.G. Lapy 1965.[74] These networks are formed layer by layer. A 1971 article by Ivakhnenko [75] describes the learning of an eight-layer deep multilayer feedback perceptron, much deeper than many later networks. In 2006, Geoffrey Hinton and Ruslan Salakhutdinov presented another way to pretrain multilayer coupled neural networks (FNNs) layer by layer, treating each layer in turn as an unsupervised bounded Boltzmann machine and then using supervised backpropagation for fine-tuning .[76] Like flat artificial neural networks, deep neural networks can model complex nonlinear relationships.
Deep learning often uses convolutional neural networks (CNN), which go back to the Neocognitron introduced by Kunihiko Fukushima in 1980[77]. In 1989, Yann LeCun and his colleagues applied backpropagation to this architecture. By the early 2000s, CNN was already processing about 10-20% of all checks issued in the United States for industrial applications.[78] Since 2011, CNN’s fast implementations on the GPU have won numerous visual pattern recognition competitions.[70]
CNNs with 12 convolutional layers were used with reinforcement learning from Deepmind’s AlphaGo Lee, a program that beat Go’s best sample in 2016. [79]
5.2. Deep recursive neural networks
Initially, deep learning was also used for sequence learning using recursive neural networks (RNNs)[80], which are theoretically Turing-complete[81] and can run arbitrary programs to process sequences of arbitrary inputs. The depth of the RNN is unlimited and depends on the length of the input sequence; RNN is therefore an example of deep learning.[70] RNNs can be trained by gradient descent[82][83][84] but suffer from the vanishing gradient problem[68][85] In 1992 it was shown that unsupervised pretraining of a stack of repetitive neural networks can speed up subsequent supervised Learning deep sequential problems.
Many researchers are now using variations of a recursive deep learning NN network called Long-Term Memory Network (LSTM) published by Hochreiter & Schmidhuber 1997. [87] LSTM is often formed from the Connectionist Time Classification (CTC). At Google, Microsoft and Baidu, this approach has revolutionized speech recognition.[89][90][91] For example, in 2015, Google Speech Recognition saw a dramatic 49% performance increase with the CTC-trained LSTM, which is now available via Google Voice to billions of smartphone users. [92] Google has also used LSTM to improve machine translation,[93] language modelling,[94] and multilingual language processing.[95] LSTM, in collaboration with CNN, has also improved automatic subtitles[96] and a host of other applications.
-
Evaluation of Progress
The games are a well-known benchmark for evaluating progress.AlphaGo around 2016 ended the era of classic board game references. Imperfect knowledge games pose new challenges for AI in game theory. Esports like StarCraft continue to provide additional public benchmarks. Many competitions and awards, such as the Imagenet Challenge, promote research in the field of artificial intelligence. Common areas of competition are artificial general intelligence, conversational skills, data mining, robot cars and robot football, and traditional games.[101]
The “mimicking game” (an interpretation of the 1950 Turing test of whether a computer can imitate a human) is now considered too intuitive to be a useful reference point. A derivative of the Turing test is the fully automated public Turing test for distinguishing computers from humans (CAPTCHA). As the name suggests, this helps determine if the user is a real person and not a computer pretending to be a human. Unlike the standard Turing test, CAPTCHA is machine-managed and human-controlled as opposed to human-managed and machine-controlled. The computer prompts the user to take a simple test and then generates a score for that test.Computers cannot fix the problem, so the test subject’s result is considered the correct solution. A common type of CAPTCHA is a test that requires you to type garbled letters, numbers, or symbols that appear in a computer-unreadable image.[103]
The proposed “universal intelligence” tests are designed to compare the performance of machines, humans, and even non-human animals on as general a set of problems as possible. At the extreme, the test-suite can contain all sorts of problems, weighted by Kolmogorov complexity; Unfortunately, these problems tend to be dominated by exhausted pattern-matching exercises where optimized AI can easily outperform human performance levels.
-
Hardware improvements
Since 2010, advances in machine learning algorithms and hardware have led to more efficient deep neural network training methods that involve multiple levels of nonlinear hidden units and a very large output level.[108] By 2019, graphics processing units (GPUs), often with AI-specific enhancements, had supplanted CPUs as the dominant method for large-scale AI training in the commercial cloud.[109] OpenAI has estimated the computing power of the hardware used in the largest deep learning projects from AlexNet (2012) to AlphaZero (2017) and found a 300,000-fold increase in required computing power, with a trendline set at 3, doubled in 4 months
