AI x Crypto: From Zero to Peak

Introduction

The recent development of the AI industry is viewed by some as the Fourth Industrial Revolution. The emergence of large models has significantly improved the efficiency of various industries, and Boston Consulting Group estimates that GPT has increased work efficiency in the U.S. by about 20%. At the same time, the generalization capabilities brought by large models are referred to as a new software design paradigm; whereas past software design relied on precise code, now it involves embedding more generalized large model frameworks into software, enabling these applications to perform better and support a wider range of input and output modalities. Deep learning technology has indeed brought about a fourth boom for the AI industry, and this wave has also spread to the cryptocurrency sector.

This report will detail the development history of the AI industry, the classification of technologies, and the impact of the invention of deep learning technology on the industry. It will then conduct an in-depth analysis of the upstream and downstream of the industrial chain in deep learning, including GPU, cloud computing, data sources, edge devices, etc., as well as their current development status and trends. After that, we will fundamentally explore the relationship between cryptocurrency and the AI industry, and sort out the pattern of the AI industrial chain related to cryptocurrency.

The Development History of the AI Industry

The AI industry began in the 1950s, and in order to achieve the vision of artificial intelligence, academia and industry have developed many schools of thought to realize artificial intelligence under different disciplinary backgrounds in different eras.

Modern artificial intelligence technology mainly uses the term "machine learning". The concept of this technology is to enable machines to iteratively improve system performance on tasks based on data. The main steps involve feeding data into algorithms, training models with this data, testing and deploying the models, and using the models to complete automated prediction tasks.

Currently, there are three major schools of machine learning: connectionism, symbolicism, and behaviorism, which respectively mimic the human nervous system, thinking, and behavior.

Currently, connectionism represented by neural networks is dominant and is also known as deep learning. The main reason is that this architecture has an input layer, an output layer, but multiple hidden layers. Once the number of layers and neurons becomes sufficiently large, there will be enough opportunity to fit complex general tasks. By inputting data, the parameters of the neurons can be continuously adjusted, and after experiencing multiple data inputs, the neuron will reach an optimal state. This is what we refer to as "great effort leads to miraculous results," and this is also the origin of the word "deep" – enough layers and neurons.

For example, it can be simply understood as constructing a function where when we input X=2, Y=3; and when X=3, Y=5. If we want this function to apply to all X, we need to continuously add the degree of this function and its parameters. For instance, I can construct a function that satisfies this condition as Y = 2X - 1. However, if there is a data point where X=2, Y=11, we need to reconstruct a function suitable for these three data points. Using a GPU for brute force cracking, we find that Y = X² - 3X + 5 is relatively suitable, but it does not need to perfectly match the data; it just needs to maintain balance and provide a roughly similar output. In this context, X², X, and X₀ represent different neurons, while 1, -3, and 5 are their parameters.

At this time, if we input a large amount of data into the neural network, we can increase the neurons and iterate parameters to fit the new data. This way, we can fit all the data.

Based on deep learning technology using neural networks, there have been multiple technical iterations and evolutions, such as the earliest neural networks in the above diagram, feedforward neural networks, RNN, CNN, and GAN, which eventually evolved into modern large models like GPT that use Transformer technology. The Transformer technology is just one evolutionary direction of neural networks, adding a converter ( Transformer ) to encode data from all modalities ( such as audio, video, images, etc., into corresponding numerical values for representation. This data is then input into the neural network, allowing the network to fit any type of data, achieving multimodality.

![Newcomer Science Popularization丨AI x Crypto: From Zero to Peak])https://img-cdn.gateio.im/webp-social/moments-0c9bdea33a39a2c07d1f06760ed7e804.webp(

The development of AI has gone through three technological waves. The first wave occurred in the 1960s, a decade after the proposal of AI technology. This wave was driven by the development of symbolic technology, which addressed the problems of general natural language processing and human-computer dialogue. During this period, expert systems were born, notably the DENRAL expert system, completed under the supervision of NASA by Stanford University. This system possesses very strong chemical knowledge and infers answers similar to those of a chemical expert through questioning. This chemical expert system can be seen as a combination of a chemical knowledge base and inference system.

After expert systems, in the 1990s, Israeli-American scientist and philosopher Judea Pearl ) proposed Bayesian networks, which are also known as belief networks. At the same time, Brooks introduced behavior-based robotics, marking the birth of behaviorism.

In 1997, IBM's Deep Blue defeated chess champion Garry Kasparov 3.5:2.5, and this victory was seen as a milestone for artificial intelligence, marking the peak of the second wave of AI development.

The third wave of AI technology occurred in 2006. The three giants of deep learning, Yann LeCun, Geoffrey Hinton, and Yoshua Bengio, proposed the concept of deep learning, an algorithm based on artificial neural networks for representation learning of data. Subsequently, deep learning algorithms gradually evolved from RNNs, GANs to Transformers and Stable Diffusion, with these two algorithms shaping this third technological wave, which is also the peak period of connectionism.

Many iconic events have gradually emerged alongside the exploration and evolution of deep learning technologies, including:

• In 2011, IBM's Watson( won the championship in the quiz show Jeopardy), defeating humans.

• In 2014, Goodfellow proposed GAN(, Generative Adversarial Network), which learns by having two neural networks compete against each other, capable of generating photorealistic images. At the same time, Goodfellow also wrote a book called "Deep Learning," known as the "Flower Book," which is one of the important introductory books in the field of deep learning.

• In 2015, Hinton and others proposed a deep learning algorithm in the journal "Nature", which immediately caused a huge response in both the academic and industrial circles.

• In 2015, OpenAI was founded, with Musk, Y Combinator president Altman, angel investor Peter Thiel(, and others announcing a joint investment of $1 billion.

• In 2016, AlphaGo, based on deep learning technology, competed against the world champion and professional 9-dan Go player Lee Sedol in a human-machine Go battle, winning with a total score of 4 to 1.

• In 2017, Hanson Robotics, a company based in Hong Kong, developed the humanoid robot Sophia, which is known as the first robot in history to receive citizenship. It possesses a wide range of facial expressions and an understanding of human language.

• In 2017, Google, which has a rich talent and technology reserve in the field of artificial intelligence, published the paper "Attention is all you need" proposing the Transformer algorithm, and large-scale language models began to emerge.

• In 2018, OpenAI released the GPT) Generative Pre-trained Transformer( built on the Transformer algorithm, which was one of the largest language models at that time.

• In 2018, Google's DeepMind team released AlphaGo based on deep learning, capable of predicting protein structures, which is seen as a significant milestone in the field of artificial intelligence.

• In 2019, OpenAI released GPT-2, which has 1.5 billion parameters.

• In 2020, OpenAI developed GPT-3, which has 175 billion parameters, 100 times more than the previous version GPT-2. The model was trained on 570GB of text and can achieve state-of-the-art performance in various NLP) tasks( such as answering questions, translation, and writing articles).

• In 2021, OpenAI released GPT-4, which has 1.76 trillion parameters, ten times that of GPT-3.

• The ChatGPT application based on the GPT-4 model was launched in January 2023, and by March, ChatGPT reached 100 million users, becoming the fastest application in history to reach 100 million users.

• In 2024, OpenAI will launch GPT-4 omni.

Note: Due to the abundance of artificial intelligence papers, various schools of thought, and differing technological evolutions, this mainly follows the historical development of deep learning or connectionism, while other schools and technologies are still in a rapid development phase.

Deep Learning Industry Chain

Current large language models are based on deep learning methods using neural networks. With GPT leading the way, a wave of artificial intelligence has emerged, attracting numerous players into this field. We have also observed a significant surge in market demand for data and computing power. Therefore, in this section of the report, we primarily explore the industrial chain of deep learning algorithms, examining how the upstream and downstream are composed in the AI industry dominated by deep learning algorithms, as well as the current situation of the upstream and downstream, their supply and demand relationships, and future developments.

First, we need to clarify that when training large models based on Transformer technology, led by GPT, there are a total of three steps.

Before training, since it is based on Transformer, the converter needs to transform text input into numerical values, a process known as "Tokenization". These numerical values are then referred to as Tokens. Under general heuristics, an English word or character can be roughly considered as one Token, while each Chinese character can be roughly considered as two Tokens. This is also the basic unit used for GPT pricing.

Step one, pre-training. By providing enough data pairs to the input layer, similar to the examples given in the first part of the report (X,Y), to find the optimal parameters for each neuron in the model, a large amount of data is required at this stage, and this process is also the most computationally intensive, as it involves repeatedly iterating the neurons to try various parameters. After a batch of data pairs has been trained, the same batch of data is generally used for a secondary training to iterate the parameters.

Step 2, fine-tuning. Fine-tuning involves training on a smaller batch of high-quality data, which significantly enhances the output quality of the model. This is because pre-training requires a large amount of data, but much of it may contain errors or be of low quality. The fine-tuning step can improve the model's quality through high-quality data.

Step three, reinforcement learning. First, a brand new model will be established, which we call the "reward model". The purpose of this model is very simple: to rank the output results. Therefore, implementing this model will be relatively simple, as the business scenario is quite vertical. After that, this model will be used to determine whether the output of our large model is of high quality, thus allowing a reward model to automatically iterate the parameters of the large model. ( However, sometimes human participation is also needed to assess the output quality of the model ).

In short, during the training process of large models, pre-training has very high requirements for the amount of data, and the GPU computing power required is also the highest. Fine-tuning requires higher quality data to improve parameters, and reinforcement learning can iteratively adjust parameters through a reward model to produce higher quality results.

In the training process, the more parameters there are, the higher the ceiling of generalization ability. For example, in the case of the function Y = aX + b, there are actually two neurons, X and X0. Therefore, the variation in parameters can only fit extremely limited data because, at its core, it remains a straight line. If there are more neurons, more parameters can be iterated, allowing for fitting more data. This is why large models achieve remarkable results, and it is also why the term 'large model' is commonly used, which essentially refers to a vast number of neurons and parameters, along with a massive amount of data, all requiring substantial computational power.

Therefore, the performance of large models is mainly determined by three aspects: the number of parameters, the amount and quality of data, and computing power. These three factors work together.