08. 06.2018

Blog of Dmitry Vassiyarov.

Binary code - where and how is it used?

Today I am especially glad to meet you, my dear readers, because I feel like a teacher who, at the very first lesson, begins to introduce the class to letters and numbers. And since we live in a world of digital technology, I will tell you what binary code is, which is their basis.

Let's start with the terminology and find out what binary means. For clarification, let’s return to our usual calculus, which is called “decimal”. That is, we use 10 digits, which make it possible to conveniently operate with various numbers and keep appropriate records.

Following this logic, the binary system provides for the use of only two characters. In our case, these are just “0” (zero) and “1” one. And here I want to warn you that hypothetically there could be other symbols in their place, but it is precisely these values, indicating the absence (0, empty) and the presence of a signal (1 or “stick”), that will help us further understand the structure of the binary code.

Why is binary code needed?

Before the advent of computers, various automatic systems were used, the operating principle of which was based on receiving a signal. The sensor is triggered, the circuit is closed and a certain device is turned on. No current in the signal circuit - no operation. It was electronic devices that made it possible to achieve progress in processing information represented by the presence or absence of voltage in a circuit.

Their further complication led to the emergence of the first processors, which also did their job, processing a signal consisting of pulses alternating in a certain way. We will not delve into the program details now, but the following is important for us: electronic devices turned out to be able to distinguish a given sequence of incoming signals. Of course, it is possible to describe the conditional combination this way: “there is a signal”; "no signal"; “there is a signal”; “there is a signal.” You can even simplify the notation: “there is”; "No"; "There is"; "There is".

But it is much easier to denote the presence of a signal with a unit “1”, and its absence with a zero “0”. Then we can use a simple and concise binary code instead: 1011.

Of course, processor technology has stepped far forward and now chips are able to perceive not just a sequence of signals, but entire programs written with specific commands consisting of individual characters.

But to record them, the same binary code is used, consisting of zeros and ones, corresponding to the presence or absence of a signal. Whether he exists or not, it doesn’t matter. For a chip, any of these options is a single piece of information, which is called a “bit” (bit is the official unit of measurement).

Conventionally, a symbol can be encoded as a sequence of several characters. Two signals (or their absence) can describe only four options: 00; 01;10; 11. This encoding method is called two-bit. But it can also be:

• Four-bit (as in the example in the paragraph above 1011) allows you to write 2^4 = 16 symbol combinations;
• Eight-bit (for example: 0101 0011; 0111 0001). At one time it was of greatest interest to programming because it covered 2^8 = 256 values. This made it possible to describe all decimal digits, the Latin alphabet and special characters;
• Sixteen-bit (1100 1001 0110 1010) and higher. But records with such a length are already for modern, more complex tasks. Modern processors use 32 and 64-bit architecture;

Frankly, there is no single official version, but it so happened that it was the combination of eight characters that became the standard measure of stored information called a “byte.” This could be applied even to one letter written in 8-bit binary code. So, my dear friends, please remember (if anyone didn’t know):

8 bits = 1 byte.

That's how it is. Although a character written with a 2 or 32-bit value can also nominally be called a byte. By the way, thanks to binary code we can estimate the volume of files measured in bytes and the speed of information and Internet transmission (bits per second).

Binary encoding in action

To standardize the recording of information for computers, several coding systems have been developed, one of which, ASCII, based on 8-bit recording, has become widespread. The values ​​in it are distributed in a special way:

• the first 31 characters are control characters (from 00000000 to 00011111). Serve for service commands, output to a printer or screen, sound signals, text formatting;
• the following from 32 to 127 (00100000 – 01111111) Latin alphabet and auxiliary symbols and punctuation marks;
• the rest, up to the 255th (10000000 – 11111111) – alternative, part of the table for special tasks and displaying national alphabets;

The decoding of the values ​​​​in it is shown in the table.

If you think that “0” and “1” are located in a chaotic order, then you are deeply mistaken. Using any number as an example, I will show you a pattern and teach you how to read numbers written in binary code. But for this we will accept some conventions:

• We will read a byte of 8 characters from right to left;
• If in ordinary numbers we use the digits of ones, tens, hundreds, then here (reading in reverse order) for each bit various powers of “two” are represented: 256-124-64-32-16-8- 4-2-1;
• Now we look at the binary code of the number, for example 00011011. Where there is a “1” signal in the corresponding position, we take the values ​​of this bit and sum them up in the usual way. Accordingly: 0+0+0+32+16+0+2+1 = 51. You can verify the correctness of this method by looking at the code table.

Now, my inquisitive friends, you not only know what binary code is, but also know how to convert the information encrypted by it.

Language understandable to modern technology

Of course, the algorithm for reading binary code by processor devices is much more complicated. But you can use it to write down anything you want:

• Text information with formatting options;
• Numbers and any operations with them;
• Graphic and video images;
• Sounds, including those beyond our hearing range;

In addition, due to the simplicity of the “presentation”, various ways of recording binary information are possible:

By changing the magnetic field by ;

The advantages of binary coding are complemented by almost unlimited possibilities for transmitting information over any distance. This is the method of communication used with spacecraft and artificial satellites.

So, today the binary number system is a language that is understood by most of the electronic devices we use. And what’s most interesting is that no other alternative is foreseen for now.

I think that the information I have presented will be quite enough for you to get started. And then, if such a need arises, everyone will be able to delve deeper into an independent study of this topic.

I will say goodbye and after a short break I will prepare for you a new article on my blog on some interesting topic.

It's better if you tell me it yourself ;)

See you soon.

If you're interested in learning how to read binary numbers, it's important to understand how binary numbers work. The binary system is known as a "base 2" numbering system, which means there are two possible numbers for each digit; one or zero. Large numbers are written by adding extra binary ones or zeros.

Understanding Binary Numbers

Knowing how to read binary files is not critical to using computers. But it is good to understand the concept to better understand how computers store numbers in memory. It also allows you to understand terms such as 16-bit, 32-bit, 64-bit and memory measurements such as bytes (8 bits).

"Reading" binary code usually means converting the binary number to the base 10 (decimal) number that people are familiar with. This conversion is fairly easy to do in your head once you understand how a binary language works.

Each digit in a binary number has a specific meaning unless the digit is zero. Once you have determined all of these values, you simply add them together to get the 10-digit decimal value of the binary number. To see how this works, take the binary number 11001010.

1. The best way to read a binary number is to start at the rightmost digit and work left. The strength of this first location is zero, that is, the value for this digit, if it is not zero, is equal to two powers of zero or one. In this case, since the digit is zero, the value for that location will be zero.

2. Then move on to the next digit. If it's one, then calculate two to the power of one. Write down this value. In this example, the value is a power of two equal to two.

3. Continue repeating this process until you reach the leftmost number.

4. To finish, all you have to do is add all these numbers together to get the total decimal value of the binary number: 128 + 64 + 0 + 0 + 8 + 0 + 2 + 0 = 202 .

The note: Another way to see this whole process in equation form is this: 1 x 2 7 + 1 x 2 6 + 0 x 2 5 + 0 x 2 4 + 1 x 2 3 + 0 x 2 2 + 1 x 2 1 + 0 x 2 0 = 20.

Binary numbers with signature

The above method works for unsigned basic binary numbers. However, computers need a way to represent negative numbers also using binary code.

Because of this, computers use signed binary numbers. In this type of system, the leftmost digit is known as the sign bit and the remaining digits are known as amplitude bits.

Reading a signed binary number is almost the same as an unsigned one, with one small difference.

1. Follow the same procedure as above for an unsigned binary number, but stop once you reach the leftmost bit.

2. To determine the sign, look at the leftmost bit. If it is one, then the number is negative. If it is zero, then the number is positive.

3. Now do the same calculations as before, but apply the appropriate sign to the number indicated by the leftmost bit: 64 + 0 + 0 + 8 + 0 + 2 + 0 = -74 .

4. Signed binary method allows computers to represent numbers that are positive or negative. However, it consumes the leading bit, meaning that large numbers require slightly more memory than unsigned binary numbers.

This lesson will cover the topic “Encoding information. Binary coding. Units of measurement of information." During it, users will be able to gain an understanding of information coding, how computers perceive information, units of measurement and binary coding.

Subject:Information around us

Lesson: Information coding. Binary coding. Units of information

This lesson will cover the following questions:

1. Coding as changing the form of information presentation.

2. How does a computer recognize information?

3. How to measure information?

4. Units of measurement of information.

In the world of codes

Why do people encode information?

1. Hide it from others (mirror cryptography of Leonardo da Vinci, military encryption).

2. Write down the information in short (shorthand, abbreviation, road signs).

3. For easier processing and transmission (Morse code, translation into electrical signals - machine codes).

Coding is the representation of information using some code.

Code is a system of symbols for presenting information.

Methods of encoding information

1. Graphic (see Fig. 1) (using drawings and signs).

Rice. 1. Signal flag system (Source)

2. Numerical (using numbers).

For example: 11001111 11100101.

3. Symbolic (using alphabet symbols).

For example: NKMBM CHGYOU.

Decoding is an action to restore the original form of information presentation. To decode, you need to know the code and encoding rules.

The means of encoding and decoding is the code correspondence table. For example, the correspondence in various number systems is 24 - XXIV, the correspondence of the alphabet with any symbols (Fig. 2).

Rice. 2. Cipher example (Source)

Examples of information encoding

An example of information coding is Morse code (see Figure 3).

Rice. 3. Morse code ()

Morse code uses only 2 symbols - a dot and a dash (short and long sound).

Another example of information encoding is the flag alphabet (see Fig. 4).

Rice. 4. Flag alphabet ()

Another example is the alphabet of flags (see Fig. 5).

Rice. 5. ABC of flags ()

A well-known example of coding is the musical alphabet (see Fig. 6).

Rice. 6. Musical alphabet ()

Consider the following problem:

Using the flag alphabet table (see Fig. 7), it is necessary to solve the following problem:

Rice. 7

Senior mate Lom passes the exam to Captain Vrungel. Help him read the following text (see Figure 8):

There are mainly two signals around us, for example:

Traffic light: red - green;

Question: yes - no;

Lamp: on - off;

It is possible - it is not possible;

Good bad;

Truth is a lie;

Back and forth;

Yes - no;

All these are signals indicating the amount of information in 1 bit.

1 bit - this is the amount of information that allows us to choose one option out of two possible ones.

Computer is an electrical machine that operates on electronic circuits. In order for the computer to recognize and understand the input information, it must be translated into computer (machine) language.

The algorithm intended for the performer must be written, that is, coded, in a language understandable to the computer.

These are electrical signals: current is passing or current is not passing.

Machine binary language - a sequence of "0" and "1". Each binary number can have the value 0 or 1.

Each digit of a machine binary code carries an amount of information equal to 1 bit.

The binary number that represents the smallest unit of information is called b it . A bit can take the value either 0 or 1. The presence of a magnetic or electronic signal in a computer means 1, the absence of 0.

A string of 8 bits is called b IT . The computer processes this string as a separate character (number, letter).

Let's look at an example. The word ALICE consists of 5 letters, each of which is represented in computer language by one byte (see Fig. 10). Therefore, Alice can be measured as 5 bytes.

Rice. 10. Binary Code (Source)

In addition to bits and bytes, there are other units of information.

Bibliography

1. Bosova L.L. Computer Science and ICT: Textbook for 5th grade. - M.: BINOM. Knowledge Laboratory, 2012.

2. Bosova L.L. Computer Science: Workbook for 5th grade. - M.: BINOM. Knowledge Laboratory, 2010.

3. Bosova L.L., Bosova A.Yu. Computer science lessons in grades 5-6: Methodological manual. - M.: BINOM. Knowledge Laboratory, 2010.

2. Festival "Open Lesson" ().

Homework

1. §1.6, 1.7 (Bosova L.L. Informatics and ICT: Textbook for grade 5).

2. Page 28, tasks 1, 4; p. 30, tasks 1, 4, 5, 6 (Bosova L.L. Informatics and ICT: Textbook for grade 5).

The set of characters with which text is written is called alphabet.

The number of characters in the alphabet is its power.

Formula for determining the amount of information: N=2b,

where N is the power of the alphabet (number of characters),

b – number of bits (information weight of the symbol).

The alphabet with a capacity of 256 characters can accommodate almost all the necessary characters. This alphabet is called sufficient.

Because 256 = 2 8, then the weight of 1 character is 8 bits.

The unit of measurement 8 bits was given the name 1 byte:

1 byte = 8 bits.

The binary code of each character in computer text takes up 1 byte of memory.

How is text information represented in computer memory?

The convenience of byte-by-byte character encoding is obvious because a byte is the smallest addressable part of memory and, therefore, the processor can access each character separately when processing text. On the other hand, 256 characters is quite a sufficient number to represent a wide variety of symbolic information.

Now the question arises, which eight-bit binary code to assign to each character.

It is clear that this is a conditional matter; you can come up with many encoding methods.

All characters of the computer alphabet are numbered from 0 to 255. Each number corresponds to an eight-bit binary code from 00000000 to 11111111. This code is simply the serial number of the character in the binary number system.

A table in which all characters of the computer alphabet are assigned serial numbers is called an encoding table.

Different types of computers use different encoding tables.

The table has become the international standard for PCs ASCII(read aski) (American Standard Code for Information Interchange).

The ASCII code table is divided into two parts.

Only the first half of the table is the international standard, i.e. symbols with numbers from 0 (00000000), up to 127 (01111111).

ASCII encoding table structure

Serial number	Code	Symbol
0 - 31	00000000 - 00011111	Symbols with numbers from 0 to 31 are usually called control symbols. Their function is to control the process of displaying text on the screen or printing, sounding a sound signal, marking up text, etc.
32 - 127	00100000 - 01111111	Standard part of the table (English). This includes lowercase and uppercase letters of the Latin alphabet, decimal numbers, punctuation marks, all kinds of brackets, commercial and other symbols. Character 32 is a space, i.e. empty position in the text. All others are reflected by certain signs.
128 - 255	10000000 - 11111111	Alternative part of the table (Russian). The second half of the ASCII code table, called the code page (128 codes, starting from 10000000 and ending with 11111111), can have different options, each option has its own number. The code page is primarily used to accommodate national alphabets other than Latin. In Russian national encodings, characters from the Russian alphabet are placed in this part of the table.

First half of the ASCII code table

Please note that in the encoding table, letters (uppercase and lowercase) are arranged in alphabetical order, and numbers are ordered in ascending order. This observance of lexicographic order in the arrangement of symbols is called the principle of sequential coding of the alphabet.

For letters of the Russian alphabet, the principle of sequential coding is also observed.

Second half of the ASCII code table

Unfortunately, there are currently five different Cyrillic encodings (KOI8-R, Windows. MS-DOS, Macintosh and ISO). Because of this, problems often arise with transferring Russian text from one computer to another, from one software system to another.

Chronologically, one of the first standards for encoding Russian letters on computers was KOI8 ("Information Exchange Code, 8-bit"). This encoding was used back in the 70s on computers of the ES computer series, and from the mid-80s it began to be used in the first Russified versions of the UNIX operating system.

From the early 90s, the time of dominance of the MS DOS operating system, the CP866 encoding remains ("CP" means "Code Page", "code page").

Apple computers running the Mac OS operating system use their own Mac encoding.

In addition, the International Standards Organization (ISO) has approved another encoding called ISO 8859-5 as a standard for the Russian language.

The most common encoding currently used is Microsoft Windows, abbreviated CP1251.

Since the late 90s, the problem of standardizing character encoding has been solved by the introduction of a new international standard called Unicode. This is a 16-bit encoding, i.e. it allocates 2 bytes of memory for each character. Of course, this increases the amount of memory occupied by 2 times. But such a code table allows the inclusion of up to 65536 characters. The complete specification of the Unicode standard includes all the existing, extinct and artificially created alphabets of the world, as well as many mathematical, musical, chemical and other symbols.

Let's try using an ASCII table to imagine what words will look like in the computer's memory.

Internal representation of words in computer memory

Sometimes it happens that a text consisting of letters of the Russian alphabet received from another computer cannot be read - some kind of “abracadabra” is visible on the monitor screen. This happens because computers use different character encodings for the Russian language.

Bit depth of binary code, Conversion of information from continuous to discrete form, Universality of binary coding, Uniform and non-uniform codes, Computer Science 7th grade Bosova, Computer Science 7th grade

1.5.1. Converting information from continuous to discrete form
To solve his problems, a person often has to transform existing information from one form of representation to another. For example, when reading aloud, information is converted from discrete (text) form to continuous (sound). During a dictation in a Russian language lesson, on the contrary, information is transformed from a continuous form (the teacher’s voice) into a discrete one (students’ notes).
Information presented in discrete form is much easier to transmit, store or automatically process. Therefore, in computer technology, much attention is paid to methods for converting information from continuous to discrete form.
Discretization of information is the process of converting information from a continuous form of representation to a discrete one.
Let's look at the essence of the information sampling process using an example.
Meteorological stations have recorders for continuous recording of atmospheric pressure. The result of their work is barograms - curves showing how pressure has changed over long periods of time. One of these curves, drawn by the device during seven hours of observation, is shown in Fig. 1.9.

Based on the information received, you can build a table containing the instrument readings at the beginning of measurements and at the end of each hour of observation (Fig. 1.10).

The resulting table does not give a completely complete picture of how the pressure changed during the observation period: for example, the highest pressure value that occurred during the fourth hour of observation is not indicated. But if you tabulate the pressure values ​​observed every half hour or 15 minutes, the new table will give a more complete picture of how the pressure changed.
Thus, we converted information presented in continuous form (barogram, curve) into discrete form (table) with some loss of accuracy.
In the future, you will become familiar with ways to discretely represent audio and graphic information.

Chains of three binary symbols are obtained by complementing two-digit binary codes on the right with the symbol 0 or 1. As a result, the code combinations of three binary symbols are 8 - twice as many as those of two binary symbols:
Accordingly, a four-bit binary allows you to get 16 code combinations, a five-bit one - 32, a six-bit one - 64, etc. The length of the binary chain - the number of characters in the binary code - is called the bit depth of the binary code.
Note that:
4 = 2 * 2,
8 = 2 * 2 * 2,
16 = 2 * 2 * 2 * 2,
32 = 2 * 2 * 2 * 2 * 2 etc.
Here, the number of code combinations is the product of a certain number of identical factors equal to the bit depth of the binary code.
If the number of code combinations is denoted by the letter N, and the bit depth of the binary code by the letter i, then the identified pattern in general form will be written as follows:
N = 2 * 2 * ... * 2.
i factors
In mathematics, such products are written as:
N = 2 i.
Entry 2 i is read as follows: “2 to the i-th power.”

Task. The leader of the Multi tribe instructed his minister to develop a binary and translate all important information into it. What size binary will be required if the alphabet used by the Multi tribe contains 16 characters? Write down all code combinations.
Solution. Since the Multi tribe alphabet consists of 16 characters, they need 16 code combinations. In this case, the length (bit depth) of the binary code is determined from the ratio: 16 = 2 i. Hence i = 4.
To write down all code combinations of four 0s and 1s, we use the diagram in Fig. 1.13: 0000, 0001, 0010, 0011, 0100, 0101, 0110,0111,1000,1001,1010,1011,1100,1101,1110,1111.

1.5.3. The versatility of binary coding
At the beginning of this section, you learned that, represented in continuous form, can be expressed using symbols in some natural or formal language. In turn, characters of an arbitrary alphabet can be converted to binary. Thus, using binary code, any natural and formal languages, as well as images and sounds, can be represented (Fig. 1.14). This means the universality of binary coding.
Binary codes are widely used in computer technology, requiring only two states of an electronic circuit - “on” (this corresponds to the number 1) and “off” (this corresponds to the number 0).
Simplicity of technical implementation is the main advantage of binary coding. The disadvantage of binary coding is the large length of the resulting code.

1.5.4. Uniform and non-uniform codes
There are uniform and non-uniform codes. Uniform codes in code combinations contain the same number of symbols, uneven ones contain a different number.
Above we looked at uniform binary codes.
An example of a non-uniform code is Morse code, in which a sequence of short and long signals is defined for each letter and number. So, the letter E corresponds to a short signal (“dot”), and the letter Ш corresponds to four long signals (four “dashes”). Uneven allows you to increase the speed of message transmission due to the fact that the most frequently occurring symbols in the transmitted information have the shortest code combinations.

The information that this symbol gives is equal to the entropy of the system and is maximum in the case when both states are equally probable; in this case, the elementary symbol conveys information 1 (two units). Therefore, the basis of optimal encoding will be the requirement that elementary characters in the encoded text occur on average equally often.

Let us present here a method for constructing a code that satisfies the stated condition; This method is known as the Shannon-Fano code. Its idea is that the encoded symbols (letters or combinations of letters) are divided into two approximately equally probable groups: for the first group of symbols, 0 is placed in the first place of the combination (the first character of the binary number representing the symbol); for the second group - 1. Next, each group is again divided into two approximately equally probable subgroups; for symbols of the first subgroup, zero is placed in second place; for the second subgroup - one, etc.

Let us demonstrate the principle of constructing the Shannon-Fano code using the material of the Russian alphabet (Table 18.8.1). Let's count the first six letters (from “-” to “t”); summing up their probabilities (frequencies), we get 0.498; all other letters (from “n” to “sf”) will have approximately the same probability of 0.502. The first six letters (from “-” to “t”) will have a binary 0 in the first place. The remaining letters (from “n” to “f”) will have a one in the first place. Next, we again divide the first group into two approximately equally probable subgroups: from “-” to “o” and from “e” to “t”; for all letters of the first subgroup in the second place we will put zero, and of the second subgroup - one. We will continue the process until exactly one letter remains in each division, which will be encoded with a certain binary number. The mechanism for constructing the code is shown in Table 18.8 .2, and the code itself is given in table 18.8.3.

Table 18.8.2.

	Binary signs

Table 18.8.3

Using Table 18.8.3, you can encode and decode any message.

As an example, let’s write the phrase “information theory” in binary code.

01110100001101000110110110000

0110100011111111100110100

1100001011111110101100110

Note that there is no need to separate the letters from each other with a special sign, since decoding is performed unambiguously even without this. You can verify this by decoding the following phrase using Table 18.8.2:

10011100110011001001111010000

1011100111001001101010000110101

010110000110110110

(“encoding method”).

However, it should be noted that any encoding error (random confusion of 0 and 1 characters) with such a code is disastrous, since decoding all text following the error becomes impossible. Therefore, this coding principle can be recommended only in cases where errors in encoding and transmitting a message are practically eliminated.

A natural question arises: is the code we have compiled, in the absence of errors, really optimal? In order to answer this question, let's find the average information per elementary symbol (0 or 1) and compare it with the maximum possible information, which is equal to one binary unit. To do this, we first find the average information contained in one letter of the transmitted text, i.e., entropy per letter:

,

where is the probability that the letter will take a certain state (“-”, o, e, a,..., f).

From the table 18.8.1 we have

(two units per letter of text).

Using table 18.8.2, we determine the average number of elementary symbols per letter

Dividing the entropy by, we obtain information per elementary symbol

(two units).

Thus, the information per character is very close to its upper limit of 1, and the code we have chosen is very close to the optimal one. Remaining within the confines of the task of encoding letters, we cannot achieve anything better.

Note that in the case of encoding simply binary numbers of letters, we would have an image of each letter with five binary characters and the information for one character would be

(two units),

i.e., noticeably less than with optimal letter coding.

However, it should be noted that coding “by letter” is not economical at all. The fact is that there is always a dependence between adjacent letters of any meaningful text. For example, after a vowel in the Russian language there cannot be “ъ” or “ь”; “I” or “yu” cannot appear after hissing ones; after several consonants in a row, the probability of a vowel increases, etc.

We know that when dependent systems are combined, the total entropy is less than the sum of the entropies of the individual systems; therefore, the information conveyed by a piece of connected text is always less than the information per character times the number of characters. Taking this circumstance into account, a more economical code can be constructed if you encode not each letter individually, but entire “blocks” of letters. For example, in a Russian text it makes sense to encode entirely some frequently occurring combinations of letters, such as “tsya”, “ayet”, “nie”, etc. The encoded blocks are arranged in descending order of frequency, like the letters in the table. 18.8.1, and binary coding is carried out according to the same principle.

In some cases, it turns out to be reasonable to encode not even blocks of letters, but entire meaningful pieces of text. For example, to relieve the telegraph during the holidays, it is advisable to encode entire standard texts with conventional numbers, such as:

“Congratulations on the New Year, I wish you good health and success in your work.”

Without dwelling specifically on block coding methods, we will limit ourselves to formulating Shannon’s theorem related here.

Let there be a source of information and a receiver connected by a communication channel (Fig. 18.8.1).

The productivity of the information source is known, i.e. the average number of binary information units coming from the source per unit of time (numerically it is equal to the average entropy of the message produced by the sources per unit of time). Let, in addition, the channel capacity be known, i.e. the maximum amount of information (for example, binary characters 0 or 1) that the channel is capable of transmitting in the same unit of time. The question arises: what should the channel capacity be for it to “cope” with its task, that is, for information to arrive from the source to the receiver without delay?

The answer to this question is given by Shannon's first theorem. Let us formulate it here without proof.

Shannon's 1st theorem

If the communication channel capacity is greater than the entropy of the information source per unit time

then it is always possible to encode a sufficiently long message so that it is transmitted by a communication channel without delay. If, on the contrary,

then the transfer of information without delay is impossible.