In-memory computing apparatus and computing method having a memory array includes a shifted weight storage, shift information storage and shift restoration circuit to restore a weigh shifted amount of shifted sum-of-products to generate multiple restored sum-of-products

The disclosure relates to a computing apparatus, and more particularly, to an in-memory computing apparatus and a computing method thereof.

Artificial Intelligence (AI) refers to the technology that presents human intelligence through computer programs. At present, it has reached a superior level in image recognition, language analysis, and board games. Taking the AI network for image recognition as an example, the convolutional neural network (CNN) is currently a widely used solution for image recognition, including many multiply-accumulate (MAC) calculations, which involve multiplying the value (weight) stored in each memory element in the memory array by the input value and summing all the products, and involve a large amount of data movement, thus resulting in high power consumption.

The disclosure relates to an in-memory computing apparatus and a computing method thereof, which may effectively reduce power consumption of the in-memory computing apparatus.

An in-memory computing apparatus of the disclosure includes a memory control circuit, a memory array, a sense circuit, and a shift restoration circuit. The memory array is coupled to the memory control circuit, and the memory control circuit controls data access of the memory array. The memory array includes a shifted weight storage area, a shift information storage area, and a shift unit amount storage area. The shifted weight storage area stores multiple shifted weight values, and provides multiple shifted sum-of-products according to multiple input signals provided by the memory control circuit through multiple first word lines. The shift information storage area stores a number of shift units of the shifted weight values, and provides the number of shift units of the shifted weight values according to multiple control signals provided by the memory control circuit through multiple second word lines. The shift unit amount storage area stores a shift unit amount, and provides a column shift unit amount according to the input signals. The column shift unit amount is equal to a sum-of-products of the input signals and the shift unit amount. The sense circuit is coupled to the memory array, and senses multiple current signals provided by the shifted weight storage area, the shift information storage area, and the shift unit amount storage area to obtain the multiple shifted sum-of-products, the number of shift units of the shifted weight values, and the column shift unit amount. The shift restoration circuit is coupled to the sense circuit, and restores weight shift amounts of the multiple shifted sum-of-products according to the number of shift units of the shifted weight values and the column shift unit amount, so as to generate multiple restored sum-of-products.

The disclosure further provides a computing method of an in-memory computing apparatus. The in-memory computing apparatus includes a memory array, and the memory array includes a shifted weight storage area, a shift information storage area, and a shift unit amount storage area. The shifted weight storage area stores multiple shifted weight values. The shift information storage area stores a number of shift units of the shifted weight values. The shift unit amount storage area stores a shift unit amount. The computing method of the in-memory computing apparatus includes the following steps. Multiple control signals are provided to the shift information storage area, so that the shift information storage area provides the number of shift units of the shifted weight values. Multiple input signals are provided to the shifted weight storage area and the shift unit amount storage area, so that the shifted weight storage area provides multiple shifted sum-of-products, and the shift unit amount storage area provides a column shift unit amount. The column shift unit amount is equal to a sum-of-products of the input signals and the shift unit amount. Weight shift amounts of the multiple shifted sum-of-products is restored according to the number of shift units of the shifted weight values and the column shift unit amount, so as to generate multiple restored sum-of-products.

Based on the above, the memory array in the embodiment of the disclosure includes the shifted weight storage area that stores the shifted weight values, the shift information storage area that stores the number of shift units, and the shift unit amount storage area that stores the shift unit amount. The shift restoration circuit may restore the weight shift amounts of the multiple shifted sum-of-products according to the number of shift units of the shifted weight values and the column shift unit amount, so as to generate the multiple restored sum-of-products. In this way, the shifted weight storage area stores the shifted weight values, and then uses the number of shift units of the shifted weight values and the column shift unit amount to restore the weight shift amounts of the multiple shifted sum-of-products, which may effectively reduce a current value on a bit line when the in-memory computing apparatus performs the sum-of-products operation, and may greatly reduce the power consumption of the in-memory computing apparatus.

FIG. 1 is a schematic diagram of an in-memory computing apparatus according to an embodiment of the disclosure.

FIG. 2 is a schematic circuit diagram of an in-memory computing apparatus according to another embodiment of the disclosure.

FIG. 3 is a schematic diagram of a shift restoration circuit according to an embodiment of the disclosure.

FIG. 4 is a schematic diagram of an in-memory computing apparatus according to another embodiment of the disclosure.

FIG. 5 is a flowchart of a computing method of an in-memory computing apparatus according to an embodiment of the disclosure.

In order for the disclosure to be more comprehensible, embodiments are described below as examples on which the disclosure may indeed be implemented. In addition, wherever possible, elements/components/steps with the same reference numerals in the drawings and embodiments represent the same or similar parts.

Hereinafter, referring to FIG. 1 , FIG. 1 is a schematic diagram of an in-memory computing apparatus according to an embodiment of the disclosure. The in-memory computing apparatus may include a memory control circuit 102 , a memory array 104 , a sense circuit 106 , and a shift restoration circuit 108 . The memory array 104 is coupled to the memory control circuit 102 and the sense circuit 106 , and the sense circuit 106 is further coupled to the shift restoration circuit 108 . The memory array 104 may include a shifted weight storage area 110 , a shift information storage area 112 , and a shift unit amount storage area 114 . The shifted weight storage area 110 stores multiple shifted weight values. The shift information storage area 112 stores a number of shift units NSF 0 to NSFm corresponding to the shifted weight values. The shift unit amount storage area 114 stores a shift unit amount.

The memory control circuit 102 may control data access of the memory array 104 . Furthermore, the memory control circuit 102 may provide multiple control signals X 0 to XFi to the shift information storage area 112 through word lines WLF 0 to WLFi, so that the shift information storage area 112 provides the number of shift units NSF 0 to NSFm of the shifted weight values according to the control signals X 0 to XFi, where i and m are positive integers. In addition, the memory control circuit 102 may provide multiple input signals X 0 to Xj to the shifted weight storage area 110 and the shift unit amount storage area 114 through word lines WL 0 to WLj (where j is a positive integer), so that the shifted weight storage area 110 provides multiple shifted sum-of-products YS 0 to YSm according to the input signals X 0 to Xj, and the shift unit storage area 114 provides a column shift unit amount Yu through a bit line BLu according to the input signals X 0 to Xj. The multiple shifted sum-of-products YS 0 to YSm are multiple sum-of-products of the input signals X 0 to Xj and the shifted weight values. For example, the multiple shifted sum-of-products YS 0 and YS 1 may be represented by Formulas (1) and (2) in the following.

Xn is the input signal of the word line WLn, and SF 0 and SF 1 are respectively shift amounts of the shifted weight values stored in memory cells on bit lines BL 0 and BL 1 . Wn 0 is a weight value of the shifted weight value stored in the memory cell corresponding to the word line WLn on the bit line BL 0 before being shifted by the shift amount SF 0 . Wn 1 is a weight value of the shifted weight value stored in the memory cell corresponding to the word line WLn on the bit line BL 1 before the shift by the shift amount SF 1 . In other words, Wn 0 and Wn 1 are the original weight values. “Wn 0 -SF 0 ” is the shifted weight value stored in the memory cell corresponding to the word line WLn on the bit line BL 0 , and “Wn 1 -SF 1 ” is the shifted weight value stored in the memory cell corresponding to the word line WLn on the bit line BL 1 . By analogy, YS 2 to YSm may also be represented in a manner similar to Formula (1) or (2). Therefore, the same details will not be repeated in the following.

The sense circuit 106 may sense current signals provided by the shifted weight storage area 110 , the shift unit amount storage area 114 , and the shift information storage area 112 to obtain the multiple shifted sum-of-products YS 0 to YSm, the number of shift units NSF 0 to NSFm of the shifted weight values corresponding to bit lines BL 0 to BLm, and the column shift unit amount Yu. The column shift unit amount Yu is equal to a sum-of-products of the input signals X 0 to Xj and the shift unit amount stored in the shift unit amount storage area 114 . The shift restoration circuit 108 may restore weight shift amounts of the multiple shifted sum-of-products YS 0 to YSm according to the number of shift units NSF 0 to NSFm and the column shift unit amount Yu, so as to generate multiple restored sum-of-products Y 0 to Ym. For example, the restored sum-of-products Y 0 may be represented by Formula (3) in the following.

As shown in Formula (3), in this embodiment, the column shift unit amount Yu is the sum-of-the products of the input signals XO to Xj and the shift unit amount (in this embodiment, the shift unit amount stored in each of the memory cells in the shift unit amount storage area 114 is set to 1; however, the disclosure is not limited thereto, and the shift unit amount may be set according to requirements), and a product value (a shift adjustment amount) of the column shift unit amount Yu and the number of shift units NSF 0 is designed to be equal to the shift amount SF 0 of the shifted weight value corresponding to the bit line BL 0 . Therefore, finally the restored sum-of-products Y 0 in which the weight shift amount has been restored may be obtained. Similarly, the restored sum-of-products Y 1 may be represented by Formula (4) in the following.

By analogy, Y 2 to Ym may also be represented in a manner similar to Formula (3) or (4). Therefore, the same details will not be repeated in the following.

In this way, the shifted weight storage area 110 stores the shifted weight values, and then uses the number of shift units NSF 0 to NSFm of the shifted weight values and the column shift unit amount Yu to restore the weight shift amounts of the multiple shifted sum-of-products YS 0 to YSm, which may effectively reduce a current value on the bit lines BL 0 to BLm when the in-memory computing apparatus performs a sum-of-products operation, and may greatly reduce power consumption of the in-memory computing apparatus.

FIG. 2 is a schematic circuit diagram of an in-memory computing apparatus according to another embodiment of the disclosure. Furthermore, the memory array 104 of the embodiment in FIG. 1 may be, for example, implemented with a NOR flash memory array as shown in FIG. 2 . The shifted weight storage area 110 includes multiple memory cells C 1 . The shift information storage area 112 includes multiple memory cells C 2 . The shift unit amount storage area 114 includes multiple memory cells C 3 . In addition, the sense circuit 106 includes sensors SA 0 to SAm and a sensor SAu.

One end of each of the memory cells C 1 , C 2 , and C 3 is coupled to the corresponding sensor through the corresponding bit line, and the other end is coupled to a source line SL. A control end is coupled to the corresponding word line. For example, one end of the memory cells C 1 and C 2 on the bit line BL 0 is coupled to the sensor SA 0 through the bit line BL 0 , and the other end is coupled to the source line SL. The control ends of the memory cells C 1 on the bit line BL 0 are coupled to the word lines WL 0 to WLj, and the control ends of the memory cells C 2 on the bit line BL 0 are coupled to the word lines WLF 0 to WLFi. In addition, one end of the memory cell C 3 on the bit line BLu is coupled to the bit line BLu, and the other end is coupled to the source line SL. The control ends are coupled to the word lines WL 0 to WLj and WLF 0 to WLFi.

The memory cells C 1 may receive the input signals X 0 to Xj from the word lines WL 0 to WLj to provide the current signals representing the multiple shifted sum-of-products YS 0 to YSm to the sensors SA 0 to SAm through the bit lines BL 0 to BLm according to the input signals X 0 to Xj. The memory control circuit 102 may adjust the shifted weight values stored in the memory cells C 1 , for example, by adjusting a threshold voltage value or a resistance value of the memory cells C 1 , so that the shifted weight values have an expected offset. The shifted weight values stored in the memory cells C 1 on the same bit line have the same shift amount. For example, the shifted weight values stored in the memory cells C 1 on the bit line BL 0 may have the same shift amount SF 0 . The memory cells C 2 may receive the control signals X 0 to XFi from the word lines WLF 0 to WLFi to provide the current signals representing the number of shift units NSF 0 to NSFm of the shifted weight values to the sensors SA 0 to SAm through the bit lines BL 0 to BLm according to the control signals X 0 to XFi. In addition, the memory cells C 3 may receive the input signals X 0 to Xj from the word lines WL 0 to WLj to provide the current signals representing the column shift unit amount Yu to the sensor SAu through the bit line BLu according to the input signals X 0 to Xj.

The sensors SA 0 to SAm and the sensor SAu are coupled to the corresponding bit lines BL 0 to BLm and BLu, and the shift restoration circuit 108 , and provide sensing results of the multiple shifted sum-of-products YS 0 to YSm, the number of shift units NSF 0 to NSFm, and the column shift unit amount Yu to the shift restoration circuit 108 , so as to restore the weight shift amounts of the multiple shifted sum-of-products YS 0 to YSm, and generate the multiple restored sum-of-products Y 0 to Ym. An implementation of the shift restoration circuit 108 may, as shown in FIG. 3 , includes a shift register 302 and an adder circuit 304 . The shift register 302 is coupled to the sensors SA 0 to SAm and the sensor SAu, and the adder circuit 304 is coupled to the shift register 302 . The shift register 302 may receive the number of shift units NSF 0 to NSFm and the column shift unit amount Yu provided by the sensors SA 0 to SAm and the sensor SAu, and generate shift adjustment amounts AD 0 to ADm used to restore the multiple shifted sum-of-products YS 0 to YSm according to the number of shift units NSF 0 to NSFm and the column shift unit amount Yu. The adder circuit 304 may add the multiple shifted sum-of-products YS 0 to YSm provided by the sensors SA 0 to SAm and the corresponding shift adjustment amounts AD 0 to ADm, so as to generate the multiple restored sum-of-products Y 0 to Ym.

Note that although the shifted weight storage area 110 and the shift information storage area 112 in the above embodiment share the bit lines BL 0 to BLm, in other embodiments, the shifted weight storage area 110 and the shift information storage area 112 may also use different bit lines. For example, in the embodiment of FIG. 4 , the shifted weight storage area 110 is coupled to the bit lines BL 0 to BLm, and the shift information storage area 112 is coupled to bit lines BLK 0 to BLKm. The shifted weight storage area 110 and the shift information storage area 112 in this embodiment do not share the bit lines. Therefore, it is not necessary to drive the shift information storage area 112 and the shifted weight storage area 110 sequentially to obtain the number of shift units NSF 0 to NSFm and the multiple shifted sum-of-products YS 0 to YSm. The shift information storage area 112 and the shifted weight storage area 110 may be driven at the same time, so that the shift information storage area 112 and the shifted weight storage area 110 provide the number of shift units NSF 0 to NSFm and the multiple shifted sum-of-products YS 0 to YSm at the same time. As a result, the computing efficiency of the in-memory computing apparatus may be effectively improved.

FIG. 5 is a flowchart of a computing method of an in-memory computing apparatus according to an embodiment of the disclosure. The in-memory computing apparatus includes a memory array. The memory array includes a shifted weight storage area, a shift information storage area, and a shift unit amount storage area. The shifted weight storage area stores multiple shifted weight values. The shift information storage area stores a number of shift units. The shift unit amount storage area stores a shift unit amount. The shifted weight storage area may include multiple first memory cells. The shift information storage area may include multiple second memory cells. The shift unit amount storage area may include multiple third memory cells. The first memory cells and the second memory cells share multiple bit lines, and the first memory cells and the third memory cells use different bit lines.

In light of the above embodiment, the computing method of the in-memory computing apparatus may include at least the following steps. First, multiple control signals are provided to the shift information storage area, so that the shift information storage area provides the number of shift units of the shifted weight values (step S 502 ). The shifted weight values stored in the first memory cells may be adjusted by adjusting a threshold voltage value or a resistance value of the first memory cells, and the shifted weight values stored in the first memory cells on the same bit line have the same shift amount. Then, multiple input signals are provided to the shifted weight storage area and the shift unit amount storage area, so that the shifted weight storage area provides multiple shifted sum-of-products, and the shift unit amount storage area provides a column shift unit amount (step S 504 ). The column shift unit amount is equal to a sum-of-products of the input signals and the shift unit amount. Finally, weight shift amounts of the multiple shifted sum-of-products are restored according to the number of shift units of the shifted weight values and the column shift unit amount, so as to generate multiple restored sum-of-products (step S 506 ). Furthermore, multiple shift adjustment amounts may be generated according to the number of shift units of the shifted weight values and the column shift unit amount, and then the multiple shifted sum-of-products and the corresponding shift adjustment amounts are respectively added, so as to generate the multiple restored sum-of-products. In addition, note that in some embodiments, the bit lines coupled to the shifted weight storage area may be different from the bit lines coupled to the shift information storage area, which may effectively improve the computing efficiency of the in-memory computing apparatus.

Based on the above, the memory array in the embodiment of the disclosure includes the shifted weight storage area that stores the shifted weight values, the shift information storage area that stores the number of shift units, and the shift unit amount storage area that stores the column shift unit amount. The shift restoration circuit may restore the weight shift amounts of the multiple shifted sum-of-products according to the number of shift units of the shifted weight values and the column shift unit amount, so as to generate the multiple restored sum-of-products. In this way, the shifted weight storage area stores the shifted weight values, and then uses the number of shift units of the shifted weight values and the column shift unit amount to restore the weight shift amounts of the shifted sum-of-products, which may effectively reduce the current value on the bit lines when the in-memory computing apparatus performs the sum-of-products operation, and may greatly reduce the power consumption of the in-memory computing apparatus. In some embodiments, the bit lines coupled to the shifted weight storage area may be different from the bit lines coupled to the shift information storage area, which may effectively improve the computing efficiency of the in-memory computing apparatus.

Although the disclosure has been described with reference to the above embodiments, it will be apparent to one of ordinary skill in the art that modifications to the described embodiments may be made without departing from the spirit and the scope of the disclosure. Accordingly, the scope of the disclosure will be defined by the attached claims and their equivalents and not by the above detailed descriptions.