Memory device and computing in memory method thereof

This application claims the benefit of U.S. provisional application Ser. No. 62/916,797, filed Oct. 18, 2019, the subject matter of which is incorporated herein by reference.

The disclosure relates in general to a memory device and a computing in memory method thereof.

In deep learning training, data movement consumes large energy cost. Ideally, computing in memory may reduce 25% energy consumption because movement of weight values is reduced.

FIG. 1 shows performance of MAC (multiplication and accumulation) by computing in memory. Weight values W 1 -W 3 are written into memory cells C 1 -C 3 . Input values I 1 -I 3 are written into word lines or bit lines. For example, in FIG. 1 , Input data I 1 -I 3 are written into word lines WL 1 -WL 3 . Cell currents of the memory cells C 1 -C 3 are I 1 *W 1 , I 2 *W 2 and I 3 *W 3 , respectively. After the cell currents are sensed by the sensing amplifier (SA) 110 , the SA 110 outputs I 1 *W 1 +I 2 *W 2 +I 3 *W 3 .

In computing in memory, taking Covolutional Neural Network (CNN) as an example, when stride operations are performed, it usually takes several cycles to complete the stride operations. Performing stride operations (stride=1) on a 3*3 array (which is a weight array) is taken as an example. FIG. 2 shows nine weights W 1 -W 9 are written into nine memory cells.

FIG. 3 shows the stride operations (stride=1) on a 3*3 array.

FIG. 4 shows nine cycles for completing the stride operations (stride=1) on a 3*3 array.

In a first cycle, inputs data I 1 -I 3 , I 6 -I 8 and I 11 -I 13 are input into the word lines WL 1 -WL 9 , respectively. The operations areas bellows.

In a second cycle, three bits are updated and thus inputs data I 6 -I 8 , I 11 -I 13 and I 16 -I 18 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In a third cycle, three bits are updated and thus inputs data I 11 -I 13 , I 16 -I 18 and I 21 -I 23 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In a fourth cycle, three bits are updated and thus inputs data I 2 -I 4 , I 7 -I 9 and I 12 -I 14 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In a fifth cycle, three bits are updated and thus inputs data I 7 -I 9 , I 12 -I 14 and I 17 -I 19 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In a sixth cycle, three bits are updated and thus inputs data I 12 -I 14 , I 17 -I 19 and I 22 -I 24 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In a seventh cycle, three bits are updated and thus inputs data I 3 -I 5 , I 8 -I 10 and I 13 -I 15 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In an eighth cycle, three bits are updated and thus inputs data I 8 -I 10 , I 13 -I 15 and I 18 -I 20 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

In a ninth cycle, three bits are updated and thus inputs data I 13 -I 15 , I 8 -I 10 and I 23 -I 25 are input into the word lines WL 1 -WL 9 , respectively. The operations are as bellows.

About the traditional CIM, there exists duplicate feeding of input feature maps. This is because the stride operation will generate a lot of input data having contents overlapping. The traditional CIM usually stores one kernel at one bit line and accordingly, input duplicate feeding is caused.

Input duplicate feeding situation will become worse when input data becomes large and stride step becomes small. Thus, how to reduce the input duplicate feeding is very important. As known, more input duplicate feeding results in more data movement, more energy consumption and reduced operation speed.

Thus, how to reduce data movement, lower energy consumption and improve operation speed are important.

According to one embodiment, provided is a computing in memory method for a memory device. The computing in memory method includes: based on a stride parameter, unfolding a kernel into a plurality of sub-kernels and a plurality of complement sub-kernels; based on the sub-kernels and the complement sub-kernels, writing a plurality of weights into a plurality of target memory cells of a memory array of the memory device; inputting an input data into a selected word line of the memory array; performing a stride operation in the memory array; temporarily storing a plurality of partial sums; and summing the stored partial sums into a stride operation result when all operation cycles are completed.

According to one embodiment, provided is a memory device comprising: a memory array; and a controller coupled to the memory array, the controller being configured for based on a stride parameter, unfolding a kernel into a plurality of sub-kernels and a plurality of complement sub-kernels; based on the sub-kernels and the complement sub-kernels, writing a plurality of weights into a plurality of target memory cells of a memory array of the memory device; inputting an input data into a selected word line of the memory array; performing a stride operation in the memory array; temporarily storing a plurality of partial sums; and summing the stored partial sums into a stride operation result when all operation cycles are completed.

FIG. 1 shows performance of MAC (multiplication and accumulation) by computing in memory.

FIG. 2 shows nine weights W 1 -W 9 are written into nine memory cells.

FIG. 3 shows the stride operations (stride=1) on a 3*3 array.

FIG. 4 shows nine cycles for completing the stride operations (stride=1) on a 3*3 array.

FIG. 5 shows a computing in memory method according to one embodiment of the application.

FIG. 6A and FIG. 6B show to unfold the kernel into a plurality of sub-kernels and a plurality of complement sub-kernels.

FIG. 7A to FIG. 7D show MAC operation according to one embodiment of the application.

FIG. 8A and FIG. 8B show that a plurality of weights are written into a plurality of memory cells of the memory array according to one embodiment of the application.

FIG. 9A to FIG. 9H shows four cycles used to complete MAC operations according to one embodiment of the application.

FIG. 10 shows a functional block diagram of the memory device according to one embodiment of the application.

FIG. 11A and FIG. 11B show that a plurality of weights are written into a plurality of memory cells of the memory array according to one embodiment of the application.

In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the disclosed embodiments. It will be apparent, however, that one or more embodiments may be practiced without these specific details. In other instances, well-known structures and devices are schematically shown in order to simplify the drawing.

Technical terms of the disclosure are based on general definition in the technical field of the disclosure. If the disclosure describes or explains one or some terms, definition of the terms is based on the description or explanation of the disclosure. Each of the disclosed embodiments has one or more technical features. In possible implementation, one skilled person in the art would selectively implement part or all technical features of any embodiment of the disclosure or selectively combine part or all technical features of the embodiments of the disclosure.

FIG. 5 shows a computing in memory method according to one embodiment of the application. As shown in FIG. 5 , in step 510 , based on a stride parameter, the kernel is unfolded into a plurality of sub-kernels and a plurality of complement sub-kernels in rows and in columns. In here, the kernel is for example but not limited by a weight matrix.

FIG. 6A and FIG. 6B show to unfold the kernel into a plurality of sub-kernels and a plurality of complement sub-kernels. In here, the stride parameter is 1 and the kernel is a 3*3 weight matrix, but the application is not limited by this.

As shown in FIG. 6A , the kernel K is stridden by zero step, one step, two steps and three steps in column direction (i.e. toward right direction) to obtain the sub-kernels SK 1 , SK 2 , SK 3 and SK 4 ; the sub-kernels SK 1 , SK 2 , SK 3 and SK 4 are stridden by zero step, one step, two steps and three steps in row direction (i.e. toward down direction) to obtain the sub-kernels SK 5 , SK 6 , SK 7 and SK 8 ; the sub-kernels SK 5 , SK 6 , SK 7 and SK 8 are by stridden zero step, one step, two steps and three steps in row direction (i.e. toward down direction) to obtain the sub-kernels SK 9 , SK 10 , SK 11 and SK 12 ; and the sub-kernels SK 9 , SK 10 , SK 11 and SK 12 are stridden by zero step, one step, two steps and three steps in row direction (i.e. toward down direction) to obtain the sub-kernels SK 13 , SK 14 , SK 15 and SK 16 . Each of the sub-kernels SK 1 -SK 16 is also a 3*3 weight matrix. The sub-kernels SK 1 -SK 16 may be a zero matrix, or the sub-kernels SK 1 -SK 16 may include at least one weight of the kernel K.

As shown in FIG. 6B , the sub-kernels SK 1 -SK 16 are complemented as a plurality of complement sub-kernels CSK 1 -CSK 16 . Taking the sub-kernel SK 1 as an example, the sub-kernel SK 1 includes nine weights W 1 -W 9 and thus after complement, the complement sub-kernel CSK 1 includes zero weight (i.e. including nine “0”). Taking the sub-kernel SK 2 as an example, the sub-kernel SK 2 includes six weights W 1 -W 6 and three “0”; and thus after complement, the complement sub-kernel CSK 2 includes three weights W 7 -W 9 and six “0”. That is, a sum of the weight number of the sub-kernel and the weight number of the corresponding complement sub-kernel is equal to the weight number of the kernel.

In general, the kernel includes an original weight matrix. When the original weight matrix is k×l (k and l both being a natural number) and the input data is N×M (N and M both being a natural number), for the stride parameter being equal to “1”, a total number of the sub-kernels is (N−k+1)×(M−l+1) and a total number of the complement sub-kernels is (N−k+1)×(M−l+1).

In step 520 , based on the sub-kernels and the complement sub-kernels, the weights are written into a plurality of target memory cells.

Taken the sub-kernel SK 1 in FIG. 6A . The sub-kernel SK 1 includes nine weights W 1 -W 9 . As for the bit line BL, the weight W 1 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 1 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 2 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 3 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 4 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 5 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 6 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 7 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 8 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 9 .

Taken the sub-kernel SK 2 in FIG. 6A . The sub-kernel SK 2 includes six weights W 1 -W 6 and three “0”. As for the bit line BL 2 , the weight W 1 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 4 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 5 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 6 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 7 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 8 ; and the weight W 6 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 9 .

In step 530 , the input data is input into the selected word lines.

In step 540 , MAC operations are performed in the memory array.

In step 550 , a respective partial sum is stored in a respective latch unit.

In step 560 , it is determined whether the corresponding complement sub-kernel is calculated (or, it is determined whether all operation cycles are completed). If yes in step 560 , then the flow proceeds to step 570 to sum the partial sums stored in the latch units to generate the MAC result. If no in step 560 , then the flow returns to step 530 .

FIG. 7A to FIG. 7D show MAC operation according to one embodiment of the application, wherein the stride parameter is set as one. The application is not limited by this. As shown in FIG. 7A to FIG. 7D , in performing stride operations, when the kernel is a 3*3 matrix and the stride parameter is one, then there are sixteen operations. In operation (a) in FIG. 7A , the input data IN 1 - 1 to IN 1 - 9 are multiplied by the weights W 1 -W 9 , respectively. In operation (b) in FIG. 7A , the input data IN 1 - 4 to IN 1 - 9 and IN 2 - 1 to IN 2 - 3 are multiplied by the weights W 1 -W 9 , respectively.

In one embodiment of the application, in order to reduce data movement, the weights W 1 -W 9 are written into target memory cells as indicated by sixteen operations shown in FIG. 7A to FIG. 7D .

FIG. 8A and FIG. 8B show that a plurality of weights are written into a plurality of memory cells of the memory array 800 according to one embodiment of the application. FIG. 8A and FIG. 8B show a part of the memory array 800 . It is assumed that, in real operations, the input data IN 1 - 1 to IN 1 - 9 are input into the word lines WL 1 to WL 9 ; the input data IN 2 - 1 to IN 2 - 9 are input into the word lines WL 10 to WL 18 ; the input data IN 3 - 1 to IN 3 - 9 are input into the word lines WL 19 to WL 27 ; and the input data IN 4 - 1 to IN 4 - 9 are input into the word lines WL 28 to WL 36 .

As shown in operation (a) of FIG. 7A , the input data IN 1 - 1 to IN 1 - 9 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 1 *W 1 +IN 1 - 2 *W 2 +IN 1 - 3 *W 3 +IN 1 - 4 *W 4 +IN 1 - 5 *W 5 +IN 1 - 6 *W 6 +IN 1 - 7 *W 7 +IN 1 - 8 *W 8 +IN 1 - 9 *W 9 . In order to write the weights W 1 to W 9 to the bit line BL 1 , the weight W 1 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 1 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 2 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 3 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 4 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 5 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 6 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 7 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 8 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 1 and the word line WL 9 . When the input data IN 1 - 1 to IN 1 - 9 are input into the word lines WL 1 to WL 9 , respectively (in the case that the word lines WL 1 to WL 9 are selected), the latch unit L 1 (which is related to the bit line BL 1 ) stores the value of IN 1 - 1 *W 1 +IN 1 - 2 *W 2 +IN 1 - 3 *W 3 +IN 1 - 4 *W 4 +IN 1 - 5 *W 5 +IN 1 - 6 *W 6 +IN 1 - 7 *W 7 +IN 1 - 8 *W 8 +IN 1 - 9 *W 9 .

As shown in operation (b) of FIG. 7A , the input data IN 1 - 4 to IN 1 - 9 and IN 2 - 1 to IN 2 - 3 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 4 *W 1 +IN 1 - 5 *W 2 +IN 1 - 6 *W 3 +IN 1 - 7 *W 4 +IN 1 - 8 *W 5 +IN 1 - 9 *W 6 +IN 2 - 1 *W 7 +IN 2 - 2 *W 8 +IN 2 - 3 *W 9 . In order to write the weights W 1 to W 9 to the bit line BL 2 , the weight W 1 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 4 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 5 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 6 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 7 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 8 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 9 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 10 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 11 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 2 and the word line WL 12 . When the input data IN 1 - 4 to IN 1 - 9 and IN 2 - 1 to IN 2 - 3 are input into the word lines WL 4 to WL 12 , respectively (in the case that the word lines WL 4 to WL 12 are selected), the latch unit L 2 (which is related to the bit line BL 2 ) stores the value of: IN 1 - 4 *W 1 +IN 1 - 5 *W 2 +IN 1 - 6 *W 3 +IN 1 - 7 *W 4 +IN 1 - 8 *W 5 +IN 1 - 9 *W 6 +IN 2 - 1 *W 7 +IN 2 - 2 *W 8 +IN 2 - 3 *W 9 .

As shown in operation (c) of FIG. 7A , the input data IN 1 - 7 to IN 1 - 9 and IN 2 - 1 to IN 2 - 6 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 7 *W 1 +IN 1 - 8 *W 2 +IN 1 - 9 *W 3 +IN 2 - 1 *W 4 +IN 2 - 2 *W 5 +IN 2 - 3 *W 6 +IN 2 - 4 *W 7 +IN 2 - 5 *W 8 +IN 2 - 6 *W 9 . In order to write the weights W 1 to W 9 to the bit line BL 3 , the weight W 1 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 7 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 8 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 9 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 10 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 11 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 12 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 13 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 14 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 3 and the word line WL 15 . When the input data IN 1 - 7 to IN 1 - 9 and IN 2 - 1 to IN 2 - 6 are input into the word lines WL 7 to WL 15 , respectively, the latch unit L 3 (which is related to the bit line BL 3 ) stores the value of: IN 1 - 7 *W 1 +IN 1 - 8 *W 2 +IN 1 - 9 *W 3 +IN 2 - 1 *W 4 +IN 2 - 2 *W 5 +IN 2 - 3 *W 6 +IN 2 - 4 *W 7 +IN 2 - 5 *W 8 +IN 2 - 6 *W 9 .

As shown in operation (d) of FIG. 7A , the input data IN 2 - 1 to IN 2 - 9 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 2 - 1 *W 1 +IN 2 - 2 *W 2 +IN 2 - 3 *W 3 +IN 2 - 4 *W 4 +IN 2 - 5 *W 5 +IN 2 - 6 *W 6 +IN 2 - 7 *W 7 +IN 2 - 8 *W 8 +IN 2 - 9 *W 9 . In order to write the weights W 1 to W 9 to the bit line BL 4 , the weight W 1 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 10 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 11 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 12 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 13 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 14 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 15 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 16 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 17 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 4 and the word line WL 18 . When the input data IN 2 - 1 to IN 2 - 9 are input into the word lines WL 10 to WL 18 , respectively, the latch unit L 4 (which is related to the bit line BL 4 ) stores the value of: IN 2 - 1 *W 1 +IN 2 - 2 *W 2 +IN 2 - 3 *W 3 +IN 2 - 4 *W 4 +IN 2 - 5 *W 5 +IN 2 - 6 *W 6 +IN 2 - 7 *W 7 +IN 2 - 8 *W 8 +IN 2 - 9 *W 9 .

As shown in operation (e) of FIG. 7A , the input data IN 1 - 2 , IN 1 - 3 , IN 3 - 1 , IN 1 - 5 , IN 1 - 6 , IN 3 - 4 , IN 1 - 8 , IN 1 - 9 and IN 3 - 7 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 2 *W 1 +IN 1 - 3 *W 2 +IN 3 - 1 *W 3 +IN 1 - 5 *W 4 +IN 1 - 6 *W 5 +IN 3 - 4 *W 6 +IN 1 - 8 *W 7 +IN 1 - 9 *W 8 +IN 3 - 7 *W 9 . In order to write the weights W 1 to W 9 to the bit line BL 5 , the weight W 1 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 2 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 3 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 19 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 5 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 6 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 22 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 8 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 9 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 5 and the word line WL 25 . When the input data IN 1 - 2 , IN 1 - 3 , IN 3 - 1 , IN 1 - 5 , IN 1 - 6 , IN 3 - 4 , IN 1 - 8 , IN 1 - 9 and IN 3 - 7 are input, the latch unit L 5 (which is related to the bit line BL 5 ) stores the value of: IN 1 - 2 *W 1 +IN 1 - 3 *W 2 +IN 3 - 1 *W 3 +IN 1 - 5 *W 4 +IN 1 - 6 *W 5 +IN 3 - 4 *W 6 +IN 1 - 8 *W 7 +IN 1 - 9 *W 8 +IN 3 - 7 *W 9 .

As shown in operation (f) of FIG. 7B , the input data IN 1 - 5 , IN 1 - 6 , IN 3 - 4 , IN 1 - 8 , IN 1 - 9 , IN 3 - 7 , IN 2 - 2 , IN 2 - 3 and IN 4 - 1 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 5 *W 1 +IN 1 - 6 *W 2 +IN 3 - 4 *W 3 +IN 1 - 8 *W 4 +IN 1 - 9 *W 5 +IN 3 - 7 *W 6 +IN 2 - 2 *W 7 +IN 2 - 3 *W 8 +IN 4 - 1 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 6 , the weight W 1 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 5 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 6 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 22 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 8 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 9 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 25 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 11 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 12 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 6 and the word line WL 28 . When the input data IN 1 - 5 , IN 1 - 6 , IN 3 - 4 , IN 1 - 8 , IN 1 - 9 , IN 3 - 7 , IN 2 - 2 , IN 2 - 3 and IN 4 - 1 are input, the latch unit L 6 (which is related to the bit line BL 6 ) stores the value of: IN 1 - 5 *W 1 +IN 1 - 6 *W 2 +IN 3 - 4 *W 3 +IN 1 - 8 *W 4 +IN 1 - 9 *W 5 +IN 3 - 7 *W 6 +IN 2 - 2 *W 7 +IN 2 - 3 *W 8 +IN 4 - 1 *W 9 .

As shown in operation (g) of FIG. 7B , the input data IN 1 - 8 , IN 1 - 9 , IN 3 - 7 , IN 2 - 2 , IN 2 - 3 , IN 4 - 1 , IN 2 - 5 , IN 2 - 6 and IN 4 - 4 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 8 *W 1 +IN 1 - 9 *W 2 +IN 3 - 7 *W 3 +IN 2 - 2 *W 4 +IN 2 - 3 *W 5 +IN 4 - 1 *W 6 +IN 2 - 5 *W 7 +IN 2 - 6 *W 8 +IN 4 - 4 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 7 , the weight W 1 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 8 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 9 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 25 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 11 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 12 : the weight W 6 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 28 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 14 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 15 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 7 and the word line WL 31 . When the input data IN 1 - 8 , IN 1 - 9 , IN 3 - 7 , IN 2 - 2 , IN 2 - 3 , IN 4 - 1 , IN 2 - 5 , IN 2 - 6 and IN 4 - 4 are input, the latch unit L 7 (which is related to the bit line BL 7 ) stores the value of: IN 1 - 8 *W 1 +IN 1 - 9 *W 2 +IN 3 - 7 *W 3 +IN 2 - 2 *W 4 +IN 2 - 3 *W 5 +IN 4 - 1 *W 6 +IN 2 - 5 *W 7 +IN 2 - 6 *W 8 +IN 4 - 4 *W 9 .

As shown in operation (h) of FIG. 7B , the input data IN 2 - 2 , IN 2 - 3 , IN 4 - 1 , IN 2 - 5 , IN 2 - 6 , IN 4 - 4 , IN 2 - 8 , IN 2 - 9 and IN 4 - 1 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 2 - 2 *W 1 +IN 2 - 3 *W 2 +IN 4 - 1 *W 3 +IN 2 - 5 *W 4 +IN 2 - 6 *W 5 +IN 4 - 4 *W 6 +IN 2 - 8 *W 7 +IN 2 - 9 *W 8 +IN 4 - 7 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 8 , the weight W 1 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 11 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 12 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 28 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 14 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 15 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 31 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 17 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 18 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 8 and the word line WL 34 . When the input data IN 2 - 2 , IN 2 - 3 , IN 4 - 1 , IN 2 - 5 , IN 2 - 6 , IN 4 - 4 , IN 2 - 8 , IN 2 - 9 and IN 4 - 1 are input, the latch unit L 8 (which is related to the bit line BL 8 ) stores the value of: IN 2 - 2 *W 1 +IN 2 - 3 *W 2 +IN 4 - 1 *W 3 +IN 2 - 5 *W 4 +IN 2 - 6 *W 5 +IN 4 - 4 *W 6 +IN 2 - 8 *W 7 +IN 2 - 9 *W 8 +IN 4 - 7 *W 9 .

As shown in operation (i) of FIG. 7C , the input data IN 1 - 3 , IN 3 - 1 , IN 3 - 2 , IN 1 - 6 , IN 3 - 4 , IN 3 - 5 , IN 1 - 9 , IN 3 - 7 and IN 3 - 8 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 3 *W 1 +IN 3 - 1 *W 2 +IN 3 - 2 *W 3 +IN 1 - 6 *W 4 +IN 3 - 4 *W 5 +IN 3 - 5 *W 6 +IN 1 - 9 *W 7 +IN 3 - 7 *W 8 +IN 3 - 8 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 9 , the weight W 1 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 3 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 19 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 20 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 6 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 22 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 23 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 9 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 25 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 9 and the word line WL 26 . When the input data IN 1 - 3 , IN 3 - 1 , IN 3 - 2 , IN 1 - 6 , IN 3 - 4 , IN 3 - 5 , IN 1 - 9 , IN 3 - 7 and IN 3 - 8 are input, the latch unit L 9 (which is related to the bit line BL 9 ) stores the value of: IN 1 - 3 *W 1 +IN 3 - 1 *W 2 +IN 3 - 2 *W 3 +IN 1 - 6 *W 4 +IN 3 - 4 *W 5 +IN 3 - 5 *W 6 +IN 1 - 9 *W 7 +IN 3 - 7 *W 8 +IN 3 - 8 *W 9 .

As shown in operation (j) of FIG. 7C , the input data IN 1 - 6 , IN 3 - 4 , IN 3 - 5 , IN 1 - 9 , IN 3 - 7 , IN 3 - 8 , IN 2 - 3 , IN 4 - 1 and IN 4 - 2 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 6 *W 1 +IN 3 - 4 *W 2 +IN 3 - 5 *W 3 +IN 1 - 9 *W 4 +IN 3 - 7 *W 5 +IN 3 - 8 *W 6 +IN 2 - 3 *W 7 +IN 4 - 1 *W 8 +N 4 - 2 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 10 , the weight W 1 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 6 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 22 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 23 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 9 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 25 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 26 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 12 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 28 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 10 and the word line WL 29 . When the input data IN 1 - 6 , IN 3 - 4 , IN 3 - 5 , IN 1 - 9 , IN 3 - 7 , IN 3 - 8 , IN 2 - 3 , IN 4 - 1 and IN 4 - 2 are input, the latch unit L 10 (which is related to the bit line BL 10 ) stores the value of: IN 1 - 6 *W 1 +IN 3 - 4 *W 2 +IN 3 - 5 *W 3 +IN 1 - 9 *W 4 +IN 3 - 7 *W 5 +IN 3 - 8 *W 6 +IN 2 - 3 *W 7 +IN 4 - 1 *W 8 +IN 4 - 2 *W 9 .

As shown in operation (k) of FIG. 7C , the input data IN 1 - 9 , IN 3 - 7 , IN 3 - 8 , IN 2 - 3 , IN 4 - 1 , IN 4 - 2 , IN 2 - 6 , IN 4 - 4 and IN 4 - 5 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 1 - 9 *W 1 +IN 3 - 7 *W 2 +IN 3 - 8 *W 3 +IN 2 - 3 *W 4 +IN 4 - 1 *W 5 +IN 4 - 2 *W 6 +IN 2 - 3 *W 7 +IN 4 - 4 *W 8 +IN 4 - 5 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 11 , the weight W 1 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 9 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 25 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 26 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 12 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 28 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 29 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 15 : the weight W 8 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 31 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 11 and the word line WL 32 . When the input data IN 1 - 9 , IN 3 - 7 , IN 3 - 8 , IN 2 - 3 , IN 4 - 1 , IN 4 - 2 , IN 2 - 6 , IN 4 - 4 and IN 4 - 5 are input, the latch unit L 11 (which is related to the bit line BL 11 ) stores the value of: IN 1 - 9 *W 1 +IN 3 - 7 *W 2 +IN 3 - 8 *W 3 +IN 2 - 3 *W 4 +IN 4 - 1 *W 5 +IN 4 - 2 *W 6 +IN 2 - 3 *W 7 +IN 4 - 4 *W 8 +IN 4 - 5 *W 9 .

As shown in operation ( 1 ) of FIG. 7C , the input data IN 2 - 3 , IN 4 - 1 , IN 4 - 2 , IN 2 - 6 , IN 4 - 4 , IN 4 - 5 , IN 2 - 9 , IN 4 - 7 and IN 4 - 8 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 2 - 3 *W 1 +IN 4 - 1 *W 2 +IN 4 - 2 *W 3 +IN 2 - 6 *W 4 +IN 4 - 4 *W 5 +IN 4 - 5 *W 6 +IN 2 - 9 *W 7 +IN 4 - 7 *W 8 +IN 4 - 8 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 12 , the weight W 1 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 12 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 28 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 29 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 15 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 31 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 32 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 18 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 34 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 12 and the word line WL 35 . When the input data IN 2 - 3 , IN 4 - 1 , IN 4 - 2 , IN 2 - 6 , IN 4 - 4 , IN 4 - 5 , IN 2 - 9 , IN 4 - 7 and IN 4 - 8 are input, the latch unit L 12 (which is related to the bit line BL 12 ) stores the value of: IN 2 - 3 *W 1 +IN 4 - 1 *W 2 +IN 4 - 2 *W 3 +IN 2 - 6 *W 4 +IN 4 - 4 *W 5 +IN 4 - 5 *W 6 +IN 2 - 9 *W 7 +IN 4 - 7 *W 8 +IN 4 - 8 *W 9 .

As shown in operation (m) of FIG. 7D , the input data IN 3 - 1 to IN 3 - 9 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 3 - 1 *W 1 +IN 3 - 2 *W 2 +IN 3 - 3 *W 3 +IN 3 - 4 *W 4 +IN 3 - 5 *W 5 +IN 3 - 6 *W 6 +IN 3 - 7 *W 7 +IN 3 - 8 *W 8 +IN 3 - 9 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 13 , the weight W 1 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 19 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 20 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 21 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 22 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 24 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 24 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 25 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 26 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 13 and the word line WL 27 . When the input data IN 3 - 1 to IN 3 - 9 are input, the latch unit L 13 (which is related to the bit line BL 13 ) stores the value of: IN 3 - 1 *W 1 +IN 3 - 2 *W 2 +IN 3 - 3 *W 3 +IN 3 - 4 *W 4 +IN 3 - 5 *W 5 +IN 3 - 6 *W 6 +IN 3 - 7 *W 7 +IN 3 - 8 *W 8 +IN 3 - 9 *W 9 .

As shown in operation (n) of FIG. 7D , the input data IN 3 - 4 to IN 3 - 9 and IN 4 - 1 to IN 4 - 3 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 3 - 4 *W 1 +IN 3 - 5 *W 2 +IN 3 - 6 *W 3 +IN 3 - 7 *W 4 +IN 3 - 8 *W 5 +IN 3 - 9 *W 6 +IN 4 - 1 *W 7 +IN 4 - 2 *W 8 +N 4 - 3 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 14 , the weight W 1 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 22 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 23 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 24 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 25 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 26 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 27 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 28 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 29 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 14 and the word line WL 30 . When the input data IN 3 - 4 to IN 3 - 9 and IN 4 - 1 to IN 4 - 3 are input, the latch unit L 14 (which is related to the bit line BL 14 ) stores the value of: IN 3 - 4 *W 1 +IN 3 - 5 *W 2 +IN 3 - 6 *W 3 +IN 3 - 7 *W 4 +IN 3 - 8 *W 5 +IN 3 - 9 *W 6 +IN 4 - 1 *W 7 +IN 4 - 2 *W 8 +IN 4 - 3 *W 9 .

As shown in operation (o) of FIG. 7D , the input data IN 3 - 7 to IN 3 - 9 and IN 4 - 1 to IN 4 - 6 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 3 - 7 *W 1 +IN 3 - 8 *W 2 +IN 3 - 9 *W 3 +IN 4 - 1 *W 4 +IN 4 - 2 *W 5 +IN 4 - 3 *W 6 +IN 4 - 4 *W 7 +IN 4 - 5 *W 8 +IN 4 - 6 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 15 , the weight W 1 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 28 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 26 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 27 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 28 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 29 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 30 : the weight W 7 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 32 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 32 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 15 and the word line WL 33 . When the input data IN 3 - 4 to IN 3 - 9 and IN 4 - 1 to IN 4 - 3 are input, the latch unit L 15 (which is related to the bit line BL 15 ) stores the value of: IN 3 - 4 *W 1 +IN 3 - 5 *W 2 +IN 3 - 6 *W 3 +IN 3 - 7 *W 4 +IN 3 - 8 *W 5 +IN 3 - 9 *W 6 +IN 4 - 1 *W 7 +IN 4 - 2 *W 8 +IN 4 - 3 *W 9 .

As shown in operation (p) of FIG. 7D , the input data IN 4 - 1 to IN 4 - 9 are multiplied by the weights W 1 to W 9 and thus the following is obtained: IN 4 - 1 *W 1 +IN 4 - 2 *W 2 +IN 4 - 3 *W 3 +IN 4 - 4 *W 4 +IN 4 - 5 *W 5 +IN 4 - 6 *W 6 +IN 4 - 7 *W 7 +IN 4 - 8 *W 8 +IN 4 - 9 *W 9 . In order to write the weights W 1 to W 9 into the bit line BL 16 , the weight W 1 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 28 ; the weight W 2 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 29 ; the weight W 3 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 30 ; the weight W 4 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 31 ; the weight W 5 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 32 ; the weight W 6 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 33 ; the weight W 7 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 34 ; the weight W 8 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 35 ; and the weight W 9 is written into a memory cell at intersection of the bit line BL 16 and the word line WL 36 . When the input data IN 4 - 1 to IN 4 - 9 are input, the latch unit L 16 (which is related to the bit line BL 16 ) stores the value of: IN 4 - 1 *W 1 +IN 4 - 2 *W 2 +IN 4 - 3 *W 3 +IN 4 - 4 *W 4 +IN 4 - 5 *W 5 +IN 4 - 6 *W 6 +IN 4 - 7 *W 7 +IN 4 - 8 *W 8 +IN 4 - 9 *W 9 .

FIG. 9A to FIG. 9H shows four cycles used to complete MAC operations according to one embodiment of the application.

As shown in FIG. 9A and FIG. 9B , in the first cycle, the word lines WL 1 to WL 9 are selected and thus the input data IN 1 - 1 to IN 1 - 9 are input into the memory array. Therefore, in the first cycle, the respective partial sum stored in the latch units L 1 to L 16 are as follows:

As shown in FIG. 9C and FIG. 9 , in the second cycle, the word lines WL 10 to WL 1 are selected and thus the input data IN 2 - 1 to IN 2 - 9 are input into the memory array. Therefore, in the second cycle, the respective partial sum stored in the latch units L 1 to L 16 are as follows (for easy understanding, the partial sum in the respective cycles are respectively listed):

As shown in FIG. 9E and FIG. 9F , in the third cycle, the word lines WL 9 to WL 27 are selected and thus the input data I 3 - 1 to IN 3 - 9 are input into the memory array. Therefore, in the third cycle, the respective partial sum stored in the latch units L 1 to L 16 are as follows (for easy understanding, the partial sum in the respective cycles are respectively listed):

As shown in FIG. 9G and FIG. 9H , in the fourth cycle, the word lines WL 28 to WL 36 are selected and thus the input data IN 4 - 1 to IN 4 - 9 are input into the memory array. Therefore, in the fourth cycle, the respective partial sum stored in the latch units L 1 to L 16 are as follows (for easy understanding, the partial sum in the respective cycles are respectively listed):

For easy understanding, the partial sums in the four cycles are added as follows (i.e. the output from the latch units L 1 to L 16 after four cycles):

FIG. 10 shows a functional block diagram of the memory device according to one embodiment of the application. The memory device 1000 according to one embodiment of the application includes a controller 1010 and a memory array 1020 . The controller 1010 is coupled to the memory array 1020 . The controller 1010 controls the memory array 1020 to perform the MAC operations in the above embodiments. A plurality of memory cells of the memory array 1020 store the weights in the above embodiments.

FIG. 11A and FIG. 11B show that a plurality of weights are written into a plurality of memory cells of the memory array 1100 according to one embodiment of the application. Compared with FIG. 8A and FIG. 8B , memory cell usage rate of the memory array 1100 of FIG. 11A and FIG. 11B is improved.

As above, advantages of embodiments of the application are reduction of data movement and faster execution time.

As above, in embodiments of the application, in stride operations, the kernel (the weight matrix) of the deep learning model are unfolded into a plurality of sub-kernels and a plurality of complement sub-kernels. The weights are written into the target memory cells based on the sub-kernels and the complement sub-kernels. Thus, the input data is efficiently reused in the memory array for reducing operation time and data movement.

Embodiments of the application are used in AI (Artificial Intelligence) field or any computing field having many MAC operations, for example but not limited by, memory data search, image processing and voice detection.

Embodiments of the application are used in different AI model design, for example but not limited by, fully connection layer model design, convolution layer model design, multilayer perceptron model design and support vector machine.

Embodiments of the application may apply any volatile memory (SRAM, DRAM) or any non-volatile memory (Resistive-RAM, Phase Change Memory, flash memory, Magnetoresistive RAM, Ferroelectric RAM and so on).

Further, in other possible embodiments of the application, the role of the bit lines and the word lines are interchangeable, that is, the input data may be input into the bit lines, which is still within the spirit and the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments. It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims and their equivalents.