Stall-free pipelined cache for statically scheduled and dispatched execution

The technical field of this invention is cache memory in data processing devices.

Cache has been widely used in digital signal processors (DSPs) as well as general purpose processors in achieving high operating clock frequency. Cache accesses are typically pipelined, thus load instructions can be issued every cycle. Due to the pipelined nature of the processing, there is a certain amount of latency for load instructions. Table 1 illustrates the pipeline stages and their functions for a typical digital signal processor.

Program fetch is performed in four pipeline stages PG, PS, PW and PR. Program decode is made up of the DP and DC pipeline stages. Execution takes place in pipeline stages E 1 to E 5 . Note that differing instructions include differing number of execute pipeline stages. Single cycle instructions including add, subtract and logical operations complete in a single execute stage (E 1 ) except for updating the SAT bit in the control status register. Multiply instructions complete in execute stage E 2 except for updating the SAT bit in the control status register. Store instructions complete in execute stage E 3 . Load instructions complete in execute stage E 5 .

FIG. 1 illustrates the functions of an example VLIW DSP including the pipeline phases of the processor. Fetch phase 100 includes the PG pipeline stage 101 , the PS pipeline stage 102 , the PW pipeline stage 103 and the PR pipeline stage 104 . In each of these pipeline stages the DSP can perform eight simultaneous commands. These commands are summarized in Table 2. The decode phase 110 includes the DP pipeline stage 105 and the DC pipeline stage 106 . Decode phase 110 also performs commands from Table 2.

FIG. 1 illustrates memory hardware external to the CPU. Program cache memory 111 stores the instructions to be performed and data cache memory 126 stores all operands in immediate use. Memory controller 125 performs program fetch control and memory controller 112 performs data transfer control. Bulk data storage resides in external memory 131 . Level-2 cache 127 provides high-speed access to data in current use.

Execute phase 120 performs all other operations including: evaluation of conditions and status; Load-Store instructions; Branch instructions; and single cycle instructions. Execute stages E 1 107 , E 2 108 prepare for the E 3 109 stage cache memory access from data cache memory 126 to retrieve the required operand.

Upon a cache hit, processing proceeds to execute stage E 4 117 and then to execute stage E 5 124 with results stored in the register file. Upon a cache miss, memory controller 112 inserts a fixed number of stall cycles via path 128 allowing data to be retrieved from level-2 cache 127 or external memory 131 . Data is returned to the pipeline via path 130 .

FIG. 2 illustrates the manner in which the pipeline is filled in an example pipeline execution of a DSP that has a four cycle load latency. Successive fetch stages can occur every clock cycle. In a given fetch packet, such as fetch packet n 200 , the fetch phase is completed in four clock cycles with the pipeline stages PG 201 , PS 202 , PW 203 and PR 204 as described in Table 1. In fetch packet n the next two clock cycles fifth 205 and sixth 206 are devoted to the program decode stage phase including dispatch stage 205 and decode stage 206 . The seventh clock cycle 207 and succeeding clock cycles of fetch packet n are devoted to the execution of the commands of the instructions of the packet. Any additional processing that may be required in processing a given packet, if not executed in the first eleven clock cycles could result in pipeline stalls or data memory stalls.

Referring back to FIG. 1 , cache memory accesses are initiated during pipeline stages E 2 108 and E 3 109 and the data is brought to the CPU boundary at pipeline stage E 4 via path 130 . At pipeline stage E 5 124 , data is written into the register file. For a cache miss 128 , the pipeline stalls CPU at pipeline stage E 3 . FIG. 3 illustrates such a stall is at 301 . This example includes two assumptions: a four cycle cache miss penalty; and the second load accesses the same cache line as the first load.

The overhead of stall cycles due to cache misses depends on the cache miss penalty and a cache hit ratio of the code executed. It is highly desirable to reduce or eliminate these stall cycles to better utilize the processing core.

Current digital signal processors utilize cache memory for storage of program and operand data. Cache storage incurs stall cycles upon cache misses in these statically scheduled processors with dispatched execution. The present invention provides flexible load latency so that cache misses can be effectively pipelined. This approach is advantageous if the code can tolerate increased load latency, which normally presents no performance penalty.

These and other aspects of this invention are illustrated in the drawings, in which:

FIG. 1 illustrates the block diagram of the functions and pipeline stages of a VLIW DSP (Prior Art);

FIG. 2 illustrates an example pipeline execution of a DSP having a four cycle load latency and how the pipeline is filled (Prior Art);

FIG. 3 illustrates an example of a pipelined cache miss assuming a four cycle load latency in which the cache inserts four stall cycles at pipeline stage E 3 (Prior Art);

FIG. 4 illustrates an example of this invention of a pipelined cache miss assuming a six cycle load latency in which the cache inserts only two stall cycles at pipeline stage E 3 ;

FIG. 5 illustrates another example of this invention of an access following a cache miss assuming an eight cycle load latency needing no stall insertions; and

FIG. 6 illustrates the functions of a VLIW DSP with a four cycle cache miss penalty and an eight cycle load latency.

Cache misses initiate CPU stalls at pipeline stage E 3 because the code is normally written with an assumption that data loaded from cache are always available in four cycles. If the code can tolerate more than four cycle latency, the number of stall cycles can be reduced or completely removed.

FIG. 4 illustrates an example of this invention where the code assumes four cycle load latency. Upon a cache miss, the cache inserts only two stall cycles 401 at E 3 . The example of FIG. 4 assumes that cache misses can be pipelined. In addition as part of the invention, a miss-under-service bit must be added to each cache line, so that a following access to the cache line being fetched can be a hit. The memory data controller 112 is modified to cause pipeline data to accumulate in additional pipeline stages inserted between E 3 109 and E 4 117 . This data is re-inserted into the pipeline starting at the fifteenth clock cycle as shown in FIG. 4 .

Table 3 shows the pipeline data re-inserted into the pipeline at the 15th clock cycle.

FIG. 5 illustrates another example of this invention where the code assumes an eight cycle load latency. In this case no stall needs to be inserted even though a following access is another miss. The example of FIG. 5 assumes that cache misses can be pipelined. Also as in the previous example, a miss-under-service bit must be added to each cache line, so that a following access to the cache line being fetched can be a hit. The memory data controller 112 is modified to cause pipeline data to be accumulated in additional pipeline stages inserted between E 3 and E 4 . This data is re-inserted into the pipeline starting at the fifteenth clock cycle as shown in FIG. 5 .

Table 4 shows the pipeline data re-inserted into the pipeline at the 15th clock cycle.

FIG. 6 illustrates the pipeline structure of the processor of FIG. 1 modified to accentuate the bypass feature available for pipeline data in the case of the examples of FIGS. 4 and 5 . Four additional pipeline stages 601 through 604 are inserted into the pipeline between pipeline stages E 3 109 and E 4 117 . Data memory controller 612 is configured to determine the number of stages to be inserted and these are coded in signal 606 to select the routing for multiplexer 605 .

The negative effects of longer load latency are: longer code schedule length; and bigger code size. The longer schedule length can be negligible if the number of loop iterations is large enough.

The load latency can be dynamically changed in the code via several means. For example, load instructions can explicitly specify the expected latency. Another possible approach is a set up instruction specifying latency for following load instructions. In this case, the latency information must go through the load pipeline and be stored in the cache controller. A further alternative embodiment includes a no-allocate load that does not stall the cache. Note that this concept can be applied to the instruction cache.

The cache miss penalty for a read miss can be worsened due to other architectural limitations. For example, a write buffer in many DSPs is always flushed for a read miss. To make this invention more effective, this kind of conflict must be handled more actively. The write buffer is not flushed unless it contains data for a read miss. Other unnecessary stalls worsening the read miss penalty must be handled similarly.

Consider a simple but extreme case where 512 units of 32-bit data are loaded and accumulated. Assume the cache miss penalty is eight cycles, the amount of data words to load is 2048 bytes or 512 data words, the cache line size is 32 bytes or eight data words and the load latency is four cycles. From the cache line size and amount of data to load, the number of cache misses is 64. Thus the number of stall cycles is 512. The total number of execution cycles without cache miss penalty is 517. This is the sum of 6 and 511 since the schedule length is 6 cycles and there are 512 pipelined iterations. In this example, the total number of cycles including cache miss stalls is 1029 (the sum of 512 and 517) when fully software pipelined. This results in about 50% of cache miss overhead.

A load latency of 12 cycles completely hides the miss penalty but increases the schedule length from 6 to 14. In this case the total number of cycles becomes 525 (the sum of 14 and 511), resulting in about two times speedup.