Method and system for latency optimized ATS usage

[Not Applicable.]

[Not Applicable]

[Not Applicable]

Certain embodiments of the invention relate to processing data. More specifically, certain embodiments of the invention relate to a method and system for latency optimized address translation system (ATS) usage.

Many modern electronic systems have memory architecture that may comprise physical memory and logical memory. For example, when addressing memory, a processor may reference a memory address, which may result in a cache hit if the memory address is associated with a cache. However, a miss can result in delay while the data associated with the memory address is fetched from another memory device, such as another level of cache, main memory, or a mass storage device.

Other memory architectures may use translation of addresses in order to access physical memory devices. For example, a processor may request data access using a virtual address that may need to be translated to a physical address. The virtual address may then be translated to a physical address before the physical memory can be accessed.

Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present invention as set forth in the remainder of the present application with reference to the drawings.

A system and/or method for latency optimized ATS usage, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.

Various advantages, aspects and novel features of the present invention, as well as details of an illustrated embodiment thereof, will be more fully understood from the following description and drawings.

FIG. 1 is a block diagram of an exemplary system for accessing memory, in accordance with an embodiment of the invention.

FIG. 2 is a flow diagram illustrating exemplary memory accesses using untranslated addresses, which may be utilized in connection with an embodiment of the invention.

FIG. 3 is a flow diagram illustrating exemplary memory accesses using translated addresses, which may be utilized in connection with an embodiment of the invention.

FIG. 4 is a flow diagram illustrating exemplary memory accesses using translated addresses, in accordance with an embodiment of the invention.

Certain embodiments of the invention may be found in a method and system for latency optimized ATS usage. Aspects of the invention may comprise communicating a memory access request using an untranslated address and also an address translation request using the same untranslated address, where the memory access request may be sent without waiting for a result of the address translation request. The memory access request and the address translation request may be made in either order.

A translation agent in an address translation system may be used to translate the untranslated address, and the translated address may be communicated to the device that made the address translation request. The translated address may also be used to make the memory access. Accordingly, by initiating the memory access request without having to wait for the communication of the translated address, or vice versa, the requesting device may reduce latency for memory accesses when using untranslated addresses. An untranslated address may also be referred to as a virtual address, and a translated address may be referred to as a physical address.

FIG. 1 is a block diagram of an exemplary system for accessing memory, in accordance with an embodiment of the invention. Referring to FIG. 1 , there is shown an electronic system 100 that may comprise communication via, for example, a PCI Express (PCIe) bus 115 . The electronic system 100 may comprise a PCI device 110 , a PCIe root complex 120 , a plurality of processors 130 . . . 132 , and a plurality of memory devices 140 . . . 142 .

The PCIe device 110 may comprise suitable logic, circuitry, and/or code that may enable issuing of PCIe read and write requests to the PCIe RC 120 via a PCIe bus 115 . In instances when the PCI device 110 supports address translation, the PCIe device 110 may comprise an address translation cache 112 that may enable storage of translated addresses.

The PCIe root complex 120 may comprise suitable logic, circuitry, and/or code that may enable processing of DMA requests, for example, from the PCIe device 110 to one or more of the plurality of memory devices 140 . . . 142 . The PCIe root complex 120 may also comprise the translation agent 122 , which may enable providing translated addresses to a device, such as, for example, the PCIe device 110 , that requests address translation. The translation agent 122 , while shown as being internal to the PCIe root complex 120 , may also be external to the PCIe root complex 120 .

The translation agent 122 may comprise suitable logic, circuitry, and/or code that may enable translation of virtual addresses to physical addresses. The address translation request may be made, for example, by the PCIe device 110 in instances when a DMA transfer may be desired. The translation agent 122 may also comprise, for example, a cache such as the address translation cache 124 for storing translated addresses.

The processors 130 . . . 132 may each have associated with it one of the memory devices 140 . . . 142 . Each of the memory devices 140 . . . 142 may comprise, for example, a volatile or non-volatile memory type. Accordingly, each processor 130 . . . 132 may control access to a corresponding one of the memory devices 140 . . . 142 .

In operation, the PCIe device 110 may need to perform a DMA transfer to one of the memory devices 140 . . . 142 . A DMA transfer may be performed using, for example, physical addresses for the memory devices 140 . . . 142 . Accordingly, the PCIe device 110 may search in the address translation cache 112 for a translation of virtual addresses, where the translated addresses may be physical addresses for the memory devices 140 . . . 142 . If the physical addresses are found in the address translation cache 112 , the PCIe device 110 may issue a DMA request to the PCIe root complex 120 with the physical addresses. The PCIe root complex 120 may then commence DMA transfer by communicating with the appropriate processor 130 . . . 132 to access data in the associated memory device 140 . . . 142 .

If there are no physical addresses that correspond to the virtual addresses, the PCIe device 110 may issue a DMA request using the virtual addresses. The PCIe root complex 120 may then translate the virtual addresses to physical addresses in order to start a DMA transfer process. The translation process may comprise, for example, communicating the virtual addresses to the translation agent 122 . The translation agent 122 may determine whether the translation is possible via entries in the address translation cache 124 . If so, the translation process may be finished. If the address translation cache 124 does not have the translation for the virtual address, the translation agent 122 may access, for example, an address translation table 141 that may be stored in one or more of the memory devices 140 . . . 142 . The resulting translated addresses may then be used by the PCIe RC 120 to perform DMA transfers as requested by the PCIe device 110 .

Although a specific PCI architecture may have been described with respect to FIG. 1 , other embodiments may also be used. For example, the address translation table 141 may be in a part of memory that may be directly accessible to the PCIe root complex 120 and/or the translation agent 122 .

FIGS. 2-4 will illustrate exemplary DMA transfer requests by the PCIe device 110 .

FIG. 2 is a flow diagram illustrating exemplary memory accesses using untranslated addresses, which may be utilized in connection with an embodiment of the invention. Referring to FIG. 2 , there is shown a memory access flow diagram 200 . The PCIe device 110 may send, for example, a memory access request 202 a to the PCIe root complex 120 . The memory access request 202 a may be made using, for example, an untranslated virtual address if a translated address is not available in the address data cache 112 .

Accordingly, the translation agent 122 in the PCIe root complex 120 may start the process of translating the virtual address from the PCIe device 110 . The translation process may comprise determining whether the translated address is available in, for example, the address translation cache 124 . If the translated address is not available in the address translation cache 124 , the translation agent 122 may access the address translation table 141 via a read request 204 a.

Accordingly, the PCIe root complex 120 may access the address translation table 141 in one or more of the memory devices 140 . . . 142 . There may be a series of memory read requests and responses 204 a . . . 204 d to and from the address translation table 141 , depending on the number of levels in the address translation table 141 . For example, if the address translation table 141 comprises N levels, the translation agent 122 may request N reads from the address translation table 141 , where each successive read request may use information from the previous read request.

After the virtual address has been translated by the translation agent 122 , the PCIe root complex 120 may make a memory access request 206 a to the appropriate memory device 140 . . . 142 using the translated address via the appropriate processor 130 . . . 132 . If the memory access request 202 a was a read request, the memory device 140 . . . 142 may respond with the requested data via the appropriate processor 130 . . . 132 in the response 206 b . The PCIe root complex 120 may then communicate the requested data to the PCIe device 110 in the response 202 b . If the memory access request 202 a was a write request, the responses 206 b and 202 b may not be needed.

In this memory access scheme described, the PCIe 120 does not communicate the translated address to the requesting PCIe device 110 . Accordingly, when using an untranslated address for memory accesses, the PCIe device 110 may incur an overhead for the translation process by the PCIe root complex 120 for each memory access.

The translation overhead may be expressed as:

Latency=( N+ 1)* T m +T p .  (1)

N may be the number of read operations needed for the translation by the translation agent 122 . T m may be the latency of read operation from the PCIe root complex 120 to the memory devices 140 . . . 142 . T p may be the round-trip time from a memory request sent by the PCIe device 110 to when a response is received from the PCIe root complex 120 . T p may be exclusive of the PCIe root complex 120 to memory devices 140 . . . 142 latency.

FIG. 3 is a flow diagram illustrating exemplary memory accesses using translated addresses, which may be utilized in connection with an embodiment of the invention. Referring to FIG. 3 , there is shown a memory access flow diagram 300 . If a translated address is not available in the address data cache 112 , the PCIe device 110 may send, for example, a translate request 302 a to the PCIe root complex 120 .

The PCIe root complex 120 may receive the translate request 302 a and communicate the translate request 302 a from the PCIe device 110 to the translation agent 122 . The translation process may first comprise determining whether the translated address is available in, for example, the address translation cache 124 . If the translated address is not available in the address translation cache 124 , the translation agent 122 may access the address translation table 141 via a read request 304 a.

Accordingly, the PCIe root complex 120 may access the address translation table 141 in one or more of the memory devices 140 . . . 142 . There may be a series of memory read requests and responses 304 a . . . 304 d to and from the address translation table 141 , depending on the number of levels in the address translation table 141 . For example, if the address translation table 141 comprises N levels, the translation agent 122 may request N reads from the address translation table 141 , where each successive read request may use information from the previous read request.

After the virtual address has been translated by the translation agent 122 , the PCIe root complex 120 may communicate the translated address to the PCIe device 110 in the response 302 b . The translated address may be stored in the address translation cache 112 by the PCIe device 110 for future use. The PCIe device 110 may also make a memory access request 306 a to the PCIe root complex 120 using the translated address. The PCIe root complex 120 may then use the translated address from the PCIe device 110 to make a memory access request 308 a to the appropriate memory device 140 . . . 142 via the appropriate processor 130 . . . 132 .

If the memory access request 306 a was a read request, the memory device 140 . . . 142 may respond with the requested data via the appropriate processor 130 . . . 132 in the response 308 b . The PCIe root complex 120 may then communicate the requested data to the PCIe device 110 in the response 306 b . If the memory access request 306 a was a write request, the responses 308 b and 306 b may not be needed.

In this instance, the requesting PCIe device 110 requested a separate translation prior to making a read request with the translated address. Accordingly, when using a translated request for an address before making a memory access, the PCIe device 110 may incur an overhead for the translation process by the PCIe root complex 120 .

The translation overhead may be expressed as:

Latency=( N+ 1)*T m +2 *T p .  (2)

N may be the number of read operations needed for the translation by the translation agent 122 . T m may be the latency of read operation from the PCIe root complex 120 to the memory devices 140 . . . 142 . T p may be the round-trip time from a memory request sent by the PCIe device 110 to when a response is received from the PCIe root complex 120 . T p may be exclusive of the PCIe root complex 120 to memory devices 140 . . . 142 latency.

Accordingly, it can be seen by comparing the latencies described by Equation (1) and Equation (2), that there is an additional latency of T p for the scenario described with respect to FIG. 3 than there is for the scenario described with respect to FIG. 2 . This is due to the need for separate translation request and memory access request with the translated address for the scenario described with respect to FIG. 3 . Hence, it can be seen that the use of address translation service prior to memory access may save PCI bus and memory bandwidth for high cache hit ratios. However, a penalty of an additional latency may be imposed when there are cache misses. Accordingly, for DMA transfers to uncached addresses, additionally latency may be present for each data transfer.

FIG. 4 is a flow diagram illustrating exemplary memory accesses using translated addresses, in accordance with an embodiment of the invention. Referring to FIG. 4 , there is shown a memory access flow diagram 400 . The PCIe device 110 may send, for example, a memory access request 402 a to the PCIe root complex 120 . The memory access request 402 a may be made using, for example, an untranslated virtual address if a translated address is not available in the address data cache 112 . After the PCIe device 110 makes the memory access request 402 a , the PCIe request may also make a translation request 404 a to the PCIe root complex 120 . The translation request 404 a may be made, for example, without waiting for a response to the memory access request 402 a.

The PCIe root complex 120 may receive an untranslated address memory access request 402 a , and may communicate the untranslated address from the PCIe device 110 to the translation agent 122 . The translation process by the translation agent 122 may comprise determining whether the translated address is available in, for example, the address translation cache 124 . If the translated address is not available in the address translation cache 124 , the translation agent 122 may access the address translation table 141 via a read request 406 a.

Additionally, the translation agent 122 may also determine whether the address that is being requested to be translated may be in the process of being translated. This may occur, for example, in instances when the memory access request 402 a using an untranslated address is received from the PCIe device 110 . The untranslated address may be in the process of being translated when translation request 404 a for the same untranslated address is received from the PCIe device 110 .

Similarly, the address translation request 404 a may be received first, and then the untranslated address memory access 402 a may be made. In either case, if the address translation cache 124 does not contain the translated address, the translation agent 122 may need to determine whether the translation required for the later received request is being made due to a previous request.

Accordingly, the PCIe root complex 120 may access the address translation table 141 in one or more of the memory devices 140 . . . 142 . There may be a series of memory read requests and responses 406 a . . . 406 d to and from the address translation table 141 , depending on the number of levels in the address translation table 141 . For example, if the address translation table 141 comprises N levels, the translation agent 122 may request N reads from the address translation table 141 , where each successive read request may use information from the previous read request.

The PCIe root complex 120 may then make a memory access request 408 a using the translated address to the appropriate memory device 140 . . . 142 via the appropriate processor 130 . . . 132 . Since the PCIe root complex 120 also received the translation request 404 a from the PCIe device 110 , the PCIe root complex 120 may communicate the translated address to the PCIe device 110 in the response 404 b after the virtual address has been translated by the translation agent 122 . The translated address may be stored in the address translation cache 112 by the PCIe device 110 for future use.

If the memory access request 402 a was a read request, the memory device 140 . . . 142 may respond with the requested data via the appropriate processor 130 . . . 132 in the response 408 b . The PCIe root complex 120 may then communicate the requested data to the PCIe device 110 in the response 402 b . If the memory access request 402 a was a write request, the responses 408 b and 402 b may not be needed.

In this scenario described with respect to FIG. 4 , the PCIe device 110 may have requested an untranslated memory read request without waiting for a response to an address translation request for the same address. Accordingly, this parallel step (using any order between the memory read request and the address translation request) may make the latency the same as the latency described by Equation 1) while still receiving a translated address for storage in the address translation cache 112 .

Accordingly, an embodiment of the invention described with respect to FIG. 4 may allow the PCIe device 110 to make a memory access to a memory device 140 . . . 142 with an untranslated address, while still receiving a translated address for future use without additional latency. Various embodiments of the invention may be used for single accesses to the memory device 140 . . . 142 , or a series of accesses such as during a DMA access.

In accordance with an embodiment of the invention, aspects of an exemplary system may comprise the PCIe device 110 that may communicate an untranslated address in a memory access request, which may be a read request or a write request. The PCIe device 110 may also communicate an address translation request for the untranslated address along with the memory access request. The memory access request and the corresponding address translation request may be made with the memory access request first or with the address translation request first.

Upon receiving the untranslated address memory access request or the address translation request for an untranslated address, the PCIe root complex may determine whether a corresponding translated address for the untranslated address may be resident in the address translation cache 122 . If so, the translated address may be sent to the PCIe device 110 . The PCIe device 110 may then receive a translated address that may correspond to the untranslated address, where the translated address may be saved, for example, in the address translation cache 112 for future use. The translated address may also be used for accessing one of the memory devices 140 . . . 142 .

Otherwise, the translation agent 122 may determine whether a translation is in progress for the untranslated address. If an address translation is not in progress for that untranslated address, then address translation process may be initiated for that untranslated address. However, if an address translation is in progress for that untranslated address, then a second address translation may not be initiated since that would duplicate the first address translation.

If the memory access was a read request, then the memory device 140 . . . 142 may return data that may correspond to the untranslated address in the memory access request from the PCIe device 110 . This data may be received by the PCIe root complex 120 , and the PCIe root complex 120 may communicate the received data to the PCIe device 110 .

Another embodiment of the invention may provide a machine-readable storage, having stored thereon, a computer program having at least one code section executable by a machine, thereby causing the machine to perform the steps as described herein for latency optimized ATS usage.

Accordingly, the present invention may be realized in hardware, software, or a combination of hardware and software. The present invention may be realized in a centralized fashion in at least one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software may be a general-purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.

The present invention may also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.

While the present invention has been described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present invention without departing from its scope. Therefore, it is intended that the present invention not be limited to the particular embodiment disclosed, but that the present invention will comprise all embodiments falling within the scope of the appended claims.