Multiple-node system power utilization management

The present invention relates generally to managing power utilization of a multiple-node system, and more particularly to increasing the individual power cap of a selected node while still ensuring that a system-wide power cap is not violated.

A data center can include tens, hundreds, or even thousands of computing devices that are managed at a single location. Such large numbers of computing devices can consume large amounts of power. The cost of power, however, continues to increase. Therefore, data center operators are attempting to better manage the power utilized by the computing devices of their data centers. Such attempts are also being made in response to customers requesting that the power consumed by their computing devices be better managed, as well as in response to requirements promulgated by various governmental agencies throughout the world. However, such power management may desirably not come at a cost of reduced performance of the computing devices in completing the tasks assigned to them.

The present invention relates to managing the power utilized by a multiple-node system that has a number of nodes. A method of one embodiment thus manages the power utilized by such a multiple-node system. Each node has a node power cap that specifies the maximum power that the node is individually able to utilize. The multiple-node system as a whole has a system-wide power cap that specifies the maximum power that the multiple-node system is able to utilize overall. The method determines that a node power cap of a selected node is to be increased. In response, where the total of the node power caps of all the nodes is equal to the system-wide power cap, the node power caps of one or more nodes are reduced so that the total of the node power caps of all the nodes is less than the system-wide power cap. The node power cap of the selected node is then increased, such that the total of the node power caps of all the nodes is equal to or less than the system-wide power cap.

A multiple-node system of an embodiment of the invention includes a number of nodes and a management component. Each node has a node power cap that specifies the maximum power that the node is individually able to utilize. The multiple-node system as a whole has a system-wide power cap that specifies the maximum power that the multiple-node system is able to utilize overall. The management component determines that a node power cap of a selected node is to be increased. In response, where the total of the node power caps of all the nodes is equal to the system-wide power cap, the management component reduces the node power caps of one or more nodes so that the total of the node power caps of all the nodes is less than the system-wide power cap. The management component then increases the node power cap of the selected node, such that the total of the node power caps of all the nodes is equal to or less than the system-wide power cap.

An article of manufacture includes a tangible computer-readable medium, and means in the medium. For instance, the tangible computer-readable medium may be a recordable data storage medium. The means in the medium may include one or more computer programs that can be executed by one or more processors. The means is for managing power utilization of a multiple-node system having a number of nodes. The means determines that a node power cap of a selected node is to be increased. In response, where the total of the node power caps of all the nodes is equal to the system-wide power cap, the means reduces the node power caps of one or more nodes so that the total of the node power caps of all the nodes is less than the system-wide power cap. The means then increases the node power cap of the selected node, such that the total of the node power caps of all the nodes is equal to or less than the system-wide power cap.

The drawings referenced herein form a part of the specification. Features shown in the drawing are meant as illustrative of only some embodiments of the invention, and not of all embodiments of the invention, unless otherwise explicitly indicated, and implications to the contrary are otherwise not to be made.

FIG. 1 is a diagram of a multiple-node system having a number of nodes, according to an embodiment of the invention.

FIG. 2 is a diagram depicting node power caps of the nodes of a multiple-node system and a system-wide power cap of the multiple-node system, according to an embodiment of the invention.

FIG. 3 is a flowchart of a method for managing power utilization of a multiple-node system having a number of nodes, according to an embodiment of the invention.

FIG. 4 is a diagram of a particular type of a multiple-node system, consistent with but more detailed than the system of FIG. 1 , according to an embodiment of the invention.

In the following detailed description of exemplary embodiments of the invention, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific exemplary embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized, and logical, mechanical, and other changes may be made without departing from the spirit or scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.

FIG. 1 shows a multiple-node system 100 , according to an embodiment of the present invention. The system 100 includes four nodes 102 A, 102 B, 102 C, and 102 D in the embodiment of FIG. 1 , which are collectively referred to as the nodes 102 . However, more generally, there are at least two nodes 102 , such as tens, hundreds, or even thousands of the nodes 102 . The system 100 also includes a management component 104 , which may be implemented in software, hardware, or a combination of software and hardware. The management component 104 and the nodes 102 are interconnected by an interconnection network 106 in one embodiment.

In one embodiment, each of the nodes 102 is a separate computing device, such as a server computing device, which includes one or more processors, memory, storage devices, and/or other types of components. In one embodiment, each of the nodes 102 is a blade computing device that is inserted into a shared or common chassis. In another embodiment, the nodes 102 include both computing device nodes and storage device nodes, installed within a shared or common chassis. In one embodiment, each of the nodes 102 includes one or more processors, where all the nodes 102 are implemented on a single integrated circuit (IC), such that the system 100 is a multiple-processor core system. In other embodiments, the nodes 102 are implemented in other ways as well.

The management component 104 manages the power utilization of the multiple-node system 100 as a whole as well as the power utilization of the individual nodes 102 of the system 100 . In particular, the system 100 has a system-wide power cap that specifies the maximum power that the system 100 is able to utilize overall. Furthermore, each of the nodes 102 has a node power cap that specifies the maximum power that the node is able to individually utilize. The individual node power cap of one node may be equal to or different than the individual node power caps of the other nodes. In general, the management component 104 modifies the individual node power caps so that the nodes 102 have the power that they need, while ensuring that the system-wide power cap is not violated.

FIG. 2 illustratively depicts the relationship between the system-wide power cap 202 of the multiple-node system 100 and the individual node power caps 204 of the nodes 102 of the system 100 , according to an embodiment of the invention. The system-wide power cap 202 is specified as Y in FIG. 2 . The individual node power caps 204 include node power caps 204 A, 204 B, 204 C, and 204 D, corresponding to the nodes 102 A 102 B, 102 C, and 102 D, respectively, and which are specified as X A , X B , X C , and X D , respectively, in FIG. 2 .

The system-wide power cap 202 is related to the individual node power caps 204 such that X A +X B +X C +X D ≦Y. That is, the total of all the node power caps 204 is always less than or equal to the system-wide power cap 202 , and is never greater than the system-wide power cap 202 . Furthermore, if the total of all the node power caps 204 is less than the system-wide power cap 202 (i.e., X A +X B +X C +X D <Y), then there is spare power capacity 206 that is not currently assigned to any of the nodes 102 , as is specifically depicted in FIG. 2 . By comparison, if the total of all the node power caps is equal to the system-wide power cap 202 (i.e., X A +X B +X C +X D =Y), then there is no spare power capacity 206 .

The management component 104 generally manages the individual node power caps 204 of the nodes 102 as follows, in relation to an example in which the node 102 A needs to have its node power cap 204 A increased by an amount Z, such that the desired X A equals the current X A plus Z. Where the total of all the node power caps 204 is less than the system-wide power cap 202 (i.e., the current X A +X B +X C +X D <Y), and the spare power capacity 206 is greater than or equal to Z, the management component 104 increases the node power cap 204 A by the amount Z, without changing any other of the node power caps 204 . This is permitted, because the current X A +Z+X B +X C +X D ≦Y.

However, where the total of all the node power caps 204 is equal to the system-wide power cap 202 (i.e., the current X A +X B +X C +X D =Y), or the total of all the node power caps 204 is less than the system-wide power cap 202 by less than Z (i.e., the current Y−X A +X B +X C +X D <Z), then the management component 104 cannot increase the node power cap 204 A by the amount Z without changing any other of the node power caps 204 . This is because if the node power cap 204 A is increased by Z without changing any other of the node power caps 204 , then the current X A +Z+X B +X C +X D >Y. As such, the system-wide power cap 202 is violated.

In this situation, therefore, the management component 104 decreases the node power cap for one or more of the other nodes 102 , so that it can increase the node power cap 204 A for the node 102 A. For example, in one embodiment the management component 104 decreases the node power cap 204 B, represented by X B , of the node 102 B by an amount V. The new X B equals the current X B minus V. The amount V is selected so that the current X A +Z+the current X B −V+X C +X D ≦Y. That is, the amount V by which the node power cap 204 B is decreased is selected so that the node power cap 204 A can be increased by the amount Z and such that the system-wide power cap 202 is not violated.

FIG. 3 shows a method 300 that is performed at least in part by the management component 104 to manage the power utilization of the multiple-node system 100 in relation to the system-wide power cap 202 and the individual node power caps 204 , according to an embodiment of the invention. Power-related data is constantly or periodically collected from the nodes 102 throughout performance of the method 300 ( 302 ). Such power-related data includes the node power caps 204 , and/or the currently utilized power levels of the nodes 102 . For example, the node power cap for a given node is specified as T, but this does not mean that the given node is currently using an amount of power equal to T. Rather, the given node currently uses an amount of power S≦T.

The method 300 determines at some point that the node power cap of a selected node is to be increased by a desired amount ( 304 ). In one embodiment, for example, the data collected in part 302 is analyzed to determine whether the node power caps of the nodes 102 are to be increased. Based on this analysis, the method 300 concludes that the power code of the selected node is to be increased by the desired amount. For example, the method 300 may determine that the selected node's current power utilization frequently hits up against its node power cap, such that desirably the node power cap for the selected node is increased so as not to hinder performance of the selected node.

In another embodiment, a request is received from the selected node to increase the node power cap of the selected node. For example, the selected node itself may determine that to achieve a desired performance level, its node power cap has to be increased. An application computer program currently running on the selected node makes this determination in one embodiment. Therefore, the selected node in this example sends a request that its node power cap be increased.

In another embodiment, it is determined that a workload currently being fulfilled by the selected node requires additional power to satisfy the workload's requirements. A workload is generally defined as any arbitrary grouping of one or more processes. Not all the processes currently being executed on the selected node have to be included in a workload. A workload is particularly defined in one embodiment as encompassing those processes that are associated with a particular workflow. In one embodiment, a workflow is a representation of business logic, such that the implementation of the business logic by process(es) of computer program(s) achieves realization of the business logic. A workload in this embodiment is the collection of processes that implement the business logic of a corresponding workflow.

In another embodiment, a workload is particularly defined as a collection of processes that satisfy a particular policy. One example of a policy specifies all processes associated with a given computer program or given computer programs. A second example of a policy specifies all processes associated with a particular workflow, in accordance with the definition of a workload in the previous paragraph. A process as used herein may be generally defined as an execution instance of a computer program or of a part of a computer program.

Where the total of all the node power caps 204 of the nodes 102 , plus the desired amount by which the node power cap for the selected node is to be increased, is less than or equal to the system-wide power cap 202 , then the node power cap of the selected node is increased by the desired amount ( 306 ). However, where the total of all the node power caps 204 of the nodes 102 , plus the desired amount by which the node power cap for the selected node is to be increased, is greater than the system-wide power cap 202 , then the following is performed in one embodiment ( 308 ). First, the node power caps of one or more nodes are reduced so that the total of all the node power caps 204 of the nodes 102 , plus the desired amount by which the node power cap for the selected node is to be increased, is less than or equal to the system-wide power cap 202 ( 310 ).

In one embodiment, part 310 of the method 300 is performed by selecting the one or more nodes of which the node power caps are to be reduced. In one embodiment, a given node of the multiple-node system 100 is selected for reduction in its node power cap where the given node is currently utilizing less power than the amount permitted by its node power cap. In another embodiment, a given node of the system 100 is selected for reduction in its node power cap where the given node has a lower priority than the priority of the selected node. In this embodiment, the nodes all have priorities, where some nodes have higher priorities than other nodes, indicating that the higher-priority nodes are to be given priority over the lower-priority nodes.

In another embodiment, a given node of the multiple-node system 100 is selected for reduction in its node power cap based on the power usage history of the given node. For example, the power usage history of the given node may suggest that typically the given node utilizes significantly less power than permitted by its node power cap. In another embodiment, a given node of the multiple-node system 100 is queried to determine whether it should be selected for reduction in its node power cap. In this embodiment, the given node itself determines whether it can have its node power cap reduced, rather than the management component 104 making this determination.

Once the one or more nodes have had their node power caps reduced so that the total of all the node power caps 204 of the nodes 102 , plus the desired amount by which the node power cap of the selected node is to be increased, is less than or equal to the system-wide power cap 202 , the method 300 increases the node power cap of the selected node by the desired amount ( 312 ). In this way, the system-wide power cap 202 is not violated. That is, after reducing the node power caps of the one or more nodes in part 310 and after increasing the node power cap of the selected node in part 312 , the total of all the node power caps 204 of the nodes 102 remains less than or equal to the system-wide power cap 202 .

FIG. 4 shows a specific type of multiple-node system 400 , according to an embodiment of the present invention. The system 400 is more detailed than but consistent with the multiple-node system 100 of FIG. 1 that has been described. The system 400 includes a chassis 402 , within which four blade computing devices 404 A, 404 B, 404 C, and 404 D, collectively referred to as the blade computing devices 404 , are installed. The blade component devices 404 correspond to the nodes 102 of FIG. 1 , and there are at least two such blade component devices 404 in general. The system 400 further includes an advanced management module (AMM) 406 , which corresponds to the management component 104 of FIG. 1 in one embodiment.

The blade computing device 404 A is depicted in exemplary detail as representative of all the blade computing devices 404 . The blade computing device 404 A includes a blade management controller (BMC) 408 , one or more processors 410 , and an operating system 412 executed by the processors 410 . The blade computing device 404 A in one embodiment includes other components, in addition to those depicted in FIG. 4 , such as hardware, including memory, network interface cards, storage devices, and/or other types of hardware, and/or such as software, including application computer programs running on the operating system 412 and executed by the processors 410 .

In one embodiment, the BMC 408 collects the power-related data from the blade computing device 404 A and transmits this data to the AMM 406 to implement part 302 of the method 300 of FIG. 3 . In one embodiment, the processors 410 each have what is referred to as a processor state, or pstate, limit, which corresponds to the node power cap of the blade computing device 404 A. The pstate limit specifies the pstate, or processor state, at which the processors 410 can operate. Thus, the greater the pstate at which the processors 410 operate, the greater the performance of the processors 410 , but the greater the power consumed by the blade computing device 404 A.

In one embodiment, a processor governor implemented within the operating system 412 determines that the processors 410 have to exceed the pstate limit, and thus the node power cap of the blade computing device 402 , as specified by the BMC 408 , and conveys this information to the AMM 406 to implement part 304 of the method 300 of FIG. 3 . In another embodiment, an application computer program running in relation to the operating system 412 determines that its response, performance, and/or service goals cannot be achieved unless the pstate of the processors 410 is increased past the pstate limit. The application computer program in this embodiment conveys this information to the AMM 306 in one embodiment, to implement part 304 of the method 300 .

In one embodiment, resource management functionality implemented by the BMC 408 and/or by the AMM 406 determines that a given workload requires more processor power to satisfy the constraints imposed by the workload, to implement part 304 of the method 300 of FIG. 3 . The AMM 406 thus reduces the node power cap of a selected blade computing device in parts 306 and 308 of the method 300 . The AMM 406 in one embodiment performs the functionality that has been described in relation to parts 306 , 310 , and 312 of the method 300 , for instance. For example, the AMM 406 in one embodiment queries the BMC 408 of the blade computing device 404 A to determine whether the blade computing device 404 A is able to operate at a lower node power cap. The BMC 408 in one embodiment in turn queries the processors 410 and/or the operating system 412 to make this determination.

In the multiple-node system 400 , therefore, in one embodiment the AMM 406 communicates with the blade computing devices 404 via their respective BMC's, such as the BMC 408 of the blade computing device 402 A. However, in another embodiment, the operating system 412 , the processor governor implemented within the operating system 412 , the processors 410 , and/or the application programs running on the operating system 412 directly communicate with the AMM 406 , without traversing the BMC 408 of the blade computing device 404 A. In general, then, the embodiment of FIG. 4 provides a specific architecture by which the power management functionality of a multiple-node system, as has been described, is achieved.

It is finally noted that, although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. For example, rather than the management component 104 being implemented as the AMM 406 in the embodiment of FIG. 4 , the management component 104 is also able to be a separate management component and not the same component as the AMM 406 . The management component 104 in such instance is an independent entity that manages the overall power budget for just the blade computing devices 404 within the chassis 402 , or for a number of devices or nodes including the devices 404 as well as other devices.

As such, in one embodiment, the management component 104 is implemented in a distributed and/or hierarchical manner. A collection of nodes that are to have their power usage managed is divided into separate groups of nodes. There is a management super-component that manages the power used by each group of nodes in total, and a management sub-component for each group of nodes that manages the power used by each node within the group of nodes in question.

For example, consider a situation in which there are twenty nodes divided into four groups of five nodes for each group. The management super-component in this case manages the power used by each group of five nodes, but does not specify how much power each node within each group is to use. Rather, for a given group of five nodes, a separate management sub-component manages the power used by each node of this group, where the total power used by all the nodes of the group is provided by the management super-component.

For example, the management super-component may specify that a given group of nodes can use a total amount of power P, but does not specify the amount of power that each node within the group can use. Rather, this is specified by the management sub-component for the group, which may specify power amounts P x for each node x=1 . . . 5. In this case, P 1 +P 2 +P 3 +P 4 +P 5 is less than or equal to P. In accordance with the foregoing, this patent application is thus intended to cover any adaptations or variations of embodiments of the present invention, such as the adaptation and variations described in this paragraph and the preceding three paragraphs. Therefore, it is manifestly intended that this invention be limited only by the claims and equivalents thereof.