Cooling system including redundant fan controllers

1. Field of the Invention

This invention relates to computer system cooling and, more particularly, to fan cooling systems having redundant fan controllers and redundant fans.

2. Description of the Related Art

Computer systems are typically available in a range of configurations which may afford a user varying degrees of reliability, availability and serviceability (RAS). In some systems, reliability may be paramount. Thus, a reliable system may include features designed to prevent failures. In other systems, availability may be important and so systems may be designed to have significant fail-over capabilities in the event of a failure. Either of these types of systems may include built-in redundancies of critical components. In addition, systems may be designed with serviceability in mind. Such systems may allow fast system recovery during system failures due to component accessibility. In critical systems, such as high-end servers and some multiple processor and distributed processing systems, a combination of the above features may produce the desired RAS level.

Many systems provide redundant backup components. In a typical redundant cooling fan system, a redundant backup fan may be powered off and if the primary fan fails, the backup fan may be powered up to take over the cooling load. However in these types of systems, a primary fan controller and or a system controller must assume that the backup fan and associated control circuitry will be operational when needed.

Various embodiments of a cooling system including redundant fan controllers are disclosed. In one embodiment, the cooling system includes a first fan controller coupled to control a first plurality of fans and a second fan controller coupled to control a second plurality of fans. During operation, the first plurality of fans and the second plurality of fans operate concurrently. The first fan controller and the second fan controller are each configured to monitor the first and the second plurality of fans and to detect a failure in any of the first and the second plurality of fans. Further, in response to detecting a failure of a fan of the first plurality of fans, the second fan controller is configured to increase an operating speed of a corresponding fan of the second plurality of fans.

FIG. 1 is a perspective view illustration of a computer system.

FIG. 2 is a front view illustration of one embodiment of the computer system of FIG. 1 .

FIG. 3 is a perspective view illustration of one embodiment of a fan tray.

FIG. 4 is a block diagram of one embodiment of a fan cooling system.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.

The illustrations shown in FIG. 1 and FIG. 2 depict one embodiment of a computer system including a fan cooling system. The fan cooling system includes several fan trays each having two layers of fans. The fans in each layer may be independently controlled by a respective fan controller.

Turning now to FIG. 1 , a perspective view illustration of one embodiment of a computer system is shown. Computer system 10 includes a plurality of system boards 20 housed by a system cabinet 30 . System cabinet 30 may include removable panels as well as a frame structure (not shown) for holding system boards 20 and other components. System boards 20 may include processor boards, input/output (I/O) boards, memory boards and system controller boards. Computer system 10 also includes various power supplies 70 , a set of upper fan trays 60 A and a set of lower fan trays 60 B.

In the illustrated embodiment, there are four fan trays 60 A above and four fan trays 60 B below system boards 20 . Each fan tray may include 12 fans. The twelve fans may be arranged in two layers of six fans each. The two layers are physically placed on top of each other such that each fan in a given layer is paired with a corresponding fan of the second layer. It is noted that although each set of fan trays 60 A-B includes four fan trays as shown. It is noted that in alternative embodiments, other numbers of fan trays may be used and each fan tray may include other numbers of fans.

As will described in greater detail below in conjunction with the descriptions of FIG. 2 - FIG. 4 , during operation, the air flows through both fans of a fan pair. The fans may have three speeds and may normally run at low speed. If the temperature of any sensed components in the system rises, all fans may be set to a high speed. Both fans of a fan pair may be running at the same time. If a single fan fails, the redundant fan in the corresponding layer of a given fan tray switches to super high-speed. This fan arrangement may provide N+1 fan redundancy, thereby allowing the system to continue to run with a failed fan or fans. In addition, if a fan controller fails, the remaining fan controller may continue to operate with its fans set to super high-speed.

Referring to FIG. 2 , a front view illustration of the computer system of FIG. 1 is shown. Components corresponding to those shown in FIG. 1 are numbered identically for simplicity and clarity. FIG. 2 further illustrates the fan tray arrangement and the airflow through computer system 10 . In the illustrated embodiment, lower fan trays 60 B pull cool air in through air inlets 280 A and 280 B in the sides of system cabinet 30 and also up through air inlet 280 C in the bottom of system cabinet 30 and through power supplies 70 . Further, lower fan trays 60 B may force air up and through system boards 20 . Upper fan trays 60 A may pull the air through system boards 20 and exhaust the warmed air out through the top of system cabinet 30 .

Turning to FIG. 3 , a perspective view of one embodiment of a fan tray arrangement of a computer system is shown. As described above, each of fan trays 360 A and 360 B may include twelve fans arranged in two layers each having six fans. Fan trays 360 A and 360 B maybe coupled to computer system 10 via fan backplane 361 . In one embodiment, fan backplane 361 is an interface to computer system 10 and may provide a feed-through for power and control signals to fan trays 360 A-B. It is noted that in alternative embodiments, fan backplane 361 may be implemented as two independent backplane units. Each of fan trays 360 A-B may be installed and removed from computer system 10 as a unit by sliding the tray in and out of system cabinet 30 . In one embodiment, each of the fan trays may be hot-swapped while the system is running. As used herein, the term “hot-swapping” refers to removing and replacing a component while the system continues to operate. It is noted that in one embodiment, fan trays 360 A-B may be installed in a computer system such as the computer system of FIG. 2 and FIG. 3 .

Referring to FIG. 4 , a block diagram of one embodiment of a fan cooling system is shown. Fan cooling system 400 is an exemplary fan cooling system such as the fan cooling system shown in computer system 10 of FIG. 1 and FIG. 2 , for example. Fan cooling system 400 includes a fan controller 450 A coupled to six fans designated 440 A through 440 F. Fan controller 450 A is coupled to control fans 450 A-F via a fan interconnect 465 A. Fan cooling system 400 also includes a fan controller 450 B coupled to six fans designated 440 G through 440 L via a fan interconnect 465 B. In one embodiment, fan interconnect 465 A-B may be implemented using cables and connectors. In alternative embodiments, fan interconnect 465 A-B may be implemented using circuit boards, connectors and cables. It is noted that although each of fan controllers 450 A-B is connected to and controls six fans, other embodiments are contemplated in which other numbers of fans may be coupled to and controlled by each of fan controllers 450 A-B. Fan controller 450 A and fan controller 450 B may be coupled to one or more system controller boards (not shown) via a communication link 451 A and 451 B, respectively. In one embodiment, communication links 451 A and 451 B may each be an I2C link, for example. In one embodiment, fan controllers 450 A-B may be mounted within fan tray 460 such as the fan trays illustrated in FIG. 1 - FIG. 3 , for example.

In the illustrated embodiment, fans 440 A-L are included in a fan tray 460 such as the fan trays illustrated in FIG. 1 - FIG. 3 , for example. However, it is contemplated that the fan trays may be any suitable enclosure for housing fans. Fans 440 A- 440 F are shown as layer 1 of fan tray 460 , while fans 440 G- 440 L are shown as layer 2 of fan tray 460 . As shown in FIG. 1 , one of the fan layers is physically placed on top of the other fan layer creating six serial two-fan pairs. Accordingly, during operation, the airflow of a fan pair passes through both fans and both fans are in operation. For example, fans 440 A and 440 G may form fan pair A and fan 440 B and 440 H may form fan pair B, and so on.

Fan controller 450 A is configured to primarily control fans 440 A-F and fan controller 450 B is configured to primarily control fans 440 G-L. In addition, each of fan controllers 450 A and 450 B may monitor the operation of fans 440 G-L. In one embodiment, each of fans 440 A-L may be configured to provide a fan fail signal indicative of the fan's operational state. For example, the fan fail signal may be a logic level signal such as a logic one or a high level to indicate that the fan is operational, while a logic zero or low level may indicate that the fan is not operational. Although it is contemplated that in other embodiments, the logic levels may be reversed to indicate a fan failure. Further, each of fans 440 A-L may be configured to provide a fan speed signal corresponding to the fan's speed. For example, in one embodiment, the speed signal may be indicative of the fan's revolutions per minute (RPM). In another embodiment, the speed signal may be a digital output and include several digitally encoded bits which correspond to the fan's speed. Accordingly, fan cooling system 400 may provide a redundant cooling system with no single point of failure.

In one embodiment, a failing fan may be a fan with a fan fail signal indicating a failure or a failing fan may be a fan that did not reach a setpoint speed. Accordingly, each of fan controllers 450 A-B may cause a given fan to operate at a particular speed (e.g., low, high and super-high speed) and may also detect whether the given fan actually attained the setpoint speed by detecting a difference between the setpoint speed and actual speed.

In one embodiment, fan controller 450 A and fan controller 450 B may each include fail-over logic 453 A and 453 B, respectively. Fan controller 450 A and fan controller 450 B may each also include an integrated circuit (not shown) such as a microcontroller, for example, which may serve as an interface to a system controller (not shown), for control and diagnostic functions. In another embodiment, fan controller 450 A and fan controller 450 B may each include other control logic (not shown) including status and control registers which may be used to control and monitor fans 440 G-L. In addition, in one embodiment, each of fan controllers 450 A-B may include a power converter unit (not shown) which may be configured to receive power from the computer system power supplies and to provide power to the fan controller board components.

In one embodiment, each of fans 440 A-L may be configured to operate at three speeds: low speed, high speed, and super high-speed. It is contemplated that in other embodiments, the fans may include other numbers of fan speeds. During operation, fan cooling system 400 may be configured to have both sets of six fans (i.e., all twelve fans) running at high speed to provide cooling air. This may be in contrast to a fan cooling system which may have backup fans that are powered off and may be free-wheeling. In one embodiment, a system controller (not shown) of computer system 10 may monitor all system functions, including the operational state of fan cooling system 400 .

In one embodiment, the system controller (not shown) may communicate with the one fan controller which it considers as the master via the appropriate communication link 451 A or 451 B. Thus, one fan controller is assumed to be the master fan controller and the second fan controller is a slave.

The operation of fan cooling system 400 may be illustrated by way of an example. Assume that fan controller 450 A is the master. As described above, fans 440 G-L are all operating at high speed to provide cooling air. Fan controller 450 A is actively controlling fans 440 A-F and fan controller 450 B is actively controlling fans 440 G-L. In addition, fan controller 450 A is monitoring the operation of fans 440 A-L. In one embodiment, fan controller 450 A may monitor the fan speed and the fan fail signals of each of fans 440 A-L and may store information corresponding to the fan operation within status registers. Similarly, fan controller 450 B may monitor the fan speed and the fan fail signals of each of fans 440 A-L and may store information corresponding to the fan operation within status registers.

Assume, for example, that fan 440 E fails. Fan controller 450 A and fan controller 450 B may both detect the failure. In response to detecting the failure, fan controller 450 B may cause the second fan (e.g., fan 440 K) of fan pair E to switch to super high-speed and fan controller 450 A may power down fan 440 E. In addition, in one embodiment, fan controller 450 A may notify the system controller (not shown) of the failure. In an alternative embodiment, system software may cause the system controller to poll the status registers of the master fan controller (e.g., fan controller 450 A) to detect failures. Further, in one embodiment, the system controller may notify maintenance personnel of the failure.

Each of fan controllers 450 A and 450 B may be interconnected and configured to detect a failure of the other fan controller. If a fan controller fails, the remaining fan controller may detect the failure and increase the speed of the fans it controls to super high-speed while the fans controlled by the failing fan controller may be powered down. For example, a power converter (not shown) on a given fan controller may fail causing the entire fan controller to malfunction. In one embodiment, each of fan controllers 450 A and 450 B may monitor the power converter voltage of the other fan controller. Thus, if the master fan controller (e.g., fan controller 450 A) power converter fails, it may be detected by the slave fan controller (e.g., fan controller 450 A) and in response to detecting the failure, the slave controller may increase the speed of all the fans it controls to super high-speed. If the slave fan controller power converter fails, the master fan controller may detect the failure and increase the speed of all the fans it controls to super high-speed. In another embodiment, a fan controller failure may be detected by the other fan controller since each fan fail signal of the failing controller may indicate a failure. In still another embodiment, each of fan controllers 450 A and 450 B may provide a heartbeat signal which may be monitored by each fan controller and by the system controller. The heartbeat signal may be indicative of the operational status of each fan controller.

As described above, if the master fan controller fails, the slave controller may increase the speed of all the fans it controls to super high-speed. In addition, fail-over logic 453 A-B may automatically enable the slave fan controller to become the master controller. Further, in one embodiment, in response to detecting the failure, the system controller may isolate the failed master fan controller and begin communicating with the slave fan controller, which is now the master fan controller. Alternatively, if the slave fan controller fails first, the master fan controller may detect the failure as described above and may also notify the system controller of the failure as described above.

In addition to increasing fan speed in response to a fan failure, fan controller 450 A and 450 B may also increase fan speed in response to commands received from the system controller. For example, the system controller may monitor the temperature within system cabinet 30 of FIG. 1 and FIG. 2 and may detect when the temperature rises beyond an acceptable limit. In response to a rise in temperature, the system controller may cause fan controllers 450 A and 450 B to increase the speed of fans 440 A-L to high speed until the temperature is within an acceptable range.

As noted above, fan controllers 450 A-B may include a microcontroller (not shown) which may be used to facilitate diagnostic functions. In one embodiment, the system controller may test the functionality of fan controllers 450 A-B by sending fan control commands to each of fan controllers 450 A-B and then monitoring whether the fan controllers responded correctly. For example, the system controller may send a command to fan controller 450 A to set fans 440 A-F to low, high and super high-speed. After each speed setting, the system controller may poll fan controller 450 A to ensure that each fan was actually able to attain the correct speed. Further, the system controller may write a value to the microcontroller of fan controller 450 A which may force a fan fail signal to indicate a fan fail condition. The system controller may then monitor the response of fan controller 450 B to ensure that the corresponding fan of the fan pair including the failed fan has been set to super high-speed. Similarly, the system controller may perform diagnostic functions on fan controller 450 B to check operation of fans 440 G-L.

Although the embodiments above have been described in considerable detail, numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.