Broadcast processing method for network system and network system

This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2006-68895, filed on Mar. 14, 2006, the entire contents of which are incorporated herein by reference.

1. Field of the Invention

The present invention relates to a broadcast processing method for performing broadcast from one node to a plurality of other nodes in a network system in which a plurality of nodes are connected via a network, and to the network system, and more particularly to a broadcast processing method for a network system for efficiently broadcasting using a plurality of network adapters in each node, and to the network system.

2. Description of the Related Art

As higher speeds for computer systems are demanded, a network type computer system, in which a plurality of nodes including computers is installed and is connected via the network, has been proposed. In the field of parallel computers, for example, data is computed in parallel by a plurality of nodes, and the processed data is exchanged via the network. Such parallel computers have several hundred-several thousand nodes if the scale becomes large.

In such a network system, the data of one node is transferred to a plurality of other nodes via the network. This is called “broadcast processing”. In this broadcast processing, mass data is sometimes transferred, and decreasing the transfer processing time is desired.

FIG. 11 and FIG. 12 are diagrams depicting a broadcast processing of a first conventional network system. As FIG. 11 shows, a plurality of (four in this case) of nodes 100 , 101 , 102 and 103 are connected via a network, which is not illustrated. Each of these nodes 100 , 101 , 102 and 103 has a plurality (three in this case) of network adapters 110 A, 110 B and 110 C to enable parallel transfers.

If one node 100 broadcasts data to the other three nodes 101 , 102 and 103 in this configuration, all transfer data is transferred from the network adapters 110 A, 110 B and 110 C of the node 100 to the corresponding network adapters 110 A, 110 B and 110 C of each node 101 , 102 and 103 in parallel.

For example, as shown in FIG. 12 , if the transfer data of 12 blocks D 0 to D 11 is transferred from the node 100 to the other three nodes 101 , 102 and 103 , the 12 blocks D 0 to D 11 are transferred from the network adapter 110 A of the node 100 to the network adapter 110 A of the node 101 , from the network adapter 110 B of the node 100 to the network adapter 110 B of the node 102 , and from the network adapter 110 C of the node 100 to the network adapter 110 C of the node 103 respectively.

In this way, according to the first prior art, the time required for broadcast processing is decreased by parallel transfer for each node 100 , 101 , 102 and 103 to have a plurality of transfer channels (network adapters).

FIG. 13 and FIG. 14 are diagrams depicting a broadcast processing of a second conventional network system. According to this method, when data is broadcasted among each node having a plurality of network adapters, just like FIG. 11 , a transmission source node 100 divides the transmission data and transfers it from each network adapter.

In other words, as FIG. 13 and FIG. 14 show, the transmission source node 100 divides the transmission data blocks D 0 to D 11 into three, and the divided blocks are transferred from the three network adapters 110 A, 110 B and 110 C of the node 100 to the network adapters 110 A, 110 B and 110 C of the node 102 , as shown in sign ( 1 ).

Since there are two nodes which hold the transmission data, the transmission node 100 transfers each of the data blocks divided into three from the three network adapters 110 A, 110 B and 110 C of the node 100 to the network adapters 110 A, 110 B and 110 C of the node 101 , as shown in sign ( 2 ). At the same time, the node 102 transfers each of the divided data blocks received from the node 100 from the three network adapters 110 A, 110 B and 110 C of the node 102 to the network adapters 110 A, 110 B and 110 C of the node 104 , as shown in sign ( 3 ).

If the transfer data length is long, the broadcast processing time is normally in proportion to the data volume that propagates the network, so a processing time to transfer 12 blocks is required for the first prior art, but in the second prior art, the data blocks are divided into three and transferred twice, so a transfer time for four blocks is required twice, in other words, a processing time becomes only 8/12=⅔ of the processing time of the first prior art (e.g. Japanese Patent Application Laid-Open No. H7-244648).

In the case of the second prior art, however, a copy of the transfer data of the broadcast source node 100 is held by the node 102 in the first transfer, and a full transfer data is transferred from two nodes at the transfer source and copy destination, that is nodes 100 and 102 , to two nodes 101 and 103 in the second transfer for broadcasting the data.

Therefore in the second prior art, the broadcast processing time is limited, and in particular when the data transfer length is long, it is difficult to decrease the broadcast processing time.

With the foregoing in view, it is an object of the present invention to provide a broadcast processing method for a network system, for decreasing the broadcast processing time, and the network system.

It is another object of the present invention to provide a broadcast processing method for a network system, for improving the performance of parallel calculation by decreasing the broadcast processing time, and the network system.

It is still another object of the present invention to provide a broadcast processing method for a network system, for decreasing the broadcast processing time even if the transfer data length is long, and the network system.

To achieve these objects, the network system of the present invention is a network system having three or more nodes each of which has a plurality of network adapters and a cross bar switch for connecting the network adapters of each of the nodes. And one of the nodes divides data to be transferred to the plurality of other nodes into a plurality of data, and transfers each of the divided data to the plurality of other nodes via a different network adapter respectively, and each of the plurality of other nodes which received the divided data transfers the received divided data to other nodes which do not receive the divided data via the network adapters.

The broadcasting processing method of the present invention is a broadcast processing method for a network system for broadcasting data from one node, having a plurality of network adapters, to a plurality of other nodes each of which has a plurality of network adapters, having a step of the one node dividing data to be transferred to the plurality of other nodes into a plurality of data, a first step of transferring each of the divided data from the one node to the plurality of other nodes via a different network adapter respectively, and a second step of transferring the received divided data from each of the plurality of other nodes which received the divided data to other nodes which do not receive the divided data via the network adapters.

In the present invention, it is preferable that each of the plurality of other nodes receives the divided data from the one node and transfers the previously received divided data to other nodes which do not receive the divided data via the network adapters.

Also in the present invention, it is preferable that the one node divides the data to be transferred by a divisor according to the number of network adapters to create the divided data.

Also in the present invention, it is preferable that the one node transfers the divided data to the plurality of nodes of which number is the same as the number of network adapters.

Also in the present invention, it is preferable that the one node divides the data by a divisor according to the number of the network adapters and the number of times of transfer, to create the divided data.

Also in the present invention, it is preferable that each of the plurality of other nodes transfer the received divided data to other nodes which do not receive the divided data via a network adapter which is predetermined among the plurality of nodes so as to prevent the overlap use of the network adapters.

Also in the present invention, it is preferable that the number of the plurality of other nodes is three or more.

Also in the present invention, it is preferable that each of the network adapters is connected with a cross bar switch via a network.

Also in the present invention, it is preferable that each of the nodes has at least a CPU, a memory and the plurality of network adapters.

Also in the present invention, it is preferable that each of the plurality of nodes has a parallel computer system for executing parallel calculation.

According to the present invention, the node at the transfer source divides the transfer data to be broadcasted, and the transfer source node transfers each of the divided data separately to a plurality of other nodes, then a node which received the divided data transfers the received data to nodes other than the transfer source node. Therefore more nodes (network adapters) can participate in data transfer in the second data transfer, high-speed transfer processing can be implemented, and the transfer processing time during broadcast can be decreased.

FIG. 1 is a block diagram depicting a network system according to an embodiment of the present invention;

FIG. 2 is a block diagram depicting the node in FIG. 1 ;

FIG. 3 is a block diagram depicting the network adapter in FIG. 1 and FIG. 2 ;

FIG. 4 is a diagram depicting the format of the transfer frame in FIG. 1 ;

FIG. 5 is a diagram depicting the broadcast processing according to the first embodiment of the present invention;

FIG. 6 is a diagram depicting the data flow according to the first embodiment in FIG. 5 ;

FIG. 7 is a flow chart depicting the processing of the transfer source node according to the first embodiment in FIG. 5 ;

FIG. 8 is a flow chart depicting the processing of the transfer destination node according to the first embodiment in FIG. 5 ;

FIG. 9 is a diagram depicting the broadcast processing according to the second embodiment of the present invention;

FIG. 10 is a diagram depicting the broadcast processing according to the third embodiment of the present invention;

FIG. 11 is a diagram depicting the broadcast processing according to the first prior art;

FIG. 12 is a diagram depicting the data flow in FIG. 11 ;

FIG. 13 is a diagram depicting the broadcast processing according to the second prior art; and

FIG. 14 is a diagram depicting the data flow in FIG. 13 .

Embodiments of the present invention will now be described in the sequence of network system configuration, first embodiment, second embodiment, third embodiment and other embodiments, but the present invention is not limited to these embodiments.

Network System Configuration

FIG. 1 is a block diagram depicting an embodiment of the network system of the present invention, FIG. 2 is a block diagram depicting the node in FIG. 1 , FIG. 3 is a block diagram depicting the network adapter in FIG. 1 and FIG. 2 , and FIG. 4 is a diagram depicting a frame format of the network system in FIG. 1 .

As FIG. 1 shows, the network system has a plurality (four in this case) of nodes 10 , 11 , 12 and 13 and three cross bar switches (SWA, SWB and SWC in figures) 20 , 21 and 22 . Each node 10 , 11 , 12 and 13 has three network adapters (A, B and C in figures) 14 A, 14 B and 14 C.

Since each node 10 , 11 , 12 and 13 has three network adapters 14 A, 14 B and 14 C, three cross bar switches 20 , 21 and 22 are installed. In other words, each of the network adapters 14 A, 14 B and 14 C of each node 10 , 11 , 12 and 13 is connected to a corresponding cross bar switch 20 , 21 and 22 .

As FIG. 2 shows, this node 10 ( 11 , 12 , 13 ) is a computer where a CPU 40 , memory 44 , IO adapter 46 and the above mentioned network adapters 14 A to 14 C are connected via a system controller 42 . The number of CPUs 40 , memories 44 and IO adapters 46 may be more than one according to the processing capability required for the node.

As FIG. 3 shows, the network adapter 14 A ( 14 B, 14 C) in FIG. 1 and FIG. 2 is comprised of a host interface control circuit 50 which is connected to a system controller 42 , a transmission control circuit 52 , a network interface control circuit 54 which is connected to the cross bar switches 20 , 21 , and 22 , and a receive control circuit 56 . The network adapter 14 A ( 14 B, 14 C) is in-charge of the data communication between nodes.

When data is transferred between nodes via the network adapter 14 A ( 14 B, 14 C), communication is performed in the frame format shown in FIG. 4 . The frame format shown in FIG. 4 is a frame format to be used for Ethernet™, and is comprised of a destination address, transmission source address, frame type (e.g. command type, data size), data, and frame checksum (e.g. CRC (Cyclic Redundancy Code)). The data length (size) of the data area is variable, and data to be transferred between nodes is divided into a plurality of frames, if necessary, and transferred.

First Embodiment

FIG. 5 is a diagram depicting the first embodiment of the broadcast processing of the present invention, FIG. 6 is a diagram depicting the data flow of the broadcast transfer in FIG. 5 , FIG. 7 is a flow chart depicting the processing of the data transfer source node in FIG. 5 , and FIG. 8 is a flow chart depicting the processing of the data receive source node.

As FIG. 5 shows, the node 10 divides the transmission block and transfers the divided transmission blocks to each node 11 , 12 and 13 via separate network adapters 14 A, 14 B and 14 C, as shown in sign ( 1 ). Then each node 11 , 12 and 13 , which received the data, transfers the received divided data to a plurality of nodes which do not receive the divided data, as shown in sign ( 2 ).

This data transfer will be described with reference to FIG. 6 , using an example of transferring the same data transfer volume in FIG. 13 and FIG. 14 . In other words, an example when the node 10 broadcasts 12 blocks of data D 0 to D 11 to the other three nodes 11 , 12 and 13 will be described. First node 10 divided the 12 blocks of data D 0 to D 11 into four. The 3 blocks of data D 0 to D 2 , D 3 to D 5 and D 6 to D 8 are transferred from each network adapter 14 A, 14 B and 14 C of the node 10 to the corresponding network adapters 14 A, 14 B and 14 C of the nodes 11 , 12 and 13 respectively.

The node 11 which received three blocks of data D 0 to D 2 transfers the received three blocks of data D 0 to D 2 from the two network adapters 14 A and 14 B to the network adapters 14 A and 14 B of the nodes 12 and 13 which do not receive the data D 0 to D 2 .

In the same way, the node 12 which received the three blocks of data D 3 to D 5 transfers the received three blocks of data D 3 to D 5 from the two network adapters 14 A and 14 C to the network adapters 14 A and 14 C of the nodes 11 and 13 which do not receive the data D 3 to D 5 .

The node 13 which received the three blocks of data D 6 to D 8 transfers the received three blocks of data D 6 to D 8 from the two network adapters 14 B and 14 C to the network adapters 14 B and 14 C of the nodes 11 and 12 which do not receive the data D 6 to D 8 .

Along with this, the node 10 transfers the remaining three blocks of data D 9 to D 11 , which is not transferred in the first transfer, from the three network adapters 14 A, 14 B and 14 C to the network adapters 14 A, 14 B and 14 C of the nodes 11 , 12 and 13 .

In this way, the transfer source node 10 divides the transfer data to be broadcasted, and transfers each divided data separately from the node 10 to the nodes 11 , 12 and 13 , then the node 10 transfers the remaining data to each node 11 , 12 and 13 . Along with this, the nodes 11 , 12 and 13 transfer the received data to the nodes 11 , 12 and 13 other than node 10 .

In this way, according to the present invention, more nodes (network adapters) participate in the data transfer in the second data transfer, so high-speed transfer processing can be implemented, and the transfer processing time during broadcast can be decreased.

In other words, according to the above mentioned second prior art, all the data is transferred from the transfer source node 10 to one other node 12 in the first transfer, and in the second transfer, all the data is transferred from the two nodes 10 and 12 , which have all the data, to the other two nodes 11 and 13 , so six network adapters participate in the transfer operation in the second transfer.

Whereas in the case of the present embodiment, data divided into four is transferred from the transfer source node 10 to the other three nodes 11 , 12 and 13 in the first transfer, and in the second transfer, necessary data is transferred from all the nodes 10 , 11 , 12 and 13 to the other two nodes 11 , 12 and 13 , so nine network adapters participate in the transfer operation in the second transfer. If the total data volume to be transferred is the same, data transfer completes more quickly as more network adapters transfer data.

In other words, data is divided in the first data transfer, and the divided data is transferred separately to a plurality of nodes so that more nodes can participate in the second data transfer. And each node transfers the received divided data to nodes which do not receive the data.

For example, comparing the transfer time with respect to the total transfer volume of the entire network between the first and second prior arts and the present embodiment, the first prior art is total transfer volume×1=1, the second prior art is total transfer volume÷3×2=⅔, and the present embodiment is total transfer volume+4×2=½. Therefore in the present embodiment, broadcast transfer can be performed in ½ the time of the first prior art, and at ¾ the time of the second prior art.

In the present embodiment, in the second transfer, the transfer relationship of the transmission network adapter and the receive network adapter is selected so as to prevent an overlap of the network adapters. For example, in the second transfer, when the node 10 performs data transfer with the network adapter 14 A of the node 11 , the network adapter 14 B of the node 12 , and the network adapter 14 C of the node 13 , then the node 11 performs data transfer with the network adapter 14 A of the node 12 and the network adapter 14 B of the node 13 , and the node 12 performs data transfer with the network adapter 14 C of the node 11 and the network adapter 14 A of the node 13 , and the node 13 performs data transfer with the network adapter 14 B of the node 11 and the network adapter 14 C of the node 12 .

By this, an overlap of network adapters can be prevented.

FIG. 7 is a flow chart depicting the processing of the transfer source node.

(S 10 ) The CPU 40 of the transfer source node 10 divides the broadcast data block into a plurality of transfer blocks. If the divisor is the number of transfer destination nodes or the number of network adapters, the above mentioned effect of the second transfer can be exhibited at the maximum level.

(S 12 ) Then the CPU 40 of the transfer source node 10 instructs the network adapters 14 A, 14 B and 14 C to transfer the divided transfer blocks to separate nodes 11 , 12 and 13 . By this, the network adapters 14 A, 14 B and 14 C of the node 10 transfer the divided transfer blocks to the network adapter 14 A of the node 11 , the network adapter 14 B of the node 12 , and the network adapter 14 C of the node 13 .

FIG. 8 is a flow chart depicting the processing of the receive source node.

(S 20 ) When a broadcast block is received, the receive source nodes 11 , 12 and 13 recognize the broadcast data by the frame type (see FIG. 4 ) of the frame.

(S 22 ) The receive source nodes 11 , 12 and 13 check the destination address and the transmission source address (see FIG. 4 ) of the frame, and decides the transfer destinations other than this.

(S 24 ) Each receive source node 11 , 12 and 13 transfers the received transfer block to the transfer destination which is the node where the transfer block should be transferred. At this time, a network adapter, predetermined in each node, is used to prevent an overlap use of the network adapters.

In this way, the broadcast transfer time can be easily decreased by the CPU 40 of each node 10 , 11 , 12 and 13 , dividing data and performing transfer processing.

Second Embodiment

FIG. 9 is a diagram depicting the broadcast processing according to the second embodiment of the present invention.

In this embodiment, broadcast processing is executed at a higher speed by overlapping the two times of data transfer in the first embodiment. FIG. 9 is an example when the node 10 broadcasts 10 blocks of data D 0 to D 9 to the other three nodes 11 , 12 and 13 .

First, the node 10 divides the data to be broadcasted into 10, D 0 to D 9 . Each one block of data D 0 , D 3 and D 6 is transferred from each network adapter 14 A, 14 B and 14 C; of the node 10 to the corresponding network adapter 14 A, 14 B and 14 C of the nodes 11 , 12 and 13 .

In the second transfer, each one block of data D 1 , D 4 and D 7 is transferred from each network adapter 14 A, 14 B and 14 C of the node 10 to the corresponding network adapter 14 A, 14 B and 14 C of the nodes 11 , 12 and 13 .

At the same time, the node 11 which received one block of data D 0 transfers the received one block of data D 0 from the two network adapters 14 A and 14 B to the network adapters 14 A and 14 B of the nodes 12 and 13 which do not receive the data D 0 .

In the same way, the node 12 which received one block of data D 3 transfers the received one block of data D 3 from the two network adapters 14 A and 14 C to the network adapters 14 A and 14 C of the nodes 13 and 11 which do not receive the data D 3 .

Also the node 13 which received one block of data D 6 transfers the received one block of data D 6 from the two network adapters 14 B and 14 C to the network adapters 14 B and 14 C of the nodes 11 and 12 which do not receive the data D 6 .

In the third data transfer, each one block of data D 2 , D 5 and D 8 is transferred from each network adapter 14 A, 14 B and 14 C of the node 10 to the corresponding network adapter 14 A, 14 B and 14 C of the nodes 11 , 12 and 13 .

At the same time, the node 11 which received one block of data D 1 in the second transfer transfers the received one block of data D 1 from the two network adapters 14 A and 14 B to the network adapters 14 A and 14 B of the nodes 12 and 13 which do not receive the data D 1 .

In the same way, the node 12 which received one block of data D 4 transfers the received one block of data D 4 from the two network adapters 14 A and 14 C to the network adapters 14 A and 14 C of the nodes 13 and 11 which do not receive the data D 4 .

Also the node 13 which received one block of data D 7 transfers the received one block of data D 7 from the two network adapters 14 B and 14 C to the network adapters 14 B and 14 C of the nodes 11 and 12 which do not receive the data D 7 .

In the fourth transfer, one block of data D 9 is transferred from each network adapter 14 A, 14 B and 14 C of the node 10 to the corresponding network adapter 14 A, 14 B and 14 C of the nodes 11 , 12 and 13 .

At the same time, the node 11 which received one block of data D 2 in the third transfer transfers the received one block of data D 2 from the two network adapters 14 A and 14 B to the network adapters 14 A and 14 B of the nodes 12 and 13 which do not receive the data D 2 .

In the same way, the node 12 which received one block of data D 5 transfers the received one block of data D 5 from the two network adapters 14 A and 14 C to the network adapters 14 A and 14 C of the nodes 13 and 11 which do not receive the data D 5 .

Also the node 13 which received one block of data D 8 transfers the received one block of data D 8 from the two network adapters 14 B and 14 C to the network adapters 14 B and 14 C of the nodes 11 and 12 which do not receive the data D 8 .

In this way, the transfer source node 10 further divides the transfer data to be broadcasted, and each divided data is separately transferred from the node 10 to the nodes 11 , 12 and 13 , then the node 10 transfers the remaining data after division to each node 11 , 12 and 13 , and the nodes 11 , 12 and 13 transfer the received data to nodes 11 , 12 and 13 other than node 10 .

In this example, the transfer data is sub-divided, so four times of transfer is required, but whereas the transfer time in the first embodiment, in which the number of times of transfer is two, is total transfer volume÷4×2= 2/4, it is total transfer volume÷10×4= 4/10 in the second embodiment. Therefore compared with the first embodiment, the time required for broadcast in the second embodiment is about ⅘.

Third Embodiment

FIG. 10 is a diagram depicting the broadcast processing according to the third embodiment of the present invention. In this embodiment, data is broadcasted from the node 0 to nodes 1 to 15 , in a network system comprised of 16 nodes 0 to 15 , each of which has three network adapters.

First, in the same manner as the first embodiment which four nodes 0 , 4 , 8 , 12 are target nodes, the node 0 divides the 12 blocks of data D 0 to D 11 into four. Each network adapter 14 A, 14 B and 14 C of the node 0 transfers each three blocks of data D 0 to D 2 , D 3 to D 5 and D 6 to D 8 to the corresponding network adapter 14 A, 14 B and 14 C of the nodes 4 , 8 and 12 .

Then the second transfer is started, where the node 4 which received the three blocks of data D 0 to D 2 transfers the received three blocks of data D 0 to D 2 from the two network adapters 14 A and 14 B to the network adapters 14 A and 14 B of the nodes 8 and 12 which do not receive the data D 0 to D 2 .

In the same way, the node 8 which received three blocks of data D 3 to D 5 transfers the received three blocks of data D 3 to D 5 from the two network adapters 14 A and 14 C to the network adapters 14 A and 14 C of the nodes 4 and 12 which do not receive the data D 3 to D 5 .

Also the node 12 which received three blocks of data D 6 to D 8 transfers the received three blocks of data D 6 to D 8 from the two network adapters 14 B and 14 C to the network adapters 14 B and 14 C of the nodes 4 and 8 which do not receive the data D 6 to D 8 .

At the same time, the node 0 transfers the remaining three blocks of data D 9 to D 11 , which is not transferred in the first transfer, from the three network adapters 14 A, 14 B and 14 C to the network adapters 14 A, 14 B and 14 C of the nodes 4 , 8 and 12 .

By this, as described in the first embodiment, four nodes 0 , 4 , 8 and 12 hold 12 blocks of data. In the third transfer, the node 0 performs the data transfer operation for nodes 1 to 3 , the node 4 for nodes 5 to 7 , the node 8 for the nodes 9 to 11 and the node 12 for the nodes 13 to 15 , in the same way as the first data transfer.

In the fourth data transfer, a transfer operation, the same as the second data transfer, is performed as nodes 0 to 3 , 4 to 7 , 8 to 11 and 12 to 15 as groups.

In this way, the maximum number of nodes that can be broadcasted in one transfer is the same as the number of network adapters, and if the number of nodes further increases, the second data transfer is performed in two steps, three steps and the like, so the number of nodes that can be broadcasted can be increased. In the case of a network system with 256 nodes, 8 times of data transfer is performed.

Other Embodiments

In the above embodiments, a network system with four nodes was used for description, but the present invention can be applied to a network system having three or more nodes. The configuration of the node described above was a computer unit having a CPU, memory, etc, but other configuration may be used.

Also the network system was described as a parallel computer, but the present invention can also be applied to other distributed processing systems and communication systems. The format of a transmission path is not limited to Ethernet™, but other network protocols can be applied.

The transfer source node divides the transfer data to be broadcasted, and transfers each divided data separately from the transfer source node to a plurality of other nodes, then the plurality of other nodes which received the transfer data transfers the received data to the nodes other than the transfer source node. In this way, more nodes (network adapters) can participate in data transfer in the second data transfer, so high-speed transfer processing can be implemented, and transfer processing time for broadcast can be decreased.