Method and apparatus for scheduling packets in a packet switch

1. Field of the Invention

This invention relates to the field of scheduling packets in a crossbar device.

2. Background

A typical crossbar switch includes multiple inputs and outputs. Packets that arrive at an input port are queued until the packet can be transferred to an output port. For purposes of the invention, there are two types of packets. A unicast packet contains addressing that selects only one output port of the packet switch. A multicast packet specifies multiple output ports of the packet switch.

When a unicast packet is pending, the packet's addressing information causes a unicast-packet-request signal to be sent the output port specified by the unicast packet. If the specified output port is free (or when it becomes free), a unicast arbiter for the output port selects and connects to a selected input port. The unicast packet is then transferred from the input port to the output port. Once the packet transfer is complete, the output port rearbitrates.

When a multicast is pending, a multicast-packet-request signals are sent by a multicast arbiter to each output port specified by the multicast packet at the input port. Once all the requested outputs are ready for the packet, the packet is simultaneously transferred to each output port. A single multicast arbiter is used for all the output ports.

One problem is related to how the output port should operate when both unicast- and multicast-packet-requests are pending at the output port. In a prior art implementation, the output has two states (unicast and multicast). Prior art packet switches can toggles between these states at the end of each transfer (if both unicast and multicast requests are pending), when the packet switch becomes totally blocked by pending multicast packets at the input ports, or in response to a timer. In prior art implementations, a synchronization delay occurs when the packet switch changes state. Thus, significant bandwidth utilization is lost every time the packet switch prepares to transfer a multicast packet. In addition, for the prior art implementations that switch state when the input ports are blocked, significant bandwidth utilization is lost as the input ports become blocked. Further, packets waiting for the blocked ports can be unacceptably delayed through the packet switch.

It would be advantageous to burst multiple multicast packets to increase the bandwidth utilization of the switch by reducing the number of multicast synchronization delays while limiting the delays caused by allowing the ports to become blocked by multicast packets.

The present invention reduces the number of multicast synchronization delays in a packet switch by determining the mix of packets pending at the input ports. When a sufficient number of multicast packets are ready for transferal, the packet switch preferably transmits a programmed number of multicast packets (or as many multicast packets that exist up to that programmed number). After transmitting these multicast packets, the packet switch resumes preferably transmitting unicast packets. Thus, the number of multicast synchronization delays is reduced over the prior art, the bandwidth utilization of the packet switch is correspondingly increased and the load due to multicast packets and unicast packet is balanced.

FIG. 1 illustrates a packet switch that transfers both unicast packets and multicast packets in accordance with a preferred embodiment;

FIG. 2 illustrates a process for improving efficiency of a packet switch that transfers both unicast packets and multicast packets in accordance with a preferred embodiment;

FIG. 3A illustrates one embodiment of the ‘determine state trigger condition’ step shown in FIG. 2;

FIG. 3B illustrates a second embodiment of the ‘determine state trigger condition’ step shown in FIG. 2;

FIG. 3C illustrates another embodiment of the ‘determine state trigger condition’ step shown in FIG. 2; and

FIG. 4 illustrates a ‘multicast packet transfer’ step shown in FIG. 2 .

FIG. 1 illustrates a packet switch, indicated by general reference character 100 , configured to transfer packets through a switch fabric 101 . The switch fabric 101 is connected to a first input port 103 and other input ports including an nth input port 105 . The first input port 103 includes a first input queue 107 that can hold the data comprising one or more of the packets sent to the first input port 103 . The nth input port 105 also includes an nth input queue 109 that can hold the data comprising one or more of the packets sent to the nth input port 105 . One skilled in the art will understand that the nth input port 105 and the nth input queue 109 can be combined into one queue. The switch fabric 101 is configured to connect any of the input ports to a first output port 111 , an nth output port 113 or any other output port dependent on the data contained within the packet. The first output port 111 is associated with a first unicast arbitrator 115 and the nth output port 113 is associated with an nth unicast arbitrator 117 . The packet switch 100 also includes a multicast arbitrator 119 and a UC/MC state control 121 .

A unicast packet at an input port causes a unicast-packet-request signal to be sent to the arbitrator of the output port specified by the content of the unicast packet. The output port arbitrator acknowledges the unicast-packet-request signal responsive to the unicast/multicast state maintained by the UC/MC state control 121 and the status of the packet requests from all the input ports and the multicast arbitrator 119 .

However, a multicast packet at an input port causes the multicast arbitrator 119 to post a multicast-packet-request signal to the set of output ports specified by the multicast packet. When the unicast/multicast state is in the state of preferentially transferring unicast packets, the output port will select one of the input ports making a unicast request (if any) of the output port and send a ready reply to the selected input port instead of responding to the multicast arbitrator 119 . When the unicast/multicast state is in the state of preferentially transferring multicast packets, the output port will send a ready reply to the multicast arbitrator 119 . When all the specified set of output ports indicate ready to the multicast arbitrator 119 the multicast arbitrator conditions the switch fabric 101 for a multicast and causes the multicast packet at the input port to be transferred across the switch fabric 101 to the specified set of output ports.

The invention increases the bandwidth that can be utilized by the packet switch by reducing the overhead incurred by the packet switch when switching from the state of preferentially transferring unicast packets to the state of preferentially transferring multicast packets. The invention does this by determining when a sufficient number of multicast packets become ready for transferal at the input ports.

A multicast packet becomes blocked at the input port when one or more of the set of output ports that are specified by the multicast packet continues to be used for unicast packet transfers. These blocked multicast packets stop subsequent packets from being transferred from the blocked input port. By waiting for multiple multicast packets to become available for transferal, the overhead of switching state from preferentially transferring unicast packets to the state of preferentially transferring multicast packets is spread across the number of multicast packets that are transferred during this state. This increases the bandwidth utilization of the packet switch.

One skilled in the art will understand that if a multicast packet is at an input port, and the output ports requested by the multicast packet happen to be available, the multicast packet is transferred regardless of the preference state of the packet switch. However, as the load on the packet switch increases, this circumstance becomes less likely.

FIG. 2 illustrates a process for switching between unicast and multicast preference states, indicated by general reference character 200 , used to arbitrate between multicast packets and unicast packets. The process 200 initiates at a ‘start’ terminal 201 during the initialization of the packet switch. After the process 200 initiates, it continues to an ‘enter unicast preference state’ step 203 . The ‘enter unicast preference state’ step 203 conditions the packet switch so that an output port arbitrator preferably responds to unicast-packet-request signals from the input ports instead of the multicast-packet-request signal from the multicast arbitrator. The process 200 continues to a ‘determine state trigger condition’ step 205 (subsequently described with respect to FIG. 3A, FIG. 3B, and FIG. 3C) that determines whether the preferential state should be changed from the unicast preference state set by the ‘enter unicast preference state’ step 203 . If the ‘trigger satisfied’ decision step 207 determines that the preferential state should not be changed, the process 200 continues to the ‘determine state trigger condition’ step 205 that repeats the determination of the state trigger condition.

However, if the state trigger condition is satisfied at the ‘trigger satisfied’ decision step 207 , the process 200 continues to an ‘enter multicast preference state’ step 209 . The ‘enter multicast preference state’ step 209 changes the state of the packet switch from the unicast preference state to a multicast preference state. At this point, the output ports preferentially reply to the multicast-packet-request signal instead of to the unicast-packet-request signals from the input ports.

The process 200 continues to a ‘synchronize packet switch for multicast operation’ step 211 that waits (if required) until the selected output ports complete unicast packet transfers and become available for multicast packets. Once the packet switch completes synchronization, the process 200 continues to a ‘burst multicast transfers’ step 213 (subsequently described with respect to FIG. 4) that transfers a number of multicast packets. After the burst of multicast packets, the process 200 continues to the ‘enter unicast preference state’ step 203 that again places the packet switch into the unicast preference state. The process 200 then repeats these steps.

The ‘determine state trigger condition’ step 205 evaluates the mix of unicast and multicast packets at the input ports. Thus, when the number of multicast packets at the input ports becomes suitable for bursting multicast transfers (or if a single multicast pack becomes sufficiently delayed), the ‘determine state trigger condition’ step 205 satisfies the trigger used to switch the packet switch from the unicast preference state to the multicast preference state.

FIG. 3A illustrates a first ‘determine state trigger condition’ process, indicated by general reference character 300 , that can be used by the ‘determine state trigger condition’ step 205 . This process uses the multicast arbitrator 119 to approximate the percentage of multicast packets that are blocking input ports. The multicast arbitrator 119 can determine this by requesting access to a specified set of output ports. If a sufficient number of these selected ports acknowledge the request, one of two conditions exists at the packet switch. The first condition occurs when the packet switch is under light load and the output ports are available because no other unicast or multicast packets have requested them. The second condition occurs when the packet switch is operating under a heavier load and multicast packets are waiting at the input ports for transferal through the packet switch (blocking subsequent packets). As more input ports are blocked by multicast packets, fewer unicast packets are processed by the packet switch as only the unblocked input ports are able to pass packets. Thus, the probability increases that a particular port is free as more input ports are blocked by multicast packets. (Remember, if a multicast packet is at an input port, and the requested set of output ports is free, the multicast packet will be transferred regardless of the preference state of the packet switch.)

The first process 300 is invoked by the ‘determine state trigger condition’ step 205 of FIG. 2 at a ‘start’ terminal 301 . In this preferred embodiment, the first process 300 continues to a ‘request output ports’ step 303 that examines a multicast packet to determine the specified set of output ports for the multicast packet and sends a multicast-packet-request signal to the specified set of output ports. Each of the specified set of output ports determines whether the port is busy. If the port is busy, it does not respond with an output port ready reply; if the port is not busy, but has a pending unicast request, the port also does not respond with an output port ready reply (because the packet switch is in the unicast preference state (see the ‘determine state trigger condition’ step 205 ); otherwise, the port responds to the multicast arbiter with an output port ready reply. The multicast arbiter receives the ready replies (if any) from the specified set of output ports at a ‘receive output port ready’ step 305 . If a sufficient percentage of the specified set of output ports do not respond with an output port ready reply, the first process 300 continues to a ‘return FALSE’ step 309 that indicates trigger unsatisfied to the ‘trigger satisfied’ decision step 207 . The first process 300 then completes through an ‘end’ terminal 311 . However if a sufficient number of the specified set of output ports respond with an output port ready reply, the first process 300 continues to a ‘return TRUE’ step 313 that indicates trigger satisfied to the ‘trigger satisfied’ decision step 207 .

The ‘received ready sufficient’ decision step 307 can be based on a specified percentage of the specified set of output ports, a specified total number of responding output ports, or other value that can be used to balance the unicast packet traffic with the multicast packet traffic through the packet switch such to maximize, in heavy packet traffic situations, the utilization of the available bandwidth of the packet switch. The first process 300 works for a lightly loaded packet switch because the switch bandwidth is already underutilized. Thus, the lost bandwidth utilization due to switching to the multicast preference state is insignificant. The first process 300 works for a heavily loaded packet switch because the lost bandwidth utilization due to switching to the multicast preference state is delayed until multiple multicast packets are blocked at the input ports. Thus, this lost bandwidth utilization is amortized over the each multicast packet sent during a burst of multicast packets.

FIG. 3B illustrates a second ‘determine state trigger condition’ process, indicated by general reference character 320 , that can be used by the ‘determine state trigger condition’ step 205 . This embodiment examines the contents of the packets in the input queue(s), determines the mix of multicast and unicast packets in the queue(s), and enables a multicast burst dependent on the determined mix. The second process 320 initiates at a ‘start’ terminal 321 from the ‘determine state trigger condition’ step 205 of FIG. 2 . The second process 320 continues to an ‘examine queued packets’ step 323 that examines the contents of the packets in the input queues. A ‘determine packet mix’ step 325 evaluates the information acquired by the ‘examine queued packets’ step 323 to determine the unicast/multicast mix of the queued packets. One embodiment examines the input packets at the head of the queue to determine whether these packets are multicast or unicast (used to determine the packet mix). Another embodiment examines the contents of a queue to determine whether there are adjacent sequences of multicast packets in the queue (that is whether a queue contains a first multicast pack immediately followed by another multicast packet). One aspect of the invention is to burst multiple multicast packets. The multicast packets that make up the burst can be blocking multiple input ports, be adjacent multicast packets queued to a single input port, or be some combination of packets that allows a burst of multicast packets to be transferred from the input ports to the output ports while the packet switch is in the multicast preference state.

A ‘burst multicast’ decision step 327 determines from the mix whether the second process 320 is to satisfy the ‘trigger satisfied’ decision step 207 . If an insufficient number of multicast packets are ready for burst transmission the second process 320 continues to a ‘return FALSE’ step 329 and exits through an ‘end’ terminal 331 . However, if a sufficient number of multicast packets are ready for burst transmission the second process 320 continues to an ‘return TRUE’ step 333 and completes through the ‘end’ terminal 331

FIG. 3C illustrates a third ‘determine state trigger condition’ process, indicated by general reference character 340 , that again is initiated from the ‘determine state trigger condition’ step 205 of FIG. 2 . at a ‘start’ terminal 341 . The third process 340 continues to a ‘timer expired’ decision step 343 that detects whether the timer has expired. If the timer has not expired the third process 340 continues to a ‘return FALSE’ step 345 and completes through an ‘end’ terminal 347 . However, if the timer has expired, the third process 340 continues to a ‘determine traffic load’ step 349 . The ‘determine traffic load’ step 349 can use the previously described techniques (such as the ‘request output ports’ step 303 and the ‘receive output port ready’ step 305 ; and the ‘examine queued packets’ step 323 and the ‘determine packet mix’ step 325 ) to determine the packet traffic load value representing the packet load value through the packet switch. In addition, the ‘determine traffic load’ step 349 can also simply monitor the number of transfers that have occurred since the last timer expiration. The third process 340 continues to a ‘reset timer step’ step 351 that resets the timer based on the load determined by the ‘determine traffic load’ step 349 . The third process 340 continues to a ‘return TRUE’ step 353 and completes through the ‘end’ terminal 347 .

One skilled in the art will understand that aspects of the third process 340 can be included with the first process 300 and the second process 320 to help protect against a single multicast packet being blocked for too long a time because no other multicast packet has been received during that time. In particular, but without limitation, a timer can be used to periodically change preference mode of the packet switch if the preference mode was not changed during the running of the timer.

FIG. 4 illustrates a multicast burst process, indicated by general reference character 400 , that transfers a number of multicast packets from the input port(s) of the packet switch to the output ports. The multicast burst process 400 is initiated at a ‘start’ terminal 401 and is invoked by the ‘burst multicast transfers’ step 213 of FIG. 2 . The multicast burst process 400 continues to a ‘determine max burst size’ step 403 that loads a specified value for the maximum number of multicast packets that are to be transferred while the packet switch is in the multicast preference state. Next, a ‘loop’ step 405 repeats some of the subsequent steps for the number of multicast packets in the burst. When the loop is complete, the multicast burst process 400 completes through an ‘end’ terminal 407 .

On each iteration of the loop, a ‘transfer multicast packet’ step 409 transfers a multicast packet from the input port to the specified set of output ports. A ‘multicast packet ready’ decision step 411 then determine whether another multicast packet is ready for transfer. If no multicast packet is ready for transfer, the multicast burst process 400 completes through the ‘end’ terminal 407 . Otherwise, the multicast burst process 400 continues to the ‘loop’ step 405 to continue bursting multicast packets. Thus, the multicast burst process 400 detects when the burst of multicast packets completes either by exhaustion of ready multicast packets at the input ports, or by assuring that the burst is no more than a maximum number of multicast packets (to not unduly delay unicast packet processing).

Because the burst of multicast packets does not require the packet switch to resynchronize between each multicast packet transfer, the bandwidth of the packet switch is better utilized and the throughput of the packet switch is increased. In a preferred embodiment, a burst contains at least two and preferably four or more multicast packets.

One skilled in the art will understand that the invention improves the efficiency of a packet switch by reducing the number of multicast synchronization delays by transferring bursts of multicast packets when possible. Such a one will also understand that, although the invention has been described using variable length packets, the invention can also be applied to fixed length packets (cells).

Although the present invention has been described in terms of the presently preferred embodiments, one skilled in the art will understand that various modifications and alterations may be made without departing from the scope of the invention. Accordingly, the scope of the invention is not to be limited to the particular invention embodiments discussed herein.