Photonic fabric chip device

Disclosed is an apparatus and method to improve the scalability of Data Center networks using mesh network topologies, switches of various radixes, tiers, and oversubscription ratios. The disclosed apparatus and method reduce the number of manual network connections, simplifying the cabling installation, and improving the flexibility and reliability of the data center at a reduced cost.

The use of optical fiber for transmitting communication signals has been rapidly growing in importance due to its high bandwidth, low attenuation, and other distinct advantages, including radiation immunity, small size, and lightweight. Datacenter architectures using optical fiber are evolving to meet the global traffic demands and the increasing number of users and applications. The rise of cloud data centers, particularly the hyperscale cloud, has significantly changed the enterprise information technology (IT) business structure, network systems, and topologies. Moreover, cloud data center requirements are impacting technology roadmaps and standardization.

The wide adoption of server virtualization and advancements in data processing and storage technologies have produced the growth of East-West traffic within the data center. Traditional three-tier switch architectures comprising Core, Aggregation, and Access (CAA) layers cannot provide the low and equalized latency channels required for East-West traffic. Moreover, since the CAA architecture utilizes spanning tree protocol to disable redundant paths and build a loop-free topology, it underutilizes the network capacity.

The Folded Clos network (FCN) or Spine-and-Leaf architecture is a better-suited topology to overcome the limitation of the three-tier CAA networks. A Clos network, a multilevel circuit switching network introduced by Charles Clos in 1953, has become very relevant today due to the use of complex optical interconnect topologies. The Folded-Clos network topology utilizes two types of switch nodes, Spine and Leaf Each Spine is connected to each Leaf. The network can scale horizontally to enable communication between a large number of servers while minimizing the latency and non-uniformity by simply adding more Spine and Leaf switches.

FIG. 1 shows an example of two FCNs with a similar number of hosts, using different radixes and levels. The higher radix, 32 in this example, connects 32 Leaf, switches in a two-layer network, as shown in part (a) of the FIGURE. The two-level FCN provides the lowest latency at the cost of requiring a denser network (512 interconnections). By using a three-layer network, the interconnection layout can become less complex or more structured. However, more switches are needed, and more latency is introduced in the network. During the last years, the need for flatter networks to address the growing traffic among machines has favored the radix increase of the switches' application-specific integrated circuits (ASICs). Currently, switch ASIC radixes can handle 256 ports at a speed of 100 Gb/s per port. ASICs with higher radixes are expected in the future.

Based on industry telecommunications infrastructure Standard TIA-942-A, the locations of leaf and spine switches can be separated by tens or hundreds of meters. Typically, Spine switches are located in the main distribution area (MDA), whereas Leaf switches are located in the equipment distribution area (EDA) or horizontal distribution area (HDA).

This architecture has been proven to deliver high-bandwidth and low latency (only two hops to reach the destination), providing low oversubscription connectivity. However, for large numbers of switches, the Spine-Leaf architecture requires a complex mesh with large numbers of fibers and connectors, which increases the cost and complexity of the installation.

Future data centers will require more flexible and adaptable networks than the traditional mesh currently implemented to accommodate highly distributed computing, machine learning (ML) training loads, high levels of virtualization, and data replication.

Traditionally the mesh fabrics such as the ones shown in FIG. 1 have been implemented over patch panels using hundreds or thousands of patch cords connections to deploy the network topology. More recently, the use of transpose boxes, as shown in the prior art and listed RSs, can help to deploy those networks while reducing installation errors. Transposed boxes implement a section of the network mesh inside a box using multiple duplex fiber connections or optical flex circuits. Optical Flex Circuits, arrays of routed fibers embedded on two layers of polymer, can provide versatile mesh deployment on a flexible substrate.

A better way to implement the meshes using planar lightwave circuits (PLC) to implement today's complex mesh topologies has been considered. Currently, most of the PLC methods involve photolithography and etching over silicon wafers. However, waveguides fabricated using photolithography limit the device fabrication to 2D configurations. Therefore, using the photolithography process, meshes needed to implement topologies similar to the ones shown in FIG. 1 will require multiple crossovers, producing high losses and crosstalk.

A more recent fabrication method is direct laser writing waveguide on glass (DLWW) using femtosecond lasers.

A femtosecond laser, which produces high-intensity pulses that last 10 s or 100 s of femtoseconds, enables efficient nonlinear absorption of several near-infrared (NIR) photons to modify the glass properties. The nonlinear absorption of NIR pulses is less damaging than UV absorption and facilitates the use of versatile optics to control the intensity spatial profile in the three dimensions inside a glass. There has been significant progress in DLWW during the last decade to reduce waveguide distortions and other defects that degrade propagation and coupling losses [4].

These most recent techniques, used for DLWW, can implement complex 3D photonic circuits (PCs), and facilitate rapid maskless prototyping. DLWW can fabricate waveguides with similar properties to single-mode fiber, SMF, which is advantageous in reducing the coupling losses. DLWW enables 3D geometries which practically eliminate the waveguide crossover and crosstalk among channels.

DLWW can be a useful technique for manufacturing 3D multichannel optical meshes. However, there are two main limitations of DLWW that require careful design of the waveguides: the propagation losses, which are higher or at best similar to the ones produced by the lithography processes mentioned previously, and bend losses caused by the relatively low index contrast of the waveguides.

Mesh DLWW small form factor designs with low loss, and negligible crosstalk to overcome the limitations mentioned above, are disclosed here.

Apparatuses having a plurality of optical duplex and parallel connector adapters, such as MPO connectors and LC adapters, where some adapters connect to network equipment in a network and others to servers or processing units such as GPUs, incorporate internal photonic circuit with a mesh. The light path of each transmitter and receivers is matched in order to provide proper optical connections from transmitting to receiving fibers, wherein complex arbitrary network topologies can be implemented.

FIG. 1 ( a ) shows a two-level FCN radix 32.

FIG. 1 ( b ) shows a three-level FCN radix 8.

FIG. 2 shows atop view of submodule 500 showing interconnection arrangements.

FIG. 3 shows twelve of twenty-four alternative meshes that can be implemented in the submodule 500 .

FIG. 4 ( a ) shows atop view of DLWW Optical Chip 700 .

FIG. 4 ( b ) shows a side view of DLWW Optical Chip 700 showing the different layers where the waveguides are written to avoid crossover.

FIG. 5 shows a top view of DLWW Optical Chip 700 .

FIG. 6 shows an exemplary of module 900 shows connection to DLWW embodiment 700 to two fiber arrays and the connection to the multifiber ports of the module.

FIG. 7 shows an exemplary of module 400 shows connection to DLWW embodiment 700 to fiber arrays and the connection to the multifiber ports of the module.

FIG. 8 illustrates a simple method for implementing networks with 16 Leaf Switches and up to 16 Spine switches, using the modules 400 .

DLWW, a relatively new technique, can enable the fabrication of optical mesh waveguides with index contrast and circular shapes closer to single-mode fiber, SMF, so that the waveguide dimensions are comparable thereby reducing the coupling losses. Moreover, 3D geometries enabled by DLWW practically eliminate the crossover of waveguides and, therefore, the optical crosstalk among channels. However, this technology has limitations, such as propagation and bending losses, tight tolerances, and requiring a careful waveguide design.

An advantage of the invention disclosed in this document is to provide small form factor meshes capable of supporting communication signals transmitted over multiple channels, with negligible crosstalk or noise penalties and limited excess losses beyond the designed split ratio.

The present invention focuses on transmitters with four and eight channels, such as 400GBASE-DR4. Several embodiments will be shown in this section. Note that all the drawings are not to scale to emphasize the main features of the devices.

FIG. 2 shows a front view of the disclosed embodiment 500 , a photonic circuit chip, which is the key element in facilitating optical network deployment, reshaping, and scaling. This embodiment is used to produce the interconnection mesh of the four groups of waveguides. As shown in this FIGURE, there are eight parallel ports, 510 , 515 , 520 , 525 , 550 , 555 , 560 , and 565 , distributed on the front and rear sides of the embodiment. Each port consists of eight waveguides labeled from a to h. The mesh of submodule 500 can be implemented in twenty-four arrangements; twelve of them are shown in FIG. 3 . The interconnection arrangements for all those configurations are described in Table I. Note that for the network deployments disclosed in this document, all the twenty-four arrangements are equivalent since they preserve the correct paths from the transmitter to receivers in the network. The one shown in the embodiment example corresponds to arrangement 632 .

FIG. 4 shows embodiment 700 , a variation of arrangement 632 where all the ports are grouped in an array of fixed separation pitches, e.g., 127 μm or 250 μm. In this embodiment, the waveguides are written at different layers inside the glass, with a separation of at least 40 μm to avoid crosstalk. Some of the ports, 515 , 520 , 525 , 550 , 555 , 560 , and 565 , are shown in the FIGURE. For example, the eight waveguides of ports 510 , 525 , 550 , and 565 are labeled in the FIGURE. From the top view of the FIGURE, it could appear that there is crossover among waveguides, e.g., waveguides 710 . However, as shown in FIG. 5 , viewed from the top with a 45-degree tilt, all adjacent waveguides maintain a minimum separation. In FIG. 5 , there might be crossovers among waveguides 720 ; however, a view of the same waveguides with a different perspective ( FIG. 4 ( a ) ) shows no crossover among them.

The preferred embodiments in this disclosure are written in glass utilizing the DLWW method. However, with some modification to the wavelength (or wavelengths) of light, the optics to focus the beam, the repetition rate and/or the power of the femtosecond laser, it is possible to write the disclosed waveguide designs directly on polymer (enabling flexible DLWW 3D optical circuits) and silicon substrates.

More advanced technology may allow the waveguide to be written utilizing multiple wavelengths or variable reputation rates as the laser applies focus through the glass—this can provide waveguides with variable width or refractive index/contrast (i.e. the difference between the refractive index of the waveguide and the surrounding glass). Changing the diametrical spot size of the laser dynamically while maintaining power density allows you to change the diameter of the waveguide, while keeping the index refraction constant or variable, depending on the application. For example, up taper or a down taper that's useful for things like mode conversion, mode matching, mode filtering, stripping out modes and even multimode waveguides.

FIG. 6 shows an example of how to use DLWW embodiment 700 or ( 500 ) in an assembly or module 800 , which is an enclosure that supports and protects the PLC 700 , have front and rear opening for optical interfaces (connectors or adaptors), and features to organize the fiber interconnections. One or more faces of the module have multiport connectors such as MPOs, MMC, or groups of SN, MDC, or alternatively, groups of duplex connectors such as LC connectors. In the FIGURE, module 900 has eight MPO ports evenly distributed on the front and back sides of the device.

The outputs of DLWW device 700 are connected to fiber arrays 830 and 850 . The fiber arrays, commonly used to couple light into and out of PLCs, consist of one or two-row arrays of optical fibers placed on a high precision v-groove chip which is covered with a lid on the top. The end face of the fiber array is optically polished, and it can be purchased with a flat or angled polished end face with or without antireflection coating (ARC). In the disclosed invention, the fibers from 830 are separated into groups, 810 , 815 , 820 , and 825 . Similarly, from the other side of the DLWW device 700 , the group of fibers from 850 is separated into groups 860 , 865 , 870 , and 875 .

The fiber groups 810 , 815 , 820 , and 825 are connected to fiber groups 910 , 915 , 920 , and 925 , respectively, using fusion or mechanical splices represented by 790 . In other designs with less restricted values for insertion loss, element 790 can also be a mated multifiber connector pair. Fibers 910 , 915 , 920 , and 925 are terminated in multifiber connectors or groups of duplex connectors placed on the front or rear face of module 800 .

Similarly, the fiber groups 850 , 855 , 860 , and 865 are connected to fiber groups 950 , 955 , 960 , and 965 , respectively, using fusion or mechanical splices or connectors represented by 790 . Fibers 950 , 955 , 960 , and 965 are terminated in multifiber connectors or groups of duplex connectors placed on the rear or front face of module 800 .

Modules with several ports and DLWW devices, the fiber groups 850 , 855 , 860 , and 865 are connected to fiber groups 950 , 955 , 960 , and 965 , respectively, using fusion or mechanical splices or connectors represented by 790 . Fibers 950 , 955 , 960 , and 965 are terminated in multifiber connectors or groups of duplex connectors placed on the rear or front face of module 800 .

Module 800 incorporates a simple mesh that can support up to four Spines and four Leaf switches. Larger meshes can be implemented using methods shown in RS25595 and DLWW device 700 . For example, a two-step mesh incorporated, by combining two sections where a mesh is applied to a group of fibers and then to individual fibers as shown in FIG. 7 and in more detail in RS25595, increases the degree of mixing of the fiber channels inside each module. This simplifies the deployment of the network since a significant part of the network complexity is moved from the structured cabling fabric to one or more modules 400 . Table II, which shows the mesh configuration of module 400 , indicates that any of the sixteen parallel ports is connected to all the other parallel ports using one duplex channel. For example, in the first column of the table it is shown that port 440 is connected to 420 a , 425 a , 430 a and 435 a . By grouping several modules 400 and following simple rules to connect a group of uplinks or downlinks horizontally or vertically the installation becomes cleaner, and cable management is highly improved. For example, FIG. 8 shows a stack of four modules 400 , each one with four photonic device 700 . Part (a) of the FIGURE shows the module side that is connected to the Leaf switches. For simplicity, we label this as the front side. Part (b) shows the opposite side of the same module 400 , the backside, which is connected to the Spine switches. It is assumed that sixteen Leaf switches, each with four MPO uplinks, need to be connected to the fabric shown in part (d) of the FIGURE. In this illustrative example, where it is assumed that sixteen Leaf switches with four uplink each need to connect to the fabric, the connection method is described as follows. Each with the uplinks of the Leaf switches are connected horizontally in groups of four until the last port of each module 400 is used. For example, 1110 and 1112 , the first and the last fourth ports of the first module 400 connect to the uplink ports of the Leaf switches L 1 and L 4 , respectively. The uplinks of the fifth Leaf switch populate the ports 1114 of the second module 400 . This method continues until the uplinks of the last Leaf switch are connected to the ports 1116 . The Spines ports are assigned at the backside of the stacked modules 400 . For example, if standalone Spine switches are used, 1120 , 1122 , and 1124 correspond to ports of the first, second, and sixteenth Spine switch, respectively, labeled as S 1 , S 2 , and S 16 .

While this invention has been described as having a preferred design, the present invention can be further modified within the spirit and scope of this disclosure. This application is therefore intended to cover any variations, uses, or adaptations of the invention using its general principles. Further, this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains and which fall within the limits of the appended claims.