About the author: Zheng Minxian, worked at Novo Information Systems (Shanghai) Co., Ltd. as a pre-sales engineer. Engaged in the promotion of open network products such as SDN and white box switches.
There are many benefits to upgrading the server interface from 10G to 25G, such as:
● Ethernet performance is 2.5 times that of 10G interfaces;
● Can provide higher port density for the server;
● Lower power consumption per GB of data transmission (more power saving);
……
Although these benefits are tempting, there are still several areas to note:
First, to ensure understanding of the meaning of 25G/100G
The transmission rate of the Ethernet interface has grown substantially linearly over the past few decades (10 times every 10 years). The wired network card of everyone’s desktop or laptop is basically a Gigabit network card. In many current data centers, the server interface is 10 Gbps (10 Mbps), and the interface between switches is 40 Gbps (40 Mbps). Most of them are considering (or are) upgrading the network to the next generation. 25G/100G network (the interface of the server is 25Gbps, and the interface between switches is 100Gbps).
A 100 Gbps connection can be broken down into four channels (25 Gbps per channel) by a Break-out Cable, so a 100G network is sometimes referred to as a 25G/100G network. Similarly, 10G networks are sometimes referred to as 10G/40G networks.
10G/40G network:
The link between the Spine switch and the Leaf (TOR) switch is 40 Gbps.
The server interface is 10Gbps.
Different Leaf switches provide 10G interfaces and port densities may not be the same.
1) Take a Leaf switch as an example. The port specification of the switch is 48 10 Gbps + 640 Gbps. The 10 G interface provided by the Leaf switch is the native 10 G interface of the Leaf switch:
2) Take a Leaf switch as an example. The port specification of the switch is 32 40 Gbps. At this time, the 10G interface provided by the Leaf switch is connected to the original 40G interface through one-four-four Break-out cable (for 6 ports). In the scenario of connecting the Spine switch, it can provide (32-6) 4=104 10G ports):
25G/100G network:
● The link between the Spine switch and the Leaf switch is 100 Gbps.
● The server interface is 25Gbps
The way in which different Leaf switches provide 25G interfaces and port density may be different:
1) Take a Leaf switch as an example. The port specification of the switch is: 48 25 Gbps + 6 100 Gbps. The 25 G interface provided by the Leaf switch is the native 25 G interface of the Leaf switch:
2) Take a Leaf switch as an example. The port specification of the switch is 32100 Gbps. At this time, the 25G interface provided by the Leaf switch is separated from the original 100G interface by one minute and four Break-out cables (for 6 ports). In the scenario of connecting the Spine switch, it can provide (32-6) 4=104 25G ports):
2. Based on the scale of business expansion, consider whether you need to upgrade
For some of the leading companies, upgrading from the current 10/40G network to 25/100G is the fastest way. After all, it can take several years from planning, new construction to a data center on the line. (It takes too long to mean that you have the risk of being caught up by your competitors). The world’s major public cloud giants are basically American companies, with special attention to some leaders in order to maintain (even to expand their lead), even if the 25G/100G equipment, optical modules, etc. are much more expensive than today, they still A few days ago, the existing data center network was gradually upgraded from 10/40G to 25/100G (of course, they also built some data centers in the same period).
▲ Forecast of the number of optical modules shipped to Amazon, Google and Microsoft
Looking at the shipments of the entire optical module market, we will find that each new optical module has not been widely used for a year or two. Think about it too, after all, the price that was just introduced was very expensive. In the past few years, domestic 25G/100G applications have been later than the major OTTs in the United States, and domestic 25G/100G are mostly used in new data centers. Nowadays (2018), many 10G data centers in China have a network bottleneck or IT equipment design life. The price of 25G/100G optical modules is much lower than the previous two years. Switching to 25G is a good choice. + performance improvement).
▲ Forecast of shipments in the entire optical module market
Third, not every server needs to be upgraded
If a 10G network in a part of your data center server (or for some time in the future) has no bottlenecks, upgrading your network to 25G will not result in performance gains. In this case, these servers can continue to use the 10G network, and then upgrade when the network is about to become a bottleneck. .
Fourth, try not to purchase a 40Gbps network card
In the past few years, 40G server NIC shipments have not been large. The price of the 25G and 40G server NICs of an e-commerce platform is shown below (excluding the optical module), and you can see that the prices are very close.
However, 40G network cards usually do not make full use of the server PCIe bus. Because many servers have PCIe 3.0 x 8 (Gen3 x8) slots, they provide 56Gbps of bandwidth. Plugging a NIC with two 25G ports into a 56Gbps PCIe slot allows for more efficient network-to-server performance (as opposed to a NIC plugged into a 40G port).
Original Article Source from http://blog.itpub.net