Our network is designed around a pure Layer 3 network topology, where we bring a routed interface to each server. However, many environments expect a Layer 2 network. To support these use cases we’ve developed a feature that allows users to create and connect Layer 2 networks to their Equinix Metal™ infrastructure.
A few notes to help set the stage:
- Availability in all datacenters.
- Available on all server types except our t1.small.x86 and c1.small.x86. The x1.small.x86 supports hybrid mode only.
- Virtual networks are confined to a single datacenter.
- There are no fees for the use of the Layer 2 feature.
Network configuration types
When converting from Layer 3 to another network type there are 3 possible configurations to choose from: hybrid mode, bonded Layer 2, or Layer 2 with a broken network bond.
In hybrid mode only one network interface is removed from bond and placed in Layer 2 mode. VLANs can then be attached to this interface for Layer 2 connectivity. This preserves Layer 3 connectivity to the server via bond0, so it can be accessed via the public IP.
Bonded Layer 2 converts the bonded network interface to pure Layer 2 mode. This means all access to the public internet is lost, and the host can only be reached by the Serial Over SSH (SOS) console. In this configuration the network bond is intact, so only one network interface will be available for attaching VLANs.
Layer 2 with a broken bond is similar to the bonded Layer 2 configuration, except the network bond is also dismantled, thus, providing two network interfaces available for VLANs.
Note: both pure Layer 2 networking configurations will permanently remove the server's IP management IP addresses. If the server is later converted back to Layer 3, new IP addresses will be assigned.
Bonding on Equinix Metal
Before we get to the nitty gritty details of the guide, it's important to understand the networking configuration of Equinix Metal servers. By default, each server has two networking interfaces that are setup in an LACP (mode 4) bond that is configured both in the Host OS and on the switch.
Equinix Metal allows users to change the networking mode of each server from the default Layer 3 bonded mode to either hybrid or Layer 2 networking. In Hybrid mode, the first interface is left in the LACP bond but the second interface is separated from the bond so that you can attach VLANs to it. In Layer 2 mode, you can either have both interface in a bonded configuration or you can have both interfaces separated so that you can attach different VLANs to each interface.
Reverting to Layer 3 from Layer 2
Reverting to Layer 3 from Layer 2 via the portal only require changing the networking mode via the server's networking page. Our API will automatically bond the network interfaces and assign a new IP address.
In the portal server configuration screen, the switch ports serving each of your servers' NICs may be independently enabled to switch one or more of your provisioned networks.
If only one VLAN is enabled on a port, packets are untagged. This means that the server's network configuration does not need to be VLAN-aware. However when two or more VLANs are enabled on a port, then packets are tagged and therefore it will be necessary to configure the server's networking accordingly.
Layer 2 Setup in the Equinix Metal Portal
Layer 2 networking is enabled in the Equinix Metal Portal from the project's "IPs and Networks" tab.
Under "Layer 2" you can add one or more networks like this:
Note that networks are local to a specific data center and that the assigned VLAN ID displayed here will be used to configure server port switching and server network setup.
When you add a network, we automatically provision it in our data center switches; however, in order for it to be made available to individual machines additional steps are required.
- Convert the server's networking mode. This will configure the server to allow attachment of your server's network interfaces to your VLAN. You can choose a mix/hybrid pure Layer 2 (with the option to break the bond or leave it intact).
- Once the network mode has been changed you will see the option to attach a new VLAN.
- Choose the network interface you wish to attach the VLAN to, but be aware that you should only choose "bond0" if you have converted the server to the bonded layer 2 networking mode.