How Ethernet Works: From Frames to Switches
Ethernet is the dominant Layer 2 networking technology — the protocol that physically connects devices within a local network. Every data center, enterprise campus, and home network relies on Ethernet to move frames between machines. When your computer sends a packet to a web server, that packet's first hop almost always travels inside an Ethernet frame. Before it reaches a router, gets encapsulated in IP, and traverses the internet via BGP, it must first cross the local Ethernet segment.
Ethernet was invented at Xerox PARC in 1973 by Robert Metcalfe, originally running at 2.94 Mbps over shared coaxial cable. Today it operates at speeds up to 400 Gbps and beyond over fiber optics and structured copper cabling. The protocol has been revised many times, but its core concepts — MAC addressing, framing, and broadcast domains — remain the foundation of virtually all local area networking.
Understanding Ethernet is essential for understanding how IP packets actually traverse physical networks, how switches build forwarding tables, how VLANs segment traffic, and why protocols like Spanning Tree exist to prevent loops in switched networks.
MAC Addresses: Layer 2 Identity
Every Ethernet device has a MAC address (Media Access Control address) — a 48-bit identifier burned into the network interface hardware at the factory. MAC addresses are written as six pairs of hexadecimal digits, separated by colons or hyphens:
00:1A:2B:3C:4D:5E
or
00-1A-2B-3C-4D-5E
The 48-bit address space gives 248 = 281,474,976,710,656 possible addresses. The address is split into two halves:
- OUI (Organizationally Unique Identifier) — The first 24 bits (3 bytes) identify the manufacturer. The IEEE assigns OUI blocks to vendors. For example,
00:1A:2Bmight belong to Cisco, while3C:22:FBbelongs to Apple. You can look up an OUI to identify the device manufacturer. - Device Identifier — The last 24 bits (3 bytes) are assigned by the manufacturer to uniquely identify each NIC they produce. Together with the OUI, this should produce a globally unique address.
Two special bits in the first byte of a MAC address carry important meaning:
- Least significant bit (bit 0) — The multicast bit. If set to 1, the address is a multicast or broadcast address (e.g.,
FF:FF:FF:FF:FF:FFis the broadcast address, where all bits are 1). If 0, it is a unicast address for a single device. - Second least significant bit (bit 1) — The locally administered bit. If set to 1, the address was assigned locally (overriding the factory address). Virtual machines, Docker containers, and software-defined networks commonly use locally administered addresses.
Unlike IP addresses, which are hierarchical and routable, MAC addresses are flat — there is no inherent structure that tells a switch how to reach a given MAC. Switches learn MAC locations dynamically by observing traffic, as we will see below. MAC addresses operate exclusively at Layer 2 and are never routed across Layer 3 boundaries.
The Ethernet Frame Format
Ethernet transmits data in discrete units called frames. Each frame wraps the higher-layer payload (typically an IP packet) with the addressing and error-checking information needed to deliver it across a single Ethernet link. The standard Ethernet II (DIX) frame format, which is used for the vast majority of traffic on modern networks, has the following structure:
Each field serves a specific purpose:
Preamble and SFD (8 bytes total)
The preamble is a 7-byte sequence of alternating 1s and 0s (10101010...) that allows the receiving NIC to synchronize its clock with the transmitter. The Start Frame Delimiter (SFD) is a single byte (10101011) that signals the actual frame data is about to begin. These 8 bytes exist at the physical layer and are stripped by the NIC before the frame reaches the operating system — network capture tools like Wireshark never show them.
Destination MAC Address (6 bytes)
The MAC address of the intended recipient. This can be a unicast address (a single device), a multicast address (a group of devices), or the broadcast address (FF:FF:FF:FF:FF:FF, meaning all devices on the local segment). Switches use this field to decide which port to forward the frame out of.
Source MAC Address (6 bytes)
The MAC address of the sending device. This is always a unicast address — you cannot send from a multicast or broadcast address. Switches use this field to learn which port a particular MAC address is reachable through, populating their MAC address table.
EtherType (2 bytes)
A 16-bit field that identifies the protocol of the encapsulated payload. The most common values are 0x0800 for IPv4, 0x86DD for IPv6, and 0x0806 for ARP. If the value is 1500 or less (0x05DC), it is interpreted as a length field instead (IEEE 802.3 framing), but in practice almost all modern Ethernet uses Ethernet II framing with EtherType values above 0x0600.
Payload (46-1500 bytes)
The actual data being carried — typically an IP packet. The minimum payload size is 46 bytes; shorter payloads are padded to meet the 64-byte minimum frame size requirement (which exists to ensure collision detection works correctly on shared media, a legacy of the original CSMA/CD design). The maximum payload of 1500 bytes is the standard Ethernet MTU (Maximum Transmission Unit). This is why IP packets over standard Ethernet are limited to 1500 bytes.
Frame Check Sequence (4 bytes)
A CRC-32 checksum computed over the destination MAC, source MAC, EtherType, and payload. The receiver recomputes the CRC and silently drops the frame if it does not match — there is no retransmission mechanism at Layer 2 (that is left to higher layers like TCP). This means Ethernet provides error detection but not error correction.
From Shared Media to Switching
Original Ethernet (10BASE5, 10BASE2) was a shared medium — all devices connected to a single coaxial cable and shared the available bandwidth. Only one device could transmit at a time. If two devices transmitted simultaneously, a collision occurred, corrupting both frames. Ethernet used CSMA/CD (Carrier Sense Multiple Access with Collision Detection) to manage access: a device would listen for silence on the wire, transmit its frame, and listen for collisions. If a collision was detected, both stations would stop, wait a random backoff period, and retry.
This shared design became a bottleneck as networks grew. The introduction of the Ethernet hub in the 1990s replaced the coaxial bus with twisted-pair star wiring (10BASE-T), but hubs were still shared media — a hub simply repeats every frame it receives out every other port, creating a single collision domain. A 10 Mbps hub with 24 ports meant all 24 devices shared 10 Mbps of bandwidth.
The Ethernet switch changed everything. Unlike a hub, a switch operates at Layer 2 and makes forwarding decisions based on MAC addresses. A switch learns which MAC addresses are reachable on which ports, and then forwards frames only to the port where the destination is located. This creates a dedicated collision domain per port — each device gets the full link bandwidth rather than sharing it. A 24-port gigabit switch can sustain 24 simultaneous full-speed conversations, each at 1 Gbps. With full-duplex links (transmit and receive on separate wire pairs), collisions are eliminated entirely. CSMA/CD is disabled on full-duplex Ethernet links.
MAC Address Table Learning
The key intelligence in an Ethernet switch is its MAC address table (also called a CAM table — Content-Addressable Memory). This table maps MAC addresses to switch ports. The switch builds and maintains this table dynamically through a simple learning process:
- Frame arrives on a port. The switch examines the source MAC address field.
- Learn the source. The switch creates (or refreshes) an entry mapping that source MAC address to the port the frame arrived on, along with a timestamp. Typical aging time is 300 seconds (5 minutes).
- Look up the destination. The switch checks the destination MAC address against its table.
- Forward, filter, or flood:
- Known unicast — If the destination MAC is in the table, the switch forwards the frame only out the associated port. If the destination port is the same port the frame arrived on, the frame is filtered (dropped) — the source and destination are on the same segment and do not need the switch.
- Unknown unicast — If the destination MAC is not in the table, the switch floods the frame out every port except the one it arrived on. When the destination replies, the switch will learn its location.
- Broadcast — Frames addressed to
FF:FF:FF:FF:FF:FFare always flooded out all ports except the ingress port. ARP requests are the most common example. - Multicast — By default, multicast frames are flooded like broadcasts. IGMP snooping can optimize this by tracking which ports have multicast listeners.
This self-learning mechanism means switches require zero configuration to work — you plug devices in and they communicate immediately. The MAC address table is volatile; entries age out after the timeout period if no frames from that MAC are seen. Table sizes vary from 8,192 entries on small desktop switches to over 100,000 on data center switches. When the table fills up, the switch can either drop the oldest entries or begin flooding unknown unicasts — a condition that creates performance problems and is actually exploited in MAC flooding attacks (where an attacker floods the switch with frames from random source MACs, filling the table and forcing the switch to flood all traffic like a hub).
Broadcast Domains and Collision Domains
Two concepts are fundamental to understanding Ethernet network segmentation:
A collision domain is the set of devices that can interfere with each other's transmissions. On a hub, all ports share one collision domain. On a switch, each port is its own collision domain (and with full-duplex, collisions are impossible). Every switch port creates a new collision domain boundary.
A broadcast domain is the set of devices that receive each other's broadcast frames. A switch, by itself, is one giant broadcast domain — a broadcast frame sent by any device reaches every other device on the switch. Broadcast domains are bounded by routers (Layer 3 devices) or by VLANs. A router does not forward Ethernet broadcasts from one interface to another.
This distinction matters enormously for network design. As a broadcast domain grows, so does the broadcast traffic every device must process. ARP requests, DHCP discovers, NetBIOS announcements, and other broadcast protocols all consume bandwidth and CPU on every device in the domain. Large flat Layer 2 networks (thousands of devices in a single broadcast domain) become noisy, unpredictable, and fragile. This is the primary motivation for VLANs and for keeping Layer 2 domains small.
VLANs and 802.1Q Tagging
A VLAN (Virtual Local Area Network) is a logical partition of a physical switch into multiple independent broadcast domains. Devices in VLAN 10 cannot communicate at Layer 2 with devices in VLAN 20 — even if they are plugged into the same physical switch. Traffic between VLANs must pass through a router (or a Layer 3 switch performing routing), just as traffic between physically separate networks would.
VLANs are implemented using the IEEE 802.1Q standard, which inserts a 4-byte tag into the Ethernet frame between the source MAC address and the EtherType field:
Standard frame: [Dst MAC][Src MAC][EtherType][Payload][FCS]
802.1Q frame: [Dst MAC][Src MAC][0x8100][TCI][EtherType][Payload][FCS]
^^^^^ ^^^
TPID Tag Control Info
The 4-byte 802.1Q tag consists of:
- TPID (Tag Protocol Identifier) — 2 bytes, always
0x8100. This value in the EtherType position tells the receiver "this is a tagged frame — the real EtherType comes after the tag." - TCI (Tag Control Information) — 2 bytes, containing:
- PCP (Priority Code Point) — 3 bits for QoS priority (0-7), used by 802.1p for traffic prioritization. Priority 6 and 7 are typically reserved for network control traffic.
- DEI (Drop Eligible Indicator) — 1 bit indicating the frame may be dropped under congestion.
- VLAN ID (VID) — 12 bits giving a VLAN identifier from 0 to 4095. VID 0 means "no VLAN" (priority-only tagging), and VID 4095 is reserved. This gives 4,094 usable VLANs — a limitation that has driven technologies like VXLAN in large data centers.
The 802.1Q tag increases the maximum frame size from 1518 to 1522 bytes. Double-tagged frames (Q-in-Q, used by service providers for VLAN stacking via 802.1ad) add another 4 bytes.
Switch ports are configured as either access ports or trunk ports:
- Access port — Assigned to a single VLAN. Frames entering are internally tagged with that VLAN's ID. Frames leaving have the tag stripped. End devices never see VLAN tags.
- Trunk port — Carries frames from multiple VLANs, with 802.1Q tags intact. Trunk ports connect switches to each other and to routers. A trunk has a "native VLAN" whose frames are sent untagged (commonly VLAN 1 by default, though best practice is to change it).
VLANs provide security isolation (devices in different VLANs cannot eavesdrop on each other's traffic), reduce broadcast domain size, and allow flexible network segmentation independent of physical topology. A server in VLAN 100 on switch A and a server in VLAN 100 on switch B are in the same broadcast domain, connected via a trunk between the switches.
Spanning Tree Protocol (STP)
Redundancy is critical in network design — a single switch failure should not bring down the entire network. The natural solution is to connect switches with multiple links, creating physical loops for failover. But Ethernet has a fatal problem with loops: broadcast storms.
When a broadcast frame enters a loop, both switches forward it to each other, each forwarding the other's copy, creating an exponentially multiplying flood that saturates all links within seconds and brings the network to a halt. Unlike IP (which has a TTL field that decrements at each hop), Ethernet frames have no hop count — they can circulate forever.
Spanning Tree Protocol (STP), defined in IEEE 802.1D, solves this by electing a root bridge and then logically blocking redundant paths to create a loop-free tree topology. The algorithm works as follows:
- Root bridge election. All switches exchange BPDUs (Bridge Protocol Data Units) containing their Bridge ID (a priority value + MAC address). The switch with the lowest Bridge ID becomes the root bridge. Administrators can influence this by setting bridge priority (default is 32768).
- Root port selection. Every non-root switch determines its root port — the port with the lowest cost path to the root bridge. Path cost is based on link speed (10 Mbps = cost 100, 100 Mbps = 19, 1 Gbps = 4, 10 Gbps = 2).
- Designated port selection. For each network segment (link between switches), the switch with the lowest root path cost becomes the designated bridge for that segment, and its port toward that segment is the designated port.
- Blocking. All ports that are neither root ports nor designated ports are placed into blocking state. They receive BPDUs but do not forward data frames, eliminating the loop.
The result is a spanning tree — a loop-free logical topology overlaid on the physical topology. If an active link fails, STP reconverges: blocked ports transition to forwarding state to restore connectivity. However, classic STP convergence is notoriously slow — it takes 30 to 50 seconds because ports must transition through listening and learning states before forwarding, during which time the network segment is down.
Rapid Spanning Tree (RSTP)
RSTP (IEEE 802.1w) dramatically improves convergence time to 1-3 seconds by introducing proposal/agreement handshakes between directly connected switches, eliminating the need for timer-based state transitions. RSTP also redefines port roles — adding alternate ports (backup root ports) and backup ports (backup designated ports) — so failover is nearly instantaneous because the replacement path is already identified.
MSTP (IEEE 802.1s, Multiple Spanning Tree Protocol) extends RSTP to support multiple spanning tree instances, each mapped to a group of VLANs. This allows different VLANs to use different active paths, improving link utilization. Without MSTP, all VLANs are forced to use the same spanning tree topology, leaving blocked redundant links completely idle.
In modern data center designs, Spanning Tree is increasingly replaced by other approaches: Multi-Chassis Link Aggregation (MLAG/VPC) allows two physical switches to appear as one, eliminating loops without blocking ports. TRILL and SPB (Shortest Path Bridging) use IS-IS routing at Layer 2. And in overlay networks like VXLAN-EVPN, the underlay is a pure Layer 3 routed fabric where STP is not needed at all.
L2 Switching vs. L3 Routing
The difference between Layer 2 switching and Layer 3 routing is fundamental to network architecture:
Layer 2 switching operates on MAC addresses. A switch examines the destination MAC, looks it up in its MAC address table, and forwards the frame out the appropriate port. Switching is fast — it can operate at line rate using specialized hardware (ASICs). But switching is limited to the local broadcast domain. Switches do not understand IP addressing, subnets, or routing protocols. They cannot reach devices outside their own Layer 2 domain.
A Layer 3 router operates on IP addresses. It receives a frame, strips the Ethernet header, examines the IP destination address, performs a routing table lookup, decrements the TTL, selects the next-hop IP, resolves the next-hop's MAC address (via ARP), builds a new Ethernet frame with new source and destination MACs, and sends it out the appropriate interface. Every Layer 3 hop completely replaces the Ethernet header. The source and destination MAC addresses in the frame at each hop are different — they represent the "this link" endpoints, not the original sender and final receiver.
This is a critical insight: MAC addresses are link-local. When your computer sends a packet to a remote web server, the Ethernet frame's destination MAC is your default gateway's (router's) MAC address, not the server's MAC. The router receives the frame, routes the IP packet, and builds a new frame with its own MAC as the source and the next hop's MAC as the destination. The end-to-end IP addresses stay the same; the link-by-link MAC addresses change at every router hop.
Modern Layer 3 switches (sometimes called multilayer switches) combine both functions in one device. They can switch traffic at wire speed within a VLAN (Layer 2) and route traffic between VLANs (Layer 3) using hardware routing tables. This is called inter-VLAN routing and eliminates the need for a separate router in most campus network designs. Data center leaf-spine architectures typically use Layer 3 switches at every tier, with all inter-switch links routed — creating a pure Layer 3 fabric where BGP or OSPF manages reachability.
ARP: Bridging L2 and L3
The Address Resolution Protocol (ARP) bridges the gap between IP addresses and MAC addresses. When a device needs to send an IP packet to another device on the same subnet, it needs the destination's MAC address to build the Ethernet frame. If it does not already have a mapping, it broadcasts an ARP request:
ARP Request (broadcast):
"Who has 192.168.1.50? Tell 192.168.1.10"
Src MAC: AA:BB:CC:DD:EE:01 Dst MAC: FF:FF:FF:FF:FF:FF
ARP Reply (unicast):
"192.168.1.50 is at AA:BB:CC:DD:EE:02"
Src MAC: AA:BB:CC:DD:EE:02 Dst MAC: AA:BB:CC:DD:EE:01
The requesting device caches the response in its ARP table (typical timeout: 60-300 seconds) so it does not need to broadcast for every packet. ARP is essential to Ethernet operation but is also inherently insecure — ARP spoofing attacks can redirect traffic by sending false ARP replies, claiming a victim's IP address belongs to the attacker's MAC. Countermeasures include Dynamic ARP Inspection (DAI) and static ARP entries for critical infrastructure.
For IPv6, ARP is replaced by NDP (Neighbor Discovery Protocol), which uses ICMPv6 multicast rather than Ethernet broadcast, and includes security extensions (SEND — Secure Neighbor Discovery).
Jumbo Frames
The standard Ethernet MTU of 1500 bytes dates from the original 1980 specification. For modern high-speed networks, this creates significant overhead: every 1500-byte payload requires at least 38 bytes of framing overhead (preamble, headers, FCS, inter-frame gap), plus CPU interrupt cost for each frame. At 10 Gbps, transmitting 1500-byte frames means the NIC must process over 800,000 frames per second — each requiring a separate interrupt, header parse, and forwarding decision.
Jumbo frames extend the MTU to 9000 bytes (or larger — there is no formal standard, but 9000 is the de facto convention). This reduces the frame count by roughly 6x for the same data volume, significantly reducing CPU overhead, interrupt load, and improving throughput for bulk transfers. The benefits are most pronounced in:
- Storage networks — iSCSI and NFS over Ethernet benefit greatly from larger frames.
- Data center interconnects — Server-to-server traffic in the same data center.
- High-performance computing — MPI and RDMA traffic between cluster nodes.
- Backup and replication — Large bulk data transfers.
The critical requirement is that every device and switch in the path must support and be configured for the same jumbo frame MTU. If any device in the path has a smaller MTU, frames will be dropped (Ethernet does not fragment — only IP does, and even then, Path MTU Discovery typically sets the DF bit to prevent fragmentation). A single misconfigured switch port that silently drops frames larger than 1500 bytes is an infuriating problem to debug because everything appears to work for small packets but fails for large transfers.
Ethernet Speeds: From 10 Mbps to 400 Gbps
Ethernet has evolved through several generations, each typically a 10x speed increase. The IEEE 802.3 standard defines each variant:
| Speed | Standard | Year | Common Media |
|---|---|---|---|
| 10 Mbps | 802.3 (10BASE-T) | 1990 | Cat 3 UTP, 100m |
| 100 Mbps | 802.3u (100BASE-TX) | 1995 | Cat 5 UTP, 100m |
| 1 Gbps | 802.3ab (1000BASE-T) | 1999 | Cat 5e/6 UTP, 100m |
| 10 Gbps | 802.3an (10GBASE-T) | 2006 | Cat 6a UTP, 100m / SFP+ fiber |
| 25 Gbps | 802.3by (25GBASE-CR/SR) | 2016 | SFP28 fiber/DAC |
| 40 Gbps | 802.3ba (40GBASE-SR4) | 2010 | QSFP+ (4x10G lanes) |
| 100 Gbps | 802.3ba (100GBASE-SR4) | 2010 | QSFP28 (4x25G lanes) |
| 200 Gbps | 802.3bs (200GBASE-SR4) | 2017 | QSFP56 (4x50G lanes) |
| 400 Gbps | 802.3bs (400GBASE-SR8) | 2017 | QSFP-DD / OSFP (8x50G lanes) |
A key pattern emerges at higher speeds: links are built by bonding multiple lanes. A 100G link is actually four 25G lanes. A 400G link is eight 50G lanes. The form factor — QSFP (Quad Small Form-factor Pluggable) or OSFP (Octal SFP) — reflects this: QSFP provides 4 lanes, QSFP-DD (Double Density) provides 8 lanes. The next generation, 800 Gbps Ethernet (IEEE 802.3df), uses 8x100G lanes and is already being deployed in hyperscaler data centers.
At 25 Gbps and above, copper cabling gives way almost entirely to fiber optics and DAC (Direct Attach Copper) cables. DAC cables are short (1-5 meter) twinax cables with transceivers permanently attached, commonly used for intra-rack connections in data centers. For longer runs, multimode fiber (up to ~100m for SR optics) or single-mode fiber (up to 10+ km for LR optics) is used.
Link Aggregation (LAG / 802.3ad)
Link Aggregation (IEEE 802.3ad, now 802.1AX) bonds multiple physical Ethernet links into a single logical link, increasing both bandwidth and redundancy. A LAG (Link Aggregation Group) of four 10G links provides 40 Gbps of aggregate bandwidth and can tolerate any individual link failure.
Traffic distribution across member links uses a hashing algorithm that considers some combination of source/destination MAC, source/destination IP, and L4 port numbers. The hash ensures that all frames belonging to a single flow follow the same physical link (preserving frame ordering), while distributing different flows across the available links. This means a single TCP connection never exceeds the speed of one member link — LAG provides aggregate bandwidth, not per-flow bandwidth.
LACP (Link Aggregation Control Protocol) dynamically negotiates LAG membership between two connected switches, detecting link failures and adding or removing member links. Without LACP, LAG must be statically configured on both ends, and misconfigurations can create loops.
Ethernet in the Data Center
Modern data center networks use Ethernet exclusively at every tier. The dominant architecture is the leaf-spine (Clos) topology:
In a leaf-spine fabric:
- Leaf switches sit at the top of each rack (ToR — Top of Rack). Servers connect to their local leaf switch, typically at 10G or 25G.
- Spine switches form the fabric core. Every leaf connects to every spine, creating a full mesh with equal-cost paths between any two leaves.
- ECMP (Equal-Cost Multi-Path) routing distributes traffic across all available spine paths, providing both load balancing and redundancy. If a spine switch fails, traffic is automatically redistributed across the remaining spines.
- No Spanning Tree. Since all inter-switch links are Layer 3 routed (typically using BGP or OSPF), there are no Layer 2 loops and STP is not needed. Layer 2 domains are confined to each individual leaf switch and its directly connected servers.
Uplinks from leaf to spine are typically 100G or 400G. The non-blocking fabric design ensures that any server can communicate with any other server at full line rate, regardless of which rack they are in. This architecture scales horizontally — adding more spine switches increases bisection bandwidth, and adding more leaf switches adds more server ports.
For workloads that require Layer 2 adjacency across racks (VM migration, for example), VXLAN (Virtual Extensible LAN) provides an overlay network that tunnels Ethernet frames inside UDP packets across the routed underlay. VXLAN uses a 24-bit VNI (VXLAN Network Identifier), supporting over 16 million logical networks — far beyond the 4,094 VLAN limit. EVPN (Ethernet VPN, defined in RFC 7432) provides a BGP-based control plane for VXLAN, distributing MAC/IP bindings across the fabric and eliminating the flood-and-learn behavior of traditional Ethernet switching.
Ethernet and the Internet Edge
Ethernet's role extends well beyond the local network. At Internet Exchange Points (IXPs), the world's networks peer with each other over Ethernet. Major IXPs like DE-CIX, AMS-IX, and LINX operate as large Ethernet switching fabrics where hundreds of networks connect their routers. Each participant connects a router to the IXP switch fabric, typically at 10G, 100G, or 400G, and establishes BGP peering sessions with other participants across the shared Ethernet infrastructure.
Even WAN connections have been "ethernetized." Metro Ethernet services (MEF standards) deliver point-to-point and multipoint WAN connectivity as Ethernet handoffs, replacing legacy technologies like Frame Relay, ATM, and SONET/SDH. Carrier Ethernet uses provider-level VLAN stacking (Q-in-Q) and MPLS transport to deliver Ethernet frames across metro and long-haul distances while maintaining the familiar Ethernet interface for customers.
At the physical layer, submarine cables that span oceans and carry the world's intercontinental internet traffic increasingly use coherent optics that are Ethernet-framed. The same Ethernet frame format designed for a building-sized network in 1973 now wraps data traveling at the speed of light through fiber-optic cables under the Atlantic Ocean.
Common Ethernet Troubleshooting
Understanding Ethernet internals helps diagnose common network problems:
- CRC errors / FCS errors — The frame's checksum does not match, indicating bit corruption. Common causes: damaged cables, failing transceivers, electromagnetic interference, speed/duplex mismatch. High CRC error rates on a port indicate a physical layer problem.
- Duplex mismatch — One side is configured for full-duplex while the other runs half-duplex (usually due to failed autonegotiation). Symptoms: the link works but with poor performance, late collisions on the half-duplex side, and FCS errors on the full-duplex side. This was a plague in the 10/100 Mbps era but is rare at Gigabit speeds where autonegotiation is mandatory.
- STP topology changes — Frequent spanning tree topology change notifications (TCN) cause the switch to flush its MAC address table, temporarily flooding all traffic. Causes: flapping links, misconfigured PortFast (STP edge ports), devices going up/down repeatedly.
- Broadcast storms — Abnormally high broadcast traffic, often caused by a Layer 2 loop (STP failure or misconfiguration). Symptoms: extremely high CPU on all devices, network unresponsive. Storm control (rate-limiting broadcasts per port) is a common mitigation.
- MTU mismatches — Some paths support jumbo frames while others do not. Packets larger than the minimum MTU in the path are silently dropped. Path MTU Discovery (PMTUD) should detect this via ICMP "Fragmentation Needed" messages, but if ICMP is blocked (a common firewall misconfiguration), the problem becomes a "black hole" — small packets work fine, large packets vanish.
- VLAN misconfiguration — A trunk port missing a VLAN in its allowed list, or an access port in the wrong VLAN. The device has link but cannot reach anything. Check
show vlanandshow interfaces trunkon the switches.
Ethernet Security Considerations
Ethernet, by design, trusts the devices connected to it. Several security features have been added to managed switches to address the resulting vulnerabilities:
- Port security — Limits the number of MAC addresses allowed on a port. If more MACs appear than configured, the port is shut down or violating frames are dropped. This mitigates MAC flooding attacks.
- 802.1X (Port-Based Network Access Control) — Requires devices to authenticate before gaining network access. The switch acts as an authenticator, communicating with a RADIUS server to verify the device's identity. Unauthenticated ports are placed in a restricted or guest VLAN.
- DHCP snooping — The switch inspects DHCP traffic and builds a binding table of IP-to-MAC-to-port mappings. Only DHCP server responses from trusted ports are allowed. This prevents rogue DHCP servers from hijacking client configurations.
- Dynamic ARP Inspection (DAI) — Uses the DHCP snooping binding table to validate ARP packets. ARP replies with mismatched IP-to-MAC bindings are dropped, preventing ARP spoofing attacks.
- Private VLANs (PVLAN) — Restricts communication between ports in the same VLAN. Isolated ports can only communicate with the promiscuous (uplink) port, not with each other. Used in hosting environments to prevent co-located servers from attacking each other at Layer 2.
- MACsec (802.1AE) — Provides line-rate encryption and integrity protection for Ethernet frames, hop by hop. MACsec encrypts everything between the source MAC and the FCS, preventing eavesdropping on the physical wire. Increasingly used in data center and campus deployments.
See It in Action
Ethernet is the foundation that every IP network is built on. You can explore how networks interconnect at the IP layer, trace the BGP AS paths that traffic follows across the internet, and see how IP subnets are organized within the globally routed address space:
- AS13335 — Cloudflare (operates a massive leaf-spine Ethernet fabric at every PoP)
- AS15169 — Google (pioneer of Clos data center network design)
- AS32934 — Facebook / Meta (runs one of the largest data center fabrics in the world)
- AS714 — Apple
Try looking up your own IP address to see which network carries your traffic, what prefix your IP falls within, and examine the Layer 3 AS path that sits on top of the Ethernet links connecting the world's networks at Internet Exchange Points.