A client with two locations needed a single, redundant and secure network between them β without depending on a single ISP or a single server. In each site we built a Linux server that does routing, VLAN trunking, dual-ISP with instant failover, an L2 VPN bridge between VLANs, the entire security stack and KVM virtualization β plus a backup server that takes over production in 15 minutes. Implemented in February 2020, on Debian 10, with Suricata 5.
The context
Initially, each location had a Mikrotik for routing and VLANs, with a single ISP. It worked, but had clear limits:
- A single ISP per site β if the provider went down, the location was isolated
- No unified L2 connectivity between the two locations β the VLANs couldn't "see" each other across sites
- Minimal perimeter security β no IPS, no dynamic attack blocking, no traffic visibility
- No server redundancy β a single point of failure per location
The client wanted the two locations to work as one network, to survive the loss of an ISP or a server, and to be genuinely protected at the perimeter.
The challenge
We had to connect the two locations at L2 β each VLAN behaving as if both sites were in the same building β with ISP and server redundancy, a strong security perimeter and no downtime on a failure. All without changing the addressing or the applications behind it.
The solution / architecture
In each site we installed a Debian Linux server that takes over and extends the role of the old Mikrotik. The same server simultaneously acts as router, VLAN trunk, VPN terminator, firewall, IPS and virtualization host.
- VLAN trunk (802.1Q) β the server has a trunk interface carrying all VLANs; on it we do the routing and internal access rules between VLANs and between sites
- Dual-ISP β two providers on each server, for internet redundancy
- OpenVPN in bridge mode (L2) β one OpenVPN instance per VLAN, so each VLAN extends at L2 across both locations
- KVM virtualization β VMs on each server, with synchronized configurations between sites
- A backup server in each location, ready to take over production
Hardware and infrastructure
Each site is a factory with hundreds of endpoints β PCs, production machines and IoT devices β all on the internal network. The main server in each location is a Dell PowerEdge R640 (14th gen, 1U, dual-CPU, 128 GB RAM), with 4x 1GbE aggregated in an LACP port-channel spread actively across two Dell N1500 switches in a stack β full redundancy at the core-switch level. If one switch fails, traffic continues uninterrupted on the other. The 2x 10GbE provide dual-ISP connectivity. The backup server in each site is a Dell PowerEdge R730 (13th gen, bought in 2017), with the same NIC configuration and connectivity β older hardware, identical architecture.
The implementation
Step 1 β The Linux server as router + VLAN trunk
The Debian server took over the Mikrotik's role, but with far more power. All VLANs arrive on the 802.1Q trunk interface, and the routing and internal access rules are done on the server β who can reach whom, between VLANs and between the two locations. Segmentation stays clean, but controlled from a single place.
Step 2 β Dual-ISP with instant failover
Each server has two ISPs. If one provider goes down, the OpenVPN connections move instantly to the other ISP β the tunnel between locations doesn't visibly drop for users. A cut cable or a provider outage no longer isolates a location.
Step 3 β L2 VPN bridge between VLANs
Connectivity between the two locations runs over OpenVPN in bridge mode, with a separate instance for each VLAN. In effect, each VLAN extends at L2 across both sites β a device in one location is on the same network as one in the other, as if they were on the same switch. Applications that assume the same local network work without changes.
Why 12 OpenVPN instances and not a single one
An objection we hear often: an L2 bridge per VLAN, with a separate OpenVPN instance for each, should be heavy. In practice it is not. We run 12 OpenVPN instances in bridge mode, one per VLAN, and the system has run smoothly in production since February 2020. The per-instance separation is actually an advantage: each VLAN has its own isolated tunnel, process and key set β if one fails or is reconfigured, the rest are unaffected. Troubleshooting is far simpler than with a single tunnel carrying all traffic. Broadcast and ARP stay isolated per VLAN. When needed, we scale by adding one more instance for a new VLAN, without touching the rest.
Step 4 β KVM virtualization with synchronized config
KVM runs on each server, and the VMs have synchronized configurations between the two locations. A service can be started in the other location with the same configuration, without manual rebuilding.
Security β a perimeter on every server
Each Linux server is also a firewall, an IPS and a visibility point. The security stack is identical in both locations:
- Suricata IPS β deep packet inspection; detects and blocks exploit attempts before they reach the internal networks
- ipset with dynamic lists β very fast kernel-level blocking of large IP lists that update dynamically (threat feeds, IP reputation) β attackers are cut off at the gate
- GeoIP filtering β country-level filtering, shrinking the attack surface from regions with no legitimate users
- nftables β the base firewall on each server, with strict access rules between VLANs and toward the outside
- Fail2ban β automatically blocks IPs attempting brute-force on services (SSH and others)
Availability and failover
Redundancy is on three levels: ISP, server and site.
- ISP β instant failover between the two providers on each server; the OpenVPN tunnels don't visibly drop
- Server β each location has a backup server that goes into production automatically 15 minutes after the main server fails, or immediately via a command that asks for confirmation
- Site β the two locations cover each other at the network and monitoring level
Why 15 minutes and not instant? Because there are maintenance windows where operating-system upgrades are done, and a planned reboot takes about 5 minutes. Instant failover would promote the backup server on every reboot β a false positive. The 15-minute threshold lets the main server finish a normal reboot without triggering failover. And when you want to take over manually, there's a command that asks for confirmation β so failover can't be triggered by accident.
Between the main and the backup server in each site there is continuous configuration synchronization: any firewall, routing or access rule changed on the main server propagates automatically to the backup server. So when the backup takes over, it does so with exactly the same configuration β no surprises, no manual reconfiguration.
Monitoring β redundant Nagios
The Linux servers also act as Nagios. Monitoring runs on the main servers in each site, but because we have two locations that see each other at L2, if a main server goes down, the Nagios in the other location takes over monitoring β you're not left blind exactly when you have an incident.
The result
- β The two locations work as a single L2 network β one VPN bridge per VLAN, transparent to applications
- β Dual-ISP with instant failover β a provider outage no longer isolates a location
- β Server redundancy β a backup that goes into production in 15 minutes or on command, with confirmation
- β A full security perimeter: Suricata IPS + ipset with dynamic lists + GeoIP + nftables + Fail2ban, identical in both sites
- β Redundant Nagios monitoring β one site covers the other
- β The old Mikrotiks consolidated into a single Linux server per site that does everything: routing, VLAN, VPN, security, virtualization
Do you have multiple locations that need to work as one?
We build redundant multi-site networks: L2/L3 connectivity between locations, dual-ISP with failover, an identical security perimeter everywhere, virtualization and a standby server that takes over production in minutes. With no dependency on a single ISP, a single server or a single location.