All articles
Network Automation by Ismail Karankin

When Network Remediation Can Run Itself

Many network repairs follow a known pattern. When diagnosis is certain and the change can be undone, automation is not reckless. It is disciplined operations.

Agent checking a BGP fault before a reversible repair

When the correct response to a fault is already known, why should execution still wait for a person? The usual answer treats automation as faster but less controlled. That framing obscures where human judgment is actually needed.

For a meaningful class of network faults, speed and control are not opposing choices. The diagnosis is clear, the response is documented, and the outcome is understood. Sending that work through a human adds delay without adding judgment. Autonomous remediation is about placing control at the appropriate layer.

Faults That Follow a Known Repair Pattern

The boundary matters. Novel failures, multi-segment degradation, and incidents whose repair depends on unseen context still call for human judgment. Deterministic automation does not cover those cases.

The overlooked point is that deterministic faults make up more of daily operations than most workflows recognize. Consider:

A BGP peer with a documented instability pattern flaps again on the same link, with the same prefixes and cadence seen in the prior four incidents. A BGP soft reset re-advertises routes without dropping the session. The diagnosis and response are both clear, leaving no creative decision to make.

An interface exceeds its CRC threshold in an environment where that pattern has tracked with SFP degradation. Shut and re-open the interface, then check whether the fault clears. A clear result calls for port inspection. A persistent error calls for hardware replacement.

An OSPF adjacency between a known switch pair fails because of a previously documented dead timer mismatch. Correcting the setting on one end is a text substitution with a known prior state and expected result.

These cases do not need a senior engineer at the execution point. That engineer already captured the response in a runbook. The practical choice is whether a tired person paged at 2am runs it, or an agent that has confirmed the known condition does.

Setting a Confidence Gate

The safety control is not merely blocking ambiguous actions. It is calculating confidence before each action, then proceeding only when that score clears a defined threshold.

A network diagnosis can be scored across several dimensions. First is fault classification: do the symptoms match a known pattern, and how closely? A BGP session in Active state for 120 seconds with a known-flappy peer is a high-confidence match. Rising interface input errors without prior history warrant less confidence.

Historical context asks whether the device, interface, or peer has shown the same condition before. Repeated incidents are easier to classify than first occurrences. With six months of device history, an agent can separate a familiar fault with a known fix from a new one.

Topology risk measures the consequence of a wrong repair. A redundant link is not equivalent to the only upstream path for a production segment. Greater blast radius should raise the confidence requirement or trigger human review.

Make Undoability Part of the Decision

High confidence alone does not justify every autonomous action. Reversibility also matters. A change that can be safely undone within seconds carries less risk than one requiring 20 minutes to restore from backup.

BGP soft reset is reversible. If the session worsens, it can be re-established manually. An interface bounce is reversible as well, and a persistent fault becomes apparent quickly. An OSPF hello timer correction is similarly easy to assess, with rollback available through one command.

Changes to security policy, routing policy, or ACL rules need a different path. Require human approval, or set an automatic rollback timer if the health check fails within a defined window. These limits belong in action classification because they are central to production trust.

Why Manual Handling Should Not Be the Default

Without autonomous remediation, many teams route every alert to a human, who diagnoses it, decides, and executes the change. The assumption is that this produces better results because a person catches what automation might miss.

That assumption holds for complex, novel incidents. Routine deterministic faults are different. Human handling is slower because it waits for an engineer to respond, and less consistent because fatigue, timing, context, and personal preference affect the result. A correct runbook applied the same way each time is more consistent.

Engineers remain essential to operations, especially when a fault is novel or consequential. Their highest-value judgment belongs in designing and governing the runbook. Once the condition and response are established, execution usually gains little from another human decision.

Lessons from Early Pilot Results

Across PhysicalLayer pilot environments, the pattern is clear: about 60% to 70% of alerts match conditions that are routine, reversible, and high-confidence. The remaining 30% to 40% need human triage because confidence is low, blast radius is high, or no known pattern fits.

Those figures are not universal. They reflect mostly mid-size enterprise networks with stable topology and documented failure history. Networks with more frequent changes, newer topology, or thinner records will likely produce a different split.

What we are confident about is that the split is not 0% versus 100% in either direction. There is a real category of deterministic faults in every production network, and the question is whether your operations workflow is set up to handle that category efficiently. For most teams, it is not. The cost is paid in MTTR, on-call load, and engineer fatigue. Autonomous remediation, applied with confidence scoring and reversibility constraints, is the correct response to that specific cost.

Put Controlled Network Repair to Work

PhysicalLayer runs repairs only when the cause is confirmed and rollback is clear.