6 Strategies Amazon S3 Engineers Can Learn from Network Engineers
Fat Finger Strikes Again
By now we are all aware of the $150M typo by an ‘authorized Amazon S3 Engineer’. On March 2nd, 2017, Amazon stated that a failure in its cloud-computing service was due to a fat finger’d command. While we don’t know if that S3 Engineer is still employed by Amazon, there probably isn’t an IT professional alive who couldn’t sympathize upon hearing the cause. And many IT professionals probably began thinking of variations of their own commands that also would lead them to a very bad day. This could obviously occur in many domains of IT where the command line interface (CLI) is used as a form of management, but where it’s particularly prevalent is in the network management space.
Getting Over Your CLI Skills
Who doesn’t know a network jock who swears by his ninja CLI skills. You can easily throw around terms like “enable mode”, “write mem”, Ctrl-Z, “banner motd”, “logging”, and “vty” to any networking professional and she’ll know exactly what you’re talking about. And who hasn’t locked himself out of a device when mucking with AAA commands or ACLs (that’s Authentication, Authorization, and Accounting and Access Control Lists if you’re still reading this!). This is just the tip of the iceberg. We’re not even talking about routing tables or protocols yet, where the fallout can be much more serious.
How to Avoid Being Fat Finger’d
So how do you prevent against the typo, the fat finger, the ID 10 Terror, the stomach dropping and sudden unresponsive puTTY session, or worse yet, the ringing mobile phone 3 minutes later? It’s called Network Automation, or sometimes called Network Change and Configuration Management. Network Automation products like Infoblox NetMRI help prevent against the fat finger in the following ways:
Minimize errors with standardized templates. All network devices change can be made with agreed upon and standard templates and scripts where the commands have been vetted and tested.
Changes can be run through an approval process so an extra set of eyes can review before a change is made. Approval can even be connected to your ticketing system (i.e. ServiceNow, Remedy, Service Manager).
All network changes are detected and recorded so if a change does cause a problem, the who, what, where, and when has been captured which helps enable a timely roll back.
Auditing and Auto-Remediation
Standard configuration settings can be audited and auto-remediated which considerably decreases the amount of time that a vulnerable change exists on the network.
Configuration Management Pipeline
A comprehensive REST API enables network changes to be orchestrated as a part of a larger predefined workflow.
CLI Session Recording
CLI sessions can be proxied through NetMRI acting as a jump host which records all CLI sessions. This provides more security and a log of all activity.
Click rather than Type
There are many other advantages of Network Automation with NetMRI including full discovery and inventory of infrastructure and end hosts, configuration compliance enforcement, rapid device and service provisioning, and mass change, so not only will you be keeping your company out of the news, you’ll be gaining operational efficiencies. So, go forth and do more with less …. just be careful with those keyboards …. Or better yet, use that finger to click instead of type. Remember this gets even more complex when enterprises are using a diverse set of environments / hybrid clouds and each environment is operated in its own silo.