fweston@gmail.com
386-843-0123
lumberjocks
[ about ]
[ misc ]
[ wishlist ]
[ resume ]
[ stats ]
[ movies ]
[ recipes ]
|
|
FWeston fweston@gmail.com 386-843-0123 lumberjocks |
[ about ] [ misc ] [ wishlist ] [ resume ] [ stats ] [ movies ] [ recipes ] |
|
InfoWorld article about network admins Posted under Miscellaneous on Thursday, July 18, 2013 @ 11:41:00 AM
Show Previous Article Show Next Article infoworld posted an article that i agree with so much, i decided to repost here. a lot of these resonate with me. credit to paul venezia. veteran network admin trait no. 1: we already know it's down few things are more annoying than having your phone blow up with automated alert messages from your monitoring systems, scrambling to dig into the issue, only to be continually bombarded with humans texting/talking/emailing/calling with the same "is x down?" question, or even worse, "the network's down!" if the outage is significant, we already know about it, and we are trying to work on it as fast as we possibly can. continued attempts to deliver elderly information will only impede that effort. veteran network admin trait no. 2: if we don't know it's down, it's probably not down conversely, if we get a message claiming, "the network's down!" yet we have not been notified by any monitoring system, then the problem is almost certainly the complaining user in question. to users, if there is any resource that cannot be contacted, whether that resource is internal to the network, on the internet, or perhaps orbiting the earth, that means the network is "down." this apparently includes 404 errors from shady websites, mistyped urls, or the lack of any sort of network connection on the user's laptop. nothing is more grating to a network admin than someone claiming the network is "down." no, it isn't -- reboot your laptop. veteran network admin trait no. 3: we will ping and test several times before digging into the problem if we begin looking into a problem, especially across a wan or long-haul link with several providers in the middle, we will reserve judgment for the first several minutes. this is because these connections are subject to the vagaries of their path, and connectivity problems can come and go like ghosts in the night. a fiber wan link that was stable a minute ago but is now exhibiting 30 percent packet loss will more than likely fix itself in short order. only after a probationary period will we start digging deeper into the issue. veteran network admin trait no. 4: believe it or not, we've tried turning it off and back on again many times, we will "fix" a problem by turning an interface off and back on again. in fact, this may be the first thing we try when troubleshooting an issue. whether it's a problem with auto-negotiation, a sketchy cable, or sunspots, you'd be surprised at how often dropping and restarting an interface will restore proper operation. while networking may be a science, it's not without its white whale. veteran network admin trait no. 5: during an outage, we're not just staring at the screen -- we're following a path in our heads if you come across a network admin working on a problem, looking at a routing table with a 1,000-yard stare, he's not experiencing performance anxiety or somehow "lost." he's running through dozens of possible scenarios in his head, following the routing and switching paths, and calculating possible problem scenarios. the uninitiated can't quite grok the way a network admin's mind works, but a parallel might be to imagine an intangible maze, then try to solve it. somewhere in that maze lies a dead end that shouldn't be there. we're looking for that -- then we can take steps to open up the path again. veteran network admin trait no. 6: we calculate subnet masks and cidr as easily as breathing i think it's safe to say that perhaps outside of the common class c netmask, the overwhelming majority of humans do not understand subnetting, or cidr. for network admins, this is as intrinsic and involuntary to our brain as breathing. it's not just knowing that a /28 is a netmask of 255.255.255.240, and that it describes 16 addresses, 14 usable, or where subnet boundaries lie. it's the ability to collapse large numbers of smaller subnets into larger descriptors in order to reduce routing table sizes, acl applications, and a wide variety of other internal networking tasks. when we see contiguous networks sliced into smaller chunks in an acl, yet with identical rules applied, we get agita. a /19 is a much better idea than a collection of /24s. the same applies to wildmasks. veteran network admin trait no. 7: we do not tolerate bugs; they are of the devil on occasion, conventional troubleshooting or building new networks run into an unexplainable blocking issue. after poring over configurations, sketching out connections, routes, and forwarding tables, and running debugs, one is brought no closer to solving the problem. this is the unholy area of networking inhabited by the software bug. network admins think of switching and routing software bugs as personal attacks, and they will usually excoriate a vendor when one is discovered. this is because before the determination is made that the problem is due to a bug, nothing makes sense whatsoever. it completely violates years of experience and knowledge, throws waste to logic, and causes immense amounts of stress and turmoil. you might think of it as if you spontaneously transmogrified into a difference species. everything you've ever known suddenly does not apply, yet here you are. veteran network admin trait no. 8: we can read live packet streams and write highly complex filters in our sleep few items in networking and computing are even remotely similar to how they're portrayed on tv and in movies. the terminal window with text rapidly scrolling past, however, is one of them. in the sys admin world, that's usually a log file tail. in the networking world, it's usually a packet dump. depending on what we're trying to do, we may quickly call up a packet capture on a live circuit and watch packets fly by in real time. it's not gibberish. we're looking for telltale signs, and we'll winnow down our capture using filters until we've found what we're looking for. speed is usually of the essence in these cases, so we're well versed in bpf filtering syntax. also, we tend to notice really strange things like ip headers that "just don't look right" and other oddities that may as well be alphabet soup to most people. some people speak klingon. we speak ip. veteran network admin trait no. 9: we take big risks all the time network admins tend to work on many remote devices. unlike server admins who can pull up a console if the server is otherwise inaccessible via the network, we usually have no such luxury. this means that changes we make to certain devices carry with them the ever-present threat of the loss of connection. basically, we're always a missed keystroke or two away from making a problem worse, or causing a big problem where none previously existed. while making a change to a remote switch and applying that change to port 34 instead of port 35, say, we could inadvertently cause a complete lack of connectivity to the remote site, taking who knows what offline until the device is power-cycled or we -- or worse, someone else -- drive over to fix it manually. ideally, this shouldn't happen in the middle of the night, but given the usual timing of network maintenance windows, it often does. oh, and we sometimes have to make changes to remote devices knowing that if anything goes slightly wrong, we will lose connectivity -- such as when changing internet providers at a remote site and reconfiguring the firewall with a script because we have no other persistent connection to it. we live with this understanding all day, every day. |