Let’s admit it. We all commit stupid sins and hope that nobody catches them. We do it in our own computers, sometime in our friends’ computers (and spend hours trying to fix them). These are small, innocent sins that doesn’t have a big effect (unless you really make a huge, stupid, irrecoverable mistake). In the case of a data center, that kind of a sin can prove to have tremendous effects, simply because a data center is a networked environment.
The first sin is the one that’s called a “cable salad” in Turkish. Yes, this is the situation that the cables are lying/hanging like guts in the slaughterhouse. I do not really know what the say about this – this is something that should never happen, but almost always happens. Cables should be given the proper attention they need. Follow the old saying, measure twice cut once. Many network administrators present the excuse that they do not have time to arrange the cables. What they don’t understand is, cable management are part of their jobs. Not just because of safety reasons, but also for effective cooling, avoiding data loss as well.
The second sin is the electricity. No generators, no battery backups, accidental power cuts, overloading a single power outlet are just everyday sins in the data center. Electricity is the blood of your data center. Without it, your thousands of dollars of investment is nothing more than a piled-up electronic trash. When designing/checking your data center, make sure that the electricity problems are avoided. All circuit breakers, alarms, turn off switches are away from daily working environment and covered preferably in an easy-to-open, transparent box, so that they are both easily accessible in an emergency and yet not accidentally operated.
The third sin is drinking. I am disciplined at sea to not approach the charts with drinks, which I carry on to the data center and never enter there with a drink. Liquid is the worst enemy of all the electronic equipment. It kills an equipment in shorter than a second without any chance of recovery. It doesn’t allow time for backups. It voids guarantees. Although these are obvious even to the non-IT people, I am really amazed to see how many administrators disregard them. Put a “no food or drink allowed” sign on your data center door and enforce the policy with zero tolerance.
The fourth sin is security. How many copies of data center door keys do you have? If you have a passcode lock, how many people know the password? In what period do you change the passcode? Did you ask the intern to do something for you and gave her the passcode? When working in the data center, do you open the door and leave it unattended? You can just imagine where this is going. If you are following these security measures in your house, why aren’t you following them in your data center?
The fifth one is the password problem. Somebody changes the system’s password and goes for his annual vacation and it will be this system that will fail. The passwords, access codes should be documents so that they are readily available when needed yet away from prying eyes during normal operations.
The sixth one is the “desktop extension.” In all the IT departments I see administrators using servers as an extension of their desktop systems. They use servers’ disk spaces as additional/external disks (map network drive), they abuse the servers’ computing powers (LAN parties) and they do not log off from the servers (as if hibernating on their laptops). Although all these seem obviously stupid, they are more common than you can imagine. These practices should be avoided with clear policies and violations should be treated at the top level with zero tolerance.
The seventh sin is overworking. All of us sucked caffeine in a day that is not sucked in a year by a big portion of the world population and felt that we can go through the night wide awake. We have spent the whole night implementing the blade servers and storage enclosures in place. We have spent the time to install the operating system and make sure that it is functioning as needed. Beyond that point we forget to stop. Just performing that small, incremental, insignificant task can wreak havoc in the data center. If everything is complete but some small here-and-there configuration issues remain, let them remain. Let the next watch make them. Or wait until the time you have slept enough. Don’t make this final command a famous last word. Don’t try to be the hero. Be smart.
Any other sins you have committed or witnessed? Hit the comments below and share your experiences.