I'm only talking my own servers. I have had to rebuild other people's servers as well, and I would still take the configuration files over documentation. At the same time, I have never managed an Oracle server, so I can't say what I would do then. I guess it comes down to managing machines where you are familiar with their process.

I agree documenting deviations can be nice, but are those deviations moving outside of the standard configuration locations?

Maybe I should not have said "/etc" but instead said "standard configuration locations" meaning the standard places where the configuration files should be. Though I will content that in most cases, that means /etc.

On Friday, May 17, 2013 12:10:56 AM JD Austin wrote:

I deal with Oracle databases and application servers a lot and while the asterisk based systems I create and support have most of their configuration in /etc other things are stored as data in the MySQL database and other places. With Oracle eBusiness Suite applications none of it is in /etc and /var except for startup scripts.

Yes I'll admit you can get away with doing what you're saying a lot especially on simple servers.. until you can't. Sometimes there is no reliable working version of the server to go to.. because the 'working version' was on a hacked server and then sometimes even your backups have the problem. Usually the stuff in /etc is pretty safe because it isn't executable but wrong settings there can open a lot of vulnerabilities; having what you deviated from the default install documented helps a lot.

I think the horse is dead here :) If you're ok trusting that what you have in /etc and /var is sufficient good luck to you.

On Thu, May 16, 2013 at 11:53 PM, Nathan England <nathan@nmecs.com> wrote:

Are you serious? Do people actually configure linux servers with configuration files outside of /etc ? Beyond LDAP and MySQL what else keeps the configuration outside of /etc ?

I'm not talking about replacing all the data from a web server or loading the data into an LDAP or MySQL server. I'm talking reconfiguration.

As far as "recreating problems from the original server" did you not read what I wrote? I said ...

[QUOTE]

A complete backup of /etc and /var from a machine that was running as intended 2 hours before the SHTF.

[/QUOTE]

Notice the *as intended*. If the machine was not running properly to begin with, that would have to dealt with before the machine goes down. If your misconfigured machine fails before you get it configured correctly, you have bigger problems my friend!

On Thursday, May 16, 2013 11:41:23 PM JD Austin wrote:

What kind of servers do you manage where knowing what is in /etc and /var is sufficient?

are you sure you aren't recreating problems from the original server on the new one?

On Thu, May 16, 2013 at 11:36 PM, Nathan England <nathan@nmecs.com> wrote:

From my experiences with server rebuilds after catastrophic failures, I absolutely have to agree. The server *is* the documentation. I have rebuilt servers using extensive documentation about how it was setup, with scripts and plans included in the documentation and why it was set up the way it was set up. I have also rebuild servers after massive failures using only data backups.

If I had to rebuild a server today, in a time crunch, and I had the choice of complete documentation of how and why vs a copy of the /etc folder.... I'd go /etc in a heart beat. I wouldn't even flinch.

Who is to say I am going to understand someone else's logic when reading their documentation. Who's to say the system wasn't built by a genius and then documented by a teenage kid still wet behind the ears having only built his first 'puter a week ago.

A complete backup of /etc and /var from a machine that was running as intended 2 hours before the SHTF.

I agree completely that this kind of attitude comes across as arrogance. But is it really arrogance if I really am just better than you? (lol - ducks)

When you build a machine for a client it is highly recommended that you well document the system. But at the end of the day, if the admin who must fix what is broken can't figure it out from looking at the code or reading the configuration files what makes you think endless reems of documentation is going to help?

I know a few guys who could figure it out, but where some of us could rebuild the machine in a couple of hours given that /etc backup, the other guy might take days...

The server *is* the documentation.

Good post Lisa!

Nathan

On Thursday, May 16, 2013 09:03:57 PM Lisa Kachold wrote:

Across the board, the number 1 worst attribute that I see working with the PLUG, technology teams, and mentoring (at or around year 3 in academics, and year 3 - 10 in IT/linux professionalism) = arrogance.

What exactly is arrogance anyway. Where is this found? Why?

It's the place in the discussion where one person dominates assuming that their position or knowledge is greater (without investigation). This is also referred to as "OneUpManShip".

It's the place in the presentation where students and PLUG peers write off the person who has taken on the role to "present on the subject" based on their ability to verbally spiel acronyms. This is referred to "Minimizing".

It's the place in the team dissemination of project roles and tasks where a member's enthusiasm is downplayed based on experience. This is referred to "Dues Hierarchy".

This is the place in the interview where the employer fails to realize all they need to do is very the work history, since everything for a Linux professional is motivated by and driven from an ethical systems administrator viewpoint (not any communications with or responsibilities disseminated from the employer); just as we are woken from sleep to work on or check systems; and jazzed beyond belief by a well engineered hardware server like IBM Blade (can you say Fiber channel switched backplane?)...

There are a great many examples where an ego based emotional assumption of or judgement is placed on our peers, our work, and even ourselves at one point or another.

The ability to understand linux systems requires a certain type of systemic theory; which can be daunting for some people; such systems integration can be hard to troubleshoot [and successfully negotiate within] without inherent abilities but can be done with a great deal of complex experience, however this is NOT ROCKET SCIENCE. So the people who do well at what we do, are usually those that find that they inherently find this easy.

It is, however, far from easy, since most of us work long hours, without adequate physical exercise and balanced stress free environments. The sheer amount of responsibility and ultimate reliance in all shops on the unique abilities of the Unix/Linux systems administrator are daunting to most once they get a full view.

However, we each learn standard process applied across the OSI stack and/or fed through the kernel/memory/processor for systems or DevOps applications performance and integration, security or troubleshooting.

Standard process, which includes a few easy to learn rules, relies on logs and linux tools, completely supplants any experience, past systems history knowledge (available on/in the server), most visio documentation or RunBooks (which should not exist unless something cannot be known by server view alone).

Ironically, to people who are not linux-ish, the statement that "The Server IS the documentation" seems incredibly arrogant, when in fact, it simplifies all the arrogant posturing and 7 deadly sin based profit from the misunderstanding of unix/linux administration and engineering.

We all intimately understand the concept of "obnosis" or Knowledge by Observation - rather than what is imparted via formal rote learning and scholastic pursuit.

What do you think? Is the adage "There is NO substitute for experience" correct or can anyone using standard process (as opposed to documented process) and NIX command line skills (yet bringing no experience) get to the finish line at the same time?

http://wiki.obnosis.com

(503) 754-4452 Android
(623) 239-3392 Skype
(623) 688-3392 Google Voice
**
it-clowns.com
Chief Clown

---------------------------------------------------
PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org
To subscribe, unsubscribe, or to change your mail settings:
http://lists.phxlinux.org/mailman/listinfo/plug-discuss