[PLUG-Devel] SVN and conf files

[Joseph Sinclair]

OK,  This is a lot, but here're my thoughts.
As I understand it, you need the following:
1) a choice of configuration file format that supports many different code platforms
2) a config file format that is amenable to automated modification (at least during a build or deploy script)
3) an idea for how to manage the whole mass in SVN

Some data points
1) XML (specifically XML-Schema) is the most standardized tool available for creating domain-specific languages (which is what a config file format is)
2) XML has built-in support for including subsets of data from other documents dynamically
3) XML is a text format that works well in SVN

Premises
1) Having a separate config file for each env is a recipe for disaster
2) Having multiple formats for configuration data makes the next person more likely to have trouble
3) Having configuration data in multiple places is likely to really irritate the next person to use them

Conclusions
1) Use XML for the config files.  Make sure your source XML for each config has a XSD schema defining it's exact syntax, well annotated to make it self-documenting.
2) Create an XML format to contain the configuration values for each environment (all in one file, to reduce the probability of sync issues).
3) Use XInclude in the main configuration template, selecting values in the machine-specific file, for each machine-specific value.
4) During build, just run the main XML through an XML parser, setting a variable to the specific env desired, and save the resultant DOM to a machine-specific XML (which may be further processed into another format if needed).
5) Since the config is part of the build output, it's not in the SVN repository.

Advantages
1) XML-based, lots of tools, easy to understand, easy to document (via annotated XSD)
2) All machine-specific values are in one file, in different sections, so it's easier to keep it all in sync
3) There is one general XML file for each cross-platform config file, and it's relatively easy to maintain
4) The config is built just like the code is built (using the same scripts/makefiles/etc...), so it's easy to treat the configs just like any other build artifact (executable, library, etc...)

I've glossed over a lot of details above (and I'm tired enough it's probably a little scattered), but that should cover the gist of it.

==Joseph++

der.hans wrote:
> moin moin,
> 
> at work we have a code base that has everything hard-coded. Major suckage.
> I'm working on fixing it.
> 
> We also have recently moved to Subversion, which is good.
> 
> We now have dev and prod environments. We might someday get to use the
> stage environment as well. The configuration we need to use in each
> environment differs somewhat. I'd like to keep the configuration files in
> SVN as well since the basic format stays the same, just some of the values
> are different.
> 
> Any suggestions on a good way to keep the base structure the same and yet
> have slightly different values?
> 
> I could throw the config files in branches and use softlinks to put them
> in place. I fear this would lead to the config files getting out of sync.
> 
> I could tokenize the things that change and build the config files in each
> environment from a skeleton that's in SVN. Twould be good to have a
> trigger hit the build mechanism every time a config skeleton changes. Yes,
> require specific OK for the trigger in prod :).
> 
> I could come up with a config file format that allows scripting and then
> just use programming structure to determine the proper config for the
> machine. Yuck.
> 
> I could add a config stanza for each machine in the config files and then
> use the hostname or environmental variables to figure out which stanza to
> use. I guess this would be OK, but again I think it leads to configs
> getting out of sync.
> 
> I could just get a job at a company that understands and values config
> files :). I'd still need a solution to this dilema, though.
> 
> Oh, BTW, I need to add config files to shell scripts, PHP scripts, Perl
> scripts, Java programs and I'm not sure what language a couple of the
> tools are written in, but probably one of those.
> 
> Some of the scripts get config info from XML files.
> 
> A couple get configs from XML files that override the configs written in
> other XML files.
> 
> Thankfully our diverse array of Oracle environments were all homogenized,
> well except the one in dev disappeared again today...
> 
> Our MySQL environments are, um, varied. They copy data back and forth and
> are all, in some way or another, different.
> 
> Mostly, the DB issues are handled by the configs for the tools, but I
> thought I'd mention them in case anybody had any gotchas they wanted to
> bring to my attention.
> 
> happy happy config config,
> 
> der.hans