Tutorial¶
We’re going to go from a simple application without configman
, to a
simple application with configman
.
Basics¶
Suppose we write an app similar to the echo
command line program in Unix
and Linux. Our app, though reverses the lettering of each word:
from __future__ import absolute_import, division, print_function
def backwards(x):
return x[::-1]
if __name__ == '__main__':
import sys
output_string = ' '.join(sys.argv[1:])
print(backwards(output_string))
When run it could look something like this:
$ ./tutorial01.py Peter was here
ereh saw reteP
Now, suppose you want to add some more options that can be selected at run time. For example, it could remove all vowels from reversed string. So you can do this:
$ ./tutorial02.py --devowel Lars was here
rh sw srL
First, let’s improve our business logic to add this new feature:
def backwards(x):
return x[::-1]
import re
vowels_regex = re.compile('[AEIOUY]', re.IGNORECASE)
def devowel(x):
return vowels_regex.sub('', x)
Now we need configman
to enable the run time option for removing the vowels
from the input string. It will take the form of a command line switch. If, at
run time, the switch is present, we’ll use the devowel
function. If the
switch is not present, we won’t use the function.
To add this option we create an option container. These containers are called namespaces and have a method that allows us to define our command line options.
Here’s a function that sets up a namespace with a single option:
from configman import Namespace, ConfigurationManager
...
def define_config():
definition = Namespace()
definition.add_option(
name='devowel',
default=False
)
Before we can use this option definition, we need to wrap it up in a
ConfigurationManager
instance that is able to cook it up for
us correctly:
...
config_manager = ConfigurationManager(definition)
config = config_manager.get_config()
That’s all! That last line returned an instance of what we call a DotDict.
It is essentially a standard Python dict that’s had its __getattr__
cross
wired with its __getitem__
method. This means that we can access the
values in the dict as if they were attributes. Watch how we access the value
of devowel
in the full example below:
from __future__ import absolute_import, division, print_function
from configman import Namespace, ConfigurationManager
def backwards(x, capitalize=False):
return x[::-1]
import re
vowels_regex = re.compile('[AEIOUY]', re.IGNORECASE)
def devowel(x):
return vowels_regex.sub('', x)
def define_config():
definition = Namespace()
definition.add_option(
name='devowel',
default=False
)
return definition
if __name__ == '__main__':
definition = define_config()
config_manager = ConfigurationManager(definition)
config = config_manager.get_config()
output_string = ' '.join(config_manager.args)
if config.devowel:
output_string = devowel(output_string)
print(backwards(output_string))
When run, you get what you expect:
$ ./tutorial02.py Peter was here
ereh saw reteP
$ ./tutorial02.py --devowel Peter was here
rh sw rtP
In the tutorial01.py
example, we fetched the command line arguments using
the reference to argv from the sys module. We couldn’t do that in the second
tutorial because sys.argv included the command line switch --devowel
. We
don’t want that as part of the output. configman
offers a version of the
command line arguments with the switches removed. That’s the
config_manager.args
reference inside the join
.
Intermediate¶
Now let’s expand some of the more powerful features of configman
to
see what it can help us with. Let’s start with the help. You invoke
the help simply by running it like this:
$ ./tutorial02.py --help
That’s set up automatically for you. As you can see, it mentions,
amongst other things, our --devowel
option there. Let’s change
the definition of the option slightly to be more helpful:
def define_config():
definition = Namespace()
definition.add_option(
name='devowel',
default=False,
doc='Removes all vowels (including Y)',
short_form='d'
)
Now, when running --help
it will explain our option like this:
-d, --devowel Removes all vowels (including Y)
Let’s add another option so that we can get our text from a file instead
of the command line. The objective is to get a file name from a --file
or -f
switch. We’ll set the default to be the empty string. If the
user doesn’t use the switch, the value for this will be the empty string:
...
definition.add_option(
name='file',
default='',
doc='Filename that contains our text',
short_form='f'
)
Excellent! The whole thing together looks like this now:
from __future__ import absolute_import, division, print_function
from configman import Namespace, ConfigurationManager
def backwards(x, capitalize=False):
return x[::-1]
import re
vowels_regex = re.compile('[AEIOUY]', re.IGNORECASE)
def devowel(x):
return vowels_regex.sub('', x)
def define_config():
definition = Namespace()
definition.add_option(
name='devowel',
default=False,
doc='Removes all vowels (including Y)',
short_form='d'
)
definition.add_option(
name='file',
default='',
doc='file name for the input text',
short_form='f'
)
return definition
if __name__ == '__main__':
definition = define_config()
config_manager = ConfigurationManager(definition)
config = config_manager.get_config()
if config.file:
with open(config.file) as f:
output_string = f.read().strip()
else:
output_string = ' '.join(config_manager.args)
if config.devowel:
output_string = devowel(output_string)
print(backwards(output_string))
And it’s executed like this:
$ cat > foo.txt
Peter works for Mozilla.^d
$ ./tutorial03.py --file foo.txt
.allizoM rof skrow reteP
$ ./tutorial03.py --file foo.txt -d
.llzM rf skrw rtP
Persistent config files¶
Our examples so far have been very much about the command line. The whole
point of using configman
is so you can use various config file formats
to provide configuration information to your programs. The real power of
configman
isn’t to wrap executable command line scripts but its ability
to work ecumenically with config files.
To get started, let’s have our program itself write a configuration file
for us. The easiest way is to use the --admin.dump_conf
option that is
automatically available. It offers different ways to output.
ini
conf
json
xml
(future enhancement, if requested)
Let’s, for the sake of this tutorial, decide to use .ini
files:
$ ./tutorial03.py --admin.dump_conf=./backwards.ini
This will write out a default configation file in ini
format.
configman
figured that out by the file extension that you specified. If
you had used ‘json’ instead, configman
would have written out a json file:
$ ./tutorial03.py --admin.dump_conf=./backwards.ini
$ cat backwards.ini
[top_level]
# name: devowel
# doc: Removes all vowels (including Y)
devowel=False
# name: file
# doc: Filename that contains our text
file=
Any of the command line switches that you specify along with the
--dump_conf
switch will appear as the new defaults in the config file
that is written:
$ python backwards.py --admin.dump_conf=./backwards.ini --file=/tmp/foo.txt
$ cat backwards.ini
[top_level]
# name: devowel
# doc: Removes all vowels (including Y)
devowel=False
# name: file
# doc: Filename that contains our text
file=/tmp/foo.txt
Next, let’s make our app always read from this file to get its defaults. To do
that, we’re going to modify what is known as the hierarchy of value sources.
configman
, when determining what values to give to your option definitions,
uses a list of sources. By default, it first checks the operating system
environment. If the names of your options match anything from the environment,
configman
will pull those values in, overriding any defaults that you
specified. Next it looks to the command line. Any values that it fetches
will override the defaults as well as the environment variables.
If this default hierarchy of value sources doesn’t suit you, you may specify your own hierarchy. In our example, we’re going to want our configuration file to be the base value source. Then we want the environment variables and finally the command line. We can specify it like this:
value_sources = ('./backwards.ini', os.environ, getopt)
configman
will walk this list, applying the values that it finds in turn.
First it will read your ini file (you may want to use an absolute path to
specify your ini file name). Second, we pass in a dict that represents the
operating system environment. Interestingly, you can use any dict-like object
that you want as a source. Third, we’re telling configman
to use the
getopt
module to read the command line. In the future, we’ll have the
argparse
module available here.
To use this value source, we must specify it in the constructor:
config_manager = ConfigurationManager(definition,
values_source_list=value_sources)
Now, the program will read from the ./backwards.ini
config file whenever
the application is run.
Suppose we change the last line of the file backwards.ini
to
instead say:
file=/tmp/bar.txt
And then create that file like this:
$ echo "Socorro" > /tmp/bar.txt
Now, our little program is completely self-sufficient:
$ ./tutorial04.py
orrocoS
Even though we’re using a config file, that doesn’t mean that we’ve eliminated the use of the command line. You can override any configuration parameter from command line:
$ ./tutorial04.py --devowel
rrcS
$ ./tutorial04.py We both work at Mozilla --file=
allizoM ta krow htob eW
More advanced options¶
We just covered how to turn a simple application to one where the
configuration is done entirely by a ini
file. Note: we could have
chosen json
or conf
instead of ini
and the program would
be completely unchanged. Only your taste of config file format
changed.