Rev 6347: (vila) Simple explanations about how to use the new config options. (Bazaar in file:///srv/pqm.bazaar-vcs.org/archives/thelove/bzr/%2Btrunk/
Patch Queue Manager
pqm at pqm.ubuntu.com
Tue Dec 6 15:39:04 UTC 2011
At file:///srv/pqm.bazaar-vcs.org/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 6347 [merge]
revision-id: pqm at pqm.ubuntu.com-20111206153904-iclcpkamx0521tdx
parent: pqm at pqm.ubuntu.com-20111206114655-fknmk11zmytthy6r
parent: v.ladeuil+lp at free.fr-20111206151304-igt3sxq09xfad74r
committer: Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Tue 2011-12-06 15:39:04 +0000
message:
(vila) Simple explanations about how to use the new config options. (Bazaar
Developers)
added:
doc/developers/new-config-rationale.txt newconfigrationale.t-20111201114009-5dp0y2jb450ct9j5-1
modified:
doc/developers/configuration.txt configuration.txt-20110408142435-korjxxnskvq44sta-1
doc/developers/plans.txt plans.txt-20090907074210-zula633ky077z6vu-8
=== modified file 'doc/developers/configuration.txt'
--- a/doc/developers/configuration.txt 2011-11-14 16:36:31 +0000
+++ b/doc/developers/configuration.txt 2011-12-01 11:40:05 +0000
@@ -1,12 +1,46 @@
Configuring Bazaar
==================
-A configuration option has:
-
-* a name: a valid python identifier (even if it's not used as an
- identifier in python itself)
-
-* a value: a unicode string or a list of unicode strings.
+.. contents::
+ :depth: 2
+
+
+The short story
+---------------
+
+As a Bazaar developer there are three things you need to know about
+configuration.
+
+1. Get a value.
+
+You construct or get a reference to a ConfigStack subclass that's relevant
+to the context where you want to look up a value. For instance, given a
+branch::
+
+ print branch.get_config_stack().get('log_format')
+
+This will look up the stack through all relevant configuration sources.
+The value returned is of the type declared for that Option and if nothing
+is specifically declared you will get the default for that option.
+
+2. Add a new option.
+
+You add a new ``Option`` to the ``option_registry``, either inside
+``bzrlib/config.py`` or during initialization of your plugin. New plugins
+should have systematic hierarchical names so that related values are grouped
+together::
+
+ option_registry.register(
+ Option('dirstate.fdatasync', default=True,
+ from_unicode=bool_from_store,
+ help="Flush dirstate changes onto physical disk? ...."))
+
+
+3. Old and new configuration code.
+
+There is (as of late 2011) some older and some newer configuration code. The
+old code has specific methods for various checks or uses classes like
+``GlobalConfig``. Don't add to to it; try to remove it.
Option
------
@@ -16,6 +50,10 @@
* name: a name: a valid python identifier (even if it's not used as an
identifier in python itself). This is also used to register the option.
+* from_unicode: a callable accepting a unicode string and returning a
+ suitable value for the option. If the string cannot be coerced it should
+ return None.
+
* default: the default value that Stack.get() should return if no
value can be found for the option.
@@ -25,15 +63,17 @@
* help: a doc string describing the option, the first line should be a
summary and can be followed by a blank line and a more detailed
- explanation.
+ explanation. This will be displayed to the user with::
-* from_unicode: a callable accepting a unicode string and returning a
- suitable value for the option. If the string cannot be coerced it should
- return None.
+ bzr help <option name>
* invalid: the action to be taken when an invalid value is encountered in a
store (during a Stack.get()).
+The value of an option is a unicode string or ``None`` if it's not
+defined. By using ``from_unicode`` you can turn this string into a more
+appropriate representation (a list of unicode strings for example).
+
Sections
--------
@@ -119,8 +159,8 @@
Stacks
------
-An option can take different values depending on the context it is used. Such
-a context can involve configuration files, options from the command line,
+An option can take different values depending on the context it is
+used. This can involve configuration files, options from the command line,
default values in bzrlib and then some.
Such a context is implemented by creating a list of ``Section`` stacked upon
=== added file 'doc/developers/new-config-rationale.txt'
--- a/doc/developers/new-config-rationale.txt 1970-01-01 00:00:00 +0000
+++ b/doc/developers/new-config-rationale.txt 2011-12-06 15:13:04 +0000
@@ -0,0 +1,778 @@
+================================
+Simplifying Bazaar Configuration
+================================
+
+Goal
+====
+
+Not all needs can be addressed by the default values used inside bzr and
+bzrlib, no matter how well they are chosen (and they are ;).
+
+Options that are rarely used don't deserve a corresponding command line
+switch in one or several commands.
+
+Many parts of ``bzrlib`` depends on some constants though and the user
+should be able to customize the behavior to suit his needs so these
+constants need to become configuration options or more generally, be easier
+to set.
+
+These options can be set from the command-line, acquired from an environment
+variable or recorded in a configuration file.
+
+To simplify writing (and reading), this document refers to the old and new
+config designs:
+* the old design is using ``Config`` as a base class for all config files,
+* the new design use ``ConfigStacks`` of ``Section`` from config ``Stores``.
+
+
+Current issues
+==============
+
+* Many parts of ``bzrlib`` declare constants and there is no way for the
+ user to look at or modify them (see http://pad.lv/832061).
+
+* The old design requires a configuration object to create, modify or delete
+ a configuration option in a given configuration file. ``bzr config``
+ makes it almost transparent for the user. Internally though, not all cases
+ are handled: only BranchConfig implements chained configs, nothing is
+ provided at the repository level and too many plugins define their own
+ section or even their own config file. (config.Stack now provides a way to
+ chain config files, BranchStack properly implements the desired behavior,
+ ``bzr config`` uses the new design).
+
+* ``locations.conf`` defines the options that need to override any setting
+ in ``branch.conf`` for both local and remotes branches (but some remote
+ branch options just ignore ``locations.conf``). Many users want a way to
+ define default values for options that are not defined in ``branch.conf``
+ (and even more users think that ``locations.conf`` provide default values,
+ see also http://pad.lv/843211 and http://pad.lv/832046). This could be
+ approximated today by *not* defining these options in ``branch.conf`` but
+ in ``locations.conf`` instead. This workaround doesn't allow a user to
+ define defaults in ``locations.conf`` and override them in
+ ``branch.conf``. (Allowing sections in 'bazaar.conf' (or introducing a new
+ defaults.conf' allowing sections) can now address that. Defining and using
+ a new file is easier as it avoids handling a migration path for
+ bazaar.conf and doesn't require banning the use of sections for special
+ purpose needs (ALIASES and BOOKMARKS for example)).
+
+* Defining a new option requires adding a new method in the ``Config``
+ object to get access to features like:
+
+ * should the option be inherited by more specific sections, (this was more
+ or less the default in the old design, it is addressed by section
+ matchers in the new one).
+
+ * should the inherited value append the relative path between the
+ section one and the location it applies to (see http://pad.lv/832013),
+
+ * the default value (including calling any python code that may be
+ required to calculate this value)(see http://pad.lv/832064),
+
+ * priority between sections in various config files (this is defined by
+ the section matcher associated with a given config store for stacks,
+ http://pad.lv/832046 is about adding a section matcher with clearer
+ semantics than the one used for locations.conf).
+
+ A related problem is that, in the actual implementation, some
+ configuration options have defined methods, others don't and this is
+ inconsistent. (Using only Stacks addresses that).
+
+* Access to the 'active' configuration option value from the command line
+ doesn't give access to the specific section. (This is only a concern if
+ the user has no other way to address a specific configuration option
+ including Store and Section when using ``bzr config``) (see http://pad.lv/725234).
+
+* Rules for configuration options are not clearly defined for remote
+ branches (they may differ between dumb and smart servers the former will
+ use the local ``bazaar.conf`` and ``locations.conf`` files while the later
+ will use (or ignore ?) the remote ones).
+
+* The features offered by the Bazaar configuration files should be easily
+ accessible to plugin authors either by supporting plugin configuration
+ options in the configuration files or allowing the plugins to define their
+ own configuration files. (Separating Section, Store and Stack starts
+ addressing that, a stack registry should complete the needed means).
+
+* While the actual configuration files support sections, they are used in
+ mutually exclusive ways that make it impossible to offer the same set of
+ features to all configuration files:
+
+ * ``bazaar.conf`` use arbitrary names for sections. ``DEFAULT`` is used
+ for global options, ``ALIASES`` are used to define command aliases,
+ plugins can define their own sections, some plugins do that
+ (``bzr-bookmarks`` use ``BOOKMARKS`` for example), some other define
+ their own sections (this is addressed with the new design by using only
+ the ``DEFAULT`` section and ignore the others. When needed, one can
+ create a specific stack to get access to a specific section).
+
+ * ``locations.conf`` use globs as section names. This provides an easy
+ way to associate a set of options to a matching working tree or
+ branch, including remote ones.
+
+ * ``branch.conf`` doesn't use any section.
+
+* There is no easy way to get configuration options for a given repository
+ or an arbitrary path. Working trees and branches are generally organized
+ in hierarchies and being able to share the option definitions is an often
+ required feature. This can also address some needs exhibited by various
+ branch schemes like looms, pipeline, colocated branches and nested
+ trees. Being able to specify options *in* a working tree could also help
+ support conflict resolution options for a given file, directory or
+ subtree (see http://pad.lv/359320).
+
+* Since sections allow different definitions for the same option, a total
+ order should be defined between sections to select the right definition
+ for a given context (paths or globs for ``locations.conf`` but other
+ schemes can be used, window names for qbzr for example). Allowing globs
+ for section names is harmful in this respect since the order is currently
+ defined as being the lexicographical one. The caveat here is that if the
+ order is always defined for a given set of sections it can change when one
+ or several globs are modified and the user may get surprising and unwanted
+ results in these cases. The lexicographical order is otherwise fine to
+ define what section is more specific than another. (This may not be a
+ problem in real life since longer globs are generally more specific than
+ shorter ones and explicit paths should also be longer than matching
+ globs. That may leave a glob and a path of equal length in a gray area but
+ in practice using ``bzr config`` should give enough feedback to address
+ them. See also http://pad.lv/832046 asking for a less magical section matcher).
+
+* Internally, configuration files (and their fallbacks, ``bazaar.conf`` and
+ ``locations.conf`` for ``branch.conf``) are read every time *one* option is
+ queried. Likewise, setting or deleting a configuration option implies
+ writing the configuration file *immediately* after re-reading the file to
+ avoid racing updates (see http://pad.lv/832042).
+
+* The current implementation use a mix of transport-based and direct file
+ systems operations (Addressed by Store implementation relying on
+ transports only).
+
+* While the underlying ``ConfigObj`` implementation provides an
+ interpolation feature, the ``bzrlib`` implementation doesn't provide an
+ easy handling of templates where other configuration options can be
+ interpolated. Instead, ``locations.conf`` (and only it) allows for
+ ``appendpath`` and ``norecurse``. (Partially implemented, cross-section
+ and cross-file interpolation still to be implemented, see
+ http://pad.lv/832013 for the remaining parts).
+
+* Inherited list values can't be modified, a more specific configuration can
+ only redefine the whole list.
+
+* There is no easy way to define dicts (the most obvious one being to use a
+ dedicated section which is already overloaded). Using embedded sections
+ for this would not be practical either if we keep using a no-name section
+ for default values. In a few known cases, a bencoded dict is stored in a
+ config value, so while this isn't user-friendly, not providing a better
+ alternative shouldn't be a concern. A possible, limited, implementation
+ can be envisioned: limiting the dict to a single level only, with simple
+ names as keys and unicode strings as values. The keys can then be mapped
+ to options prefixed with the dict name.
+
+
+Proposed implementation
+=======================
+
+
+Configuration files definition
+------------------------------
+
+While of course configurations files can be versioned they are not intended
+to be accessed in sync with the files they refer to (one can imagine
+handling versioned properties this way but this is *not* what the bazaar
+configuration files are targeted at). ``bzr`` will always refer to
+configuration files as they exist on disk when an option is queried or set.
+
+The configuration files are generally local to the file system but some of
+them can be accessed remotely (``branch.conf``, ``repo.conf``).
+
+
+Naming
+------
+
+Option names are organized into a name space for a given configuration file
+(or a set of related configuration files). One such set includes
+``bazaar.conf``, ``locations.conf``, ``branch.conf``, etc. Plugins can
+define their own sets for their own needs.
+
+Using a name space is meant to help:
+
+* avoid collisions between bzr and plugins and between plugins,
+
+* discover the available options and making them easier to remember,
+
+* organise the documentation for the option set.
+
+Using valid python identifiers is recommended but not enforced (but we may
+do so in the future).
+
+The option name space is organized by topic:
+
+* bzrlib options are grouped by topic (``branch``, ``tree``, ``repo``)
+
+* plugins are encouraged (but not required) to prefix their specific options
+ with their name (``qbzr.`` for qbzr)
+
+* collisions are detected at registration time so users are protected from
+ incompatibilities between plugins,
+
+* options that need to be used by several plugins (or shared between ``bzr``
+ core and plugins) should be discussed but these discussions are already
+ happening so the risk of misuse is low enough.
+
+Value
+-----
+
+All option values are text. They are provided as Unicode strings to API
+users with some refinements:
+
+* boolean values can be obtained for a set of acceptable strings (yes/no,
+ y/n, on/off, etc), (implemented with the ``from_unicode`` parameter)
+
+* a list of strings from a value containing a comma separated list of
+ strings.
+
+Since the configuration files can be edited by the user, ``bzr`` doesn't
+expect their content to be valid at all times. Instead, the code using
+options should be ready to handle *invalid* values by warning the user and
+falling back to a default value.
+
+Likely, if an option is not defined in any configuration file, the code
+should fallback to a default value (helpers should be provided by the API to
+handle common cases: warning the user, getting a particular type of value,
+returning a default value)(most of that is now handled at Option definition).
+
+This also ensures compatibility with values provided via environment
+variables or from the command line (where no validation can be expected
+either)(done in the new design, some cases missing, see http://pad.lv/832064).
+
+
+Option expansion
+----------------
+
+Some option values can be templates and contain references to other
+options. This is especially useful to define URLs in sections shared for
+multiple branches for example. It can also be used to describe commands
+where some parameters are set by ``bzrlib`` at runtime.
+
+Since option values are text-only, and to avoid clashing with other option
+expansion (also known as interpolation) syntaxes, references are enclosed
+with curly brackets::
+
+ push_location = lp:~{launchpad_username}/bzr/{nick}
+
+In the example above, ``launchpad_username`` is an already defined
+configuration option while ``nick`` is the branch nickname and is set when a
+configuration applies to a given branch.
+
+The interpolation implementation should accept an additional dict so that
+``bzrlib`` or plugins can define references that can be expanded without
+being existing configuration options::
+
+ diff_command={cmd} {cmd_opts} {file_a} {file_b}
+
+There are two common errors that should be handled when handling interpolation:
+
+* loops: when a configuration value refers to itself, directly or indirectly,
+
+* undefined references: when a configuration value refers to an unknown option.
+
+The loop handling can be modified to allow cross-sections and cross-files
+interpolation: if an option refers to itself (directly or indirectly) during
+an expansion, the fallback sections or files can be queried for its value.
+
+This allows list values to refer to the definition in the less specific
+configurations::
+
+ bazaar.conf:
+ debug_flags = hpss
+
+ branch.conf for mybranch:
+ debug_flags = {debug_flags}, hpssdetail
+
+ $ bzr -d mybranch config debug_flags
+ hpss, hpssdetail
+
+Undefined references are detected if they are not defined in any
+configuration. This will trigger errors while displaying the value. Diagnosing
+typos should be doable in this case.
+
+Configuration file syntax
+-------------------------
+
+The configuration file is mostly an ``ini-file``. It contains ``name = value``
+lines grouped in sections. A section starts with a string enclosed in squared
+brackets ('[section_name]`), this string uniquely identifies the section in
+the file. Comments are allowed by prefixing them with the '#' character.
+
+A section is named by the path (or some other unuique identifier) it should
+apply to (more examples below).
+
+When sections are used, they provide a finer grain of configuration by
+defining option sets that apply to some working trees, branches,
+repositories (or any kind of context) or part of them. The relationship
+between a given context and the sections it applies to is defined by the
+config file.
+
+So far, Bazaar uses a glob in ``locations.conf`` and select the sections
+that apply to a given url (or a local path).
+
+The subset is defined by the common leading path or a glob.
+
+Different kinds of section names can be defined:
+
+* a full url: used to described options for remote branches and
+ repositories (LocationMatcher supports this).
+
+* local absolute path: used for working trees, branches or repositories
+ on the local disks (LocationMatcher supports this).
+
+* relative path: the path is relative to the configuration file and can be
+ used for colocated branches or threads in a loom, i.e any working tree,
+ branch or repository that is located in a place related to the
+ configuration file path. Some configuration files may define this path
+ relationship in specific ways to make them easier to use (i.e. if a config
+ file is somewhere below ``.bzr`` and refers to threads in a loom for
+ example, the relative path would be the thread name, it doesn't have to be
+ an *exact* relative path, as long as its interpretation is unambiguous and
+ clear for the user) (No section matchers support this so far, needs to
+ file a bug)
+
+Section matching
+----------------
+
+Section names define another name space (than the option names one) with an
+entirely different purpose: in a given configuration file, for a given
+context, only some sections will be relevant and will be queried in a
+specific order.
+
+This matching is specific to each config file and implemented by the
+SectionMatcher objects.
+
+Whatever this matching does, the user can observe the results with the ``bzr
+config`` command which displays the sections in the order they are queried.
+
+LocationMatcher
+~~~~~~~~~~~~~~~
+
+The context here is either:
+
+* an URL,
+
+* a local path.
+
+Note that for both the provided context and the section names, if an URL uses
+a ``file:///`` form, it is converted to a local path.
+
+The sections names can use globs for each path component
+(i.e. ``/dir/*/subdir`` is allowed but ``/dir/\*\*/subdir`` will match
+``/dir/a/subdir`` but not ``/dir/a/b/subdir``.
+
+The reason is that the ordering is defined by sorting the section names
+matching the context on the number of path components followed by the path
+itself in lexicographical order. This results in most specific sections being
+queried before the more generic ones.
+
+PathMatcher
+~~~~~~~~~~~
+
+``LocationMatcher`` has some obscure (for unaware users) edge cases and
+limitations that can be surprising. ``PathMatcher`` aims at addressing these
+issues by providing simpler rules while still giving full control to the
+user (http://pad.lv/832046).
+
+The context here is a local path, absolute or relative. If the path is
+relative it is interpreted from the file base directory.
+
+Note that 'base directory' for configuration files in Bazaar directories is
+really:
+
+* the home directory for files under ``~/.bazaar``,
+
+* the ``.bzr`` base directory for files under ``.bzr``.
+
+The order is the one observed in the file so most generic values are specified
+first and most specific ones last. As such, the order in the file is the
+opposite of the one displayed by ``bzr config`` which displays most specific
+values first. This seems to be the most natural order in both cases.
+
+A section matches if the section name is a prefix of the context path
+(relative paths being converted to absolute on the fly).
+
+The Option object
+-----------------
+
+(copied from a recent version of bzr.dev for easier reading, refer to the
+original for an up to date version)
+
+The Option object is used to define its properties:
+
+* name: a name: a valid python identifier (even if it's not used as an
+ identifier in python itself). This is also used to register the option.
+
+* default: the default value that Stack.get() should return if no
+ value can be found for the option.
+
+* default_from_env: a list of environment variables. The first variable set
+ will provide a default value overriding 'default' which remains the
+ default value if *no* environment variable is set.
+
+* help: a doc string describing the option, the first line should be a
+ summary and can be followed by a blank line and a more detailed
+ explanation.
+
+* from_unicode: a callable accepting a unicode string and returning a
+ suitable value for the option. If the string cannot be coerced it should
+ return None.
+
+* invalid: the action to be taken when an invalid value is encountered in a
+ store (during a Stack.get()).
+
+The Section object
+------------------
+
+Options are grouped into sections which share some properties with the well
+known dict objects:
+
+* the key is the name,
+* you can get, set and remove an option,
+* the value is a unicode string.
+
+MutableSection is needed to set or remove an option, ReadOnlySection should
+be used otherwise.
+
+The Store object
+----------------
+
+This is an implementation-level object that should rarely be used directly.
+
+* it can be local or remote
+
+* locking
+
+ All lock operations should be implemented via transport objects. (True for
+ Store).
+
+* option life cycle
+
+ Working trees, branches and repositories should define a config attribute
+ following the same life cycle as their lock: the associated config file is
+ read once and written once if needed. This should minimize the file system
+ accesses or the network requests. There is no known racing scenarios for
+ configuration options, changing the existing implementation to this less
+ constrained one shouldn't introduce any. Yet, in order to detect such
+ racing scenarios, we can add a check that the current content of the
+ configuration file is the expected one before writing the new content and
+ emit warnings if differences occur. The checks should be performed for the
+ modified values only. As of today (and in the foreseeable future), the
+ size of the configuration files are small enough to be kept in memory (see
+ http://pad.lv/832042).
+
+The Stack object
+-----------------
+
+This the object that provides access to the needed features:
+
+* getting an option value,
+
+* setting an option value,
+
+* deleting an option value,
+
+* handling a list of configuration files and for each of them a section
+ matcher providing the sections that should be tried in the given order to
+ find an option.
+
+* handling a Store and a section where option creation, modification and
+ deletion will occur.
+
+Depending on the files involved, a working tree, branch or repository object
+(or more generally a context) should be provided to access the corresponding
+configuration files. Note that providing a working tree object also
+implicitly provides the associated branch and repository object so only one
+of them is required (or none for configuration files specific to the user
+like ``bazaar.conf``).
+
+Getting an option value
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Depending on the option, there are various places where it can be defined
+and several ways to override these settings when needed.
+
+The following lists all possible places where a configuration option can
+be defined, but some options will make sense in only some of them. The
+first to define a value for an option wins (None is therefore used to
+express that an option is not set).
+
+* command-line
+ ``-Ooption=value`` see http://pad.lv/491196.
+
+* ``~/.bazaar/locations.conf``
+
+ When an option is set in ``locations.conf`` it overrides any other
+ configuration file. This should be used with care as it allows setting a
+ different value than what is recommended by the project
+
+* ``tree`` (Not Implemented Yet)
+
+ The options related to the working tree.
+
+ This includes all options related to commits, ignored files, junk files,
+ etc.
+
+ Note that the sections defined there can use relative paths if some
+ options should apply to a subtree or some specific files only.
+
+ See http://pad.lv/430538 and http://pad.lv/654998.
+
+* ``branch`` located in ``.bzr/branch/branch.conf``
+
+ The options related to the branch.
+
+ Sections can be defined for colocated branches or loom threads.
+
+* ``repository`` (Not Implemented Yet)
+
+ The options related to the repository.
+
+ Using an option to describe whether or not a repository is shared could
+ help address http://pad.lv/342119 but this will probably requires a format
+ bump).
+
+* ``project`` (Not Implemented Yet)
+
+ The options common to all branches and working trees for a project.
+
+* ``organization`` (Not Implemented Yet)
+
+ The options common to all branches and working trees for an organization.
+
+ See http://pad.lv/419854.
+
+* ``system`` (Not Implemented Yet but see http://pad.lv/419854 and
+ https://code.launchpad.net/~thomir/bzr/add-global-config/+merge/69592)
+
+ The options common to all users of a system (may be /etc/bzr/defaults
+ or /usr/local/etc/bzr/defaults or
+ /Library/Preferences/com.canonical.defaults or c:\windows\bazaar.conf
+ (someone fix this one please ;) depending on the OS).
+
+* ``bazaar.conf``
+
+ The options the user has selected for the host he is using.
+
+ Sections can be defined for both remote and local branches to define
+ default values (i.e. the most common use of ``locations.conf`` today).
+
+* default (implemented by the OptionRegistry)
+
+ The options defined in the ``bzr`` source code.
+
+ This will be implemented via the Option objects.
+
+Plugins can define additional configuration files as they see fit and
+insert them in this list, see their documentation for details.
+
+Compatibility
+=============
+
+There are ways to keep the same files while ensuring compatibility via various
+tricks but there are cases where using new files to replace the old ones is
+definitely easier:
+
+* no need to ensure that the new files are correctly handled by old bzr
+ versions,
+
+* making it clear for users that there is a switch and let them migrate at
+ their own pace.
+
+The known cases so far are described below.
+
+Obvious at this point:
+
+* Branch provides ``get_config`` for the old design and ``get_config_stack``
+ for the new design so that both designs are supported. Once the store
+ sharing is implemented, we may want to use an attribute for the stack and
+ deprecate both ``get_config`` and ``get_config_stack``.
+
+* Sections names in ``bazaar.conf`` are arbitrary (except ``DEFAULT``) so
+ it's easier to leave the file untouched and let plugin authors and users
+ migrate away (or not) from them. For ``bzr`` itself, that means
+ ``DEFAULT`` is the only section used for most of the options and provides
+ user defaults. ``ALIASES`` requires a specific stack but only the ``bzr
+ alias`` command cares about that.
+
+* Option policies should be deprecated:
+
+ * The ``norecurse`` policy is useless, all options are recursive by
+ default. If specific values are needed for specific paths, they can just
+ be defined as such (in the appropriate sections or files).
+
+ * The ``appendpath`` policy should be implemented via interpolation and a
+ ``relpath`` option provided by the configuration framework
+ (http://pad.lv/832013).
+
+* Section order in ``locations.conf`` has issues which make a migration to a
+ different way to organize the sections (hence the file content) far easier
+ with a new file.
+
+* ``locations.conf`` is really for overrides but many users have been using it
+ to provide defaults. There is no way to know if the whole content has been
+ used for defaults or overrides or a mix of both. So there is no way to
+ migrate this automatically.
+
+Unclear at this point:
+
+* [BOOKMARKS] section can be replaced by ``bookmarks.xxx`` options (the
+ bookmarks plugins already uses ``bookmarks_xxx`` in branch.conf since no
+ sections were supported there). The easiest here is probably to just merge
+ the plugin into core and use the appropriate option names consistently. A
+ ``config:`` directory service may even be better as any option can be used
+ as a bookmark. This allows things like::
+
+ [/whatever/path]
+ my_push = lp:<launchpad.login>/xxx/{nick}
+ web_site=ftp://example.com/
+
+ bzr push config:web_site
+
+ Which means we completely replace the plugin and don't need to care about
+ migrating the section.
+
+* [ALIASES] section can be replaced by corresponding bzr.alias.xxx
+ options. This could be automated by creating the corresponding options ?
+
+* I don't know about other sections, feedback welcome. Plugin authors are
+ encouraged to migrate to the new name space scheme by prefixing their
+ options with their plugin name.
+
+Notes
+=====
+
+These are random notes about concepts, ideas or issues not implemented yet.
+
+Developer facing concepts
+-------------------------
+
+Option
+~~~~~~
+
+* list of allowed Config IDs (this allows a list of possible config files in
+ bazaar.conf only option and use it while bootstrapping the config
+ creations).
+
+* blacklist of config IDs (some options *can't* be stored (modified) by the
+ user)
+
+An alternative is to just let the devs decide which stack they use for a
+given option, ``stacked_on_location`` for example is said to relate to the
+branch only and changing it or setting it in a different config file may not
+be appropriate. This may not be a good example as there is also the
+``default_stack_on`` option which can be set only in ``control.conf``
+though...
+
+Stack
+~~~~~
+
+* a lazy cache for the option values (should be reset on modifications as
+ interpolations will make it tricky to update incrementally) (see FIXME in
+ config.py Stack.get()))
+
+* ensures that the Stores involved generate as less IOs as possible (see
+ http://pad.lv/832042)
+
+* ensures that the transaction is the object life time (i.e. modifications
+ will be taken into account *iff* they are committed explicitly).
+
+StackRegistry
+~~~~~~~~~~~~~
+
+* ensures that a config ID is a unique identifier
+* register Stacks
+
+Store
+~~~~~
+
+* ensures that the transaction is the object life time (i.e. modifications
+ will be taken into account *iff* they are committed explicitly).
+
+Examples
+--------
+
+store examples:
+~~~~~~~~~~~~~~~
+
+* ConfigObj (bazaar.conf)
+
+* DB (<scheme>://bazaar.launchpad.net/bazaar.conf)
+
+
+Why and when locking config files matter
+----------------------------------------
+
+This is relevant for http://pad.lv/832042.
+
+``bzr`` behavior, as well as the objects it acts upon, is configured via a
+set of so-called configuration files.
+
+These files allow to define working trees, branches and repositories, their
+relationships and how ``bzr`` should handle them.
+
+The default behavior of ``bzr`` is aimed at making this configuration as
+transparent as possible by keeping track of how these objects are created
+and modified when they are used. In short, they are useless until you want
+to change the default behavior in some specific context.
+
+We mostly **read** config options. Therefore all we care about is to
+guarantee that:
+
+* we get a valid config file at all times when reading,
+
+* we always leave a valid config file when writing (via the rename dance)
+
+From there, conceptually, all operations can clearly define whether or not
+they need to modify a config file and do so only when they succeed. All
+modifications occurring during such an operation are delayed until the very
+end of the operation.
+
+Now, we want to minimize the overlapping times where one bzr operation has
+changed a value and another concurrent operation is unaware of this
+modification.
+
+These overlapping periods are *as of today* rare.
+
+The only known case, http://pad.lv/525571 has been fixed in bzr-2.1.3. The
+bug there was triggered when two processes tried to write the same config
+file at the same time leaving an invalid file in the end.
+
+Such a period can be recognized and detected though: when changing an option
+value, if the preserved original value is different in the config file,
+someone else modified it and the operation can be invalid because it relied
+on the original value.
+
+For the sake of the example, if an option value represent a global unique ID
+via a simple counter (very bad idea), if two operations try to increment it,
+both will use the same value that won't be unique anymore. Checking the
+value present in the file when trying to save the updated value with
+identify such a collision.
+
+An assumption is floating around: it should be enough to report when an
+operation is modifying an already modified option and observe that no-one
+reports such occurrences.
+
+Note that this assumption is made in a context where *no* known scenarios
+exist in the bzr code base not in any plugin (for a best effort value of
+'any', feedback highly welcome, bug reports even ;)
+
+With this in mind, we can change the definition of config options, stores
+and stacks to ensure that:
+
+* a config file is read only once when in read access,
+
+* a config file is read only once and written only once when in write
+ access, adding the check mentioned above will require *one* additional
+ read.
+
+A reader can then safely assume that reading a config file gives it a valid
+(and coherent) definition of the configuration when the operation
+starts. All the operation has to do is to declare which config files may be
+modified by an operation (whether or not we can be liberal on this 'may be'
+is yet to be defined).
=== modified file 'doc/developers/plans.txt'
--- a/doc/developers/plans.txt 2011-10-14 01:18:16 +0000
+++ b/doc/developers/plans.txt 2011-12-01 11:41:07 +0000
@@ -2,29 +2,12 @@
=====
.. toctree::
- :hidden:
+ :maxdepth: 1
- performance-roadmap
+ Performance roadmap <performance-roadmap>
colocated-branches
feature-flags
+ new-config-rationale
tortoise-strategy
improved_chk_index
-
-* `Performance roadmap <performance-roadmap.html>`_ |--| The roadmap
- for fixing performance in bzr over the next few releases.
-
-* `Co-located branches <colocated-branches.html>`_ |--| Planned(?) support
- for storing multiple branches in one file-system directory.
-
-* `Bazaar Windows Shell Extension Options <tortoise-strategy.html>`_ |--|
- Implementation strategy for Bazaar Windows Shell Extensions, aka
- TortoiseBzr.
-
-* `CHK Optimized index <improved_chk_index.html>`_
-
-
-.. |--| unicode:: U+2014
-
-..
- vim: ft=rst tw=74 ai
More information about the bazaar-commits
mailing list