Rev 6336: Add the devnotes/configurations.txt as doc/developers/new-config-rationale.txt with minimal edits to use http:/pad.lv to reference bugs. in file:///home/vila/src/bzr/reviews/doc/

Vincent Ladeuil v.ladeuil+lp at free.fr
Thu Dec 1 11:41:07 UTC 2011


At file:///home/vila/src/bzr/reviews/doc/

------------------------------------------------------------
revno: 6336
revision-id: v.ladeuil+lp at free.fr-20111201114107-bfjulsggilgm6iei
parent: v.ladeuil+lp at free.fr-20111201114005-4ctbmk5nnzoqeh2b
committer: Vincent Ladeuil <v.ladeuil+lp at free.fr>
branch nick: doc
timestamp: Thu 2011-12-01 12:41:07 +0100
message:
  Add the devnotes/configurations.txt as doc/developers/new-config-rationale.txt with minimal edits to use http:/pad.lv to reference bugs.
-------------- next part --------------
=== added file 'doc/developers/new-config-rationale.txt'
--- a/doc/developers/new-config-rationale.txt	1970-01-01 00:00:00 +0000
+++ b/doc/developers/new-config-rationale.txt	2011-12-01 11:41:07 +0000
@@ -0,0 +1,778 @@
+================================
+Simplifying Bazaar Configuration
+================================
+
+Goal
+====
+
+Not all needs can be addressed by the default values used inside bzr and
+bzrlib, no matter how well they are chosen (and they are ;).
+
+Options that are rarely used don't deserve a corresponding command line
+switch in one or several commands.
+
+Many parts of ``bzrlib`` depends on some constants though and the user
+should be able to customize the behavior to suit his needs so these
+constants need to become configuration options or more generally, be easier
+to set.
+
+These options can be set from the command-line, acquired from an environment
+variable or recorded in a configuration file.
+
+To simplify writing (and reading), this document refers to the old and new
+config designs:
+* the old design is using ``Config`` as a base class for all config files,
+* the new design use ``ConfigStacks`` of ``Section`` from config ``Stores``.
+
+
+Current issues
+==============
+
+* Many parts of ``bzrlib`` declare constants and there is no way for the
+  user to look at or modify them (see http://pad.lv/832061).
+
+* The old design requires a configuration object to create, modify or delete
+  a configuration option in a given configuration file.  ``bzr config``
+  makes it almost transparent for the user. Internally though, not all cases
+  are handled: only BranchConfig implements chained configs, nothing is
+  provided at the repository level and too many plugins define their own
+  section or even their own config file. (config.Stack now provides a way to
+  chain config files, BranchStack properly implements the desired behavior,
+  ``bzr config uses the new design).
+
+* ``locations.conf`` defines the options that need to override any setting
+  in ``branch.conf`` for both local and remotes branches (but some remote
+  branch options just ignore ``locations.conf``). Many users want a way to
+  define default values for options that are not defined in ``branch.conf``
+  (and even more users think that ``locations.conf`` provide default values,
+  see also http://pad.lv/843211 and http://pad.lv/832046). This could be
+  approximated today by *not* defining these options in ``branch.conf`` but
+  in ``locations.conf`` instead. This workaround doesn't allow a user to
+  define defaults in ``locations.conf`` and override them in
+  ``branch.conf``. (Allowing sections in 'bazaar.conf' (or introducing a new
+  defaults.conf' allowing sections) can now address that. Defining and using
+  a new file is easier as it avoids handling a migration path for
+  bazaar.conf and doesn't require banning the use of sections for special
+  purpose needs (ALIASES and BOOKMARKS for example)).
+
+* Defining a new option requires adding a new method in the ``Config``
+  object to get access to features like:
+
+  * should the option be inherited by more specific sections, (this was more
+    or less the default in the old design, it is addressed by section
+    matchers in the new one).
+
+  * should the inherited value append the relative path between the
+    section one and the location it applies to (see http://pad.lv/832013),
+
+  * the default value (including calling any python code that may be
+    required to calculate this value)(see http://pad.lv/832064),
+
+  * priority between sections in various config files (this is defined by
+    the section matcher associated with a given config store for stacks,
+    http://pad.lv/832046 is about adding a section matcher with clearer
+    semantics than the one used for locations.conf).
+
+  A related problem is that, in the actual implementation, some
+  configuration options have defined methods, others don't and this is
+  inconsistent. (Using only Stacks addresses that).
+
+* Access to the 'active' configuration option value from the command line
+  doesn't give access to the specific section. (This is only a concern if
+  the user has no other way to address a specific configuration option
+  including Store and Section when using ``bzr config``) (see http://pad.lv/725234).
+
+* Rules for configuration options are not clearly defined for remote
+  branches (they may differ between dumb and smart servers the former will
+  use the local ``bazaar.conf`` and ``locations.conf`` files while the later
+  will use (or ignore ?) the remote ones).
+
+* The features offered by the Bazaar configuration files should be easily
+  accessible to plugin authors either by supporting plugin configuration
+  options in the configuration files or allowing the plugins to define their
+  own configuration files. (Separating Section, Store and Stack starts
+  addressing that, a stack registry should complete the needed means).
+
+* While the actual configuration files support sections, they are used in
+  mutually exclusive ways that make it impossible to offer the same set of
+  features to all configuration files:
+
+  * ``bazaar.conf`` use arbitrary names for sections. ``DEFAULT`` is used
+    for global options, ``ALIASES`` are used to define command aliases,
+    plugins can define their own sections, some plugins do that
+    (``bzr-bookmarks`` use ``BOOKMARKS`` for example), some other define
+    their own sections (this is addressed with the new design by using only
+    the ``DEFAULT`` section and ignore the others. When needed, one can
+    create a specific stack to get access to a specific section).
+
+  * ``locations.conf`` use globs as section names. This provides an easy
+    way to associate a set of options to a matching working tree or
+    branch, including remote ones.
+
+  * ``branch.conf`` doesn't use any section.
+
+* There is no easy way to get configuration options for a given repository
+  or an arbitrary path. Working trees and branches are generally organized
+  in hierarchies and being able to share the option definitions is an often
+  required feature. This can also address some needs exhibited by various
+  branch schemes like looms, pipeline, colocated branches and nested
+  trees. Being able to specify options *in* a working tree could also help
+  support conflict resolution options for a given file, directory or
+  subtree (see http://pad.lv/359320).
+
+* Since sections allow different definitions for the same option, a total
+  order should be defined between sections to select the right definition
+  for a given context (paths or globs for ``locations.conf`` but other
+  schemes can be used, window names for qbzr for example). Allowing globs
+  for section names is harmful in this respect since the order is currently
+  defined as being the lexicographical one. The caveat here is that if the
+  order is always defined for a given set of sections it can change when one
+  or several globs are modified and the user may get surprising and unwanted
+  results in these cases. The lexicographical order is otherwise fine to
+  define what section is more specific than another. (This may not be a
+  problem in real life since longer globs are generally more specific than
+  shorter ones and explicit paths should also be longer than matching
+  globs. That may leave a glob and a path of equal length in a gray area but
+  in practice using ``bzr config`` should give enough feedback to address
+  them. See also http://pad.lv/832046 asking for a less magical section matcher).
+
+* Internally, configuration files (and their fallbacks, ``bazaar.conf`` and
+  ``locations.conf`` for ``branch.conf``) are read every time *one* option is
+  queried. Likewise, setting or deleting a configuration option implies
+  writing the configuration file *immediately* after re-reading the file to
+  avoid racing updates (see http://pad.lv/832042).
+
+* The current implementation use a mix of transport-based and direct file
+  systems operations (Addressed by Store implementation relying on
+  transports only).
+
+* While the underlying ``ConfigObj`` implementation provides an
+  interpolation feature, the ``bzrlib`` implementation doesn't provide an
+  easy handling of templates where other configuration options can be
+  interpolated. Instead, ``locations.conf`` (and only it) allows for
+  ``appendpath`` and ``norecurse``. (Partially implemented, cross-section
+  and cross-file interpolation still to be implemented, see
+  http://pad.lv/832013 for the remaining parts).
+
+* Inherited list values can't be modified, a more specific configuration can
+  only redefine the whole list.
+
+* There is no easy way to define dicts (the most obvious one being to use a
+  dedicated section which is already overloaded). Using embedded sections
+  for this would not be practical either if we keep using a no-name section
+  for default values. In a few known cases, a bencoded dict is stored in a
+  config value, so while this isn't user-friendly, not providing a better
+  alternative shouldn't be a concern. A possible, limited, implementation
+  can be envisioned: limiting the dict to a single level only, with simple
+  names as keys and unicode strings as values. The keys can then be mapped
+  to options prefixed with the dict name.
+
+
+Proposed implementation
+=======================
+
+
+Configuration files definition
+------------------------------
+
+While of course configurations files can be versioned they are not intended
+to be accessed in sync with the files they refer to (one can imagine
+handling versioned properties this way but this is *not* what the bazaar
+configuration files are targeted at). ``bzr`` will always refer to
+configuration files as they exist on disk when an option is queried or set.
+
+The configuration files are generally local to the file system but some of
+them can be accessed remotely (``branch.conf``, ``repo.conf``).
+
+
+Naming
+------
+
+Option names are organized into a name space for a given configuration file
+(or a set of related configuration files). One such set includes
+``bazaar.conf``, ``locations.conf``, ``branch.conf``, etc. Plugins can
+define their own sets for their own needs.
+
+Using a name space is meant to help:
+
+* avoid collisions between bzr and plugins and between plugins,
+
+* discover the available options and making them easier to remember,
+
+* organise the documentation for the option set.
+
+Using valid python identifiers is recommended but not enforced (but we may
+do so in the future).
+
+The option name space is organized by topic:
+
+* bzrlib options are grouped by topic (``branch``, ``tree``, ``repo``)
+
+* plugins are encouraged (but not required) to prefix their specific options
+  with their name (``qbzr.`` for qbzr)
+
+* collisions are detected at registration time so users are protected from
+  incompatibilities between plugins,
+
+* options that need to be used by several plugins (or shared between ``bzr``
+  core and plugins) should be discussed but these discussions are already
+  happening so the risk of misuse is low enough.
+
+Value
+-----
+
+All option values are text. They are provided as Unicode strings to API
+users with some refinements:
+
+* boolean values can be obtained for a set of acceptable strings (yes/no,
+  y/n, on/off, etc), (implemented with the ``from_unicode`` parameter)
+
+* a list of strings from a value containing a comma separated list of
+  strings.
+
+Since the configuration files can be edited by the user, ``bzr`` doesn't
+expect their content to be valid at all times. Instead, the code using
+options should be ready to handle *invalid* values by warning the user and
+falling back to a default value.
+
+Likely, if an option is not defined in any configuration file, the code
+should fallback to a default value (helpers should be provided by the API to
+handle common cases: warning the user, getting a particular type of value,
+returning a default value)(most of that is now handled at Option definition).
+
+This also ensures compatibility with values provided via environment
+variables or from the command line (where no validation can be expected
+either)(done in the new design, some cases missing, see http://pad.lv/832064).
+
+
+Option expansion
+----------------
+
+Some option values can be templates and contain references to other
+options. This is especially useful to define URLs in sections shared for
+multiple branches for example. It can also be used to describe commands
+where some parameters are set by ``bzrlib`` at runtime.
+
+Since option values are text-only, and to avoid clashing with other option
+expansion (also known as interpolation) syntaxes, references are enclosed
+with curly brackets::
+
+  push_location = lp:~{launchpad_username}/bzr/{nick}
+
+In the example above, ``launchpad_username`` is an already defined
+configuration option while ``nick`` is the branch nickname and is set when a
+configuration applies to a given branch.
+
+The interpolation implementation should accept an additional dict so that
+``bzrlib`` or plugins can define references that can be expanded without
+being existing configuration options::
+
+  diff_command={cmd} {cmd_opts} {file_a} {file_b}
+
+There are two common errors that should be handled when handling interpolation:
+
+* loops: when a configuration value refers to itself, directly or indirectly,
+
+* undefined references: when a configuration value refers to an unknown option.
+
+The loop handling can be modified to allow cross-sections and cross-files
+interpolation: if an option refers to itself (directly or indirectly) during
+an expansion, the fallback sections or files can be queried for its value.
+
+This allows list values to refer to the definition in the less specific
+configurations::
+
+  bazaar.conf:
+    debug_flags = hpss
+
+  branch.conf for mybranch:
+    debug_flags = {debug_flags}, hpssdetail
+
+  $ bzr -d mybranch config debug_flags
+  hpss, hpssdetail
+
+Undefined references are detected if they are not defined in any
+configuration. This will trigger errors while displaying the value. Diagnosing
+typos should be doable in this case.
+
+Configuration file syntax
+-------------------------
+
+The configuration file is mostly an ``ini-file``. It contains ``name = value``
+lines grouped in sections. A section starts with a string enclosed in squared
+brackets ('[section_name]`), this string uniquely identifies the section in
+the file. Comments are allowed by prefixing them with the '#' character.
+
+A section is named by the path (or some other unuique identifier) it should
+apply to (more examples below).
+
+When sections are used, they provide a finer grain of configuration by
+defining option sets that apply to some working trees, branches,
+repositories (or any kind of context) or part of them. The relationship
+between a given context and the sections it applies to is defined by the
+config file.
+
+So far, Bazaar uses a glob in ``locations.conf`` and select the sections
+that apply to a given url (or a local path).
+
+The subset is defined by the common leading path or a glob.
+
+Different kinds of section names can be defined:
+
+* a full url: used to described options for remote branches and
+  repositories (LocationMatcher supports this).
+
+* local absolute path: used for working trees, branches or repositories
+  on the local disks (LocationMatcher supports this).
+
+* relative path: the path is relative to the configuration file and can be
+  used for colocated branches or threads in a loom, i.e any working tree,
+  branch or repository that is located in a place related to the
+  configuration file path. Some configuration files may define this path
+  relationship in specific ways to make them easier to use (i.e. if a config
+  file is somewhere below ``.bzr`` and refers to threads in a loom for
+  example, the relative path would be the thread name, it doesn't have to be
+  an *exact* relative path, as long as its interpretation is unambiguous and
+  clear for the user) (No section matchers support this so far, needs to
+  file a bug)
+
+Section matching
+----------------
+
+Section names define another name space (than the option names one) with an
+entirely different purpose: in a given configuration file, for a given
+context, only some sections will be relevant and will be queried in a
+specific order.
+
+This matching is specific to each config file and implemented by the
+SectionMatcher objects.
+
+Whatever this matching does, the user can observe the results with the ``bzr
+config`` command which displays the sections in the order they are queried.
+
+LocationMatcher
+~~~~~~~~~~~~~~~
+
+The context here is either:
+
+* an URL,
+
+* a local path.
+
+Note that for both the provided context and the section names, if an URL uses
+a ``file:///`` form, it is converted to a local path.
+
+The sections names can use globs for each path component
+(i.e. ``/dir/*/subdir`` is allowed but ``/dir/\*\*/subdir`` will match
+``/dir/a/subdir`` but not ``/dir/a/b/subdir``. 
+
+The reason is that the ordering is defined by sorting the section names
+matching the context on the number of path components followed by the path
+itself in lexicographical order. This results in most specific sections being
+queried before the more generic ones.
+
+PathMatcher
+~~~~~~~~~~~
+
+``LocationMatcher`` has some obscure (for unaware users) edge cases and
+limitations that can be surprising. ``PathMatcher`` aims at addressing these
+issues by providing simpler rules while still giving full control to the
+user (http://pad.lv/832046).
+
+The context here is a local path, absolute or relative. If the path is
+relative it is interpreted from the file base directory.
+
+Note that 'base directory' for configuration files in Bazaar directories is
+really:
+
+* the home directory for files under ``~/.bazaar``,
+
+* the ``.bzr`` base directory for files under ``.bzr``.
+
+The order is the one observed in the file so most generic values are specified
+first and most specific ones last. As such, the order in the file is the
+opposite of the one displayed by ``bzr config`` which displays most specific
+values first. This seems to be the most natural order in both cases.
+
+A section matches if the section name is a prefix of the context path
+(relative paths being converted to absolute on the fly).
+
+The Option object
+-----------------
+
+(copied from a recent version of bzr.dev for easier reading, refer to the
+original for an up to date version)
+
+The Option object is used to define its properties:
+
+* name: a name: a valid python identifier (even if it's not used as an
+  identifier in python itself). This is also used to register the option.
+
+* default: the default value that Stack.get() should return if no
+  value can be found for the option.
+
+* default_from_env: a list of environment variables. The first variable set
+  will provide a default value overriding 'default' which remains the
+  default value if *no* environment variable is set.
+
+* help: a doc string describing the option, the first line should be a
+  summary and can be followed by a blank line and a more detailed
+  explanation.
+
+* from_unicode: a callable accepting a unicode string and returning a
+  suitable value for the option. If the string cannot be coerced it should
+  return None.
+
+* invalid: the action to be taken when an invalid value is encountered in a
+  store (during a Stack.get()).
+
+The Section object
+------------------
+
+Options are grouped into sections which share some properties with the well
+known dict objects:
+
+* the key is the name,
+* you can get, set and remove an option,
+* the value is a unicode string.
+
+MutableSection is needed to set or remove an option, ReadOnlySection should
+be used otherwise.
+
+The Store object
+----------------
+
+This is an implementation-level object that should rarely be used directly.
+
+* it can be local or remote
+
+* locking
+
+  All lock operations should be implemented via transport objects. (True for
+  Store).
+
+* option life cycle
+
+  Working trees, branches and repositories should define a config attribute
+  following the same life cycle as their lock: the associated config file is
+  read once and written once if needed. This should minimize the file system
+  accesses or the network requests. There is no known racing scenarios for
+  configuration options, changing the existing implementation to this less
+  constrained one shouldn't introduce any. Yet, in order to detect such
+  racing scenarios, we can add a check that the current content of the
+  configuration file is the expected one before writing the new content and
+  emit warnings if differences occur. The checks should be performed for the
+  modified values only. As of today (and in the foreseeable future), the
+  size of the configuration files are small enough to be kept in memory (see
+  http://pad.lv/832042).
+
+The Stack object
+-----------------
+
+This the object that provides access to the needed features:
+
+* getting an option value,
+
+* setting an option value,
+
+* deleting an option value,
+
+* handling a list of configuration files and for each of them a section
+  matcher providing the sections that should be tried in the given order to
+  find an option.
+
+* handling a Store and a section where option creation, modification and
+  deletion will occur.
+
+Depending on the files involved, a working tree, branch or repository object
+(or more generally a context) should be provided to access the corresponding
+configuration files. Note that providing a working tree object also
+implicitly provides the associated branch and repository object so only one
+of them is required (or none for configuration files specific to the user
+like ``bazaar.conf``).
+
+Getting an option value
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Depending on the option, there are various places where it can be defined
+and several ways to override these settings when needed.
+
+The following lists all possible places where a configuration option can
+be defined, but some options will make sense in only some of them. The
+first to define a value for an option wins (None is therefore used to
+express that an option is not set).
+
+* command-line
+  ``-Ooption=value`` see http://pad.lv/491196.
+
+* ``~/.bazaar/locations.conf``
+
+  When an option is set in ``locations.conf`` it overrides any other
+  configuration file. This should be used with care as it allows setting a
+  different value than what is recommended by the project
+
+* ``tree`` (Not Implemented Yet)
+
+  The options related to the working tree.
+
+  This includes all options related to commits, ignored files, junk files,
+  etc.
+
+  Note that the sections defined there can use relative paths if some
+  options should apply to a subtree or some specific files only.
+
+  See http://pad.lv/430538 and http://pad.lv/654998.
+
+* ``branch`` located in ``.bzr/branch/branch.conf``
+
+  The options related to the branch.
+
+  Sections can be defined for colocated branches or loom threads.
+
+* ``repository`` (Not Implemented Yet)
+
+  The options related to the repository.
+
+  Using an option to describe whether or not a repository is shared could
+  help address http://pad.lv/342119 but this will probably requires a format
+  bump).
+
+* ``project`` (Not Implemented Yet)
+
+  The options common to all branches and working trees for a project.
+
+* ``organization`` (Not Implemented Yet)
+
+  The options common to all branches and working trees for an organization.
+
+  See http://pad.lv/419854.
+
+* ``system`` (Not Implemented Yet but see http://pad.lv/419854 and
+  https://code.launchpad.net/~thomir/bzr/add-global-config/+merge/69592)
+
+  The options common to all users of a system (may be /etc/bzr/defaults
+  or /usr/local/etc/bzr/defaults or
+  /Library/Preferences/com.canonical.defaults  or c:\windows\bazaar.conf
+  (someone fix this one please ;) depending on the OS).
+
+* ``bazaar.conf``
+
+  The options the user has selected for the host he is using.
+
+  Sections can be defined for both remote and local branches to define
+  default values (i.e. the most common use of ``locations.conf`` today).
+
+* default (implemented by the OptionRegistry)
+
+  The options defined in the ``bzr`` source code.
+
+  This will be implemented via the Option objects.
+
+Plugins can define additional configuration files as they see fit and
+insert them in this list, see their documentation for details.
+
+Compatibility
+=============
+
+There are ways to keep the same files while ensuring compatibility via various
+tricks but there are cases where using new files to replace the old ones is
+definitely easier:
+
+* no need to ensure that the new files are correctly handled by old bzr
+  versions,
+
+* making it clear for users that there is a switch and let them migrate at
+  their own pace.
+
+The known cases so far are described below.
+
+Obvious at this point:
+
+* Branch provides ``get_config`` for the old design and ``get_config_stack``
+  for the new design so that both designs are supported. Once the store
+  sharing is implemented, we may want to use an attribute for the stack and
+  deprecate both ``get_config`` and ``get_config_stack``.
+
+* Sections names in ``bazaar.conf`` are arbitrary (except ``DEFAULT``) so
+  it's easier to leave the file untouched and let plugin authors and users
+  migrate away (or not) from them. For ``bzr`` itself, that means
+  ``DEFAULT`` is the only section used for most of the options and provides
+  user defaults. ``ALIASES`` requires a specific stack but only the ``bzr
+  alias`` command cares about that.
+
+* Option policies should be deprecated:
+
+  * The ``norecurse`` policy is useless, all options are recursive by
+    default. If specific values are needed for specific paths, they can just
+    be defined as such (in the appropriate sections or files).
+
+  * The ``appendpath`` policy should be implemented via interpolation and a
+    ``relpath`` option provided by the configuration framework
+    (http://pad.lv/832013).
+
+* Section order in ``locations.conf`` has issues which make a migration to a
+  different way to organize the sections (hence the file content) far easier
+  with a new file.
+
+* ``locations.conf`` is really for overrides but many users have been using it
+  to provide defaults. There is no way to know if the whole content has been
+  used for defaults or overrides or a mix of both. So there is no way to
+  migrate this automatically.
+
+Unclear at this point:
+
+* [BOOKMARKS] section can be replaced by ``bookmarks.xxx`` options (the
+  bookmarks plugins already uses ``bookmarks_xxx`` in branch.conf since no
+  sections were supported there). The easiest here is probably to just merge
+  the plugin into core and use the appropriate option names consistently. A
+  ``config:`` directory service may even be better as any option can be used
+  as a bookmark. This allows things like::
+
+    [/whatever/path]
+    my_push = lp:<launchpad.login>/xxx/{nick}
+    web_site=ftp://example.com/
+
+    bzr push config:web_site
+
+  Which means we completely replace the plugin and don't need to care about
+  migrating the section.
+
+* [ALIASES] section can be replaced by corresponding bzr.alias.xxx
+  options. This could be automated by creating the corresponding options ?
+
+* I don't know about other sections, feedback welcome. Plugin authors are
+  encouraged to migrate to the new name space scheme by prefixing their
+  options with their plugin name.
+
+Notes
+=====
+
+These are random notes about concepts, ideas or issues not implemented yet.
+
+Developer facing concepts
+-------------------------
+
+Option
+~~~~~~
+
+* list of allowed Config IDs (this allows a list of possible config files in
+  bazaar.conf only option and use it while bootstrapping the config
+  creations). 
+
+* blacklist of config IDs (some options *can't* be stored (modified) by the
+  user)
+
+An alternative is to just let the devs decide which stack they use for a
+given option, ``stacked_on_location`` for example is said to relate to the
+branch only and changing it or setting it in a different config file may not
+be appropriate. This may not be a good example as there is also the
+``default_stack_on`` option which can be set only in ``control.conf``
+though...
+
+Stack
+~~~~~
+
+* a lazy cache for the option values (should be reset on modifications as
+  interpolations will make it tricky to update incrementally) (see FIXME in
+  config.py Stack.get()))
+
+* ensures that the Stores involved generate as less IOs as possible (see
+  http://pad.lv/832042)
+
+* ensures that the transaction is the object life time (i.e. modifications
+  will be taken into account *iff* they are committed explicitly).
+
+StackRegistry
+~~~~~~~~~~~~~
+
+* ensures that a config ID is a unique identifier
+* register Stacks
+
+Store
+~~~~~
+
+* ensures that the transaction is the object life time (i.e. modifications
+  will be taken into account *iff* they are committed explicitly).
+
+Examples
+--------
+
+store examples:
+~~~~~~~~~~~~~~~
+
+* ConfigObj (bazaar.conf)
+
+* DB (<scheme>://bazaar.launchpad.net/bazaar.conf)
+
+
+Why and when locking config files matter
+----------------------------------------
+
+This is relevant for http://pad.lv/832042.
+
+``bzr`` behavior, as well as the objects it acts upon, is configured via a
+set of so-called configuration files.
+
+These files allow to define working trees, branches and repositories, their
+relationships and how ``bzr`` should handle them.
+
+The default behavior of ``bzr`` is aimed at making this configuration as
+transparent as possible by keeping track of how these objects are created
+and modified when they are used. In short, they are useless until you want
+to change the default behavior in some specific context.
+
+We mostly **read** config options. Therefore all we care about is to
+guarantee that:
+
+* we get a valid config file at all times when reading,
+
+* we always leave a valid config file when writing (via the rename dance)
+
+From there, conceptually, all operations can clearly define whether or not
+they need to modify a config file and do so only when they succeed. All
+modifications occurring during such an operation are delayed until the very
+end of the operation.
+
+Now, we want to minimize the overlapping times where one bzr operation has
+changed a value and another concurrent operation is unaware of this
+modification.
+
+These overlapping periods are *as of today* rare.
+
+The only known case, http://pad.lv/525571 has been fixed in bzr-2.1.3. The
+bug there was triggered when two processes tried to write the same config
+file at the same time leaving an invalid file in the end.
+
+Such a period can be recognized and detected though: when changing an option
+value, if the preserved original value is different in the config file,
+someone else modified it and the operation can be invalid because it relied
+on the original value.
+
+For the sake of the example, if an option value represent a global unique ID
+via a simple counter (very bad idea), if two operations try to increment it,
+both will use the same value that won't be unique anymore. Checking the
+value present in the file when trying to save the updated value with
+identify such a collision.
+
+An assumption is floating around: it should be enough to report when an
+operation is modifying an already modified option and observe that no-one
+reports such occurrences.
+
+Note that this assumption is made in a context where *no* known scenarios
+exist in the bzr code base not in any plugin (for a best effort value of
+'any', feedback highly welcome, bug reports even ;)
+
+With this in mind, we can change the definition of config options, stores
+and stacks to ensure that:
+
+* a config file is read only once when in read access,
+
+* a config file is read only once and written only once when in write
+  access, adding the check mentioned above will require *one* additional
+  read.
+
+A reader can then safely assume that reading a config file gives it a valid
+(and coherent) definition of the configuration when the operation
+starts. All the operation has to do is to declare which config files may be
+modified by an operation (whether or not we can be liberal on this 'may be'
+is yet to be defined).

=== modified file 'doc/developers/plans.txt'
--- a/doc/developers/plans.txt	2011-10-14 01:18:16 +0000
+++ b/doc/developers/plans.txt	2011-12-01 11:41:07 +0000
@@ -2,29 +2,12 @@
 =====
 
 .. toctree::
-   :hidden:
+   :maxdepth: 1
 
-   performance-roadmap
+   Performance roadmap <performance-roadmap>
    colocated-branches
    feature-flags
+   new-config-rationale
    tortoise-strategy
    improved_chk_index
 
-
-* `Performance roadmap <performance-roadmap.html>`_ |--| The roadmap
-  for fixing performance in bzr over the next few releases.
-
-* `Co-located branches <colocated-branches.html>`_ |--| Planned(?) support
-  for storing multiple branches in one file-system directory.
-
-* `Bazaar Windows Shell Extension Options <tortoise-strategy.html>`_ |--|
-  Implementation strategy for Bazaar Windows Shell Extensions, aka
-  TortoiseBzr.
-
-* `CHK Optimized index <improved_chk_index.html>`_
-
-
-.. |--| unicode:: U+2014
-
-..
-   vim: ft=rst tw=74 ai



More information about the bazaar-commits mailing list