multipackage.repo module

Main entry point for dealing with repositories.

Summary

Classes:

Component
EnvironmentVariable
Message
Repository	High-level representation of an entire repository.

Reference

class multipackage.repo.Message(file, message, suggestion, type)

Bases: tuple

file

Alias for field number 0

message

Alias for field number 1

suggestion

Alias for field number 2

type

Alias for field number 3

class multipackage.repo.Component(name, relative_path, options)

Bases: tuple

name

Alias for field number 0

options

Alias for field number 2

relative_path

Alias for field number 1

class multipackage.repo.EnvironmentVariable(name, subsystem, contexts, usage, required, verify)

Bases: tuple

contexts

Alias for field number 2

name

Alias for field number 0

required

Alias for field number 4

subsystem

Alias for field number 1

usage

Alias for field number 3

verify

Alias for field number 5

class multipackage.repo.Repository(path)

Bases: object

High-level representation of an entire repository.

This is the main class inside multipackage that processes a complete repository. It reads two files from the repository:

• .multipackage/settings.json: The overall settings for this repository
• .multipackage/manifest.json: A manifest file of all of the files multipackage is managing including hashes to check for changes.

Using these files, the Repository class will load in the correct repository template that informs it what other files are managed and how to parse the settings stored inside settings.json.

Parameters:path (str) – A path to the repository root directory.

DEFAULT_TEMPLATE = 'pypi_package'

MULTIPACKAGE_DIR = '.multipackage'

SETTINGS_VERSION = '0.1'

SCRIPT_DIR = '.multipackage/scripts'

SETTINGS_FILE = '.multipackage/settings.json'

MANIFEST_FILE = '.multipackage/manifest.json'

COMPONENT_FILE = '.multipackage/components.txt'

COMPONENT_REGEX = '^(?P<package>[a-zA-Z_0-9]+):\\s*(?P<path>[\\.a-zA-Z_\\-0-9\\\\/]+)(?P<options>(\\s*,\\s*[a-zA-Z_0-9_]+\\s*=\\s*[a-zA-Z_0-9_]+)+)?$'

clean

Whether we have any errors.

settings_changed

Whether our settings file on disk has been changed.

error(file, message, suggestion)

Record an error.

warning(file, message, suggestion)

Record a warning.

info(file, message, suggestion)

Record an info message.

iter_messages(message_type)

Iterate over all messages of a given type.

The message_type parameter must be one of info, warning or error and those messages will be returned in the order they were added to the Repository.

Parameters:message_type (str) – The type of message that you want to iterate over. One of “info”, “warning”, or “error”. Returns:An iterable of Message objects. Return type:iterable of Message

count_messages(message_type)

Count how many messages of a given type there are.

Parameters:message_type (str) – The type of message that you want to iterate over. One of “info”, “warning”, or “error”. Returns:The number of messages of the given type. Return type:int

add_subsystem(subsystem)

Add a subsystem to this repository.

required_secret(name, subsystem, usage, verify=None, context='all')

Add a required environment variable.

See the documentation for Repository.add_secret().

optional_secret(name, subsystem, usage, verify=None, context='all')

Add an optional environment variable.

See the documentation for Repository.add_secret().

add_secret(name, subsystem, usage, required, verify=None, context='all')

Add an optional or required environment variable.

Environment variables are needed when Repository.update() is called and are used to convey secret information that must be encrypted and stored in the repository itself. The way this information is used and what specific environment variables are required is 100% decided by the RepositoryTemplate that is managing the repository.

This method exists for the RepositoryTemplate to convey to the Repository class what environment variables it needs and whether they are required or optional. If the relevant subsystem has a method of verifying the contents of the environment variable, it can also pass a callable function in the verify parameter that will be used to verify the value of the environment variable.

This method can be called multiple times with the same name parameter by different subsystems if they both use the same enviornment variable.

The context parameter must be a comma separated list with each entry being one of “update”, “all”, “build”, “deploy”. The context parameter controls when an environment variable is needed. It is used to get lists of all environment variables that might be needed during, e.g. building or deploying a project. Only variables marked for a given context are provided when that context runs.

Parameters:

• name (str) – The name of the environment variable
• subsystem (str) – The name of the subsystem that needs the variable
• usage (str) – A short, one line description of how the variable is used by the subsystem.
• required (bool) – Whether the variable is optional or required.
• verify (callable) – Optional callable with signature: callable(name, value) that is called with the name and value of the environment variable as strings and should return None if the variable is valid. If the variable is invalid, it should return a str instance with a short message describing why the variable’s contents are not valid.
• context (list of str) – A comma separated list of contexts during which this environment variable is needed. Supported contexts are update, all, build an deploy.

iter_secrets()

Iterate over all declared secrets.

get_secret(name)

Get the value of a secret from the user’s environment.

If the secret is marked as required and it is not present, an exception is raised, otherwise None is returned when the value is not present.

Parameters:

name (str) – The name of the secret that was previously registered with add_secret(), required_secret(), or optional_secret().

Returns:

The secret’s value or None if it is optional and not present.

Return type:

str

Raises:

• InvalidEnvironmentError – If the secret is required and not present.
• IntenalError – If name was never declared as a secret.

iter_context(context, include_empty=True)

Iterate over environment variables applicable to a given context.

The context parameter must be one of ‘update’, ‘build’, or ‘deploy’ and will be used to select environment variables needed in that context.

The variables will be yielded in sorted order by name for consistency.

Parameters:

• context (str) – The context we wish to get variables for. Must be one of update, build or deploy. Variables marked with all will be returned for all contexts.
• include_empty (bool) – Also yield None for optional environement variables that are not defined in the user’s ENVIRONMENT.

Returns:

An iterable of variable name and value.

Each variable that is marked with the passed context will be returned as part of the iterable in stable sorted order. If the variable is marked as optional and it is not defined in the process’s environment, then a None value will be yielded if include_empty is True (the default behvaior), otherwise the variable will be skipped as if it were not declared at all.

Return type:

iterable of str, str

get_setting(name, default=<object object>)

Get the value of a setting from the options dictionary.

The name of the setting can be a series of dot separated identifiers that will be resolved recursively into subkeys.

For example, “doc.deploy” would be the key “deploy” in a dictionary assigned to the key “doc”. “doc” would return the entire doc dictionary.

Parameters:

• name (str) – The name of the setting that we want to fetch.
• default (object) – If the setting is optional, you can specify a default value. The default behavior is to raise an exception when a setting is missing.

Returns:

The value of the setting.

Return type:

str

set_setting(name, value)

Change or set an option in settings.json.

The change is flushed out to the settings file immediately.

Parameters:

• name (str) – The name of the setting you want to set or change.
• value (object) – A json serializable value.

initialize(clean=False, platforms=('macos', 'windows', 'linux'), python_versions=('2.7', '3.6'))

Initialize or reinitialize this repository.

ensure_lines(relative_path, lines, match=None, present=True, multi=False, delimiter_start='#', delimiter_end='')

Ensure that the following lines are in the given file.

This will create or update a ManagedSection and ensure that the following lines are all present in that file. The lines are assumed to be unordered so they are added independently of each other.

This method is idempotent. It will only add a line to the file if it does not already exist.

The lines are automatically saved to disk.

Parameters:

• relative_path (str) – The relative path to the file that we want to update.
• lines (list of str) – A list of strings to add to the given file if they do not exist.
• present (bool) – If True, ensure lines are present (the default), if False, ensure lines are absent.
• match (list of str) – A list of regular expressions to use to match against a line. If passed it must have the same length as lines and will be paired with each line in lines in order to determine which line matches.
• multi (bool) – If true, allow for matching and updating multiple lines for each line in lines. If False, InternalError is raised if there are multiple lines matching a given line.
• delimiter_start (str) – The character string that starts a managed file section block header line. This is usually a comment character like ‘#’.
• delimiter_end (str) – Optional charcter string that ends a managed file section block header line. This defaults to None but can be set to a value if comments need to be explicitly closed such as delimiter_start=”<!– “, delimiter_end=” –>”.

ensure_directory(relative_path, gitkeep=False)

Ensure that a given directory exists.

Parameters:

• relative_path (str) – The relative path to the directory that we wish to check.
• gitkeep (bool) – Also insert an unmanaged .gitkeep file to ensure the directory is saved in the git repo history.

ensure_script(relative_path, source, present=True, overwrite=True)

Ensure that a script file is copied.

This function will copy or remove the given script file. All script files come from the multipackage/data/scripts folder.

Parameters:

• relative_path (str) – The relative path to the file that we want to update.
• source (str) – The name of the template file to render
• present (bool) – If True, ensure lines are present (the default), if False, ensure lines are absent.
• overwrite (bool) – Overwrite the file if it exists already. Setting this to False will ensure that the file is added only if is absent, allowing for initializing files that are not present without subsequently overwriting them.

ensure_template(relative_path, template, variables=None, present=True, overwrite=True, raw=False, filters=None)

Ensure that the contents of a given file match a template.

This function will render the given template shipped with the multipackage package.

Parameters:

• relative_path (str) – The relative path to the file that we want to update.
• template (str) – The name of the template file to render
• variables (dict) – A set of substitution variables to fill into the template.
• present (bool) – If True, ensure lines are present (the default), if False, ensure lines are absent.
• overwrite (bool) – Overwrite the file if it exists already. Setting this to False will ensure that the file is added only if is absent, allowing for initializing files that are not present without subsequently overwriting them.
• raw (bool) – Do not actually fill in the template, just copy it into place. This is useful if the file itself is a jinja2 template that should be rendered later.
• filters (dict) – Optional dictionary of callable filters that will be passed to jinja2 for use in the template formatting.

update()

Update all of the managed files in this multipackage installation.

This method delegates to all of the enabled multipackage subsystems to actually update each subcomponent.