The crawlster.config module

Configuration classes

class crawlster.config.Configuration(options=None)

Configuration object that stores key-value pairs of options

class crawlster.config.JsonConfiguration(file_path)

Reads the configuration from a json file

Configuration options

class crawlster.config.ConfigOption(validators, default=None, required=False)

Class for configuration option definitions

class crawlster.config.Required(validators)

A required value that the user must provide.

class crawlster.config.NumberOption(default=None, required=False, extra_validators=None)

A numeric option

class crawlster.config.StringOption(default=None, required=False, extra_validators=None)

A string option

class crawlster.config.ListOption(default=None, required=False, extra_validators=None)

A list/tuple option

class crawlster.config.ChoiceOption(choices, default=None, required=False, extra_validators=None)

An option whose value must be one from the specified choices

class crawlster.config.UrlOption(default=None, required=False, extra_validators=None)

An option whose value must be a valid URL