Detakon Module API

This is the pydoc code for detakon module.

class detakon.converter.Detakon(detamap, source, destination, *args, **kargs)

detakon uses a detakon map to convert data.

_load_detamap(detamap) → dict

Process object passed as detamap and return dictionary detamap.

Parameters:

  • self – Object reference.

  • detamap – Either a dictionary, JSON stream/string, or file path (string or pathlib.Path) to JSON file.

Returns:

dict of detamap

Return type:

dict

_process_defaults(row)

Add default values to any missing column or empty string.

_process_operations(row)

Process all operations from self.operations in order that operations appear in list.

_source_reader() → Generator | dict | list

Validate source type, and return a generator object if possible, otherwise return full object in accepted format.

Source type should be defined in detamap under Source map. Argument key defines what the source argument passed to Detakon() is (such as a filepath), and type key defines what sub-type to apply to that (such as filepath provided is a str or path object).

Intent to add remote file, or result of API calls - giving consideration to add ability, or require calling application to submit data directly.

Parameters:

self – Object reference.

Returns:

Generator object of dictionaries, or list/dictionary of dictionaries if generator not possible.

Return type:

Generator | dict

get_destination()

Return the current destination.

Returns:

self.destination

get_source()

Return the current source.

Returns:

self.source

Return type:

str | path | dict | list

process() → None

Processes conversion for the currently loaded Detamap, data source, and destination.

set_destination(destination) → None

Change destination or output.

set_detamap(detamap=None) → None

For interactive sessions to change the detamap. Pass new detamap as argument.

If no argument is provided, the original detamap will be reloaded; if the original detamap was a file, this will update the with any changes from the file.

Parameters:

detamap – New detamap. Default to reloading self.original_detamap (which is stored during __init__).

set_source(source) → None

Change the data source.

detakon.operations.cast_type(value, data_type: str, *args, language_map: dict = {'Defaults': 'Defaults', 'Mappings': 'Mappings', 'Operations': 'Operations', 'Output': 'Output', 'Source': 'Source', 'append': 'append', 'argument': 'argument', 'arguments': 'args', 'cast': ['cast', 'converttype', 'convert type', 'type cast', 'typecast'], 'cast_boolean': ['bool', 'boolean'], 'cast_decimal': ['decimal'], 'cast_float': ['float', 'double'], 'cast_int': ['int', 'integer', 'long'], 'cast_string': ['str', 'string'], 'create field': ['create', 'new', 'create field', 'new field'], 'duplicate': ['duplicate'], 'duplicate_rows': ['duplicate_rows', 'duplicate rows', 'duplicate-row', 'duplicate_row', 'duplicate row', 'duplicate-rows', 'union duplicates'], 'exclude': ['exclude'], 'fields': 'fields', 'filepath': ['filepath'], 'filter_boolean': ['bool', 'boolean', 'truthiness', 'truthy', 'falsy'], 'filter_equal': ['equal', '=', '==', 'isequal', 'is equal'], 'filter_greater_or_equal': ['ge', 'greater or equal', 'greater than or equal', '>=', '≥'], 'filter_greater_than': ['gt', 'greaterthan', 'greater than', '>'], 'filter_in': ['in', 'contains', 'substring'], 'filter_less_or_equal': ['le', 'less or equal', 'less than or equal', '<=', '≤'], 'filter_less_than': ['lt', 'lessthan', 'less than', '<'], 'filter_none': ['isnone', 'none'], 'filter_not_equal': ['not equal', 'notequal', '!=', '~=', '<>', 'not equals to', 'not ='], 'filter_not_in': ['not in', 'notin'], 'hashmap': ['hashmap', 'dictionary', 'dict'], 'include': ['include'], 'join': ['join'], 'keyword_arguments': 'kwargs', 'omit_heading': 'omit_heading', 'outofplace': ['change place', 'change_place', 'changeplace', 'change-place', 'outofplace', 'out of place', 'out-of-place', 'out_of_place', 'not-in-place', 'not in place', 'not_in_place'], 'slice': ['slice'], 'type': 'type'}, **kwargs)

Cast value into the given type. If type does not match an expected value raise a ValueError.

For clarity, please see shorter version of function header below:

detakon.operations.cast_type(value, data_type: str, *args, language_map: dict = load_language("en-us"), **kwargs)

Types values for casting, and accepted aliases:

  • int: “int”, “integer”, “long”

  • float: “float”, “double”

  • Decimal: “decimal” (specifically the python type decimal.Decimal)

  • bool: “bool”, “boolean”

  • str: “str”, “string”

Parameters:

  • value – value from source to cast into new type

  • data_type – type to cast value to

  • language_map – The language map to use for comparation aliases. Defaults to “en-us”.

Returns:

value as new type

detakon.operations.change_place(field: str, row: dict, destination: str, operation: str, args, kwargs, language_map: dict) → dict

Changes an in-place operation into an out-of-place operation.

Calls the specified operation on a copy of the row, and merge the specified row from the copy back into the original with a different key.

Parameters:

  • field – Field used as data source.

  • row – A dictionary of data that holds the field to be operated on.

  • destination – The name of the field to store the result of the operation.

  • *args –

    Arguments to be passed to the operation.

  • **kwargs –

    Keyword arguments to be passed to the operation.

detakon.operations.create_field(field: str, row: dict, *args, **kwargs) → dict

Add field to source dictionary. By default new fields are empty strings.

If args are supplied: * List of length one: Assign first value to field. * List of length greater than one: Assign list of args to field.

Parameters:

  • field – The name of the field to be created.

  • row – A dictionary of data that holds the current state of the row.

  • *args –

    If supplied, args[0] will be used to set the value of the field.

  • **kwargs –

    Keyword arguments to be passed.

detakon.operations.duplicate(field: str, row: dict, *args, **kwargs) → dict

Creates new fields with the value contained in field. Each value in args will be the name of the new fields created by duplicating field.

Parameters:

  • field – The name of the field to be duplicated.

  • row – A dictionary of data that holds the current state of the row.

  • *args –

    A list of fields be created with the duplicated value.

  • **kwargs –

    Keyword arguments to be passed.

detakon.operations.duplicate_row(field: str, row: dict, *args, **kwargs) → dict

Causes row to be appended to output multiple times based on value of field. Value must be int or castable into int.

Adds a special key to row called detakon_duplicate_rows that contains an int quantity.

Parameters:

  • field – Field with the value specifying quantity of times to output row.

  • row – A dictionary of data that holds the current state of the row.

  • *args –

    Additional arguments being passed.

  • **kwargs –

    Keyword arguments to be passed.

detakon.operations.filter(row_value: dict, comparison: str, comparison_value, *args, language_map: dict = {'Defaults': 'Defaults', 'Mappings': 'Mappings', 'Operations': 'Operations', 'Output': 'Output', 'Source': 'Source', 'append': 'append', 'argument': 'argument', 'arguments': 'args', 'cast': ['cast', 'converttype', 'convert type', 'type cast', 'typecast'], 'cast_boolean': ['bool', 'boolean'], 'cast_decimal': ['decimal'], 'cast_float': ['float', 'double'], 'cast_int': ['int', 'integer', 'long'], 'cast_string': ['str', 'string'], 'create field': ['create', 'new', 'create field', 'new field'], 'duplicate': ['duplicate'], 'duplicate_rows': ['duplicate_rows', 'duplicate rows', 'duplicate-row', 'duplicate_row', 'duplicate row', 'duplicate-rows', 'union duplicates'], 'exclude': ['exclude'], 'fields': 'fields', 'filepath': ['filepath'], 'filter_boolean': ['bool', 'boolean', 'truthiness', 'truthy', 'falsy'], 'filter_equal': ['equal', '=', '==', 'isequal', 'is equal'], 'filter_greater_or_equal': ['ge', 'greater or equal', 'greater than or equal', '>=', '≥'], 'filter_greater_than': ['gt', 'greaterthan', 'greater than', '>'], 'filter_in': ['in', 'contains', 'substring'], 'filter_less_or_equal': ['le', 'less or equal', 'less than or equal', '<=', '≤'], 'filter_less_than': ['lt', 'lessthan', 'less than', '<'], 'filter_none': ['isnone', 'none'], 'filter_not_equal': ['not equal', 'notequal', '!=', '~=', '<>', 'not equals to', 'not ='], 'filter_not_in': ['not in', 'notin'], 'hashmap': ['hashmap', 'dictionary', 'dict'], 'include': ['include'], 'join': ['join'], 'keyword_arguments': 'kwargs', 'omit_heading': 'omit_heading', 'outofplace': ['change place', 'change_place', 'changeplace', 'change-place', 'outofplace', 'out of place', 'out-of-place', 'out_of_place', 'not-in-place', 'not in place', 'not_in_place'], 'slice': ['slice'], 'type': 'type'}, **kwargs) → bool

Take a string indicating a comparison to make, and a value that comparison will be made to, and return a bool indicating if that comparison is met. Designed for use in exclude and include filter operations.

For clarity, please see shorter version of function header below:

detakon.operations.filter(row_value: dict, comparison: str, comparison_value, *args, language_map: dict = load_language("en-us"), **kwargs) → bool

Filter comparison values and accepted aliases:

  • equal: “equal”, “=”, “==”, “isequal”, “is equal”

  • not equal: “not equal”, “notequal”, “!=”, “~=”, “<>”, “not equals to”, “not =”

  • in: “in”, “contains”, “substring”

  • not in: “not in”, “notin”

  • greater than: “gt”, “greaterthan”, “greater than”, “>”

  • less than: “lt”, “lessthan”, “less than”, “<”

  • greater than or equal: “ge”, “greater or equal”, “greater than or equal”, “>=”, “≥”

  • less than or equal: “le”, “less or equal”, “less than or equal”, “<=”, “≤”

  • boolean: “bool”, “boolean”, “truthiness”, “truthy”, “falsy”

  • none: “isnone”, “none”

Parameters:

  • row_value – The value from the data source row to compare against comparison_value.

  • comparison – The filter comparison operator to be used for the comparison.

  • comparison_value – The value to compare the source value against.

  • language_map – The language map to use for comparation aliases. Defaults to “en-us”.

Returns:

bool

detakon.operations.hashmap(field: str, row: dict, *args, **kwargs) → dict

Convert value of field in row to new value from dictionary args[0] if value of field matches key in dict args[0] and return updated row.

Parameters:

  • field – The key for the value being value being converted.

  • row – A dictionary of data that holds the field to be converted.

  • *args –

    List of which args[0] is expected to be a dictionary of mappings from old value to new value. Keys are case sensitive.

  • **kwargs –

    Keyword arguments to be passed.

detakon.operations.slice_field(field: str, row: dict, *args, **kwargs) → dict

Perform a slice on a field from a row of data and return the row with the updated sliced field.

Parameters:

  • field – The key for the value being sliced.

  • row – A dictionary of data that holds the field to be sliced.

  • *args –

    Arguments to be passed to python’s slice object.

  • **kwargs –

    Keyword arguments to be passed to python’s slice object.

detakon.operations.string_join(field: str, row: dict, *args, **kwargs) → dict

Calls str.join on list in specified field and uses args[0] as the separator for the resulting string. Return the updated row.

Parameters:

  • field – Field used as data source. Must contain a list that str.join can operate on.

  • row – A dictionary of data that holds the field to be operated on.

  • *args –

    List of which args[0] is expected to be a string to be used as a separator.

  • **kwargs –

    Keyword arguments to be passed.