graphid.util.util_misc module¶
- graphid.util.util_misc.aslist(sequence)[source]¶
Ensures that the sequence object is a Python list. Handles, numpy arrays, and python sequences (e.g. tuples, and iterables).
- Parameters:
sequence (sequence) – a list-like object
- Returns:
list_ - sequence as a Python list
- Return type:
Example
>>> s1 = [1, 2, 3] >>> s2 = (1, 2, 3) >>> assert aslist(s1) is s1 >>> assert aslist(s2) is not s2 >>> aslist(np.array([[1, 2], [3, 4], [5, 6]])) [[1, 2], [3, 4], [5, 6]] >>> aslist(range(3)) [0, 1, 2]
- class graphid.util.util_misc.classproperty(fget=None, fset=None, fdel=None, doc=None)[source]¶
Bases:
property
Decorates a method turning it into a classattribute
References
https://stackoverflow.com/questions/1697501/python-staticmethod-with-property
- graphid.util.util_misc.estarmap(func, iter_, **kwargs)[source]¶
Eager version of it.starmap from itertools
Note this is inefficient and should only be used when prototyping and debugging.
- graphid.util.util_misc.delete_dict_keys(dict_, key_list)[source]¶
Removes items from a dictionary inplace. Keys that do not exist are ignored.
- Parameters:
dict_ (dict) – dict like object with a __del__ attribute
key_list (list) – list of keys that specify the items to remove
Example
>>> dict_ = {'bread': 1, 'churches': 1, 'cider': 2, 'very small rocks': 2} >>> key_list = ['duck', 'bread', 'cider'] >>> delete_dict_keys(dict_, key_list) >>> result = ub.urepr(dict_, nl=False) >>> print(result) {'churches': 1, 'very small rocks': 2}
- graphid.util.util_misc.delete_items_by_index(list_, index_list, copy=False)[source]¶
Remove items from
list_
at positions specified inindex_list
The originallist_
is preserved ifcopy
is True- Parameters:
list_ (list)
index_list (list)
copy (bool) – preserves original list if True
Example
>>> list_ = [8, 1, 8, 1, 6, 6, 3, 4, 4, 5, 6] >>> index_list = [2, -1] >>> result = delete_items_by_index(list_, index_list) >>> print(result) [8, 1, 1, 6, 6, 3, 4, 4, 5]
- graphid.util.util_misc.make_index_lookup(list_, dict_factory=<class 'dict'>)[source]¶
- Parameters:
list_ (list) – assumed to have unique items
- Returns:
mapping from item to index
- Return type:
Example
>>> list_ = [5, 3, 8, 2] >>> idx2_item = make_index_lookup(list_) >>> result = ub.urepr(idx2_item, nl=False, sort=1) >>> assert list(ub.take(idx2_item, list_)) == list(range(len(list_))) >>> print(result) {2: 3, 3: 1, 5: 0, 8: 2}
- graphid.util.util_misc.cprint(text, color=None)[source]¶
provides some color to terminal output
- Parameters:
text (str)
color (str)
- Example0:
>>> import pygments.console >>> msg_list = list(pygments.console.codes.keys()) >>> color_list = list(pygments.console.codes.keys()) >>> [cprint(text, color) for text, color in zip(msg_list, color_list)]
- Example1:
>>> import pygments.console >>> print('line1') >>> cprint('line2', 'red') >>> cprint('line3', 'blue') >>> cprint('line4', 'magenta') >>> cprint('line5', 'reset') >>> cprint('line5', 'magenta') >>> print('line6')
- graphid.util.util_misc.ensure_iterable(obj)[source]¶
- Parameters:
obj (scalar or iterable)
- Returns:
obj if it was iterable otherwise [obj]
- Return type:
it3erable
- Timeit:
%timeit util.ensure_iterable([1]) %timeit util.ensure_iterable(1) %timeit util.ensure_iterable(np.array(1)) %timeit util.ensure_iterable([1]) %timeit [1]
Example
>>> obj_list = [3, [3], '3', (3,), [3,4,5]] >>> result = [ensure_iterable(obj) for obj in obj_list] >>> result = str(result) >>> print(result) [[3], [3], ['3'], (3,), [3, 4, 5]]
- graphid.util.util_misc.highlight_regex(str_, pat, reflags=0, color='red')[source]¶
FIXME Use pygments instead
- graphid.util.util_misc.setdiff(list1, list2)[source]¶
returns list1 elements that are not in list2. preserves order of list1
- Parameters:
list1 (list)
list2 (list)
- Returns:
new_list
- Return type:
Example
>>> list1 = ['featweight_rowid', 'feature_rowid', 'config_rowid', 'featweight_forground_weight'] >>> list2 = [u'featweight_rowid'] >>> new_list = setdiff(list1, list2) >>> result = ub.urepr(new_list, nl=False) >>> print(result) ['feature_rowid', 'config_rowid', 'featweight_forground_weight']
- graphid.util.util_misc.all_dict_combinations(varied_dict)[source]¶
- Parameters:
varied_dict (dict) – a dict with lists of possible parameter settings
- Returns:
dict_list a list of dicts correpsonding to all combinations of params settings
- Return type:
Example
>>> varied_dict = {'logdist_weight': [0.0, 1.0], 'pipeline_root': ['vsmany'], 'sv_on': [True, False, None]} >>> dict_list = all_dict_combinations(varied_dict) >>> result = str(ub.urepr(dict_list)) >>> print(result) [ {'logdist_weight': 0.0, 'pipeline_root': 'vsmany', 'sv_on': True}, {'logdist_weight': 0.0, 'pipeline_root': 'vsmany', 'sv_on': False}, {'logdist_weight': 0.0, 'pipeline_root': 'vsmany', 'sv_on': None}, {'logdist_weight': 1.0, 'pipeline_root': 'vsmany', 'sv_on': True}, {'logdist_weight': 1.0, 'pipeline_root': 'vsmany', 'sv_on': False}, {'logdist_weight': 1.0, 'pipeline_root': 'vsmany', 'sv_on': None}, ]
- graphid.util.util_misc.replace_nones(list_, repl=-1)[source]¶
Recursively removes Nones in all lists and sublists and replaces them with the repl variable
- Parameters:
list_ (list)
repl (obj) – replacement value
- Returns:
list
Example
>>> list_ = [None, 0, 1, 2] >>> repl = -1 >>> repl_list = replace_nones(list_, repl) >>> result = str(repl_list) >>> print(result) [-1, 0, 1, 2]
- graphid.util.util_misc.take_percentile_parts(arr, front=None, mid=None, back=None)[source]¶
Take parts from front, back, or middle of a list
Example
>>> arr = list(range(20)) >>> front = 3 >>> mid = 3 >>> back = 3 >>> result = take_percentile_parts(arr, front, mid, back) >>> print(result) [0, 1, 2, 9, 10, 11, 17, 18, 19]
- graphid.util.util_misc.snapped_slice(size, frac, n)[source]¶
Creates a slice spanning n items in a list of length size at position frac.
- Parameters:
size (int) – length of the list
frac (float) – position in the range [0, 1]
n (int) – number of items in the slice
- Returns:
slice object that best fits the criteria
- Return type:
- SeeAlso:
take_percentile_parts
Example:
Example
>>> # DISABLE_DOCTEST >>> print(snapped_slice(0, 0, 10)) >>> print(snapped_slice(1, 0, 10)) >>> print(snapped_slice(100, 0, 10)) >>> print(snapped_slice(9, 0, 10)) >>> print(snapped_slice(100, 1, 10)) pass
- graphid.util.util_misc.get_timestamp(format_='iso', use_second=False, delta_seconds=None, isutc=False, timezone=False)[source]¶
- Parameters:
format_ (str) – (tag, printable, filename, other)
use_second (bool)
delta_seconds (None)
- Returns:
stamp
- Return type:
Example
>>> format_ = 'printable' >>> use_second = False >>> delta_seconds = None >>> stamp = get_timestamp(format_, use_second, delta_seconds) >>> print(stamp) >>> assert len(stamp) == len('15:43:04 2015/02/24')
- graphid.util.util_misc.isect(list1, list2)[source]¶
returns list1 elements that are also in list2. preserves order of list1
intersect_ordered
- Parameters:
list1 (list)
list2 (list)
- Returns:
new_list
- Return type:
Example
>>> list1 = ['featweight_rowid', 'feature_rowid', 'config_rowid', 'featweight_forground_weight'] >>> list2 = [u'featweight_rowid'] >>> result = isect(list1, list2) >>> print(result) ['featweight_rowid']
- graphid.util.util_misc.safe_extreme(arr, op, fill=nan, finite=False, nans=True)[source]¶
Applies an exterme operation to an 1d array (typically max/min) but ensures a value is always returned even in operations without identities. The default identity must be specified using the fill argument.
- Parameters:
arr (ndarray) – 1d array to take extreme of
op (func) – vectorized operation like np.max to apply to array
fill (float) – return type if arr has no elements (default = nan)
finite (bool) – if True ignores non-finite values (default = False)
nans (bool) – if False ignores nans (default = True)
- graphid.util.util_misc.safe_argmax(arr, fill=nan, finite=False, nans=True)[source]¶
Doctest
>>> assert safe_argmax([np.nan, np.nan], nans=False) == 0 >>> assert safe_argmax([-100, np.nan], nans=False) == 0 >>> assert safe_argmax([np.nan, -100], nans=False) == 1 >>> assert safe_argmax([-100, 0], nans=False) == 1 >>> assert np.isnan(safe_argmax([]))
- graphid.util.util_misc.safe_max(arr, fill=nan, finite=False, nans=True)[source]¶
- Parameters:
arr (ndarray) – 1d array to take max of
fill (float) – return type if arr has no elements (default = nan)
finite (bool) – if True ignores non-finite values (default = False)
nans (bool) – if False ignores nans (default = True)
Example
>>> arrs = [[], [np.nan], [-np.inf, np.nan, np.inf], [np.inf], [np.inf, 1], [0, 1]] >>> arrs = [np.array(arr) for arr in arrs] >>> fill = np.nan >>> results1 = [safe_max(arr, fill, finite=False, nans=True) for arr in arrs] >>> results2 = [safe_max(arr, fill, finite=True, nans=True) for arr in arrs] >>> results3 = [safe_max(arr, fill, finite=True, nans=False) for arr in arrs] >>> results4 = [safe_max(arr, fill, finite=False, nans=False) for arr in arrs] >>> results = [results1, results2, results3, results4] >>> result = ('results = %s' % (ub.urepr(results, nl=1, sv=1),)) >>> print(result) results = [ [nan, nan, nan, inf, inf, 1], [nan, nan, nan, nan, 1.0, 1], [nan, nan, nan, nan, 1.0, 1], [nan, nan, inf, inf, inf, 1], ]
- graphid.util.util_misc.safe_min(arr, fill=nan, finite=False, nans=True)[source]¶
Example
>>> arrs = [[], [np.nan], [-np.inf, np.nan, np.inf], [np.inf], [np.inf, 1], [0, 1]] >>> arrs = [np.array(arr) for arr in arrs] >>> fill = np.nan >>> results1 = [safe_min(arr, fill, finite=False, nans=True) for arr in arrs] >>> results2 = [safe_min(arr, fill, finite=True, nans=True) for arr in arrs] >>> results3 = [safe_min(arr, fill, finite=True, nans=False) for arr in arrs] >>> results4 = [safe_min(arr, fill, finite=False, nans=False) for arr in arrs] >>> results = [results1, results2, results3, results4] >>> result = ('results = %s' % (ub.urepr(results, nl=1, sv=1),)) >>> print(result) results = [ [nan, nan, nan, inf, 1.0, 0], [nan, nan, nan, nan, 1.0, 0], [nan, nan, nan, nan, 1.0, 0], [nan, nan, -inf, inf, 1.0, 0], ]
- graphid.util.util_misc.stats_dict(list_, axis=None, use_nan=False, use_sum=False, use_median=False, size=False)[source]¶
- Parameters:
list_ (listlike) – values to get statistics of
axis (int) – if list_ is ndarray then this specifies the axis
- Returns:
- stats: dictionary of common numpy statistics
(min, max, mean, std, nMin, nMax, shape)
- Return type:
OrderedDict
- Examples0:
>>> # xdoctest: +IGNORE_WHITESPACE >>> import numpy as np >>> axis = 0 >>> np.random.seed(0) >>> list_ = np.random.rand(10, 2).astype(np.float32) >>> stats = stats_dict(list_, axis, use_nan=False) >>> result = str(ub.urepr(stats, nl=1, precision=4, with_dtype=True)) >>> print(result) { 'mean': np.array([0.5206, 0.6425], dtype=np.float32), 'std': np.array([0.2854, 0.2517], dtype=np.float32), 'max': np.array([0.9637, 0.9256], dtype=np.float32), 'min': np.array([0.0202, 0.0871], dtype=np.float32), 'nMin': np.array([1, 1], dtype=np.int32), 'nMax': np.array([1, 1], dtype=np.int32), 'shape': (10, 2), }
- Examples1:
>>> import numpy as np >>> axis = 0 >>> rng = np.random.RandomState(0) >>> list_ = rng.randint(0, 42, size=100).astype(np.float32) >>> list_[4] = np.nan >>> stats = stats_dict(list_, axis, use_nan=True) >>> result = str(ub.urepr(stats, precision=1, sk=True)) >>> print(result) {mean: 20.0, std: 13.2, max: 41.0, min: 0.0, nMin: 7, nMax: 3, shape: (100,), num_nan: 1,}