obsplus.utils.misc module

Misc. ObsPlus utilities.

class obsplus.utils.misc.DummyFile

Bases: object

Dummy class to mock std out interface but go nowhere.

flush()

do nothing

write(x)

do nothing

class obsplus.utils.misc.ObjectWrapper(data)

Bases: object

A class for wrapping objects.

This is useful so array-like things can be packaged into numpy arrays and pandas dataframes.

data

obsplus.utils.misc.apply_to_files_or_skip(func, directory)

Generator for applying func to all files in directory.

Skip any files that raise an exception.

Parameters:

• func (Callable) – Any callable that takes a file path as the only input.

• directory (Union[str, Path]) – A directory that exists.

Yields:

outputs of func

obsplus.utils.misc.deprecated_callable(func=None, replacement_str=None)

Mark a function as deprecated.

Whenever it is used a userwarning will be issued. You can optionally provide a string to indicate which function should be used in its place.

Parameters:

• func –

• replacement_str –

obsplus.utils.misc.get_instances_from_tree(object, cls)

Get all instances in an object tree.

Simply uses yield_obj_parent_attr() under the hood.

obsplus.utils.misc.get_progressbar(max_value, min_value=None, *args, **kwargs)

Get a progress bar object using the ProgressBar2 library.

Fails gracefully if bar cannot be displayed (eg if no std out). Args and kwargs are passed to ProgressBar constructor.

Parameters:

• max_value – The highest number expected

• min_value – The minimum number of updates required to show the bar

Return type:

Optional

obsplus.utils.misc.get_version_tuple(version_str)

Convert a semantic version string to a tuple.

Parameters:

version_str (str) – A version of the form “x.y.z”. Google semantic versioning for more details.

Return type:

Tuple[int, int, int]

obsplus.utils.misc.getattrs(obj, col_set, default_value=nan)

Parse an object for a collection of attributes, return a dict of values.

If obj does not have a requested attribute, or if its value is None, fill with the default value.

Parameters:

• obj (object) – Any object.

• col_set (Collection) – A sequence of attributes to extract from obj.

• default_value (object) – If not attribute is found fill with this value.

Return type:

dict

obsplus.utils.misc.hash_directory(path, match='*', exclude=None, hidden=False)

Calculate the sha256 hash of all files in a directory.

Parameters:

• path (Union[Path, str]) – The path to the directory

• match (str) – A unix-style matching string

• exclude (Union[str, Collection[str], None]) – A list of unix style strings to exclude

• hidden – If True skip all files starting with a .

Return type:

A dict containing paths and md5 hashes.

obsplus.utils.misc.hash_file(path)

Calculate the sha256 hash of a file.

Reads the file in chunks to allow using large files. Taken from this stack overflow answer: http://bit.ly/2Jqb1Jr

Parameters:

path (Union[str, Path]) – The path to the file to read.

Return type:

A str of hex for file hash

obsplus.utils.misc.iter_files(paths, ext=None, mtime=None, skip_hidden=True)

use os.scan dir to iter files, optionally only for those with given extension (ext) or modified times after mtime

Parameters:

• paths (Union[str, Iterable[str]]) – The path to the base directory to traverse. Can also use a collection of paths.

• ext (str or None) – The extensions to map.

• mtime (int or float) – Time stamp indicating the minimum mtime.

• skip_hidden (bool) – If True skip files or folders (they begin with a ‘.’)

Yields:

Paths, as strings, meeting requirements.

Return type:

Iterable[str]

obsplus.utils.misc.iterate(obj)

Return an iterable from any object.

If string, do not iterate characters, return str in tuple .

obsplus.utils.misc.read_file(file_path, funcs=(<function read_csv>, ))

For a given file_path, try reading it with each function in funcs.

Parameters:

• file_path – The path to the file to read

• funcs – A tuple of functions to try to read the file (starting with first)

Return type:

Optional[Any]

obsplus.utils.misc.register_func(list_or_dict, key=None)

Decorator for registering a function name in a list or dict.

If list_or_dict is a list only append the name of the function. If it is as dict append name (as key) and function as the value.

Parameters:

• list_or_dict (Union[list, dict]) – A list or dict to which the wrapped function will be added.

• key (Optional[str]) – The name to use, if different than the name of the function.

obsplus.utils.misc.replace_null_nlsc_codes(obspy_object, null_codes=(None, '--', 'None', 'nan', 'null', nan), replacement_value='')

Iterate an obspy object and replace nullish nslc codes with some value.

Operates in place, but also returns the original object.

Parameters:

• obspy_object (TypeVar(any_type)) – An obspy catalog, event, (or any sub element), stream, trace, inventory, etc.

• null_codes – The codes that are considered null values and should be replaced.

• replacement_value – The value with which to replace the null_codes.

Return type:

TypeVar(any_type)

obsplus.utils.misc.strip_prefix(some_str, prefixes)

Strip a prefix of a string.

Return type:

str

obsplus.utils.misc.suppress_warnings(category=<class 'Warning'>)

Context manager for suppressing warnings.

Parameters:

category – The types of warnings to suppress. Must be a subclass of Warning.

obsplus.utils.misc.try_read_catalog(catalog_path, **kwargs)

Try to read a events from file, if it raises return None

obsplus.utils.misc.validate_version_str(version_str)

Check the version string is of the form x.y.z.

If the version string is not valid raise a ValueError.

obsplus.utils.misc.yield_obj_parent_attr(obj, cls=None, is_attr=None, has_attr=None, basic_types=False)

Recurse an object, yield a tuple of object, parent, attr.

Useful when data need to be changed or the provided DataFrame extractors don’t quite perform the desired task. Can also be used to extract relationships between entities in object trees to build a connecting graph.

Parameters:

• obj – The object to recurse through attributes of lists, tuples, and other instances.

• cls – Only return instances of cls if not None, dont filter on types.

• is_attr – Only return objects stored as attr_name, if None return all.

• has_attr – Only return objects that have attribute has_attr, if None return all.

• basic_types – If True, yield non-sequence basic types (int, float, str, bool).

Return type:

Generator[Tuple[Any, Any, str], None, None]

Examples

>>> # --- get all picks from a catalog object
>>> import obsplus
>>> import obspy.core.event as ev
>>> cat = obsplus.load_dataset('bingham_test').event_client.get_events()
>>> picks = []  # put all the picks in a list.
>>> for pick, _, _ in yield_obj_parent_attr(cat, cls=ev.Pick):
...     picks.append(pick)
>>> assert len(picks)

>>> # --- yield all objects which have resource identifiers
>>> objects = []  # list of (rid, parent)
>>> RID = ev.ResourceIdentifier
>>> for rid, parent, attr in yield_obj_parent_attr(cat, cls=RID):
...     objects.append((str(rid), parent))
>>> assert len(objects)

>>> # --- Create a dict of {resource_id: [(attr, parent), ...]}
>>> from collections import defaultdict
>>> rid_mapping = defaultdict(list)
>>> for rid, parent, attr in yield_obj_parent_attr(cat, cls=RID):
...     rid_mapping[str(rid)].append((attr, parent))
>>> # count how many times each resource_id is referred to
>>> count = {i: len(v) for i, v in rid_mapping.items()}