hosh - Operable HaSH

This library provides a richer approach to hashing.
Here, hash digests can be combined through a reversible operation. In practice, this means that, e.g., a sequence of data transformation steps can be identified by a single hash, instead of a list of hashes. We call each such identifier a `hosh` (operable hash). A handful of colored pretty printing methods is also featured to ease visual comparison between hashes.

Basic Usage

The library is mainly represented by the class Hosh which provides some math operators and useful methods.

Multiplication

A product of hoshes generates a new hosh as shown below, where sequences of bytes (b"···") are provided as arguments to simulate some binary objects intended to be hashed. Here, multiplication is non commutative (probabilistically speaking - see our paper).

Examples

from hosh.theme import HTML
from hosh import Hosh, ø, setup  # ø is a shortcut for identity (AltGr+O in most keyboards)

# Hoshes (operable hash-based elements) can be multiplied.
a = Hosh(content=b"Some large binary content...")
b = Hosh(content=b"Some other binary content. Might be, e.g., an action or another large content.")
c = a * b
print(f"{a} * {b} = {c}")

8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 * 7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = z3EgxfisgqbNXBd0eqDuFiaTblBLA5ZAUbvEZgOh


# Colored:
print(f"{a.html} * {b.html} = {c.html}")

8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 * 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = 
z3EgxfisgqbNXBd0eqDuFiaTblBLA5ZAUbvEZgOh


# Global setup for light HTML (web) colors instead of dark ANSI (terminal).
setup(dark_theme=False, format=HTML)
Hosh.__str__ = Hosh.__repr__  # <- a little trick to allow print() in colors
print(~b)

Q6OjmYZSJ8pB3ogBVMKBOxVp-oZ80czvtUrSyTzS


# Multiplication can be reverted by the inverse hosh. Zero is the identity hosh.
print(f"{b} * {~b} = {b * ~b} = 0")

7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 * 
Q6OjmYZSJ8pB3ogBVMKBOxVp-oZ80czvtUrSyTzS = 
0000000000000000000000000000000000000000 = 0


print(f"{b} * {ø} = {b * ø} = b")

7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 * 
0000000000000000000000000000000000000000 = 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = b


print(f"{c} * {~b} = {c * ~b} = {a} = a")

z3EgxfisgqbNXBd0eqDuFiaTblBLA5ZAUbvEZgOh * 
Q6OjmYZSJ8pB3ogBVMKBOxVp-oZ80czvtUrSyTzS = 
8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 = 
8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 = a


print(f"{~a} * {c} = {~a * c} = {b} = b")

RNvSdLI-5RiBBGL8NekctiQofWUIeYvXFP3wvTFT * 
z3EgxfisgqbNXBd0eqDuFiaTblBLA5ZAUbvEZgOh = 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = b


# Division is shorthand for reversion.
print(f"{c} / {b} = {c / b} = a")

z3EgxfisgqbNXBd0eqDuFiaTblBLA5ZAUbvEZgOh / 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = 
8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 = a


# Hosh multiplication is not expected to be commutative.
print(f"{a * b} != {b * a}")

z3EgxfisgqbNXBd0eqDuFiaTblBLA5ZAUbvEZgOh != 
wwSd0LaGvuV0W-yEOfgB-yVBMlNLA5ZAUbvEZgOh


# Hosh multiplication is associative.
print(f"{a * (b * c)} = {(a * b) * c}")

RuTcC4ZIr0Y1QLzYmytPRc087a8cbbW9Nj-gXxAz = 
RuTcC4ZIr0Y1QLzYmytPRc087a8cbbW9Nj-gXxAz


# An element can be multiplied by itself a certain number of times by the `^` operator.
print(f"a³ = {a ^ 3} = {a * a * a}")

a³ = 
pFBLMSKyBffSXQSdmgPgW4cgWFbH4wwd2Niuxp2p = 
pFBLMSKyBffSXQSdmgPgW4cgWFbH4wwd2Niuxp2p

Addition

The order of elements has no meaning in some scenarios, e.g.: items inside a bag; or, steps that produce the same result despite which one is performed before another one. This requires a commutative operation - provided by the + operator.

Examples

from hosh.theme import HTML
from hosh import Hosh, ø, setup  # ø=identity

# Enable HTML colors. See 'multiplication' example for details.
setup(dark_theme=False, format=HTML)
Hosh.__str__ = Hosh.__repr__

# Hoshes can be added.
a = Hosh(content=b"Some large binary content...")
b = Hosh(content=b"Some other binary content. Might be, e.g., an action or another large content.")
c = a + b
print(f"{a} + {b} = {c}")

8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 + 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = 
4i8YqqVswkwbMv2NKCB.p2JpsuLLA5ZAUbvEZgOh


# Addition can be reverted by subtraction. Zero is the identity hosh.
# Warning: unary operators `+b` and `-b` are not related to addition nor inversion.
# They are reserved for the 'lift' operation - see advanced example later in documentation.
print(f"{b} - {b} = {b - b} = 0")

7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 - 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = 
0000000000000000000000000000000000000000 = 0


print(f"{b} + {ø} = {b + ø} = b")

7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 + 
0000000000000000000000000000000000000000 = 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = b


print(f"{c} - {a} = {c - a} = {b} = b")

4i8YqqVswkwbMv2NKCB.p2JpsuLLA5ZAUbvEZgOh - 
8CG9so9N1nQ59uNO8HGYcZ4ExQW5Haw4mErvw8m8 = 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = 
7N-L-10JS-H5DN0-BXW2e5ENWFQFVWswyz39t8s9 = b


# Hosh addition is always commutative.
print(f"{a + b} == {b + a}")

4i8YqqVswkwbMv2NKCB.p2JpsuLLA5ZAUbvEZgOh == 
4i8YqqVswkwbMv2NKCB.p2JpsuLLA5ZAUbvEZgOh


# Hosh addition is associative.
print(f"{a + (b + c)} = {(a + b) + c}")

lywZuVigAtLiwnN02fb.9C7PUYecbbW9Nj-gXxAz = 
lywZuVigAtLiwnN02fb.9C7PUYecbbW9Nj-gXxAz

Advanced Usage

This package works well out of the box for most usages. However, some scenarios can take advantage of the math features embbeded in the library.

Element types

Some elements are commutative for multiplication. They are rare enough to be probabilistically inexistent for a normal usage of hoshes. However, their subgroup is still large enough to provide interesting features.

Examples

from hosh.theme import HTML
from hosh import Hosh, ø, setup  # ø=identity

print("ongoing documentation work!")

ongoing documentation work!

Lifting

hosh provides a lifting mechanism inspired by a category theory concept used in Haskell. Lifting is achieved by applying a unary operator to map a commutative element to a non commutative subgroup. Operator + maps to higher lexicographic rank, whilst operator - maps to a lower lexicographic rank. This enables us to temporarily impose order when combining two commutative elements.

Examples

from hosh.theme import HTML
from hosh import Hosh, ø, setup  # ø=identity

print("ongoing documentation work!")

ongoing documentation work!

Real-world Usage

A full fledged application using hoshes is hdict - presented below. More information can be found at: Presentation, packages (hosh, hdict) or code repositories (hosh, hdict).

Author Statement (CRediT)

Davi Pereira-Santos: Conceptualization, Investigation, Software, Validation, Writing, Review & Editing. Gabriel Dalforno Silvestre: Formal analysis, Software (groups prospection), Writing (Sections 4.2 and 5). André C. P. L. F. Carvalho: Supervision, Funding acquisition, Resources, Review & Editing.

Acknowledgment

The seminal work that inspired the evolving ideias in this library was supported by CNPq and FAPESP [grant numbers 2013/07375-0, 2019/01735-0 (CEPID CeMEAI)]. We are also grateful for the initial advice from Mark Gritter and Jyrki Lahtonen in some topics of group theory.

hdict { a unique data structure }

Overview


Shortly: A data structure based on a novel identification paradigm useful for frictionless computing, experiments, distributed data, among others.
Formally: Hosh-based cacheable lazy dict with predictable/deterministic universally (probabilistically guaranteed) unique identifiers.

The Concept


`hdict` is like an ordinary `dict` with `str` keys. Each entry, called field, and the `hdict` itself, are identified by 40-digit hashes (see hosh). A `hdict` object (say `d`) provides two extra entries: `_id` (hdict identifier) and `_ids` (field identifiers), accessible through square brackets or through the shortcuts `d.id` and `d.ids`.

Examples

from hdict import hdict
from hosh.theme import HTML
from hosh import setup, Hosh

# For better integration within the documentation, we change the color theme.
setup(dark_theme=False, format=HTML)

# From named arguments.
d = hdict(x=5, y=7, z=10)

# From a dict object.
d = hdict({"x": 5, "y": 7, "z": 10})

# From an empty 'hdict' object.
d = hdict() >> {"x": 5} >> {"y": 7, "z": 10}

# All three options have the same result.
d.show()

{
    x: 5,
    y: 7,
    z: 10,
    _id: 
BN-3Q3Z.2Q.9nsbIYnOI75HT7xhgjvF6wErwBPTn,
    _ids: {
        x: 
ecvgo-CBPi7wRWIxNzuo1HgHQCbdvR058xi6zmr2,
        y: 
eJCW9jGsdZTD6-AD9opKwjPIOWZ4R.T0CG2kdyzf,
        z: 
u-Yykj2nDtKaUMGzfqScX5Y14qC7eqJrO7lXrJ1m
    }
}

A field contains a value or a function application. A field pointing to an application is only evaluated on demand, i.e., lazily.

This documentation is still an ongoing work. It has been growing steadly. Last update: Feb/2023

Short syntax

...

hosh - Operable HaSH

Basic Usage

Multiplication

Addition

Advanced Usage

Element types

Lifting

Real-world Usage

Author Statement (CRediT)

Acknowledgment

hdict { a unique data structure }

Overview

The Concept

This documentation is still an ongoing work. It has been growing steadly. Last update: Feb/2023

More advanced topics will be added in the near future.

Short syntax