Python API

The LNN API uses an intuitive pythonic approach for learning, reasoning and interacting with neural representations of first-order logic knowledge bases.

The API follows a hierarchy of interacting components, allowing designers to construct and interact with knowledge at a desired level of granularity.

Hierarchical Class Structure

  1. Model

    A model provides a context or theory of reference for which reasoning or training can be applied.

  2. Symbolic Nodes

    Nodes are human-interpretable containers that follow a predefined category of behavior or compute according to the type of node being specified. These nodes may directly be instantiated by the designer as prior knowledge or extracted from templates to fit the data.

  3. Neural Computations

    The underlying computations used by real-valued logical operations are weighted in nature, typically using parameterised extensions of the standard t-norms: Łukasiewicz, Gödel and Product logic

Using the LNN

Symbolic Structure

  1. Dynamic Models

    • An LNN model can be initialised as an “empty container” and populated on-demand with the knowledge and data required to compute over. 

      from lnn import Model

model = Model()

    • This functionality becomes of special interest in programmatic environments that may have discoverable information that requires reasoning over the new information while simultaneously retaining previously stored or inferred facts.

    • Model knowledge and facts can be initiated with model constructors or populated with content on-demand

  2. Knowledge Represention

    1. Predicates are the base building-blocks for a grounded first-order logic system like the LNN. Predicates provide both properties of and relations between objects. Unary predicates provide properties and n(>1)-ary predicates provided relations. Predicates in LNNs explicitly store tables of information along with truth values associated with each row of the truth table. 

      from lnn import Predicates

Smokes, Cancer = Predicates('Smokes', 'Cancer')
Friends = Predicates('Friends', arity=2)

      • The arity (default = 1) represents the number of columns in the table, where the rows will be filled by facts in the table.

      • The inputs for each row of the truth table form a unique Grounding that tells the predicate which row is being operated on.

      • Predicates (and Propositions ) require names, e.g., 'Smokes' , which is used to identify the node within the scope of the model

    2. Where standard neural networks abstractly model knowledge into a neural architecture, LNNs do so explicitly, using first-order logic formulae

      • An explicit representation constructs neurons directly from the formulae, using a 1-to-1 mapping to construct the model neurons. The model architecture is therefore symbolically defined and precise, as defined by the required semantic expression.

      • First we instantiate a free variable 

        from lnn import Variables

x, y = Variables('x', 'y')

        so that “grounding management” understands how to compose subformulae during table joining operations

        • FOL formula can be constructed using neural connectives as follows: 

          from lnn import Implies, Iff

Smoking_causes_Cancer = Implies(Smokes(x), Cancer(x))
Smokers_befriend_Smokers = Implies(Friends(x, y), Iff(Smokes(x), Smokes(y)))

          • All nodes requiring handles for lookup or directly for symbolic operations, should include a name via the kwarg

          Root level formulae can be collected and inserted into the model en masse: 
          from lnn import World

formulae = [
    Smoking_causes_Cancer
    Smokers_befriend_Smokers
]
model.add_knowledge(*formulae, world=World.AXIOM)

          • Additional World assumptions include: [World.OPEN, World.CLOSED] to enforce that formulae follow the open/closed world assumption

          • The compositional structure of the LNN will recursively build up a connective tree structure per root formula and the insertion of a root will also include all connected subformulae.

          • Formulae can also be modified to follow a truth world assumption using the kwarg, i.e., OPEN or CLOSED world assumption .

            • This places restrictions on the symbolic truths to only consider worlds where the facts are initialised as UNKNOWN or FALSE accordingly.

            • By default, LNNs make no assumption of perfect knowledge and opt for OPEN assumptions on all formulae unless otherwise specified.
              e.g. A safer assumption from the above scenario may assume 

              Cancer = Predicate('Cancer', world=World.CLOSED)

              to prevent false negatives from downward inference, opting to trust the data instead of assuming the formulae perfectly describes the data

          • Alternatively a formula can be defined as an AXIOM , thereby limiting truths to worlds where facts are assumed TRUE

    3. Connectives are “neural” due to their parameterised computation or implementation as a weighted real-value t-norm.

      • The base neural connectives in the LNN are And , Or , Implies

      • Note that Not gates, albeit a connective, is not neural in nature

      • Compound connectives, e.g., Iff , Xor , etc. are implemented from these 4 connectives

      • The parameterised configuration of each neuron can be overloaded using the neuron kwarg. See [neuron module] documentation for more.

  3. Tables of Data 

    from lnn import Fact

# add data to the model
model.add_data({
    Friends: {
        ('Anna', 'Bob'): Fact.TRUE,
        ('Bob', 'Anna'): Fact.TRUE,
        ('Anna', 'Edward'): Fact.TRUE,
        ('Edward', 'Anna'): Fact.TRUE,
        ('Anna', 'Frank'): Fact.TRUE,
        ('Frank', 'Anna'): Fact.TRUE,
        ('Bob', 'Chris'): Fact.TRUE},
    Smokes.name: {
        'Anna': Fact.TRUE,
        'Edward': Fact.TRUE,
        'Frank': Fact.TRUE,
        'Gary': Fact.TRUE},
    Cancer.name: {
        'Anna': Fact.TRUE,
        'Edward': Fact.TRUE}
    })
model.print()

    Output: 

    *************************************************************
                             LNN Model

 AXIOM  Implies: Smokers befriend Smokers('x', 'y') 

 OPEN   Iff: Iff_0('x', 'y') 

 OPEN   Implies: Implies_2('y', 'x') 

 OPEN   Implies: Implies_1('x', 'y') 

 OPEN   Predicate: Friends
        ('Bob', 'Anna')                     TRUE (1.0, 1.0)
        ('Anna', 'Bob')                     TRUE (1.0, 1.0)
        ('Anna', 'Edward')                  TRUE (1.0, 1.0)
        ('Frank', 'Anna')                   TRUE (1.0, 1.0)
        ('Bob', 'Chris')                    TRUE (1.0, 1.0)
        ('Edward', 'Anna')                  TRUE (1.0, 1.0)
        ('Anna', 'Frank')                   TRUE (1.0, 1.0)

 AXIOM  Implies: Smokers have Cancer(x) 

 CLOSED Predicate: Cancer
        'Edward'                            TRUE (1.0, 1.0)
        'Anna'                              TRUE (1.0, 1.0)
 
 OPEN   Predicate: Smokes
        'Frank'                             TRUE (1.0, 1.0)
        'Edward'                            TRUE (1.0, 1.0)
        'Anna'                              TRUE (1.0, 1.0)
        'Gary'                              TRUE (1.0, 1.0)

 ************************************************************

  4. Reasoning

    • Using the above formulae and specified facts, inference runs across the entire model until convergence - inferring all possible facts that can be inferred.

    • inference can be computed over the entire model by calling 

      model.infer()

  5. Learning

    • Learning in the LNN can be done with supervisory signals but also under self supervision, using the rules to enforce logical consistency.

    • Supervisory learning uses labelled targets on truth bounds, which can be set as follows: 

      from lnn import Loss

# add supervisory targets
model.add_labels({
    'Smokes': {
        'Ivan': Fact.TRUE,
        'Nick': Fact.TRUE}
    })

# train the model and output results
model.train(losses=Loss.SUPERVISED)
model.print(params=True)

    • Self-supervisory signals can also enforce logical consistency, i.e., ensuring that the lower bounds do not cross above the upper bounds. This can be set by adding additional losses to the kwarg, to jointly optimise multiple loss functions. 

      losses=[Loss.SUPERVISED, Loss.CONTRADICTION]
model.train(losses=losses)

    • The following are additional losses that may be of interest: losses

      • Logical loss

        • Logical losses enforce the logical constraints, see Section E for more details.

          • Enforcement of constraints are required for environments that require a ‘hard-logic’ interpretation of neurons, however, this presents the LNN with a non-trivial optimisation problem - especially in the classical case where noise is present

          • A soft-logic is however more desirable in most situation, allowing accuracy to be a more prominent goal than interpretability - which is the case for most machine learning scenarios.

      • Uncertainty loss

        • Uncertainty losses are simply the upper - lower bounds, allowing the parameter configuration to tighten bounds and thereby minimise the amount of uncertainty within the output truths.

          • This configuration is more readily used within scenarios that have non-classical inputs, i.e., uncertainty bounds on real-valued (0, 1) truths - excluding the extremes.

    • Losses can also be appended to the training list, enabling multi-task learning across the model

  6. Interpretability

    • Printing

      • You would have noticed that the LNN is symbolically interpretable for every subformula in the model, which can easily be extracted using a print out: 

        model.print()

        This prints out the groundings and respective truths for each node in the graph.

      • The above printout may, however, be very large for models with a large amount of knowledge and data. A subset of the model can also be printed out: 

        model.print(source=Smokers_befriend_Smokers)

        This uses the source node as a root, and a print out will be done for all nodes that are chilrden of the root.

      • Alternatively, a print out can be done for just a single node 

        Smokers_befriend_Smokers.print()

    • States

      • For every truth value associated with a grounding, there is an associated state (from 9 possible classes).

      • If the states of truths need to be extracted, this allows the state of a formula or a particular grounding to extracted 

        Cancer.state('Edward')

        returns a single state construct, whereas 

        Cancer.state()

        returns a dictionary of states, keyed by the groundings

Activation Configuration

Neural Configuration

The LNN offers a generic framework for computing symbolic connectives via different classes of neural computations according to the user specification. We can therefore keep the symbolic containers the same and switch out the underlying activations for different results.

Neural parameters:

parameter

type

type

NeuralActivation

Some design choices

  • While the LNN can handle large graphs, the current strategy of unguided reasoning or reasoning over all possible points, can be time intensive.

    • Bottlenecks being addressed include:

      • loading large numbers of formulae sequentially

      • efficiency of grounding management for large tables and deep formulae

  • “Grounding management” or the composition of subformulae tables according to the variable configuration is therefore kept to a minimal by opting for inner-joins instead of outer/product joins.

  • Albeit more efficient, these joins can still be computationally intensive (2N^2 in some situations) for large n-ary tables. The length of connectives should therefore be kept as compact as possible.