CauseNet and CGF: Saving and Loading a Causal Graph¶

CauseNet is a large causal knowledge graph mined from the web and Wikipedia: each edge is a cause -> effect relation between two concepts, backed by the sentences it was extracted from.

causalatee.graph can load CauseNet directly and convert it to CGF, a compact, memory-mappable binary format for causal graphs (see the CGF 1.0 specification). This notebook loads a small CauseNet sample, saves it as a .cgf file, and loads that file back.

Setup¶

causalatee depends on fastavro (used to encode/decode CGF metadata records), so no extra install flags are needed.

In [ ]:

%pip install -q causalatee

%pip install -q causalatee

Load a CauseNet sample¶

load_causenet accepts a local path or an HTTP(S) URL, with gzip/bzip2 detected automatically. The full CauseNet precision graph has millions of edges, so pass limit to bound how many JSONL lines are read -- this is an eager loader that materializes everything it reads as Python objects.

In [ ]:

from causalatee.graph import load_causenet, load_cgf, save_cgf

CAUSENET_PRECISION_URL = "https://groups.uni-paderborn.de/wdqa/causenet/causality-graphs/causenet-precision.jsonl.bz2"

graph = load_causenet(CAUSENET_PRECISION_URL, limit=50)
print(f"Loaded {len(graph.nodes)} nodes and {len(graph.edges)} edges")

from causalatee.graph import load_causenet, load_cgf, save_cgf CAUSENET_PRECISION_URL = "https://groups.uni-paderborn.de/wdqa/causenet/causality-graphs/causenet-precision.jsonl.bz2" graph = load_causenet(CAUSENET_PRECISION_URL, limit=50) print(f"Loaded {len(graph.nodes)} nodes and {len(graph.edges)} edges")

Expected output:

Loaded 49 nodes and 50 edges

In [ ]:

node = graph.get_node("accident")
top_effects = sorted(node.outgoing_edges(), key=lambda edge: edge.support, reverse=True)[:3]
for edge in top_effects:
    print(f"{edge.cause.concept!r} -> {edge.effect.concept!r} (support={edge.support})")

node = graph.get_node("accident") top_effects = sorted(node.outgoing_edges(), key=lambda edge: edge.support, reverse=True)[:3] for edge in top_effects: print(f"{edge.cause.concept!r} -> {edge.effect.concept!r} (support={edge.support})")

Expected output:

'accident' -> 'death' (support=38)
'accident' -> 'injury' (support=32)
'accident' -> 'injuries' (support=27)

Save as CGF¶

save_cgf accepts any causalatee.graph.Graph, not just CauseNet. It streams nodes and edges through a disk-backed sort rather than holding everything in memory, and by default reuses the schemas the adapter defines for its own metadata (here, CauseNetEdgeMetadataLean is not yet in play -- this is the CauseNet adapter's own default schema, which embeds each relation's full supporting-sentence provenance in the sources field).

In [ ]:

import os

save_cgf(graph, "causenet-sample.cgf")

print(f"{os.path.getsize('causenet-sample.cgf'):,} bytes")

import os save_cgf(graph, "causenet-sample.cgf") print(f"{os.path.getsize('causenet-sample.cgf'):,} bytes")

Expected output:

17,717,600 bytes

That is large for just 50 edges: CauseNet's sources provenance can include thousands of full supporting sentences for well-attested relations like accident -> death. Keep this in mind when converting a larger slice of CauseNet -- see below for how to drop it.

Load the CGF file back¶

load_cgf memory-maps the file; nodes and edges are resolved lazily rather than being loaded into Python objects up front. Passing validate=True additionally performs a full structural and metadata check (sorted IDs, in-bounds offsets, valid Avro records for every node and edge).

In [ ]:

with load_cgf("causenet-sample.cgf", validate=True) as mapped:
    print(f"Mapped {len(mapped.nodes)} nodes and {len(mapped.edges)} edges")

    node = mapped.get_node("accident")
    top_effects = sorted(node.outgoing_edges(), key=lambda edge: edge.metadata["support"], reverse=True)[:3]
    for edge in top_effects:
        print(f"{edge.source.id!r} -> {edge.target.id!r} (support={edge.metadata['support']})")

with load_cgf("causenet-sample.cgf", validate=True) as mapped: print(f"Mapped {len(mapped.nodes)} nodes and {len(mapped.edges)} edges") node = mapped.get_node("accident") top_effects = sorted(node.outgoing_edges(), key=lambda edge: edge.metadata["support"], reverse=True)[:3] for edge in top_effects: print(f"{edge.source.id!r} -> {edge.target.id!r} (support={edge.metadata['support']})")

Expected output:

Mapped 49 nodes and 50 edges
'accident' -> 'death' (support=38)
'accident' -> 'injury' (support=32)
'accident' -> 'injuries' (support=27)

Note that a CGFGraph node's metadata is a plain dict decoded from Avro on access, so edge.metadata["support"] replaces the edge.support convenience property CauseNetEdge exposed.

Shrinking the file with a custom metadata schema¶

save_cgf lets you override either adapter schema, which is the documented way to drop fields you do not need -- for example, keeping only support and leaving out sources entirely:

In [ ]:

lean_edge_schema = {
    "type": "record",
    "name": "CauseNetEdgeMetadataLean",
    "namespace": "causalatee.graph.causenet",
    "fields": [{"name": "support", "type": "long"}],
}

save_cgf(graph, "causenet-sample-lean.cgf", edge_metadata_schema=lean_edge_schema)
print(f"{os.path.getsize('causenet-sample-lean.cgf'):,} bytes")

lean_edge_schema = { "type": "record", "name": "CauseNetEdgeMetadataLean", "namespace": "causalatee.graph.causenet", "fields": [{"name": "support", "type": "long"}], } save_cgf(graph, "causenet-sample-lean.cgf", edge_metadata_schema=lean_edge_schema) print(f"{os.path.getsize('causenet-sample-lean.cgf'):,} bytes")

Expected output:

4,400 bytes

Dropping the per-edge provenance shrinks this 50-edge sample from ~17 MB to ~4 KB. edge.metadata now only contains the fields declared in the lean schema:

In [ ]:

with load_cgf("causenet-sample-lean.cgf") as mapped:
    edge = next(iter(mapped.get_node("accident").outgoing_edges()))
    print(edge.metadata)

with load_cgf("causenet-sample-lean.cgf") as mapped: edge = next(iter(mapped.get_node("accident").outgoing_edges())) print(edge.metadata)

Expected output:

{'support': 38}