ReaLitE: Knowledge Graph Embedding Model that Enriches Relation Embeddings using Numeric Literals

Introduction

This repository contains the implementation associated with the paper Klironomos A., Zhou B., Zheng Z., Gad-Elrab M., Paulheim H., Kharlamov E. ReaLitE: Enrichment of Relation Embeddings in Knowledge Graphs using Numeric Literals accepted in ESWC 2025. The paper presents a novel Knowledge Graph Embedding (KGE) model, namely ReaLitE, that demonstrates improved performance compared to the state of the art both in link prediction and node classification tasks.

Table of Contents

1. Overview
2. Dependencies
3. Structure
4. Preparation
5. Reproduction of paper's results

Overview

Our contributions are summarized below:

1. We propose ReaLitE, an approach that can be combined with any vanilla KGE method with relation embeddings. In addition, we demonstrate the integration of our method into existing KGE frameworks, highlighting its versatility and ease of adoption.
2. We experiment with different methods of aggregating numeric literals, including an automated method to learn a combination of multiple aggregation types.
3. We evaluate ReaLitE extensively and compare it with state-of-the-art in two tasks: link prediction and node classification. For the former, we evaluate on the standard setting of link prediction, along with a more granular relation-focused evaluation. The results show that our approach is comparable or superior compared to the state-of-the-art methods, particularly on the numeric literals with higher correlation and long-tail relations.

Dependencies

• Dependencies for the outsourced and modified codebase in MKGA/
  • Details in the original repo
• Dependencies for the codebase in experiments/
  • Python 3.10.13
  • Poetry 1.7.1 (https://python-poetry.org/docs/#installation)
  • The remaining dependencies are in pyproject.toml and are installed using poetry (See section Preparation).

Structure

This project consists of a main codebase (in experiments/) and 2 outsourced codebases. The top-level structure is shown below:

├── MKGA    # outsourced and modified codebase for training and testing KGE models on node classification
├── pykeen-with-realite  # outsourced and modified codebase for training and testing KGE models on link prediction, incl. proposed KGE model
└── experiments # codebase for link prediction evaluation as presented in the paper

More detailed structure of the main codebase (experiments/):

├── datasets        # download locations for datasets used for link prediction
│   ├── fb15k237    # FB15k-237 dataset enhanced with numeric literals
│   ├── yago15k     # YAGO15k dataset enhanced with numeric literals
├── results             # link prediction results for ReaLitE incl. the best found configurations
│   ├── FB15K237Literal
│   ├── YAGO15KLiteral
└── model_evaluation        # scripts for extended link prediction evaluation
    └── trained_model_tests # results of additional link prediction tests
        ├── test_with_relation_filter_kga # results of relation-focused link prediction tests for existing KGE models
        ├── test_with_relation_filter_pykeen # results of relation-focused link prediction tests for ReaLitE

The implementation of ReaLitE can be found in pykeen-with-realite/src/pykeen/models/multimodal/ in the following files:

├── base.py
├── complex_realite_variations.py
├── conve_realite_variations.py
├── distmult_realite_variations.py
├── rotate_realite_variations.py
├── transe_realite_variations.py
└── tucker_realite_variations.py

Preparation

1. Run poetry install
2. For downloading the datasets and preparing the KGA codebase, follow the instructions in the Preparation_README.md file.

Reproduction of Paper's Results

Link Prediction Overall Evaluation

python experiments/reproduce_pipeline.py <dataset> <model>

• Options for <dataset>: YAGO15KLiteral or FB15K237Literal
• Options for <model>: TransEReaLitE, DistMultReaLitE, ComplExReaLitE, RotatEReaLitE or TuckERReaLitE

Link Prediction Relation-Focused Evaluation

Evaluation of Existing KGE Models

⚠️ Required: Training of existing KGE models as per provided instructions.

1. Activate Environment: Ensure you are in the Python environment for the main ReaLitE project, not the KGA environment.
2. Navigate: Change your current directory to the root of the ReaLitE project.
3. Run: Execute the following script. This script internally calls the KGA environment (using the KGA_ENV_PYTHON_PATH you configured) to perform tests using the trained KGA models.

   python experiments/model_evaluation/relation_focused_test_kga.py

Evaluation of ReaLitE

⚠️ Required: Training the ReaLitE model. This should be done using either the configurations from the paper (see Link Prediction Overall Evaluation) or the PyKEEN interface.

Note: Step 1 will generate (and overwrite) experiments/model_evaluation/best_runs.csv file, so for each dataset these steps should be repeated.

1. Identify Best Model Instances: Find the best trained model instances per vanilla KGE model on a specific dataset.

   python experiments/model_evaluation/best_runs_finder.py <dataset>

   Options for <dataset>: YAGO15KLiteral or FB15K237Literal
2. Perform Relation-Focused Testing: Execute relation-specific tests using the identified model instances.

   Note: This step is configured for the YAGO15KLiteral dataset. To run it for other datasets, you need to modify the YAGO15K_RELS_WITH_HIGH_LITERAL_CORR, YAGO15K_SYMMETRIC_RELS, and DATASET_CLASS_TO_EVALUATE variables in the script.

   python experiments/model_evaluation/relation_focused_test_pykeen.py

Node Classification

1. Configure: Modify the contents of MKGA/config/multiple_realite.yaml. Choose the dataset(s) and ReaLitE model variation(s) you want to evaluate by tweaking the dataload and embed properties.
2. Overwrite: Copy the contents of MKGA/config/multiple_realite.yaml and use them to overwrite the contents of MKGA/config/multiple.yaml.
3. Activate Environment: Switch to your Python environment created for the MKGA/ codebase.
4. Navigate: Change your current directory to MKGA/src/.

   cd MKGA/src/

   (Adjust path relative to your current location if needed)
5. Run: Execute the auto-evaluation script:

   python autoevaluate.py

License

This software is open-sourced under the AGPL-3.0 license. See the LICENSE file for details. For a list of open source components included in this project, see the file 3rd-party-licenses.txt.

Citation

If you use our software in your scientific work, please cite our paper:

@article{klironomos2025realite,
  title={ReaLitE: Enrichment of Relation Embeddings in Knowledge Graphs using Numeric Literals},
  author={Klironomos, Antonis and Zhou, Baifan and Zheng, Zhuoxun and Mohamed, Gad-Elrab and Paulheim, Heiko and Kharlamov, Evgeny},
  journal={arXiv preprint arXiv:2504.00852},
  year={2025}
}