U4RASD Takes 1st Place at KnowledgeGraphEval-2026: 94.06 micro-F1 on Arabic

Team U4RASD won Subtask 2 at KnowledgeGraphEval-2026, a competition on directed relation extraction from Arabic text. Its final micro-F1 was 94.06. The score came from an ensemble of three systems: Yehia-RE, Qwen-RE and the TypedPair encoder. The best single model, Yehia-RE, stopped at 92.98, which is 1.08 points lower.

What the systems had to do

The input is an Arabic sentence and a subject–object pair. The output is one of 40 predefined relations, or the no_relation label if the entities are not related. Direction matters: "A owns B" and "B owns A" count as different answers. The no_relation label makes the task harder than ordinary classification. The system has to tell the absence of a relation from a weakly expressed one, and it must not invent a relation where none exists.

This is a typical step in building knowledge graphs. Triples are extracted from a text corpus, and the accuracy of that extraction determines the quality of the whole graph. Arabic adds difficulty: rich morphology, clitics that attach to words, and word order that depends on the genre of the text.

Two camps: encoders and LLMs

The team built two types of systems and compared them. The encoder models received typed entity markers in the sentence, that is, markup indicating the type of the subject and the object. The solution is split into two heads: one answers whether a relation exists at all, and the other picks its type. This decoupling takes the no_relation pressure off the relation-type classifier.

The second line is LLMs. The authors fine-tuned Yehia and Qwen3.5 with QLoRA, a method that makes it possible to tune large models on limited hardware. The key technique is ontology-constrained prediction: the model picks a label from the permitted set instead of generating free text. In the abstract, the Yehia-RE variant is labeled full-vocabulary. The abstract does not reveal the details of the comparison with the constrained version, so these numbers alone do not show how much each technique contributed. The authors themselves, however, name constrained prediction as one of the main lessons of the work.

The numbers

• Ensemble of Yehia-RE + Qwen-RE + TypedPair: micro-F1 94.06, first place in Subtask 2.
• Yehia-RE (full-vocabulary): 92.98.
• Best encoder: 92.69.
• Qwen-RE: 91.94.
• The ensemble's margin over the best single system: 1.08 points; over the best encoder: 1.37; over Qwen-RE: 2.12.

The most interesting result here is the encoder. It trails Yehia-RE by only 0.29 points and beats Qwen-RE by 0.75. For a narrow task with a fixed label set, a compact model with well-designed input markup remains a serious competitor to a fine-tuned LLM. That matters because an encoder is typically cheaper and faster at inference. The abstract contains no such measurements, so this is a general consideration, not a result from the paper.

It is also worth noting that the ensemble won through diversity. The encoder and the generative models make different kinds of errors, and combining them yields a gain you cannot get by simply adding another model from the same family. The authors themselves frame their conclusion as the importance of complementary approaches together with constrained prediction.

What to take from this

If you are building relation extraction for Arabic or another lower-resource language, the paper suggests several practical steps. First, test a strong encoder with typed markers: it is a cheap baseline that an LLM will have to genuinely beat. Second, when you use an LLM, constrain its output to the label set from the ontology instead of leaving generation free. QLoRA is suitable for fine-tuning and lowers the memory barrier to entry. Finally, if a gain of one to two points on the leaderboard justifies the cost, build an ensemble from models with different architectures.

The limitations are also clear. This is a description of one system in one competition, and the abstract gives neither the training set size, nor inference time, nor a per-class breakdown. Before transferring the approach to your own data, you will need to measure these parameters yourself.