Skip to main content

SEMScore

A metric that measures semantic similarity between candidate (generated) text and reference text using embeddings and cosine similarity.

  • Inputs: candidate (generated) text and reference text.
  • Returns: a single SEMScore value.

Insights​

SEMScoreInference
closer to 1more semantically similar
closer to 0unrelated
negative scoresemantically opposite

Example​

from vero.metrics import SEMScore

#example inputs
#chunks_list = ["The cat sat on the mat.", "The dog barked at the mailman."]
#answers_list = ["A cat is sitting on a mat and a dog is barking at the mailman."]
with SEMScore() as ss:
    sem_results = [ss.evaluate(chunk, ans) for chunk, ans in zip(chunks_list, answers_list)]
print(sem_results)

Output​

0.92