pyevoc.features.emoji_labelling

Emoji description labelling with codepoint fallback.

This module enriches term-level EVOC outputs with emoji descriptions. It is intended to run after term-level statistics and, typically, after concreteness labelling.

The expected input is a dataframe containing at least:

• term

• upos

The module adds:

• emoji_description

• emoji_in_lookup

• emoji_match_method

• emoji_match_similarity

Matching strategy

Emoji matching is performed through several increasingly permissive passes:

1. exact glyph;

2. exact codepoints;

3. normalised glyph;

4. normalised codepoints;

5. conservative codepoint-similarity fallback.

The normalisation removes variation selectors and dangling zero-width joiners, while preserving the visible base emoji sequence.

class pyevoc.features.emoji_labelling.EmojiLabellingConfig(lookup_file='emoji_lookup.csv', models_dir=None, term_col='term', upos_col='upos', emoji_upos='EMOJI', min_similarity=0.8, verbose=True)

Bases: object

Configuration for emoji description labelling.

Parameters:

• lookup_file (str)

• models_dir (str | Path | None)

• term_col (str)

• upos_col (str)

• emoji_upos (str)

• min_similarity (float)

• verbose (bool)

lookup_file: str = 'emoji_lookup.csv'

models_dir: str | Path | None = None

term_col: str = 'term'

upos_col: str = 'upos'

emoji_upos: str = 'EMOJI'

min_similarity: float = 0.8

verbose: bool = True

pyevoc.features.emoji_labelling.resolve_emoji_lookup_path(lookup_file='emoji_lookup.csv', *, models_dir=None)

Resolve the emoji lookup file path.

Parameters:

• lookup_file (str | Path)

• models_dir (str | Path | None)

Return type:

Path

pyevoc.features.emoji_labelling.available_emoji_lookup_paths(lookup_file='emoji_lookup.csv')

Return a diagnostic table of detected emoji lookup locations.

Parameters:

lookup_file (str)

Return type:

DataFrame

pyevoc.features.emoji_labelling.load_emoji_lookup(path=None, *, models_dir=None, lookup_file='emoji_lookup.csv', encoding='utf-8')

Load an emoji lookup table.

Parameters:

• path (str | Path | None)

• models_dir (str | Path | None)

• lookup_file (str)

• encoding (str)

Return type:

DataFrame

pyevoc.features.emoji_labelling.emoji_codepoints_hex(value)

Return Unicode codepoints for an emoji/string as uppercase hex tokens.

Parameters:

value (object)

Return type:

str

pyevoc.features.emoji_labelling.normalise_emoji_glyph(value)

Conservatively normalise emoji glyphs for lookup matching.

The function removes variation selectors FE0E/FE0F and dangling zero-width joiners, while preserving the visible base emoji sequence.

Parameters:

value (object)

Return type:

str

pyevoc.features.emoji_labelling.codepoint_tokens(value)

Return codepoint tokens excluding variation selectors and ZWJ.

Parameters:

value (object)

Return type:

list[str]

pyevoc.features.emoji_labelling.codepoint_jaccard(a, b)

Return Jaccard similarity between two emoji codepoint sets.

Parameters:

• a (object)

• b (object)

Return type:

float

pyevoc.features.emoji_labelling.prepare_emoji_lookup(lookup)

Prepare a normalised emoji lookup table.

Parameters:

lookup (DataFrame)

Return type:

DataFrame

pyevoc.features.emoji_labelling.add_emoji_descriptions(term_stats_df, *, lookup=None, config=None, models_dir=None, emoji_lookup_file=None, min_similarity=None)

Add emoji descriptions to a term-level dataframe.

Returns:

(out, coverage_df, method_counts_df, missing_df).

Return type:

tuple

Parameters:

• term_stats_df (DataFrame)

• lookup (DataFrame | None)

• config (EmojiLabellingConfig | None)

• models_dir (str | Path | None)

• emoji_lookup_file (str | None)

• min_similarity (float | None)

pyevoc.features.emoji_labelling.label_emojis(terms, *, lookup=None, term_col='term', upos_col='upos', description_col='emoji_description', return_diagnostics=False)

Backward-compatible wrapper for emoji labelling.

Parameters:

• terms (DataFrame)

• lookup (DataFrame | None)

• term_col (str)

• upos_col (str)

• description_col (str)

• return_diagnostics (bool)

Return type:

DataFrame | tuple[DataFrame, DataFrame, DataFrame, DataFrame]