pyevoc.features.foregrounding#

Token-level structural foregrounding and positional salience.

This module computes the token-level indicators used by PyEvoc to reconstruct the AOE-like salience component of computational Hierarchical Evocation Analysis.

The module is metadata-preserving: every input column is retained and new salience-related columns are appended. Identifiers such as user_id, doc_id, time and source are therefore not removed or modified.

class pyevoc.features.foregrounding.ForegroundingWeights(opening_sentence=0.35, emphasis=0.25, list_or_quote=0.2, intensification=0.2)[source]#

Bases: object

Weights used to compute structural foregrounding salience.

Parameters:
opening_sentence: float = 0.35#
emphasis: float = 0.25#
list_or_quote: float = 0.2#
intensification: float = 0.2#
normalised()[source]#

Return a copy of the weights scaled so that they sum to 1.

Raises:

ValueError – If the weights sum to zero or a negative value.

Return type:

ForegroundingWeights

as_dict()[source]#

Return the weights as a plain dict keyed by indicator name.

Return type:

dict[str, float]

class pyevoc.features.foregrounding.ForegroundingConfig(doc_col='doc_id', user_col='user_id', time_col='time', source_col='source', token_col='text', sentence_col='sentence_id', position_col='position', opening_sentence_value=1, emphasis_pattern=re.compile('(?:\\*\\*|__|<b>|</b>|<strong>|</strong>|_{2,}|\\*{2,})', re.IGNORECASE), list_or_quote_pattern=re.compile('^\\s*(?:-|\\*|>|•|–|—|\\d+[\\.\\)]|[a-zA-Z][\\.\\)])'), intensification_pattern=re.compile('(?:[A-Z]{3,}|!{2,}|\\?{2,}|[!?]{3,})'), weights=ForegroundingWeights(opening_sentence=0.35, emphasis=0.25, list_or_quote=0.2, intensification=0.2), keep_component_scores=True, show_progress=True)[source]#

Bases: object

Configuration for token-level foregrounding indicators.

Parameters:
doc_col: str = 'doc_id'#
user_col: str = 'user_id'#
time_col: str = 'time'#
source_col: str = 'source'#
token_col: str = 'text'#
sentence_col: str = 'sentence_id'#
position_col: str = 'position'#
opening_sentence_value: int = 1#
emphasis_pattern: Pattern = re.compile('(?:\\*\\*|__|<b>|</b>|<strong>|</strong>|_{2,}|\\*{2,})', re.IGNORECASE)#
list_or_quote_pattern: Pattern = re.compile('^\\s*(?:-|\\*|>|•|–|—|\\d+[\\.\\)]|[a-zA-Z][\\.\\)])')#
intensification_pattern: Pattern = re.compile('(?:[A-Z]{3,}|!{2,}|\\?{2,}|[!?]{3,})')#
weights: ForegroundingWeights = ForegroundingWeights(opening_sentence=0.35, emphasis=0.25, list_or_quote=0.2, intensification=0.2)#
keep_component_scores: bool = True#
show_progress: bool = True#
pyevoc.features.foregrounding.validate_token_metadata(tokens, *, doc_col='doc_id', user_col='user_id', time_col='time', require_user=False, require_time=False)[source]#

Validate that the token table preserves key metadata.

Parameters:

  • tokens (DataFrame)

  • doc_col (str)

  • user_col (str)

  • time_col (str)

  • require_user (bool)

  • require_time (bool)

Return type:

None

pyevoc.features.foregrounding.add_token_position(tokens, *, doc_col='doc_id', position_col='position', overwrite=False)[source]#

Add or recompute one-based token positions within documents.

Parameters:

  • tokens (DataFrame)

  • doc_col (str)

  • position_col (str)

  • overwrite (bool)

Return type:

DataFrame

pyevoc.features.foregrounding.add_foregrounding_indicators(tokens, *, sentence_col='sentence_id', token_col='text', opening_sentence_value=1, emphasis_pattern=EMPHASIS_RE, list_or_quote_pattern=LIST_OR_QUOTE_RE, intensification_pattern=INTENSIFICATION_RE)[source]#

Add binary foregrounding indicators to a token table.

Parameters:

  • tokens (DataFrame)

  • sentence_col (str)

  • token_col (str)

  • opening_sentence_value (int)

  • emphasis_pattern (Pattern)

  • list_or_quote_pattern (Pattern)

  • intensification_pattern (Pattern)

Return type:

DataFrame

pyevoc.features.foregrounding.compute_structural_salience(tokens, weights=ForegroundingWeights(), *, keep_component_scores=True, show_progress=True)[source]#

Compute weighted structural salience r_str.

Parameters:
Return type:

DataFrame

pyevoc.features.foregrounding.add_positional_salience(tokens, doc_col='doc_id', position_col='position', *, output_col='r_pos', show_progress=True)[source]#

Compute positional salience r_pos within documents.

Parameters:

  • tokens (DataFrame)

  • doc_col (str)

  • position_col (str)

  • output_col (str)

  • show_progress (bool)

Return type:

DataFrame

pyevoc.features.foregrounding.add_salience_indicators(tokens, *, config=None)[source]#

Add foregrounding, structural salience and positional salience.

Parameters:
Return type:

DataFrame

pyevoc.features.foregrounding.foregrounding_diagnostics(tokens, *, doc_col='doc_id', user_col='user_id', time_col='time')[source]#

Return diagnostics for foregrounding and salience indicators.

Parameters:

  • tokens (DataFrame)

  • doc_col (str)

  • user_col (str)

  • time_col (str)

Return type:

DataFrame

pyevoc.features.foregrounding.metadata_coverage(tokens, *, required_metadata=None)[source]#

Return a diagnostic table describing metadata availability.

Parameters:

  • tokens (DataFrame)

  • required_metadata (list[str] | None)

Return type:

DataFrame

pyevoc.features.foregrounding.assert_metadata_preserved(before, after, *, metadata_cols=None)[source]#

Raise an error if metadata columns were lost during transformation.

Parameters:

  • before (DataFrame)

  • after (DataFrame)

  • metadata_cols (list[str] | None)

Return type:

None