pyevoc.analysis

EVOC quadrant assignment, collocation and entity analysis, and temporal stability diagnostics.

PyEvoc analysis layer.

This subpackage contains EVOC quadrant assignment, compact HTML reports, dependency-based collocations, named-entity n-grams, temporal stability and quadrant-mobility analysis. Public objects are loaded lazily for documentation and faster package import.

class pyevoc.analysis.CollocationEntityConfig(output_dir='evoc_outputs', entity_n=2, min_freq=3, min_docs=3, min_users=3, g2_alpha=0.001, caps_thr=0.6, min_caps_obs=20, include_collocations=True, include_entities=True, resolve_overlap=True, prefer_overlap='evidence', write_html=True, html_max_colloc_rows=60, html_max_entity_rows=60, html_max_overlap_rows=50, doc_col='doc_id', user_col='user_id', sentence_col='sentence_id', token_id_col='token_id', token_col='token', lemma_col='lemma', upos_col='upos', surface_col='token', head_col='head_token_id', dep_rel_col='dep_rel', colloc_relations=<factory>, false_entity_terms=<factory>, stop_words=<factory>, verbose=True)

Bases: object

Configuration for collocation and named-entity extraction.

Parameters:

• output_dir (str | Path)

• entity_n (int)

• min_freq (int)

• min_docs (int)

• min_users (int)

• g2_alpha (float)

• caps_thr (float)

• min_caps_obs (int)

• include_collocations (bool)

• include_entities (bool)

• resolve_overlap (bool)

• prefer_overlap (str)

• write_html (bool)

• html_max_colloc_rows (int)

• html_max_entity_rows (int)

• html_max_overlap_rows (int)

• doc_col (str)

• user_col (str)

• sentence_col (str)

• token_id_col (str)

• token_col (str)

• lemma_col (str)

• upos_col (str)

• surface_col (str)

• head_col (str)

• dep_rel_col (str)

• colloc_relations (set[str])

• false_entity_terms (set[str])

• stop_words (set[str])

• verbose (bool)

caps_thr: float = 0.6

dep_rel_col: str = 'dep_rel'

doc_col: str = 'doc_id'

entity_n: int = 2

g2_alpha: float = 0.001

head_col: str = 'head_token_id'

html_max_colloc_rows: int = 60

html_max_entity_rows: int = 60

html_max_overlap_rows: int = 50

include_collocations: bool = True

include_entities: bool = True

lemma_col: str = 'lemma'

min_caps_obs: int = 20

min_docs: int = 3

min_freq: int = 3

min_users: int = 3

output_dir: str | Path = 'evoc_outputs'

prefer_overlap: str = 'evidence'

resolve_overlap: bool = True

sentence_col: str = 'sentence_id'

surface_col: str = 'token'

token_col: str = 'token'

token_id_col: str = 'token_id'

upos_col: str = 'upos'

user_col: str = 'user_id'

verbose: bool = True

write_html: bool = True

colloc_relations: set[str]

false_entity_terms: set[str]

stop_words: set[str]

class pyevoc.analysis.QuadrantConfig(minimal_freq=2, round_digits=2, focal_upos=<factory>, quadrant_order=<factory>, diffusion_basis='user_penetration', term_col='term', upos_col='upos', term_type_col='term_type', n_docs_col='n_docs', n_posts_col='n_posts', n_users_col='n_users', r_pos_col='R_pos', r_struct_col='R_struct', salience_col='S', rank_col='Rank', freq_col='freq_for_quadrant', user_penetration_col='user_penetration', document_penetration_col='document_penetration', post_penetration_col='post_penetration', posts_per_user_col='posts_per_user', concreteness_score_col='concreteness_score', concreteness_label_col='concreteness_label', concreteness_in_lexicon_col='concreteness_in_lexicon', emoji_description_col='emoji_description', human_readable=True, generate_html=False, html_output_dir='evoc_outputs', html_top_n=20, html_max_width_px=950, verbose=True)

Bases: object

Configuration for EVOC quadrant assignment and optional HTML reporting.

Parameters:

• minimal_freq (int)

• round_digits (int)

• focal_upos (set[str])

• quadrant_order (list[str])

• diffusion_basis (str)

• term_col (str)

• upos_col (str)

• term_type_col (str)

• n_docs_col (str)

• n_posts_col (str)

• n_users_col (str)

• r_pos_col (str)

• r_struct_col (str)

• salience_col (str)

• rank_col (str)

• freq_col (str)

• user_penetration_col (str)

• document_penetration_col (str)

• post_penetration_col (str)

• posts_per_user_col (str)

• concreteness_score_col (str)

• concreteness_label_col (str)

• concreteness_in_lexicon_col (str)

• emoji_description_col (str)

• human_readable (bool)

• generate_html (bool)

• html_output_dir (str | Path)

• html_top_n (int)

• html_max_width_px (int)

• verbose (bool)

concreteness_in_lexicon_col: str = 'concreteness_in_lexicon'

concreteness_label_col: str = 'concreteness_label'

concreteness_score_col: str = 'concreteness_score'

diffusion_basis: str = 'user_penetration'

document_penetration_col: str = 'document_penetration'

emoji_description_col: str = 'emoji_description'

freq_col: str = 'freq_for_quadrant'

generate_html: bool = False

html_max_width_px: int = 950

html_output_dir: str | Path = 'evoc_outputs'

html_top_n: int = 20

human_readable: bool = True

minimal_freq: int = 2

n_docs_col: str = 'n_docs'

n_posts_col: str = 'n_posts'

n_users_col: str = 'n_users'

post_penetration_col: str = 'post_penetration'

posts_per_user_col: str = 'posts_per_user'

r_pos_col: str = 'R_pos'

r_struct_col: str = 'R_struct'

rank_col: str = 'Rank'

round_digits: int = 2

salience_col: str = 'S'

term_col: str = 'term'

term_type_col: str = 'term_type'

upos_col: str = 'upos'

user_penetration_col: str = 'user_penetration'

verbose: bool = True

focal_upos: set[str]

quadrant_order: list[str]

class pyevoc.analysis.TemporalStabilityConfig(n_periods=4, time_period_mode='equal_posts', custom_time_breaks=None, custom_time_cuts=None, period_labels=None, right=True, alpha=0.5, max_rank_value=5.0, use_users_for_frequency=True, focal_upos=<factory>, round_digits=2, diffusion_multiplier=100.0, time_col='time', doc_col='doc_id', user_col='user_id', term_col='term', upos_col='upos', r_pos_col='r_pos', r_str_col='r_str', expected_min_timestamp=None, expected_max_timestamp=None, warn_on_time_range_mismatch=False, output_dir='evoc_outputs', write_html_report=True, show_tables=False, verbose=True)

Bases: object

Configuration for temporal stability analysis.

Parameters:

• n_periods (int)

• time_period_mode (str)

• custom_time_breaks (list[str] | None)

• custom_time_cuts (list[str] | None)

• period_labels (list[str] | None)

• right (bool)

• alpha (float)

• max_rank_value (float)

• use_users_for_frequency (bool)

• focal_upos (list[str])

• round_digits (int)

• diffusion_multiplier (float)

• time_col (str)

• doc_col (str)

• user_col (str)

• term_col (str)

• upos_col (str)

• r_pos_col (str)

• r_str_col (str)

• expected_min_timestamp (str | None)

• expected_max_timestamp (str | None)

• warn_on_time_range_mismatch (bool)

• output_dir (str | Path)

• write_html_report (bool)

• show_tables (bool)

• verbose (bool)

alpha: float = 0.5

custom_time_breaks: list[str] | None = None

custom_time_cuts: list[str] | None = None

diffusion_multiplier: float = 100.0

doc_col: str = 'doc_id'

expected_max_timestamp: str | None = None

expected_min_timestamp: str | None = None

max_rank_value: float = 5.0

n_periods: int = 4

output_dir: str | Path = 'evoc_outputs'

period_labels: list[str] | None = None

r_pos_col: str = 'r_pos'

r_str_col: str = 'r_str'

right: bool = True

round_digits: int = 2

show_tables: bool = False

term_col: str = 'term'

time_col: str = 'time'

time_period_mode: str = 'equal_posts'

upos_col: str = 'upos'

use_users_for_frequency: bool = True

user_col: str = 'user_id'

verbose: bool = True

warn_on_time_range_mismatch: bool = False

write_html_report: bool = True

focal_upos: list[str]

pyevoc.analysis.aggregate_named_entities(entities, text_col='text', type_col='type')

Backward-compatible utility for aggregating pre-extracted NER tables.

Parameters:

• entities (DataFrame)

• text_col (str)

• type_col (str)

Return type:

DataFrame

pyevoc.analysis.assign_evoc_quadrants(term_stats_df, *, minimal_freq=2, focal_upos=None, quadrant_order=None, round_digits=2, diffusion_basis='user_penetration', generate_html=False, html_output_dir='evoc_outputs', html_top_n=20, html_max_width_px=950, config=None)

Assign EVOC quadrants using relative AFE and mean AOE thresholds.

Parameters:

• term_stats_df (DataFrame) – Term-level dataframe.

• minimal_freq (int) – Minimal absolute frequency used to retain terms for quadrant assignment.

• focal_upos (set[str] | None) – POS categories retained for EVOC assignment.

• quadrant_order (list[str] | None) – Ordered quadrant labels.

• round_digits (int) – Number of digits used for rounded threshold comparisons.

• diffusion_basis (str) – Relative diffusion variable used to compute AFE thresholds. Supported values are user_penetration, document_penetration, post_penetration and diffusion_penetration.

• generate_html (bool) – If True, write compact EVOC HTML reports by UPOS category.

• html_output_dir (str | Path) – Directory where HTML files are written.

• html_top_n (int) – Number of terms displayed in each quadrant card.

• html_max_width_px (int) – Maximum width of the generated HTML page.

• config (QuadrantConfig | None) – Optional complete configuration. If supplied, explicit keyword arguments above are ignored unless they are already encoded in config.

Returns:

evoc_quadrants_df, quadrant_counts_df and pos_thresholds_round_df.

Return type:

tuple[pandas.DataFrame, pandas.DataFrame, pandas.DataFrame]

Notes

For backward compatibility, the function always returns three objects. When HTML reports are generated, their paths are stored in evoc_quadrants_df.attrs["html_outputs"].

pyevoc.analysis.assign_quadrants(term_stats_df, *, config=None)

Alias for assign_evoc_quadrants.

Parameters:

• term_stats_df (DataFrame)

• config (QuadrantConfig | None)

Return type:

tuple[DataFrame, DataFrame, DataFrame]

pyevoc.analysis.build_evoc_html_for_upos(evoc_quadrants_df, upos, pos_thresholds_round_df, *, output_dir='evoc_outputs', top_n=20, max_width_px=950)

Write the compact EVOC HTML report for one UPOS category.

Parameters:

• evoc_quadrants_df (DataFrame)

• upos (str)

• pos_thresholds_round_df (DataFrame)

• output_dir (str | Path)

• top_n (int)

• max_width_px (int)

Return type:

str

pyevoc.analysis.build_time_periods(df, *, time_col='time', doc_col='doc_id', user_col='user_id', n_periods=4, mode='equal_posts', custom_time_breaks=None, custom_time_cuts=None, period_labels=None, right=True, time_range_reference_df=None, expected_min_timestamp=None, expected_max_timestamp=None, warn_on_time_range_mismatch=False)

Assign observations to temporal periods.

Supported modes

custom

Use user-defined breaks or internal cuts.

equal_days or equal_width

Generate periods with approximately equal calendar duration.

equal_posts or equal_count

Generate periods with approximately equal numbers of unique posts.

quarters

Generate calendar-quarter periods.

Parameters:

• df (DataFrame)

• time_col (str)

• doc_col (str)

• user_col (str)

• n_periods (int)

• mode (str)

• custom_time_breaks (list[str] | None)

• custom_time_cuts (list[str] | None)

• period_labels (list[str] | None)

• right (bool)

• time_range_reference_df (DataFrame | None)

• expected_min_timestamp (object | None)

• expected_max_timestamp (object | None)

• warn_on_time_range_mismatch (bool)

Return type:

tuple[DataFrame, DataFrame, list[Timestamp], dict[str, object]]

pyevoc.analysis.compute_period_quadrants(tokens_df, evoc_quadrants_df, pos_thresholds_round_df=None, *, alpha=0.5, max_rank_value=5.0, use_users_for_frequency=True, focal_upos=None, round_digits=2, diffusion_multiplier=100.0)

Reconstruct period-specific term quadrants.

Parameters:

• tokens_df (DataFrame)

• evoc_quadrants_df (DataFrame)

• pos_thresholds_round_df (DataFrame | None)

• alpha (float)

• max_rank_value (float)

• use_users_for_frequency (bool)

• focal_upos (list[str] | None)

• round_digits (int)

• diffusion_multiplier (float)

Return type:

DataFrame

pyevoc.analysis.compute_stability_metrics(all_period_quadrants, n_periods)

Compute term-level and transition-level temporal stability metrics.

Parameters:

• all_period_quadrants (DataFrame)

• n_periods (int)

Return type:

dict[str, object]

pyevoc.analysis.compute_temporal_engagement(periodised_df, *, time_col='time', doc_col='doc_id', user_col='user_id')

Compute engagement diagnostics at post/user level.

Parameters:

• periodised_df (DataFrame)

• time_col (str)

• doc_col (str)

• user_col (str)

Return type:

DataFrame

pyevoc.analysis.extract_collocations(tokens, *, config=None, **kwargs)

Backward-compatible wrapper returning only dependency collocations.

Parameters:

• tokens (DataFrame)

• config (CollocationEntityConfig | None)

Return type:

DataFrame

pyevoc.analysis.extract_collocations_and_entities(tokens_df, *, config=None, output_dir=None, include_collocations=None, include_entities=None, entity_n=None, min_freq=None, min_docs=None, min_users=None, g2_alpha=None, resolve_overlap=None, prefer_overlap=None, write_html=None)

Extract collocations and/or named entities through one wrapper.

This is the recommended public API. The user can run only collocations, only named entities, or both.

Parameters:

• tokens_df (DataFrame)

• config (CollocationEntityConfig | None)

• output_dir (str | Path | None)

• include_collocations (bool | None)

• include_entities (bool | None)

• entity_n (int | None)

• min_freq (int | None)

• min_docs (int | None)

• min_users (int | None)

• g2_alpha (float | None)

• resolve_overlap (bool | None)

• prefer_overlap (str | None)

• write_html (bool | None)

Return type:

dict[str, object]

pyevoc.analysis.extract_dependency_collocations(tokens_df, *, config=None, min_freq=None, min_docs=None, min_users=None, alpha=None)

Extract dependency-based collocations.

Parameters:

• tokens_df (DataFrame)

• config (CollocationEntityConfig | None)

• min_freq (int | None)

• min_docs (int | None)

• min_users (int | None)

• alpha (float | None)

Return type:

DataFrame

pyevoc.analysis.extract_named_entity_ngrams(tokens_df, *, config=None, entity_n=None, min_freq=None, min_docs=None, min_users=None, alpha=None)

Extract contiguous PROPN named-entity n-grams.

Parameters:

• tokens_df (DataFrame)

• config (CollocationEntityConfig | None)

• entity_n (int | None)

• min_freq (int | None)

• min_docs (int | None)

• min_users (int | None)

• alpha (float | None)

Return type:

DataFrame

pyevoc.analysis.generate_evoc_html_reports(evoc_quadrants_df, pos_thresholds_round_df, *, output_dir='evoc_outputs', top_n=20, max_width_px=950, upos_values=None)

Generate compact EVOC HTML reports for the requested UPOS categories.

Parameters:

• evoc_quadrants_df (DataFrame)

• pos_thresholds_round_df (DataFrame)

• output_dir (str | Path)

• top_n (int)

• max_width_px (int)

• upos_values (list[str] | tuple[str, ...] | None)

Return type:

dict[str, str]

pyevoc.analysis.quadrant_summary_by_pos(evoc_quadrants_df, *, upos_col='upos', quadrant_col='quadrant')

Return quadrant counts by UPOS category.

Parameters:

• evoc_quadrants_df (DataFrame)

• upos_col (str)

• quadrant_col (str)

Return type:

DataFrame

pyevoc.analysis.quadrant_trajectories(period_terms, term_col='term', period_col='period', quadrant_col='quadrant')

Backward-compatible utility returning quadrant trajectories.

Parameters:

• period_terms (DataFrame)

• term_col (str)

• period_col (str)

• quadrant_col (str)

Return type:

DataFrame

pyevoc.analysis.run_temporal_stability_analysis(df, evoc_quadrants_df, pos_thresholds_round_df=None, *, n_periods=4, time_period_mode='equal_posts', custom_time_breaks=None, custom_time_cuts=None, period_labels=None, alpha=0.5, max_rank_value=5.0, use_users_for_frequency=True, focal_upos=None, round_digits=2, diffusion_multiplier=100.0, time_range_reference_df=None, count_reference_df=None, expected_min_timestamp=None, expected_max_timestamp=None, warn_on_time_range_mismatch=False, output_dir='evoc_outputs', write_html_report=True, show_tables=True, config=None)

Run the complete temporal stability analysis.

Parameters:

• df (DataFrame)

• evoc_quadrants_df (DataFrame)

• pos_thresholds_round_df (DataFrame | None)

• n_periods (int)

• time_period_mode (str)

• custom_time_breaks (list[str] | None)

• custom_time_cuts (list[str] | None)

• period_labels (list[str] | None)

• alpha (float)

• max_rank_value (float)

• use_users_for_frequency (bool)

• focal_upos (list[str] | None)

• round_digits (int)

• diffusion_multiplier (float)

• time_range_reference_df (DataFrame | None)

• count_reference_df (DataFrame | None)

• expected_min_timestamp (object | None)

• expected_max_timestamp (object | None)

• warn_on_time_range_mismatch (bool)

• output_dir (str | Path)

• write_html_report (bool)

• show_tables (bool)

• config (TemporalStabilityConfig | None)

Return type:

dict[str, object]

pyevoc.analysis.write_temporal_report_html(results, output_file, *, max_rows=80)

Write a compact HTML report for temporal stability analysis.

Parameters:

• results (dict[str, object])

• output_file (str | Path)

• max_rows (int)

Return type:

str

Modules

• pyevoc.analysis.collocations_entities
  • CollocationEntityConfig
  • normalise_annotation_columns()
  • is_valid_term()
  • prepare_tokens()
  • score_phrase_table()
  • extract_dependency_collocations()
  • extract_named_entity_ngrams()
  • compute_caps_evidence()
  • resolve_ne_colloc_overlap()
  • write_html_table()
  • extract_collocations_and_entities()
  • extract_collocations()
  • aggregate_named_entities()
• pyevoc.analysis.quadrants
  • QuadrantConfig
  • format_value()
  • format_int()
  • concreteness_class()
  • get_threshold_values()
  • quadrant_rule_label()
  • select_top_terms()
  • build_quadrant_block()
  • build_evoc_html_for_upos()
  • generate_evoc_html_reports()
  • assign_evoc_quadrants()
  • assign_quadrants()
  • quadrant_summary_by_pos()
• pyevoc.analysis.temporal_stability
  • TemporalStabilityConfig
  • format_float()
  • format_int()
  • quadrant_entropy()
  • gini_coefficient()
  • safe_std()
  • safe_spearman()
  • quarter_label_from_timestamp()
  • quarter_start_utc()
  • next_quarter_start_utc()
  • validate_time_range()
  • normalise_threshold_table()
  • build_threshold_display_table()
  • build_time_periods()
  • compute_period_quadrants()
  • add_term_diffusion_summary()
  • compute_stability_metrics()
  • compute_temporal_engagement()
  • dataframe_to_html_table()
  • write_temporal_report_html()
  • run_temporal_stability_analysis()
  • quadrant_trajectories()