NickZaitsev/ru-normalizr

ru-normalizr — лучший нормализатор русского текста без LLM. Приводит числа, даты, время, сокращения, римские цифры, символы и латиницу в русские буквы для использования в TTS и NLP.

22Stars on the repository
19Mods indexed here, across every type
25d agoLast push, which is what freshness is scored on
MITLicence, which decides whether bodies are shown

NickZaitsev/ru-normalizr

Agent

A unit parser that uses one shared registry to recognize measurement units and their accepted spellings, including some safe case corrections.

22 25d ago A 0 tokens original MIT

locale-encoding

03

NickZaitsev/ru-normalizr

Agent

The windows-latest CI runner is an English Windows image, so locale.getencoding() there is cp1252, which cannot represent Cyrillic at all. The primary development machine is a Russian Windows install with cp1251, which can. Ubuntu cells are UTF-8.

22 25d ago A 0 tokens original MIT

mutation-audit

04

NickZaitsev/ru-normalizr

Agent

A one-time mutation audit using mutmut, a tool that changes source code to test whether the test suite detects those changes.

22 25d ago A 0 tokens original MIT

mutation-windows

05

NickZaitsev/ru-normalizr

Agent

Guidance for running mutmut mutation audits on Windows, where the tool changes source files and can leave processes and temporary files behind.

22 25d ago A 0 tokens original MIT

negative-numbers

06

NickZaitsev/ru-normalizr

Agent

Agent "negative-numbers" from NickZaitsev/ru-normalizr, covering unary minus vs. hyphen, the failure mode, the one exception: equations and rule for new patterns.

22 25d ago A 0 tokens original MIT

NickZaitsev/ru-normalizr

Agent

MorphAnalyzer.parse() orders candidates by a non-contextual dictionary score. The first candidate is therefore only a fallback, never contextual disambiguation. Callers that know a syntactic role or domain lexeme must use the shared linear selectparse() policy and constrain POS, normal form, gender, animacy, number…

22 25d ago A 0 tokens original MIT

stage-ordering

11

NickZaitsev/ru-normalizr

Agent

Every stage is a set of greedy regexes over the whole text. There is no arbitration: whichever stage reaches a span of characters first claims it, and the stage that would have matched a longer, more specific span gets whatever debris is left. Debris is worse than no match at all, because it is usually still numeric…

22 25d ago A 0 tokens original MIT

version-numbers

12

NickZaitsev/ru-normalizr

Agent

Agent "version-numbers" from NickZaitsev/ru-normalizr, covering dotted numbers: version vs. fraction, what counts as a version, why the old behaviour was broken and the pymorphy trap behind model numbers.

22 25d ago A 0 tokens original MIT