Two linguistic engines (TS-LLM vs Python-spaCy)

Parent: key-designs

The same conceptual outputs — lemma, POS, verb table, separable-verb reconstruction, grammar structure — are computed by two parallel engines, not a producer/consumer pipeline. The live client path is LLM-first and does not call the Python service for word lookup.

TS core (@lucerna/core) — the always-present client brain, any language. Owns tokenization/elision, all-forms matching, verb-form harvesting, the raw↔display geometry, and the POS taxonomy/colors/11-language labels. All content generation is delegated to a user-configured LLM (api/llmLookup.ts): basic lookup, verb tables, full inflected form-sets, sentence analysis (the grammar-structure string), POS coloring. The practical hard part is not trusting LLM output: cleanLLMJson carves the JSON substring from first {/[ to last }/] (killing fences/preamble) then runs jsonrepair (trailing commas, single quotes, truncation), with tolerant array extraction (sentences|result|analysis|data|items or any top-level array) and HTTP-status→localized-error mapping.

Python nlp-api — the deterministic, offline engine, DE+EN only. spaCy (de_core_news_lg / en_core_web_lg) does tokenize+lemmatize; a dependency-parse analyzer (infrastructure/grammar/dep_parse_analyzer.py) produces genuinely rich grammar: clause type (Hauptsatz vs Nebensatz via subordinators / cp/rc deps), tense inference (Perfekt/Plusquamperfekt/Futur/Präteritum from Partizip-II + aux morphology), word order (Inversion / verb-final), features (Passiv, Konjunktiv II, modal, negation, reflexive), then a subject/accusative/dative constituent breakdown — surfaced as a Chinese-annotated string. A Wiktionary/Postgres dictionary (preferring Chinese-meaning rows) and a CompositeTranslator (NLLB / Opus-MT by language pair) round it out.

Why both exist (the design logic). The LLM engine works for any language with rich target-language meanings but needs a user API key and JSON-repair; the spaCy engine is deterministic, offline, key-free, but bounded to DE+EN. German separable verbs are the clearest illustration of the split: a prefix-strip heuristic in TS (separableVerbService.ts) vs a true dependency-parse svp reconstruction in Python (spacy_tokenizer.py — finds the svp-dep token whose head is the verb, rebuilds prefix+lemma, capitalizes German noun lemmas). Same problem, two fidelities — the client can always answer, the server answers correctly for its two languages.