The limitations of TeX for typesetting text are well-known. Typesetting coloured text, letterspacing, hypertext, and hanging punctuation is much easier in pdfTeX. Omega, XeTeX and LuaTeX solve the 256 glyphs restriction. Every one of these extensions support right-to-left typesetting. OpenType fonts can be used natively in XeTeX and probably LuaTeX.
But is there any visible development related to typesetting math? Of course, support for right-to-left typesetting in TeX extensions allows typesetting RTL texts with LTR mathematics, but it does not affect the inside of the mathematical formula.
Except e-TeX there was no well-known (to a user of LaTeX) development of the programming-related features of TeX. Now other languages, including Lua, are preferred to the TeX macro-based language. In my opinion, this results from lack of competition (can any non-TeX-based program typeset \primes30 as a sequence of the first thirty prime numbers?) and small visibility of the need to program texts. Haskell or Python with a typesetting library could be better than TeX, if it is still possible to design pragmatically error-free programs.
This would clearly be incompatible with TeX. But addition of a new primitive command also makes a program incompatible with TeX. Try compiling this input with both TeX and e-TeX:
\csname TeXXeTstate\endcsname1 Hello \csname beginR\endcsname world\csname endR\endcsname. \bye
The results will be different, although it is not so strange to use \csname with undefined commands. (I used slightly similar code in macros for generating parametric text when the parameters could be unspecified due to a bug.)
Therefore I believe that writing a program which could typeset mathematical books with the quality of TeX, but with different design (a library in Python with separate modules for different things) would be useful. Obviously, all existing documents could be typeset with existing tools, but the new ones could be written more simply for this program.
Math typesetting contains clearly redundant commands, although they cannot be written as macros. The command \abovewithdelims can represent any generalized fraction, but some specific commands like \over are builtins. They lead also to a larger problem – generalized fractions make the style of previous material dependent on the later one. Therefore TeX has a separate glue type for math (mu depends on the current style) and the command \mathchoice does four times more work than it could without it. But most mathematics is written with LaTeX \frac command which does not have this problems. But primitive generalized fraction commands do not have to be used to make TeX’s work more complicated.
Texts written with plain TeX use very small amount of macros. So rewriting them in Python would make simpler and easier to improve code. Maybe it would be a nice exercise in learning TeX to write a typesetter based on the above ideas, just like kernels are written for their authors to learn Unix.
