OSDN -- Develop and Download Open Source Software

Ticket #47321
Ticket List Submit New Ticket RSS

\leavevmode together with \everypar{\strut} causes \strutbox to have zero dimensions

Open Date: 2023-02-07 06:14 Last Update: 2023-04-09 07:56

Reporter:
jfbu
Owner:
(None)
Type:
Bugs
Status:
Closed
Component:
(None)
MileStone:
(None)
Priority:
5 - Medium
Severity:
5 - Medium
Resolution:
None
File:
None

Details

  • The original issue was in build of Japanese version of Python docs which uses Sphinx LaTeX.
  • It got reported at <https://groups.google.com/g/sphinx-users/c/9sKyElODU5I>
  • There the problem was reduced to a small example with titlesec, an unnumbered section and \texttt.
  • The present file reduces it even further and includes analysis of cause of the problem.

Regards, Jean-François B. (jfbu)

Details:

  1. \leavevmode expands to \unhbox \voidb@x
  2. \unhbox is modified in luatexja-core.sty, and \unhcopy also (the original primitive is preserved as \ltj@@orig@unhbox)
  3. the modified \unhbox uses temporary count register \ltj@tempcnta and the modified \unhcopy uses also this registr
  4. when the \everypar tokens cause an \unhcopy this may cause the \ltj@tempcnta to be altered hence the \unhbox will end up as collateral emptying some other box. With \strut it ends up being the \strutbox.

Additional remarks:

  • I wondered if there was an interaction with the "para" hooks from LaTeX of about 2 years ago, but then I accessed a final TeXLive2019 which did not have yet the "para" hooks and the problem was there (and much easier to analyse via \tracingmacros1). Here is the log trace with my TL2019. We can see that \ltj@tempcnta ends up being overwritten when the \strut expands to an \unhcopy
  • I wonder if there is a problem with \ltj@reset@globaldefs being used but not \ltj@restore@globaldefs but I am not knowledgeable in lualatex. 
  1.  % ****************************** log trace with TeXLive 2019
  2.  % ****************************** (without the "para" hooks)
  3.  % ****************************** \unhbox/\unhcopy slightly different with TL2022
  4.  
  5.  \leavevmode ->\unhbox \voidb@x 
  6.  
  7.  \unhbox ->\ltj@reset@globaldefs \afterassignment \ltj@@unhbox \ltj@tempcnta 
  8.  
  9.  \ltj@reset@globaldefs ->\luafunction \ltj@reset@globaldefs@inner 
  10.  
  11.  \ltj@@unhbox ->\directlua {luatexja.direction.unbox_check_dir()}\ltj@@orig@unhb
  12.  ox \ltj@tempcnta 
  13.  
  14.  \eh@prepar ->
  15.  
  16.  \strut ->\protect \strut  
  17.  
  18.  \strut  ->\relax \ifnum \ltjgetparameter {direction}=1 \ifmmode \copy \dstrutbo
  19.  x \else \unhcopy \dstrutbox \fi \else \ifnum \ltjgetparameter {direction}=4 \if
  20.  mmode \copy \ystrutbox \else \unhcopy \ystrutbox \fi \else \ifmmode \copy \tstr
  21.  utbox \else \unhcopy \tstrutbox \fi \fi \fi 
  22.  
  23.  \ltjgetparameter #1->\directlua {luatexja.base.start_time_measure('get_par')}\i
  24.  fcsname ltj@@array@param/#1\endcsname \expandafter \ltx@firstoftwo \else \expan
  25.  dafter \ltx@secondoftwo \fi {\ltj@@getparam@two {#1}}{\ltj@@getparam@one {#1}}
  26.  #1<-direction
  27.  
  28.  \ltx@secondoftwo #1#2->#2
  29.  #1<-\ltj@@getparam@two {direction}
  30.  #2<-\ltj@@getparam@one {direction}
  31.  
  32.  \ltj@@getparam@one #1->\directlua {luatexja.ext_get_parameter_unary('#1')}
  33.  #1<-direction
  34.  
  35.  \ltjgetparameter #1->\directlua {luatexja.base.start_time_measure('get_par')}\i
  36.  fcsname ltj@@array@param/#1\endcsname \expandafter \ltx@firstoftwo \else \expan
  37.  dafter \ltx@secondoftwo \fi {\ltj@@getparam@two {#1}}{\ltj@@getparam@one {#1}}
  38.  #1<-direction
  39.  
  40.  \ltx@secondoftwo #1#2->#2
  41.  #1<-\ltj@@getparam@two {direction}
  42.  #2<-\ltj@@getparam@one {direction}
  43.  
  44.  \ltj@@getparam@one #1->\directlua {luatexja.ext_get_parameter_unary('#1')}
  45.  #1<-direction
  46.  
  47.  \unhcopy ->\ltj@reset@globaldefs \afterassignment \ltj@@unhcopy \ltj@tempcntb 
  48.  
  49.  \ltj@reset@globaldefs ->\luafunction \ltj@reset@globaldefs@inner 
  50.  
  51.  \ltj@@unhcopy ->\directlua {luatexja.direction.unbox_check_dir(true)}\ltj@@orig
  52.  @unhcopy \ltj@tempcntb \directlua {luatexja.direction.uncopy_restore_whatsit()}
  53.  
  54.  \eh@postpar ->

File demonstrating the issue 

  1.  \documentclass{ltjsbook}
  2.  % TO BE COMPILED WITH lualatex OF A TEXLIVE 2022 INSTALLATION
  3.  
  4.  % Februrary 6, 2023
  5.  % partial analysis by jfbu:
  6.  
  7.  % Bug: the \strut in the \everypar causes the \unhbox \ltj@tempcnta
  8.  % to modify the value of \ltj@tempcnta, because it is overwritten
  9.  % by a usage of \unhcopy.  Then the box which is unhbox'ed is not
  10.  % \voidb@x but ends up referring to \strutbox !
  11.  
  12.  % This bug was observed under these conditions:
  13.  % - usage of titlesec
  14.  % - for either \section* (i.e. unnumbered) or say \subsubsection
  15.  %   (with secnumdepth=2)
  16.  % - with a heading starting with \texttt{...} (not \ttfamily)
  17.  
  18.  % Indeed titlesec puts a \strut in \everypar, and if the title
  19.  % is unnnumbered the first thing will be a \leavevmode inserted
  20.  % by \texttt.  A numbered entry uses \noindent I think and does
  21.  % not trigger this bug.  A \ttfamily does no \leavevmode and does
  22.  % not trigger the bug.
  23.  
  24.  % Build failure arises with Sphinx in case a table containing
  25.  % a multirow merged cell because Sphinx wants to use the
  26.  % height of the \strutbox to make a division, so this causes
  27.  % a division by zero which is reported by TeX as an arithmetic
  28.  % overflow.
  29.  
  30.  % Original issue was reported at
  31.  % <https://groups.google.com/g/sphinx-users/c/9sKyElODU5I>
  32.  
  33.  \begin{document}
  34.  
  35.  START OF TEST
  36.  
  37.  \the\ht\strutbox
  38.  
  39.  \everypar{\strut}
  40.  
  41.  % \tracingmacros1 
  42.  
  43.  \leavevmode TEST
  44.  
  45.  % \tracingmacros0
  46.  
  47.  %\showthe\ht\strutbox
  48.  
  49.  \the\ht\strutbox% HAS BECOME ZERO PT!
  50.  
  51.  % To confirme the analysis I modify \unhcopy
  52.  
  53.  \makeatletter
  54.  
  55.  % this is syntax for a TeXLive 2022, uptodate as of Feb 6, 2022
  56.  
  57.  \protected\def\ltj@@unhcopy{%
  58.      \ltj@reset@globaldefs
  59.      \afterassignment\ltj@@unhcopy@
  60.      \ltj@tempcntb %<<<<<<<<<<<<<<<<<<<<<< cntb  in place of cnta
  61.  }
  62.  \protected\def\ltj@@unhcopy@{%
  63.    \directlua{luatexja.direction.unbox_check_dir(true)}%
  64.    \ltj@@orig@unhcopy\ltj@tempcntb %<<<<<<<<<<<<<<<<< cntb in place of cnta
  65.    \directlua{luatexja.direction.uncopy_restore_whatsit()}%
  66.  }
  67.  \let\unhcopy\ltj@@unhcopy
  68.  
  69.  RESET STRUTBOX
  70.  
  71.  \normalsize % triggers reset of \strutbox
  72.  
  73.  \the\ht\strutbox
  74.  
  75.  \everypar{\strut}
  76.  
  77.  \leavevmode TEST WITH PATCH
  78.  
  79.  %\showthe\ht\strutbox
  80.  
  81.  \the\ht\strutbox% IS NOW UNMODIFIED
  82.  
  83.  \end{document}

Ticket History (3/7 Histories)

2023-02-07 06:14 Updated by: jfbu
  • New Ticket "\leavevmode together with \everypar{\strut} causes \strutbox to have zero dimensions" created
2023-02-07 06:34 Updated by: jfbu
Comment

By accident I posted the TeX trace when using a patched version of \unhcopy which used \ltj@tempcntb. In truth the real TeX trace ends like this 

  1.  \unhcopy ->\ltj@reset@globaldefs \afterassignment \ltj@@unhcopy \ltj@tempcnta 
  2.  
  3.  \ltj@reset@globaldefs ->\luafunction \ltj@reset@globaldefs@inner 
  4.  
  5.  \ltj@@unhcopy ->\directlua {luatexja.direction.unbox_check_dir(true)}\ltj@@orig
  6.  @unhcopy \ltj@tempcnta \directlua {luatexja.direction.uncopy_restore_whatsit()}

i.e. it uses \ltj@tempcnta which originates the issue.

It was bit mind-boggling to me that \ltj@tempcnta value would change between start and end of \unhbox \ltj@tempcnta but this is what happens in the above indeed: \unhcopy reassigns \ltj@tempcnta before it has been used by the \unhbox original TeX primitive...

2023-02-07 20:20 Updated by: h7k
Comment

Thanks for the report. This is mind-boggling to me, too...

a patched version of \unhcopy which used \ltj@tempcntb

I come up with another approach: guarding LuaTeX-ja's \unhcopy by \begingroup...\endgroup (similar guards with \unvcopy, \unhbox, \unvbox). 

  1.  \let\ltj@@orig@unhcopy\unhcopy
  2.  \protected\def\ltj@@unhcopy{\begingroup\ltj@reset@globaldefs\afterassignment\ltj@@unhcopy@\ltj@tempcnta}
  3.  \protected\def\ltj@@unhcopy@{%
  4.    \directlua{luatexja.direction.unbox_check_dir(true)}%
  5.    \ltj@@orig@unhcopy\ltj@tempcnta
  6.    \directlua{luatexja.direction.uncopy_restore_whatsit()}\endgroup}

This makes an assignment to \ltj@tempcnta by \unhcopy: 

  1.  \ltj@@unhbox@ ->\ltj@@lua@unboxcheckdir \ltj@@orig@unhbox \ltj@tempcnta \endgro
  2.  up 
  3.  {expandable luacall 51}
  4.  {\unhbox} % <== "\unhbox \voidb@x" (\leavevmode) in vmode
  5.  % ... start a paragraph and re-read this command
  6.  
  7.  ...
  8.  
  9.  \strut ->\protect \strut  
  10.  
  11.  ...
  12.  
  13.  \unhcopy ->\begingroup \ltj@reset@globaldefs \afterassignment \ltj@@unhcopy@ \l
  14.  tj@tempcnta 
  15.  {\begingroup}
  16.  {reassigning \nolocalwhatsits=0}
  17.  {reassigning \nolocaldirs=0}
  18.  {luacall 32}
  19.  {reassigning \globaldefs=0}
  20.  {\afterassignment}
  21.  {\count187}
  22.  {changing \count187=10}
  23.  {into \count187=11}
  24.  
  25.  \ltj@@unhcopy@ ->\directlua {luatexja.direction.unbox_check_dir(true)}\ltj@@ori
  26.  g@unhcopy \ltj@tempcnta \directlua {luatexja.direction.uncopy_restore_whatsit()
  27.  }\endgroup 
  28.  {\directlua}
  29.  {\unhcopy}
  30.  {\directlua}
  31.  {\endgroup}
  32.  {restoring \count187=10}
  33.  
  34.  ...
  35.  
  36.  {\unhbox} % <== "\unhbox \voidb@x" (\leavevmode) in hmode
2023-02-07 23:08 Updated by: jfbu
Comment

Répondre à h7k

Thanks for the report. This is mind-boggling to me, too...

a patched version of \unhcopy which used \ltj@tempcntb

I come up with another approach: guarding LuaTeX-ja's \unhcopy by \begingroup...\endgroup (similar guards with \unvcopy, \unhbox, \unvbox).

I think the scope limiting approach via groups is fine (in theory it has a limitation to 255 levels of nesting if I remember correctly from TeX, I do not know LuaTeX, anyway this is not a truly limiting factor in practice). At any rate, my patch with \ltj@tempcntb was only for me to confirm I had understood the issue, but it is definitely not a robust fix, for example \unhbox can be used in the \everypar too, not only \unhcopy. Your groups if I understand correctly do fix such a situation too of nested usage, that's the whole point.

I had thought of still another approach like this, via some \expandafter. 

  1.  \protected\def\ltj@@unhbox@{\ltj@@lua@unboxcheckdir\expandafter\ltj@@orig@unhbox\the\ltj@tempcnta\relax}

Other things should probably be modified in this spirit for example 

  1.  \protected\def\ltj@@unhcopy@{%
  2.     \directlua{luatexja.direction.unbox_check_dir(true)}%
  3.     \expandafter\ltj@@orig@unhcopy\the\ltj@tempcnta\relax
  4.     \directlua{luatexja.direction.uncopy_restore_whatsit()}}

and perhaps similar changes to \unvbox/\unvcopy. After all, \everypar tokens may cause many things to happen.

As I don't know the LuaTeX-ja code base I do not know if the \begingroup/\endgroup may have any side effect but I am quite confident that the above will not have any.

I wanted to show a trace (where I had only patched ltj@unhbox@) but to show what happens I had to replace \ltj@@orig@unhbox with one more layer picking up the digits tokens until \relax to fetch them to the primitive \unhbox 

  1.  \leavevmode ->\unhbox \voidb@x 
  2.  
  3.  \unhbox ->\ltj@reset@globaldefs \afterassignment \ltj@@unhbox@ \ltj@tempcnta 
  4.  {luacall 32}
  5.  {\afterassignment}
  6.  {\count187}
  7.  
  8.  \ltj@@unhbox@ ->\ltj@@lua@unboxcheckdir \expandafter \debug@unhbox \the \ltj@te
  9.  mpcnta \relax 
  10.  
  11.  \debug@unhbox #1\relax ->\ltj@@orig@unhbox #1\relax 
  12.  
  13.  #1<-10
  14.  {\unhbox}

(I used 

  1.  \tracingcommands1
  2.  \tracingmacros1

)

2023-02-07 23:21 Updated by: jfbu
Comment

Répondre à jfbu

Répondre à h7k

Thanks for the report. This is mind-boggling to me, too...

a patched version of \unhcopy which used \ltj@tempcntb

I come up with another approach: guarding LuaTeX-ja's \unhcopy by \begingroup...\endgroup (similar guards with \unvcopy, \unhbox, \unvbox).

>...

I had thought of still another approach like this, via some \expandafter. {{{ code latex \protected\def\ltj@@unhbox@{\ltj@@lua@unboxcheckdir\expandafter\ltj@@orig@unhbox\the\ltj@tempcnta\relax} }}}

A variant is to insert a space token rather than a \relax I am not sure which is more efficient, but anyhow the \expandafter is probably the costlier bit.

To use a space token as delimiter of the explicit digit tokens one can do: 

  1.  \def\ltj@@unhbox@ #1% will redefine itself with #1 = space token
  2.  {%
  3.  \protected\def\ltj@@unhbox@{\ltj@@lua@unboxcheckdir\expandafter\ltj@@orig@unhbox\the\ltj@tempcnta#1}
  4.  }%
  5.  \ltj@@unhbox@{ }

or alternatively with an \edef and some \unexpanded.

(Edited, 2023-02-07 23:31 Updated by: jfbu)
2023-02-10 19:37 Updated by: h7k
Comment

jfbu への返信

I had thought of still another approach like this, via some \expandafter. {{{ code latex \protected\def\ltj@@unhbox@{\ltj@@lua@unboxcheckdir\expandafter\ltj@@orig@unhbox\the\ltj@tempcnta\relax} }}}

A variant is to insert a space token rather than a \relax I am not sure which is more efficient, but anyhow the \expandafter is probably the costlier bit.

Thanks. It seems that your \expandafter...\the\ltj@tempcnta\relax approach is slightly faster than \expandafter...\the\ltj@tempcnta(space token) and \begingroup...\endgroup approaches.

I'll adopt \expandafter...\the\ltj@tempcnta\relax approach to other commands in few days.

2023-04-09 07:56 Updated by: h7k
  • Status Update from Open to Closed
Comment

This issue is fixed in 20230211.0, so I close this ticket.

Attachment File List

No attachments

Edit

You are not logged in. I you are not logged in, your comment will be treated as an anonymous post. » Login