Ticket #39288

Quotation of patterns is not POSIXly correct

Open Date: 2019-06-02 16:27 Last Update: 2020-09-21 23:50

Reporter:
Owner:
Type:
Status:
Closed
Component:
MileStone:
(None)
Priority:
5 - Medium
Severity:
5 - Medium
Resolution:
Fixed
File:
None
Vote
Score: 0
No votes
0.0% (0/0)
0.0% (0/0)

Details

var="\z"
case ${var} in
(${var}) printf M;;
(*) printf X;;
esac

This should not match because the backslash in the pattern should be special (an escape character).


Pull request: https://github.com/magicant/yash/pull/8

Ticket History (3/13 Histories)

2019-06-02 16:27 Updated by: magicant
  • New Ticket "Quotation of patterns is not POSIXly correct" created
2019-06-02 16:35 Updated by: magicant
Comment

Other examples

P="x\?x"; var='x?xyyy';printf '%s\n' ${var#${P}}
P="x\?x"; var='x?xyyy';printf '%s\n' "${var#${P}}"
P="\[a]"; var='[a]bc';printf '%s\n' ${var#$P}
P="\\[a]"; var='[a]bc';printf '%s\n' ${var#$P}
P="\\?"; var='1-2-3-\?';printf '%s\n' ${var%-${P}}
2019-06-09 11:43 Updated by: magicant
  • Owner Update from (None) to magicant
  • Component Update from (None) to shell-main
2020-08-30 23:36 Updated by: magicant
Comment

In word expansion in the simple command, backslashes are correctly handled. So, to fix this bug, we should do the same thing to the case pattern?

$ v='\z'
$ printf '%s\n' \z $v
z
\z
2020-09-01 23:20 Updated by: magicant
Comment

The problem is not that simple.

Currently the expand_four function returns \\z for ${var}, which is indistinguishable from literal \z. We need to tell if the backslash was part of the original word or coming from expansion, since the quote removal rule requires to remove only the former.

Idea: Can we (ab)use the splittability string to tell whether the backslash is subject to quote removal?

2020-09-02 22:04 Updated by: magicant
Comment

XCU 2.6.1:

The pathname resulting from tilde expansion shall be treated as if quoted to prevent it being altered by field splitting and pathname expansion.

XCU 2.6.2:

If a parameter expansion occurs inside double-quotes: ... Field splitting shall not be performed on the results of the expansion.

XCU 2.6.3:

If a command substitution occurs inside double-quotes, field splitting and pathname expansion shall not be performed on the results of the substitution.

XCU 2.6.5:

the shell shall scan the results of expansions and substitutions that did not occur in double-quotes for field splitting and multiple fields can result.

To sum up, field splitting is performed on the results of unquoted expansions (but not tilde expansion).


XCU 2.6.7:

The quote characters ( <backslash>, single-quote, and double-quote) that were present in the original word shall be removed unless they have themselves been quoted.

This is almost the same as "the quote characters resulting from expansions", but including the results from tilde expansion. (sigh)

2020-09-04 13:08 Updated by: magicant
Comment

Yash prints 1-2-3 for IFS=-; echo "${unset-1\-2-3}" but other shells (bash, dash, ksh, mksh, zsh) correctly prints 1\-2-3. Maybe this issue can be handled in this ticket?

Note that all the shells including yash correctly prints 1-2 3 for IFS=-; echo ${unset-1\-2-3} and that the behavior is not well-defined for IFS=-; echo "${unset-"1\-2-3"}".

2020-09-10 21:53 Updated by: magicant
Comment

XCU 2.13.1

If a pattern ends with an unescaped <backslash>, it is unspecified whether the pattern does not match anything or the pattern is treated as invalid.

Ouch. Yash's behavior is neither of the two. I need to fix some tests, too.

But, wait, existing shells (including yash) print match for the following code, which is contrary to POSIX...:

bs='\'
case \\ in ${bs}) echo match; esac
(Edited, 2020-09-10 21:57 Updated by: magicant)
2020-09-15 00:40 Updated by: magicant
  • Details Updated
2020-09-20 23:19 Updated by: magicant
Comment

The main issue was fixed in r4073

2020-09-21 16:30 Updated by: magicant
Comment

magicant への返信

Yash prints 1-2-3 for IFS=-; echo "${unset-1\-2-3}" but other shells (bash, dash, ksh, mksh, zsh) correctly prints 1\-2-3. Maybe this issue can be handled in this ticket? Note that all the shells including yash correctly prints 1-2 3 for IFS=-; echo ${unset-1\-2-3} and that the behavior is not well-defined for IFS=-; echo "${unset-"1\-2-3"}".

I'll handle this in #40751.

2020-09-21 16:48 Updated by: magicant
  • Status Update from Open to Closed
  • Resolution Update from None to Fixed
Comment

magicant への返信

XCU 2.13.1

If a pattern ends with an unescaped <backslash>, it is unspecified whether the pattern does not match anything or the pattern is treated as invalid.

Ouch. Yash's behavior is neither of the two. I need to fix some tests, too. But, wait, existing shells (including yash) print match for the following code, which is contrary to POSIX...: {{{ bs='\' case \\ in ${bs}) echo match; esac }}}

New ticket: #40752.

2020-09-21 23:50 Updated by: magicant
Comment

The behavior of pattern matching (used for other than pathname expansion) is going to be stated more clearly in the future version of POSIX. In addition, the new behavior also applies to pattern matching in pathname expansion, which was not implemented in r4073 → See #40755.

https://www.austingroupbugs.net/view.php?id=1234#c4564

Attachment File List

No attachments

Edit

You are not logged in. I you are not logged in, your comment will be treated as an anonymous post. » Login