Yet Another Shell

Pass test of dot with echo disabled

In a configuration where the echo built-in is disabled, a test case for
the dot built-in was failing because the echo built-in was not found in
the dot script after $PATH had been modified.

$PATH is no longer modified, instead the dot script is renamed so that
it is much unlikely for the name to be found in unmodified $PATH.

Pass test of kill with help disabled

In a configuration where the help built-in is disabled, the test case
for the --help option of the kill built-in was failing.

Eliminate occasional test failure in wait-y

Pass arith-y Valgrind tests

Some recently added tests in arith-y.tst were failing when tested with
Valgrind because of program name mismatch. This commit hides the program
name by wrapping commands with the eval built-in.

completion/git-grep: Don't complete --threads option argument

The --threads option requires a number. It is pointless to complete
pathnames.

completion/git-grep: Support Git 2.19.1

Test POSIX mode behaviors

Some tests have been missing that check the behaviors of the
posixly-correct option. This commit adds them.

Document all POSIXly-correct behavior

All the effects of the POSIXly-correct mode are now documented in
posix.txt and the built-in command manual pages.

Reject keywords after redirection (#38617)

More test on alias substitution of esac

Reject IO_NUMBER as redirection operand (#38594)

POSIX requires the redirection operand to be a WORD, not IO_NUMBER.

Such an IO_NUMBER is rejected only in the POSIXly-correct mode because
it does not actually make sense to reject it. It can only be parsed as a
redirection operand anyway.

Reject invalid semicolon after alias substitution

This is a fix-up for r3710 and r3809. Invalid semicolons have been
rejected since r3710, but r3809 failed to consider the case where alias
substitution occurs after a newline.

Don't alias-substitute "do" in for command (#38575)

More strict tests of alias substitution

tests/builtins-y: Avoid race condition

The bg built-in fails if the specified job has already terminated. For
the bg built-in that is expected to succeed, we need to ensure the job
has not yet terminated.

doc: Fix broken macro markup

We should have a word separator before a link:...[...] macro so that the
macro is recognized as expected.

doc/builtin: Link to list of built-ins

doc/posix: Remove outdated description

Since r3620 unclosed here-document contents are always rejected, not
only in the POSIXly-correct mode.

parser: Reorder result construction

Allocate the results later for locality.

parser: Clear reparse flag earlier

Now the `reparse' flag is true only while stack unwinding is begin done.
The `reparse' flag is cleared just after stack unwinding ends.

Remove token_based_parser branch

Merge token_based_parser branch

This commit merges a massive refactoring in the parser.

The parser is now implemented in a token-based algorithm. The entire
parser has been split into the lexical tokenizer and the LL(1) syntax
parser. With the new parser architecture, it would be possible to
implement a parser for the [[ command.

Test alias substitution followed by operator character

po: Update error messages and translations

doc/fgrammar: Rewrite formal syntax

Reorder parser functions

Now functions are arranged in sections.

Refactor successive word parser

The simple command is now parsed by the new parse_simple_command_tokens
function. Successive words are now parsed by the new parse_words
function. The parse_assignments_and_redirects, parse_words_and_redirects
and parse_words_to_paren functions are removed in favor of them.

The is_command_delimiter_tokentype function is removed as it is no
longer used.

Remove unused functions in parser.c

The parse_word_to function is renamed to parse_word, which is one of the
functions removed in this commit.

Optimize candidate name check in alias substitution

This refactoring commit moves the
is_token_delimiter_char(buf->contents[j])
and
is_redir_fd(&buf->contents[i])
calls out of the substitute_alias_range function. These calls are
omitted in the psubstitute_alias function since the token has already
been delimited and categorized.

The psubstitute_alias function no longer uses the count_name_length
function to find the end of the token. Now the token passed from
psubstitute_alias to substitute_alias_range can contain non-alias-name
characters, but such a token will not be substituted anyway since no
alias can be defined with such a name.

The is_alias_name_char function is now defined with an internal linkage.

Simplify count_name_length function