Map: core / syntax
core/src/syntax/ turns source text into the surface AST — the only place that
sees raw bytes and bare words.
lexer.rs—lex(source) -> Result<Vec<Lexeme>, LexError>; theTokenenum. The innermost open delimiter (DelimKind:Brace,Bracket,Expr) is the lexer’s whole context: it decides whether a newline separates, whether,punctuates, and — inside$[…]only — whether<><=>=!=&&||and the arithmetic characters areToken::Op(Operator)(a closed enum overir::BinaryOpplusAnd,Or,Assign) rather than a redirect, refused punctuation or word characters. A closer that is not the innermost opener’s isLexErrorKind::Mismatched(L0006), a definite error. The bash reflexes${…}and$(cmd args)are refused atscan_dollar. Outside$[…]a word starting with&is refused namingspawn { … }, one starting&&naming;, while a&inside a word is ordinary (?a=1&b=2); a bare$is refused naming the three things it can open. A splice inside"…"(scan_splice) is the token stream the same text has outside the string, so"!$d"and!$dare one form;$(name)ends it, and every other splice continues into[key]. An unterminated string names the exact closer it still wants,StringForm::closing()— a bumped literal wants its#run back after the', so the bare reflex never closes it. Folding fd 1 onto fd 2 (1>&2, and>&2, which is the same redirect spelled short) is refused at the lexer, pointing at thewarnbuiltin, and at2>&1for a program holding the direction backwards: a diagnostic is a verb here, not a second name for the byte channel. A bare word is shaped once:Word::TildebyTildePath::parse_for,Word::Pathwhenpath::names_a_path_forfinds a separator, elseWord::Plain, both under the lexer’sPathRules—HOSTthroughlex_with, either set throughlex_forin tests — soC:\xis a path under Windows rules and a plain word under Posix. A redirect list is bound stream by stream,Redirects::bindrefusing a second binding of stdin, stdout or stderr at the second redirect (redirects-are-bindings).syntax.rs— the module root:CONTROL_OPERATORS(try,guard,within,grant,audit,spawn,gather) andFLOW_KEYWORDS(if,elsif,else,let,return,case), read by the parser and, throughis_control_operator/is_keyword, by the checker, highlighters, and the help registry’s documentation-completeness test;NESTING_DEPTH_LIMIT.parser.rs—parse(source) -> Result<Vec<Stmt>, ParseError>;parse_withcarries aFileId. A||after a pipe is refused naming?. The words rule is checked once per unit byend_unit.[:]is the empty map and the only literal opening with:; otherwise a bracket’s keys decide — any data key ("…"or$k,MapKeyForm::Data) makes a map, labels alone anAst::Record, no entry at all a list.parse_static_keyadmits a name or a single-quoted word; a tag is refused there for literal and pattern alike, since a tag keys nothing. The items are read in one pass, each lifting the literal to at least its own kind, so no token-level lookahead has to skip a spread’s nested brackets. Every item is one atom (parse_entry_value): a command written as one — an argument,|,?or redirect after the value — is P0003 (ParseErrorKind::EntryRuns), whose helpsyntax/report.rsdraws fromEntryReading. Every other refusal of a bracket’s shape is a P0001 whose help shows what to write (ParseErrorKind::Plain { hint }, set byParseError::with_hint):parse_bracket_itemsreads the items of every[…]— literal, options,casearms, list and map pattern — and names them (Bracketed) when a,is missing or extra;stray_coloncatches akey: valuewritten outside[ ]. The$[…]body is a Pratt parser whose operands areparse_atom’s atoms, plus(…)grouping and the prefixes-andnot; an operator in operand position, or two operands with no operator between them, each earn an error naming the gap; a bare non-numeral word under-, arithmetic, or ordering (numeric_operand) is refused as the string it is, asking whether$namewas meant.<<is the here-string redirect; the bash spellings (<<<, a glued heredoc<<EOF) earn targeted diagnostics naming ral’s form. A run of separators collapses to one token,Token::Semiwhen it contains a;and the softToken::Newlineotherwise: a;is hard, never crossed by a pipeline or chain continuation.parse_patternis the one place a duplicate binder is refused — a pattern binds all its names or none, solet [same, same] = [1, 2]is a parse error at the pattern’s span, while a repeat across curried parameters stays ordinary shadowing.ast.rs— the surface AST.Astis the expression node andStmtthe statement node; the enum is deliberately wide and flat (ast-stays-flat). The operator formsBinary,Negate,Not,And,Orare ordinary variants withAstoperands;$[…]itself leaves no node (expression-block-is-a-lexical-mode).Ast::Redirectedholds a stage’s trailing redirects, one node for every stage form;CallandScopecarry none (redirects-are-bindings).Ast::Unitis the()literal — punctuation denoting the unit value, like[]and[:], so not a word.Ast::Casecarriesarms: Vec<CaseArm>, a finite list of tag-and-body alternatives the parser hands on whole (case-is-syntax-try-is-not).WordLiteral::classifydecides by the numeral grammar: purely lexical, never a round trip through printing, so007and1.50are numerals while1e5, which merely happens to f64-parse, is text (a-numeral-denotes-its-number).numeral.rs— the single home of the numeral grammar (prefix); the lexer’s numeral scanning andWordLiteral::classifyboth read it.group.rs— pre-pass detecting mutually recursive binding groups, consumed by the elaborator.quote.rs— bare-word classification (is_bare_word,quote_word, returning aCow), shared with the REPL and theral-quotebuiltin. It is the dual ofWordLiteral::classify: a string may go bare only where the numeral grammar declines it, since a bare007would read back as the number 7.free_refs.rs— free-reference scans.Pattern,BinaryOpandRedirectsare not here: the IR owns them, and the parser builds them (redirect_word/redirect_duptranslate a redirect token).highlight.rs—classify(src), token classes (Class) read off the lexer for the exarch and synod highlighters, so neither carries a second parser of its own.
Recursion is bounded by a single shared depth cap, so adversarial nesting
rejects cleanly instead of overflowing the stack. One NESTING_DEPTH_LIMIT
governs both stages.
- The lexer enforces it at
scan_token_group, the sole recursion through bracketed token groups. - The parser’s three mutually-recursive sub-grammars each descend through
exactly one guarded chokepoint, all counting against
Parser::nested’s shareddepth: values throughparse_primary, expressions throughparse_expr_operand(the unary prefixes-/notand parenthesised sub-expressions bottom out here), patterns throughparse_pattern(list and map patterns recurse back through it per element).
The seven reserved control operators are recognised
at this layer, spawn’s arity elastic (one operand, or a title and the body)
and gather’s one body; everything else is a library binding. See docs/SPEC.md §3,
§17.1 for the grammar.