Introduction

A top-down parser builds the parse tree from the root down, in preorder, and in doing so produces a leftmost derivation of the input. At each step it holds a leftmost nonterminal and must choose one of its productions. If the choice can be wrong, the parser has to back up and try another alternative (backtracking). If the next input token always determines the choice, the parser is predictive and runs in linear time.

The grammars for which one token of lookahead always suffices are the LL(1) grammars: input scanned from Left to right, Leftmost derivation, 1 token of lookahead. An LL(1) parser can be written by hand as a set of recursive procedures (recursive descent) or driven by a table and an explicit stack. Both use the same table, built from FIRST and FOLLOW sets.

Prerequisites

  • FIRST and FOLLOW Sets: definitions and computation; the expression grammar and its sets are reused here.
  • The grammar must first be free of left recursion and left-factored (see the note Grammar Transformations for Parsing).

Core concepts

TermMeaning
Recursive descentOne procedure per nonterminal; the procedure for AA chooses an AA-production and calls the procedures for its body
BacktrackingTrying alternatives in turn and undoing input consumption on failure; exponential in the worst case
Predictive parserRecursive descent or table-driven parser that never backtracks, choosing by the lookahead
Parsing table M[A,a]M[A, a]The production to use when AA is on top of the stack and aa is the next token; an empty entry means a syntax error
LL(1) grammarA grammar whose table has at most one production in every entry

The LL(1) conditions

A grammar is LL(1) if and only if, for every pair of distinct productions A→α∣βA \to \alpha \mid \beta:

  1. FIRST(α)∩FIRST(β)=∅\text{FIRST}(\alpha) \cap \text{FIRST}(\beta) = \emptyset (no terminal begins strings of both; this includes ε\varepsilon, so at most one of them is nullable);
  2. if β⇒∗ε\beta \Rightarrow^{*} \varepsilon, then FIRST(α)∩FOLLOW(A)=∅\text{FIRST}(\alpha) \cap \text{FOLLOW}(A) = \emptyset (and symmetrically when α\alpha is nullable).

Condition 2 is the one most often forgotten. It says: when the parser could choose the nullable alternative because the next token is something that may follow AA, that token must not also start the other alternative.

Algorithms

Building the table

Text
BUILD-LL1-TABLE(G)
    for each production A -> alpha
        for each terminal a in FIRST(alpha) - {eps}
            add A -> alpha to M[A, a]
        if eps in FIRST(alpha)
            for each b in FOLLOW(A)          (b may be $)
                add A -> alpha to M[A, b]
    every entry still empty is an error entry

The grammar is LL(1) exactly when no entry receives two productions. Time: one FIRST-of-string per production and one FOLLOW lookup per nullable production, so linear in the grammar size times the number of terminals.

The table-driven parser

Pseudocode uses a stack whose top is written on the right, $ as the bottom marker and as the end of the input, and a for the current input token.

Text
LL1-PARSE(M, S, input)
    push $; push S
    a <- first token of input$
    loop
        X <- top of stack
        if X = $ and a = $ then accept
        else if X is a terminal
            if X = a then pop; a <- next token
            else error
        else if M[X, a] = X -> Y1 Y2 ... Yk
            pop; push Yk, ..., Y2, Y1        (Y1 ends on top)
        else error

Each step either matches and consumes a token or expands a nonterminal. For a fixed reduced LL(1) grammar, the total number of expansions in a successful parse is linear in the input length (with a grammar-dependent constant), so total time and stack space are O(n+1). Expansions between consecutive matches need not be bounded by a constant: one nullable cleanup burst can be linear in the input already consumed.

Recursive descent for the same grammar

Text
PROC-E():   PROC-T(); PROC-E1()
PROC-E1():  if a = '+' then MATCH('+'); PROC-T(); PROC-E1()
            else if a in { ')', '$' } then return          (E' -> eps)
            else error
PROC-T():   PROC-F(); PROC-T1()
PROC-T1():  if a = '*' then MATCH('*'); PROC-F(); PROC-T1()
            else if a in { '+', ')', '$' } then return     (T' -> eps)
            else error
PROC-F():   if a = '(' then MATCH('('); PROC-E(); MATCH(')')
            else if a = 'id' then MATCH('id')
            else error

The sets tested before returning on ε\varepsilon are exactly the FOLLOW sets; the call stack of the procedures plays the role of the explicit stack.

Worked examples

The table for the expression grammar

Grammar: E -> T E', E' -> + T E' | eps, T -> F T', T' -> * F T' | eps, F -> ( E ) | id. Using the FIRST and FOLLOW sets (FOLLOW(E') ={),$}= \{), \text{\textdollar}\}, FOLLOW(T') ={+,),$}= \{+, ), \text{\textdollar}\}):

id+*()$
EEE -> T E'E -> T E'
E′E'E' -> + T E'E' -> epsE' -> eps
TTT -> F T'T -> F T'
T′T'T' -> epsT' -> * F T'T' -> epsT' -> eps
FFF -> idF -> ( E )

13 of the 30 entries are filled and none has two productions, so the grammar is LL(1). The ε\varepsilon-entries sit exactly in the FOLLOW columns.

Parsing id + id * id

StepStack (top at right)InputAction
1$ Eid + id * id $E -> T E'
2$ E' Tid + id * id $T -> F T'
3$ E' T' Fid + id * id $F -> id
4$ E' T' idid + id * id $match id
5$ E' T'+ id * id $T' -> eps
6$ E'+ id * id $E' -> + T E'
7$ E' T ++ id * id $match +
8$ E' Tid * id $T -> F T'
9$ E' T' Fid * id $F -> id
10$ E' T' idid * id $match id
11$ E' T'* id $T' -> * F T'
12$ E' T' F ** id $match *
13$ E' T' Fid $F -> id
14$ E' T' idid $match id
15$ E' T'$T' -> eps
16$ E'$E' -> eps
17$$accept

There are 11 expansions (one per interior node of the parse tree, counting the ε\varepsilon-productions) and 5 matches (one per token), 16 moves before acceptance. The productions in the "Action" column, read top to bottom, are the leftmost derivation.

LL(1) table and parse trace for id + id \* id
Figure 1. The LL(1) parsing table of the expression grammar (13 filled entries) and the 17-row stack trace for id + id * id, with the five match steps marked.

Detecting an error: id + * id

StepStack (top at right)InputAction
1$ Eid + * id $E -> T E'
2$ E' Tid + * id $T -> F T'
3$ E' T' Fid + * id $F -> id
4$ E' T' idid + * id $match id
5$ E' T'+ * id $T' -> eps
6$ E'+ * id $E' -> + T E'
7$ E' T ++ * id $match +
8$ E' T* id $M[T,∗]M[T, *] is empty: error

The error is reported on *, the first token that cannot continue any sentence: id + is a prefix of valid input, id + * is not. This is the valid-prefix property of LL(1) parsing: the parser never consumes a token that cannot be part of a correct program. It may apply ε\varepsilon-expansions (step 5) before it notices an error, but it never shifts past one.

Is the grammar LL(1)?

GrammarLL(1)?Reason
S -> a S b | epsYesFIRST(aSbaSb) ={a}= \{a\}, FOLLOW(SS) ={b,$}= \{b, \text{\textdollar}\}, disjoint
S -> a B | a C, B -> b, C -> cNoBoth alternatives start with a (left-factor to fix)
S -> a A | b, A -> c A | epsYesFIRST(cAcA) ={c}= \{c\}, FOLLOW(AA) ={$}= \{\text{\textdollar}\}
S -> A b, A -> b | epsNob is in FIRST(bb) and in FOLLOW(AA): condition 2 fails
S -> A | B, A -> a | eps, B -> b | epsNoBoth alternatives of SS are nullable; the grammar is ambiguous for the empty string
S -> A a A b | B b B a, A -> eps, B -> epsYesFIRST(AaAbAaAb) ={a}= \{a\}, FIRST(BbBaBbBa) ={b}= \{b\}
S -> i E t S S' | a, S' -> e S | eps, E -> bNoe is in FIRST(eSeS) and in FOLLOW(S′S')
Any left-recursive grammar, for example E -> E + T | TNoFIRST(E+TE + T) contains FIRST(TT)
S -> a S a | b S b | epsNoFOLLOW(SS) ={a,b,$}= \{a, b, \text{\textdollar}\} meets FIRST(aSaaSa) and FIRST(bSbbSb): conflicts at M[S,a]M[S, a] and M[S,b]M[S, b], although the grammar is unambiguous
S -> ( S ) S | epsYesFIRST((S)S(S)S) ={(}= \{(\}, FOLLOW(SS) ={),$}= \{), \text{\textdollar}\}
S -> a S b S | b S a S | epsNoFOLLOW(SS) ={a,b,$}= \{a, b, \text{\textdollar}\}; conflicts at M[S,a]M[S, a] and M[S,b]M[S, b], and the grammar is ambiguous (abab has two trees)
S -> b S', S' -> a S' | epsYesFIRST(aS′aS') ={a}= \{a\}, FOLLOW(S′S') ={$}= \{\text{\textdollar}\}

The S -> i E t S S' row is the dangling-else grammar: the conflict is in M[S′,e]M[S', e]. Hand-written parsers resolve it by always choosing S′→eSS' \to e S, which attaches the else to the nearest if.

Important properties and observations

  • LL(1) implies unambiguous. Each string has at most one leftmost derivation, because every step is forced. Hence an ambiguous grammar is never LL(1).
  • LL(1) implies not left-recursive (for grammars without useless symbols). A left-recursive alternative shares FIRST with the alternative that ends the recursion.
  • Language versus grammar. "LL(1)" is a property of a grammar. A language is called LL(1) if some LL(1) grammar generates it. Rewriting can turn a non-LL(1) grammar into an LL(1) one for the same language, but not always: some deterministic context-free languages have no LL(kk) grammar for any kk.
  • Relation to LR. Every LL(1) grammar is LR(1). It need not be SLR(1): S -> A a A b | B b B a, A -> eps, B -> eps is LL(1) but its SLR(1) table has a reduce-reduce conflict, because FOLLOW(AA) == FOLLOW(BB) ={a,b}= \{a, b\}. See the note CLR(1) and LALR(1) Parsing for the full picture.
  • Every regular language has an LL(1) grammar: the right-linear grammar read off a DFA has one production per state and symbol, so the next token always decides.
  • Error recovery. In panic mode the parser skips input until a token in a synchronising set, typically FOLLOW of the nonterminal on top of the stack, then pops that nonterminal and continues.

Common misconceptions.

  • "A grammar with ε\varepsilon-productions cannot be LL(1)." It can; the expression grammar above has two.
  • "Left factoring and left-recursion removal always give LL(1)." They are necessary steps, not a guarantee; the dangling-else grammar stays non-LL(1).
  • "The number of parsing steps equals the number of tokens." It equals expansions plus matches; expansions include every ε\varepsilon-production applied.
  • "An unambiguous grammar without left recursion is LL(1)." S -> a S a | b S b | eps is both, and still has two conflicts: the parser cannot tell where the middle of the string is.
  • "An empty table entry means the grammar is not LL(1)." Empty entries are error entries; only an entry with two productions is a conflict.

GATE examination perspective

  • Fill a table entry. M[A,a]M[A, a] contains A→αA \to \alpha if a∈FIRST(α)a \in \text{FIRST}(\alpha), or if α\alpha is nullable and a∈FOLLOW(A)a \in \text{FOLLOW}(A). Questions ask for the entries of one row or for which entries hold ε\varepsilon-productions.
  • Decide LL(1). Check both conditions for every pair of alternatives. A conflict through FOLLOW (condition 2) is the classic trap.
  • Count moves or the stack contents. At a given step of the trace; keep the top of the stack at a consistent end and push bodies in reverse.
  • Grammar classes. LL(1) grammars are unambiguous, not left-recursive, and LR(1). Not every LL(1) grammar is SLR(1) or LR(0).
  • Recursive descent. Questions may show recursive procedures and ask which grammar they parse, or which input causes an infinite loop (left recursion).
  • Where is the error detected? Trace until an empty entry or a terminal mismatch; by the valid-prefix property, the offending token is the first one that cannot extend a valid prefix.
  • Nullable alternatives check against FOLLOW of the head. For S→aSa∣εS \to a S a \mid \varepsilon, the test uses FOLLOW(SS), which contains a because of the inner SS in aSaaSa.

Summary

Sources

  1. A. V. Aho, M. S. Lam, R. Sethi, J. D. Ullman, Compilers, Principles, Techniques, and Tools, 2nd ed., Pearson Addison-Wesley, 2007: Section 2.4 (top-down and predictive parsing, left recursion), Section 4.4.1 (recursive-descent parsing), Section 4.4.3 (LL(1) grammars and the parsing-table construction), Section 4.4.4 (nonrecursive predictive parsing), Section 4.4.5 (error recovery in predictive parsing).

Next steps