# Verible SystemVerilog Parser Design

Verible uses traditional tools like Flex and Yacc/Bison, but *not* in the usual
manner where the generated `yyparse()` function calls `yylex()` directly.
Instead, the generated lexer and parser are completely decoupled.

## Lexer

The lexer is generated using Flex. It supports both SystemVerilog
language-proper lexical tokens as well as preprocessing directives, macros, and
other directives, all-in-one. The Flex-generated code is then wrapped into an
interface that returns tokens one-by-one, until ending with a special EOF token.

## Contextualizer

The contextualizer is a pass over the token stream produced by the lexer that
can help disambiguate tokens with multiple interpretations. This in turn, helps
simplify the grammar in the parser implementation in ways that would not be
possible with a context-free LR parser alone.

## Preprocessor

There is no standalone preprocessor yet.

## Filter

Token streams can be filtered into token stream views in various ways. One of
the most useful filters hides comments and attributes so that the resulting view
can be parsed.

## Parser

The parser is generated using Bison, using the LALR(1) algorithm. The grammar
implemented is unconventional in that it explicitly handles preprocessing
constructs, directives, macros -- this allows it to operate on a
limited-but-useful subset of *unpreprocessed* code. The parser constructs a
concrete syntax tree (CST) that captures all non-comment tokens, including
punctuation and syntactic sugar.