Google Git
Sign in
foss-fpga-tools/third_party/verible/f6824d78b8e0db61f59340c543f4b9c3b7e02aa5/./verible/common/text
tree: 35d7730d6791e0eb08afc92b0193e3725a9507b2
  1. BUILD
  2. concrete-syntax-leaf.cc
  3. concrete-syntax-leaf.h
  4. concrete-syntax-leaf_test.cc
  5. concrete-syntax-tree.cc
  6. concrete-syntax-tree.h
  7. concrete-syntax-tree_test.cc
  8. config-utils.cc
  9. config-utils.h
  10. config-utils_test.cc
  11. constants.h
  12. macro-definition.cc
  13. macro-definition.h
  14. macro-definition_test.cc
  15. parser-verifier.cc
  16. parser-verifier.h
  17. parser-verifier_test.cc
  18. README.md
  19. symbol-ptr.h
  20. symbol.cc
  21. symbol.h
  22. syntax-tree-context.h
  23. syntax-tree-context_test.cc
  24. text-structure-test-utils.cc
  25. text-structure-test-utils.h
  26. text-structure.cc
  27. text-structure.h
  28. text-structure_test.cc
  29. token-info-json.cc
  30. token-info-json.h
  31. token-info-json_test.cc
  32. token-info-test-util.cc
  33. token-info-test-util.h
  34. token-info-test-util_test.cc
  35. token-info.cc
  36. token-info.h
  37. token-info_test.cc
  38. token-stream-view.cc
  39. token-stream-view.h
  40. token-stream-view_test.cc
  41. tree-builder-test-util.cc
  42. tree-builder-test-util.h
  43. tree-builder-test-util_test.cc
  44. tree-compare.cc
  45. tree-compare.h
  46. tree-compare_test.cc
  47. tree-context-visitor.cc
  48. tree-context-visitor.h
  49. tree-context-visitor_test.cc
  50. tree-utils.cc
  51. tree-utils.h
  52. tree-utils_test.cc
  53. visitors.h
verible/common/text/README.md

Text Structural Representation Libraries

At the heart of language-tooling libraries and applications lie various structural representations of text, and the functions that operate on them. This directory contains language-agnostic data structures like:

  • Tokens: annotated substrings of a body of text, often what a lexer produces.
    • Token streams: iterable representations of lexer output, including filtered views thereof.
  • Syntax trees: represent how parsers understand and organize code hierarchically.

Key Concepts

absl::string_views do not just represent text, but they represent position within a larger body of text, by virtue of comparing their begin and end bounds. This concept is leveraged heavily to avoid unnecessary string copying. A base string_view that represents a body of text and serve as the basis for interchanging between substring-views and byte-offsets relative to the start of the base.