# Module macros

This module contains the interface to the compiler's abstract syntax tree (AST). Macros operate on this tree.

# The AST in Nimrod

This section describes how the AST is modelled with Nimrod's type system. The AST consists of nodes (`PNimrodNode`) with a variable number of children. Each node has a field named `kind` which describes what the node contains:

    type
      TNimrodNodeKind = enum ## kind of a node; only explanatory
        nnkNone,             ## invalid node kind
        nnkEmpty,            ## empty node
        nnkIdent,            ## node contains an identifier
        nnkIntLit,           ## node contains an int literal (example: 10)
        nnkStrLit,           ## node contains a string literal (example: "abc")
        nnkNilLit,           ## node contains a nil literal (example: nil)
        nnkCaseStmt,         ## node represents a case statement
        ...                  ## many more

      PNimrodNode = ref TNimrodNode
      TNimrodNode {.final.} = object
        case kind: TNimrodNodeKind       ## the node's kind
        of nnkNone, nnkEmpty, nnkNilLit:
          nil                            ## node contains no additional fields
        of nnkCharLit..nnkInt64Lit:
          intVal: biggestInt             ## the int literal
        of nnkFloatLit..nnkFloat64Lit:
          floatVal: biggestFloat         ## the float literal
        of nnkStrLit..nnkTripleStrLit:
          strVal: string                 ## the string literal
        of nnkIdent:
          ident: TNimrodIdent            ## the identifier
        of nnkSym:
          symbol: PNimrodSymbol          ## the symbol (after symbol lookup phase)
        else:
          sons: seq[PNimrodNode]         ## the node's sons (or children)

For the `PNimrodNode` type, the `[]` operator has been overloaded: `n[i]` is `n`'s `i`-th child.

To specify the AST for the different Nimrod constructs, the notation `nodekind(son1, son2, ...)` or `nodekind(value)` or `nodekind(field=value)` is used.

Some child may be missing. A missing child is a node of kind `nnkEmpty`; a child can never be nil.

# Leaf nodes/Atoms

A leaf of the AST often corresponds to a terminal symbol in the concrete syntax.

Nimrod expression | corresponding AST
--- | ---
`42` | `nnkIntLit(intVal = 42)`
`42'i8` | `nnkInt8Lit(intVal = 42)`
`42'i16` | `nnkInt16Lit(intVal = 42)`
`42'i32` | `nnkInt32Lit(intVal = 42)`
`42'i64` | `nnkInt64Lit(intVal = 42)`
`42.0` | `nnkFloatLit(floatVal = 42.0)`
`42.0'f32` | `nnkFloat32Lit(floatVal = 42.0)`
`42.0'f64` | `nnkFloat64Lit(floatVal = 42.0)`
`"abc"` | `nnkStrLit(strVal = "abc")`
`r"abc"` | `nnkRStrLit(strVal = "abc")`
`"""abc"""` | `nnkTripleStrLit(strVal = "abc")`
`' '` | `nnkCharLit(intVal = 32)`
`nil` | `nnkNilLit()`
`myIdentifier` | `nnkIdent(ident = !"myIdentifier")`
`myIdentifier` | after lookup pass: `nnkSym(symbol = ...)`

Identifiers are `nnkIdent` nodes. After the name lookup pass these nodes get transferred into `nnkSym` nodes. However, a macro receives an AST that has not been checked for semantics and thus the identifiers have not been looked up. Macros should deal with `nnkIdent` nodes and do not need to deal with `nnkSym` nodes.

# Calls/expressions

## Command call

Concrete syntax:

    echo "abc", "xyz"

AST:

    nnkCommand(nnkIdent(!"echo"), nnkStrLit("abc"), nnkStrLit("xyz"))

## Call with `()`

Concrete syntax:

    echo("abc", "xyz")

AST:

    nnkCall(nnkIdent(!"echo"), nnkStrLit("abc"), nnkStrLit("xyz"))

## Infix operator call

Concrete syntax:

    "abc" & "xyz"

AST:

    nnkInfix(nnkIdent(!"&"), nnkStrLit("abc"), nnkStrLit("xyz"))

## Prefix operator call

Concrete syntax:

    ? "xyz"

AST:

    nnkPrefix(nnkIdent(!"?"), nnkStrLit("abc"))

## Postfix operator call

Note: There are no postfix operators in Nimrod. However, the `nnkPostfix` node is used for the asterisk export marker `*`:

Concrete syntax:

    identifier*

AST:

    nnkPostfix(nnkIdent(!"*"), nnkIdent(!"identifier"))

## Call with named arguments

Concrete syntax:

    writeln(file=stdout, "hallo")

AST:

    nnkCall(nnkIdent(!"writeln"),
            nnkExprEqExpr(nnkIdent(!"file"), nnkIdent(!"stdout")),
            nnkStrLit("hallo"))

## Dereference operator `^`

Concrete syntax:

    x^

AST:

    nnkDerefExpr(nnkIdent(!"x"))

## Addr operator

Concrete syntax:

    addr(x)

AST:

    nnkAddr(nnkIdent(!"x"))

## Cast operator

Concrete syntax:

    cast[T](x)

AST:

    nnkCast(nnkIdent(!"T"), nnkIdent(!"x"))

## Object access operator `.`

Concrete syntax:

    x.y

AST:

    nnkDotExpr(nnkIdent(!"x"), nnkIdent(!"y"))

## Array access operator `[]`

Concrete syntax:

    x[y]

AST:

    nnkBracketExpr(nnkIdent(!"x"), nnkIdent(!"y"))

## Parentheses

Parentheses for affecting operator precedence or tuple construction are built with the `nnkPar` node.

Concrete syntax:

    (1, 2, (3))

AST:

    nnkPar(nnkIntLit(1), nnkIntLit(2), nnkPar(nnkIntLit(3)))

## Curly braces

Curly braces are used as the set constructor.

Concrete syntax:

    {1, 2, 3}

AST:

    nnkCurly(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3))

## Brackets

Brackets are used as the array constructor.

Concrete syntax:

    [1, 2, 3]

AST:

    nnkBracket(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3))

## Ranges

Ranges occur in set constructors, case statement branches or array slices.

Concrete syntax:

    1..3

AST:

    nnkRange(nnkIntLit(1), nnkIntLit(3))

## If expression

The representation of the if expression is subtle, but easy to traverse.

Concrete syntax:

    if cond1: expr1 elif cond2: expr2 else: expr3

AST:

    nnkIfExpr(
      nnkElifExpr(cond1, expr1),
      nnkElifExpr(cond2, expr2),
      nnkElseExpr(expr3)
    )

# Statements

## If statement

The representation of the if statement is subtle, but easy to traverse. If there is no `else` branch, no `nnkElse` child exists.

Concrete syntax:

    if cond1:
      stmt1
    elif cond2:
      stmt2
    elif cond3:
      stmt3
    else:
      stmt4

AST:

    nnkIfStmt(
      nnkElifBranch(cond1, stmt1),
      nnkElifBranch(cond2, stmt2),
      nnkElifBranch(cond3, stmt3),
      nnkElse(stmt4)
    )

## When statement

Like the `if` statement, but the root has the kind `nnkWhenStmt`.

## Assignment

Concrete syntax:

    x = 42

AST:

    nnkAsgn(nnkIdent(!"x"), nnkIntLit(42))

## Statement list

Concrete syntax:

    stmt1
    stmt2
    stmt3

AST:

    nnkStmtList(stmt1, stmt2, stmt3)

## Case statement

Concrete syntax:

    case expr1
    of expr2, expr3..expr4:
      stmt1
    of expr5:
      stmt2
    elif cond1:
      stmt3
    else:
      stmt4

AST:

    nnkCaseStmt(
      expr1,
      nnkOfBranch(expr2, nnkRange(expr3, expr4), stmt1),
      nnkOfBranch(expr5, stmt2),
      nnkElifBranch(cond1, stmt3),
      nnkElse(stmt4)
    )

The `nnkElifBranch` and `nnkElse` parts may be missing.

## While statement

Concrete syntax:

    while expr1:
      stmt1

AST:

    nnkWhileStmt(expr1, stmt1)

## For statement

Concrete syntax:

    for ident1, ident2 in expr1:
      stmt1

AST:

    nnkForStmt(ident1, ident2, expr1, stmt1)

## Try statement

Concrete syntax:

    try:
      stmt1
    except e1, e2:
      stmt2
    except e3:
      stmt3
    except:
      stmt4
    finally:
      stmt5

AST:

    nnkTryStmt(
      stmt1,
      nnkExceptBranch(e1, e2, stmt2),
      nnkExceptBranch(e3, stmt3),
      nnkExceptBranch(stmt4),
      nnkFinally(stmt5)
    )

## Return statement

Concrete syntax:

    return expr1

AST:

    nnkReturnStmt(expr1)

## Yield statement

Like `return`, but with `nnkYieldStmt` kind.

## Discard statement

Like `return`, but with `nnkDiscardStmt` kind.

## Continue statement

Concrete syntax:

    continue

AST:

    nnkContinueStmt()

## Var section

To be written.

## Const section

To be written.

## Type section

To be written.

## Procedure declaration

To be written.

## Iterator declaration

To be written.

## Template declaration

To be written.

## Macro declaration

To be written.

# Special node kinds

There are several node kinds that are used for semantic checking or code generation. These are accessible from this module, but should not be used. Other node kinds are especially designed to make AST manipulations easier. These are explained here.

To be written.

# Types

    TNimrodNodeKind* = enum
      nnkNone, nnkEmpty, nnkIdent, nnkSym, nnkType, nnkCharLit, nnkIntLit,
      nnkInt8Lit, nnkInt16Lit, nnkInt32Lit, nnkInt64Lit, nnkUIntLit, nnkUInt8Lit,
      nnkUInt16Lit, nnkUInt32Lit, nnkUInt64Lit, nnkFloatLit, nnkFloat32Lit,
      nnkFloat64Lit, nnkFloat128Lit, nnkStrLit, nnkRStrLit, nnkTripleStrLit,
      nnkNilLit, nnkMetaNode, nnkDotCall, nnkCommand, nnkCall, nnkCallStrLit,
      nnkExprEqExpr, nnkExprColonExpr, nnkIdentDefs, nnkVarTuple, nnkInfix,
      nnkPrefix, nnkPostfix, nnkPar, nnkCurly, nnkCurlyExpr, nnkBracket,
      nnkBracketExpr, nnkPragmaExpr, nnkRange, nnkDotExpr, nnkCheckedFieldExpr,
      nnkDerefExpr, nnkIfExpr, nnkElifExpr, nnkElseExpr, nnkLambda, nnkDo,
      nnkAccQuoted, nnkTableConstr, nnkBind, nnkClosedSymChoice, nnkOpenSymChoice,
      nnkHiddenStdConv, nnkHiddenSubConv, nnkHiddenCallConv, nnkConv, nnkCast,
      nnkStaticExpr, nnkAddr, nnkHiddenAddr, nnkHiddenDeref, nnkObjDownConv,
      nnkObjUpConv, nnkChckRangeF, nnkChckRange64, nnkChckRange, nnkStringToCString,
      nnkCStringToString, nnkAsgn, nnkFastAsgn, nnkGenericParams, nnkFormalParams,
      nnkOfInherit, nnkModule, nnkProcDef, nnkMethodDef, nnkConverterDef,
      nnkMacroDef, nnkTemplateDef, nnkIteratorDef, nnkOfBranch, nnkElifBranch,
      nnkExceptBranch, nnkElse, nnkMacroStmt, nnkAsmStmt, nnkPragma, nnkPragmaBlock,
      nnkIfStmt, nnkWhenStmt, nnkForStmt, nnkParForStmt, nnkWhileStmt, nnkCaseStmt,
      nnkTypeSection, nnkVarSection, nnkLetSection, nnkConstSection, nnkConstDef,
      nnkTypeDef, nnkYieldStmt, nnkTryStmt, nnkFinally, nnkRaiseStmt, nnkReturnStmt,
      nnkBreakStmt, nnkContinueStmt, nnkBlockStmt, nnkStaticStmt, nnkDiscardStmt,
      nnkStmtList, nnkImportStmt, nnkFromStmt, nnkIncludeStmt, nnkBindStmt,
      nnkMixinStmt, nnkCommentStmt, nnkStmtListExpr, nnkBlockExpr, nnkStmtListType,
      nnkBlockType, nnkTypeOfExpr, nnkObjectTy, nnkTupleTy, nnkRecList, nnkRecCase,
      nnkRecWhen, nnkRefTy, nnkPtrTy, nnkVarTy, nnkConstTy, nnkMutableTy,
      nnkDistinctTy, nnkProcTy, nnkEnumTy, nnkEnumFieldDef, nnkArglist, nnkPattern,
      nnkReturnToken

    TNimNodeKinds* = set[TNimrodNodeKind]

    TNimrodTypeKind* = enum
      ntyNone, ntyBool, ntyChar, ntyEmpty, ntyArrayConstr, ntyNil, ntyExpr, ntyStmt,
      ntyTypeDesc, ntyGenericInvokation, ntyGenericBody, ntyGenericInst,
      ntyGenericParam, ntyDistinct, ntyEnum, ntyOrdinal, ntyArray, ntyObject,
      ntyTuple, ntySet, ntyRange, ntyPtr, ntyRef, ntyVar, ntySequence, ntyProc,
      ntyPointer, ntyOpenArray, ntyString, ntyCString, ntyForward, ntyInt, ntyInt8,
      ntyInt16, ntyInt32, ntyInt64, ntyFloat, ntyFloat32, ntyFloat64, ntyFloat128

    TNimTypeKinds* = set[TNimrodTypeKind]

    TNimrodSymKind* = enum
      nskUnknown, nskConditional, nskDynLib, nskParam, nskGenericParam, nskTemp,
      nskType, nskConst, nskVar, nskProc, nskMethod, nskIterator, nskConverter,
      nskMacro, nskTemplate, nskField, nskEnumField, nskForVar, nskModule, nskLabel,
      nskStub

    TNimSymKinds* = set[TNimrodSymKind]

    TNimrodIdent* = object of TObject

  represents a Nimrod identifier in the AST

    PNimrodType* {.compilerproc.} = ref TNimrodType

  represents a Nimrod type in the compiler; currently this is not very useful as there is no API to deal with Nimrod types.

    PNimrodSymbol* {.compilerproc.} = ref TNimrodSymbol

  represents a Nimrod symbol in the compiler; a symbol is a looked-up ident.

    TBindSymRule* = enum
      brClosed,                   ## only the symbols in current scope are bound
      brOpen, ## open wrt overloaded symbols, but may be a single
              ## symbol if not ambiguous (the rules match that of
              ## binding in generics)
      brForceOpen                 ## same as brOpen, but it will always be open even
                                  ## if not ambiguous (this cannot be achieved with
                                  ## any other means in the language currently)

  specifies how `bindSym` behaves

# Consts

    nnkLiterals* = {nnkCharLit .. nnkNilLit}

    nnkCallKinds* = {nnkCall, nnkInfix, nnkPrefix, nnkPostfix, nnkCommand,
                     nnkCallStrLit}

# Procs

    proc `[]`*(n: PNimrodNode; i: int): PNimrodNode {.magic: "NChild".}

  get n's i'th child.

    proc `[] =`*(n: PNimrodNode; i: int; child: PNimrodNode) {.magic: "NSetChild".}

  set n's i'th child to child.

    proc `!`*(s: string): TNimrodIdent {.magic: "StrToIdent".}

  constructs an identifier from the string s

    proc `$`*(i: TNimrodIdent): string {.magic: "IdentToStr".}

  converts a Nimrod identifier to a string

    proc `$`*(s: PNimrodSymbol): string {.magic: "IdentToStr".}

  converts a Nimrod symbol to a string

    proc `==`*(a, b: TNimrodIdent): bool {.magic: "EqIdent", noSideEffect.}

  compares two Nimrod identifiers

    proc `==`*(a, b: PNimrodNode): bool {.magic: "EqNimrodNode", noSideEffect.}

  compares two Nimrod nodes

    proc len*(n: PNimrodNode): int {.magic: "NLen".}

  returns the number of children of n.

    proc add*(father, child: PNimrodNode) {.magic: "NAdd".}

  adds the child to the father node

    proc add*(father: PNimrodNode; children: varargs[PNimrodNode]) {.
        magic: "NAddMultiple".}

  adds each child of children to the father node

    proc del*(father: PNimrodNode; idx = 0; n = 1) {.magic: "NDel".}

  deletes n children of father starting at index idx.

    proc kind*(n: PNimrodNode): TNimrodNodeKind {.magic: "NKind".}

  returns the kind of the node n.

    proc intVal*(n: PNimrodNode): biggestInt {.magic: "NIntVal".}

    proc floatVal*(n: PNimrodNode): biggestFloat {.magic: "NFloatVal".}

    proc symbol*(n: PNimrodNode): PNimrodSymbol {.magic: "NSymbol".}

    proc ident*(n: PNimrodNode): TNimrodIdent {.magic: "NIdent".}

    proc typ*(n: PNimrodNode): PNimrodType {.magic: "NGetType".}

    proc strVal*(n: PNimrodNode): string {.magic: "NStrVal".}

    proc `intVal =`*(n: PNimrodNode; val: biggestInt) {.magic: "NSetIntVal".}

    proc `floatVal =`*(n: PNimrodNode; val: biggestFloat) {.magic: "NSetFloatVal".}

    proc `symbol =`*(n: PNimrodNode; val: PNimrodSymbol) {.magic: "NSetSymbol".}

    proc `ident =`*(n: PNimrodNode; val: TNimrodIdent) {.magic: "NSetIdent".}

    proc `typ =`*(n: PNimrodNode; typ: PNimrodType) {.magic: "NSetType".}

    proc `strVal =`*(n: PNimrodNode; val: string) {.magic: "NSetStrVal".}

    proc newNimNode*(kind: TNimrodNodeKind; n: PNimrodNode = nil): PNimrodNode {.
        magic: "NNewNimNode".}

    proc copyNimNode*(n: PNimrodNode): PNimrodNode {.magic: "NCopyNimNode".}

    proc copyNimTree*(n: PNimrodNode): PNimrodNode {.magic: "NCopyNimTree".}

    proc error*(msg: string) {.magic: "NError".}

  writes an error message at compile time

    proc warning*(msg: string) {.magic: "NWarning".}

  writes a warning message at compile time

    proc hint*(msg: string) {.magic: "NHint".}

  writes a hint message at compile time

    proc newStrLitNode*(s: string): PNimrodNode {.compileTime.}

  creates a string literal node from s

    proc newIntLitNode*(i: biggestInt): PNimrodNode {.compileTime.}

  creates a int literal node from i

    proc newFloatLitNode*(f: biggestFloat): PNimrodNode {.compileTime.}

  creates a float literal node from f

    proc newIdentNode*(i: TNimrodIdent): PNimrodNode {.compileTime.}

  creates an identifier node from i

    proc newIdentNode*(i: string): PNimrodNode {.compileTime.}

  creates an identifier node from i

    proc bindSym*(ident: string; rule: TBindSymRule = brClosed): PNimrodNode {.
        magic: "NBindSym".}

  creates a node that binds ident to a symbol node. The bound symbol may be an overloaded symbol. If `rule == brClosed` either an `nkClosedSymChoice` tree is returned or `nkSym` if the symbol is not ambiguous. If `rule == brOpen` either an `nkOpenSymChoice` tree is returned or `nkSym` if the symbol is not ambiguous. If `rule == brForceOpen` always an `nkOpenSymChoice` tree is returned even if the symbol is not ambiguous.

    proc callsite*(): PNimrodNode {.magic: "NCallSite".}

  returns the AST if the invokation expression that invoked this macro.

    proc toStrLit*(n: PNimrodNode): PNimrodNode {.compileTime.}

  converts the AST n to the concrete Nimrod code and wraps that in a string literal node

    proc lineinfo*(n: PNimrodNode): string {.magic: "NLineInfo".}

  returns the position the node appears in the original source file in the form filename(line, col)

    proc parseExpr*(s: string): PNimrodNode {.magic: "ParseExprToAst".}

  Compiles the passed string to its AST representation. Expects a single expression.

    proc parseStmt*(s: string): PNimrodNode {.magic: "ParseStmtToAst".}

  Compiles the passed string to its AST representation. Expects one or more statements.

    proc getAst*(macroOrTemplate: expr): PNimrodNode {.magic: "ExpandToAst".}

  Obtains the AST nodes returned from a macro or template invocation. Example:

      macro FooMacro() =
        var ast = getAst(BarTemplate())

    proc expectKind*(n: PNimrodNode; k: TNimrodNodeKind) {.compileTime.}

  checks that n is of kind k. If this is not the case, compilation aborts with an error message. This is useful for writing macros that check the AST that is passed to them.

    proc expectMinLen*(n: PNimrodNode; min: int) {.compileTime.}

  checks that n has at least min children. If this is not the case, compilation aborts with an error message. This is useful for writing macros that check its number of arguments.

    proc expectLen*(n: PNimrodNode; len: int) {.compileTime.}

  checks that n has exactly len children. If this is not the case, compilation aborts with an error message. This is useful for writing macros that check its number of arguments.

    proc newCall*(theProc: PNimrodNode; args: varargs[PNimrodNode]): PNimrodNode {.
        compileTime.}

  produces a new call node. theProc is the proc that is called with the arguments `args[0..]`.

    proc newCall*(theProc: TNimrodIdent; args: varargs[PNimrodNode]): PNimrodNode {.
        compileTime.}

  produces a new call node. theProc is the proc that is called with the arguments `args[0..]`.

    proc newCall*(theProc: string; args: varargs[PNimrodNode]): PNimrodNode {.
        compileTime.}

  produces a new call node. theProc is the proc that is called with the arguments `args[0..]`.

    proc nestList*(theProc: TNimrodIdent; x: PNimrodNode): PNimrodNode {.compileTime.}

  nests the list x into a tree of call expressions: `[a, b, c]` is transformed into `theProc(a, theProc(c, d))`.

    proc treeRepr*(n: PNimrodNode): string {.compileTime.}

  Convert the AST n to a human-readable tree-like string.

  See also repr and lispRepr.

    proc lispRepr*(n: PNimrodNode): string {.compileTime.}

  Convert the AST n to a human-readable lisp-like string,

  See also repr and treeRepr.

# Macros

    macro dumpTree*(s: stmt): stmt

  Accepts a block of nimrod code and prints the parsed abstract syntax tree using the toTree function. Printing is done at compile time.

  You can use this as a tool to explore the Nimrod's abstract syntax tree and to discover what kind of nodes must be created to represent a certain expression/statement.

    macro dumpLisp*(s: stmt): stmt

  Accepts a block of nimrod code and prints the parsed abstract syntax tree using the toLisp function. Printing is done at compile time.

  See dumpTree.

# Templates

    template emit*(s: expr): stmt

  accepts a single string argument and treats it as nimrod code that should be inserted verbatim in the program Example:

  > emit("echo " & '"' & "hello world".toUpper & '"')

Generated: 2012-09-23 21:47:53 UTC