...

Package compile

import "go.starlark.net/internal/compile"
Overview
Index

Overview ▾

Package compile defines the Starlark bytecode compiler. It is an internal package of the Starlark interpreter and is not directly accessible to clients.

The compiler generates byte code with optional uint32 operands for a virtual machine with the following components:

  • a program counter, which is an index into the byte code array.
  • an operand stack, whose maximum size is computed for each function by the compiler.
  • an stack of active iterators.
  • an array of local variables. The number of local variables and their indices are computed by the resolver. Locals (possibly including parameters) that are shared with nested functions are 'cells': their locals array slot will contain a value of type 'cell', an indirect value in a box that is explicitly read/updated by instructions.
  • an array of free variables, for nested functions. Free variables are a subset of the ancestors' cell variables. As with locals and cells, these are computed by the resolver.
  • an array of global variables, shared among all functions in the same module. All elements are initially nil.
  • two maps of predeclared and universal identifiers.

Each function has a line number table that maps each program counter offset to a source position, including the column number.

Operands, logically uint32s, are encoded using little-endian 7-bit varints, the top bit indicating that more bytes follow.

Constants

Increment this to force recompilation of saved bytecode files.

const Version = 13

Variables

Disassemble causes the assembly code for each function to be printed to stderr as it is generated.

var Disassemble = false

func PrintOp

func PrintOp(fn *Funcode, pc uint32, op Opcode, arg uint32)

PrintOp prints an instruction. It is provided for debugging.

type Binding

A Binding is the name and position of a binding identifier.

type Binding struct {
    Name string
    Pos  syntax.Position
}

type Bytes

The type of a bytes literal value, to distinguish from text string.

type Bytes string

type Funcode

A Funcode is the code of a compiled Starlark function.

Funcodes are serialized by the encoder.function method, which must be updated whenever this declaration is changed.

type Funcode struct {
    Prog *Program
    Pos  syntax.Position // position of def or lambda token
    Name string          // name of this function
    Doc  string          // docstring of this function
    Code []byte          // the byte code

    Locals                []Binding // locals, parameters first
    Cells                 []int     // indices of Locals that require cells
    Freevars              []Binding // for tracing
    MaxStack              int
    NumParams             int
    NumKwonlyParams       int
    HasVarargs, HasKwargs bool
    // contains filtered or unexported fields
}

func (*Funcode) Position

func (fn *Funcode) Position(pc uint32) syntax.Position

Position returns the source position for program counter pc.

type Opcode

type Opcode uint8

"x DUP x x" is a "stack picture" that describes the state of the stack before and after execution of the instruction.

OP<index> indicates an immediate operand that is an index into the specified table: locals, names, freevars, constants.

const (
    NOP Opcode = iota // - NOP -

    // stack operations
    DUP  //   x DUP x x
    DUP2 // x y DUP2 x y x y
    POP  //   x POP -
    EXCH // x y EXCH y x

    // binary comparisons
    // (order must match Token)
    LT
    GT
    GE
    LE
    EQL
    NEQ

    // binary arithmetic
    // (order must match Token)
    PLUS
    MINUS
    STAR
    SLASH
    SLASHSLASH
    PERCENT
    AMP
    PIPE
    CIRCUMFLEX
    LTLT
    GTGT

    IN

    // unary operators
    UPLUS  // x UPLUS x
    UMINUS // x UMINUS -x
    TILDE  // x TILDE ~x

    NONE      // - NONE None
    TRUE      // - TRUE True
    FALSE     // - FALSE False
    MANDATORY // - MANDATORY Mandatory	     [sentinel value for required kwonly args]

    ITERPUSH     //       iterable ITERPUSH     -  [pushes the iterator stack]
    ITERPOP      //              - ITERPOP      -    [pops the iterator stack]
    NOT          //          value NOT          bool
    RETURN       //          value RETURN       -
    SETINDEX     //        a i new SETINDEX     -
    INDEX        //            a i INDEX        elem
    SETDICT      // dict key value SETDICT      -
    SETDICTUNIQ  // dict key value SETDICTUNIQ  -
    APPEND       //      list elem APPEND       -
    SLICE        //   x lo hi step SLICE        slice
    INPLACE_ADD  //            x y INPLACE_ADD  z      where z is x+y or x.extend(y)
    INPLACE_PIPE //            x y INPLACE_PIPE z      where z is x|y
    MAKEDICT     //              - MAKEDICT     dict

    // control flow
    JMP     //            - JMP<addr>     -
    CJMP    //         cond CJMP<addr>    -
    ITERJMP //            - ITERJMP<addr> elem   (and fall through) [acts on topmost iterator]

    CONSTANT     //                 - CONSTANT<constant>  value
    MAKETUPLE    //         x1 ... xn MAKETUPLE<n>        tuple
    MAKELIST     //         x1 ... xn MAKELIST<n>         list
    MAKEFUNC     // defaults+freevars MAKEFUNC<func>      fn
    LOAD         //   from1 ... fromN module LOAD<n>      v1 ... vN
    SETLOCAL     //             value SETLOCAL<local>     -
    SETGLOBAL    //             value SETGLOBAL<global>   -
    LOCAL        //                 - LOCAL<local>        value
    FREE         //                 - FREE<freevar>       cell
    FREECELL     //                 - FREECELL<freevar>   value       (content of FREE cell)
    LOCALCELL    //                 - LOCALCELL<local>    value       (content of LOCAL cell)
    SETLOCALCELL //             value SETLOCALCELL<local> -           (set content of LOCAL cell)
    GLOBAL       //                 - GLOBAL<global>      value
    PREDECLARED  //                 - PREDECLARED<name>   value
    UNIVERSAL    //                 - UNIVERSAL<name>     value
    ATTR         //                 x ATTR<name>          y           y = x.name
    SETFIELD     //               x y SETFIELD<name>      -           x.name = y
    UNPACK       //          iterable UNPACK<n>           vn ... v1

    // n>>8 is #positional args and n&0xff is #named args (pairs).
    CALL        // fn positional named                CALL<n>        result
    CALL_VAR    // fn positional named *args          CALL_VAR<n>    result
    CALL_KW     // fn positional named       **kwargs CALL_KW<n>     result
    CALL_VAR_KW // fn positional named *args **kwargs CALL_VAR_KW<n> result

    OpcodeArgMin = JMP
    OpcodeMax    = CALL_VAR_KW
)

func (Opcode) String

func (op Opcode) String() string

type Program

A Program is a Starlark file in executable form.

Programs are serialized by the Program.Encode method, which must be updated whenever this declaration is changed.

type Program struct {
    Loads     []Binding     // name (really, string) and position of each load stmt
    Names     []string      // names of attributes and predeclared variables
    Constants []interface{} // = string | int64 | float64 | *big.Int | Bytes
    Functions []*Funcode
    Globals   []Binding // for error messages and tracing
    Toplevel  *Funcode  // module initialization function
}

func DecodeProgram

func DecodeProgram(data []byte) (_ *Program, err error)

DecodeProgram decodes a compiled Starlark program from data.

func Expr

func Expr(expr syntax.Expr, name string, locals []*resolve.Binding) *Program

Expr compiles an expression to a program whose toplevel function evaluates it.

func File

func File(stmts []syntax.Stmt, pos syntax.Position, name string, locals, globals []*resolve.Binding) *Program

File compiles the statements of a file into a program.

func (*Program) Encode

func (prog *Program) Encode() []byte

Encode encodes a compiled Starlark program.