Skip to content
Quick links:   Flags   Verbs   Functions   Glossary   Release docs

The REPL

The Miller REPL (read-evaluate-print loop) is an interactive counterpart to record-processing using the put/filter language. (A REPL is anything that evaluates what you type into it -- like python with no arguments, or Ruby's irb, or node with no arguments, etc.)

Miller's REPL isn't a source-level debugger which lets you execute one source-code statement at a time -- however, it does let you operate on one record at a time. Further, it lets you use "immediate expressions", namely, you can interact with the Miller programming language without having to provide data from an input file.

mlr repl


[mlr] 1 + 2
3

[mlr] apply([1,2,3,4,5], func(e) {return e ** 3})
[1, 8, 27, 64, 125]

[mlr] :open example.csv

[mlr] :read

[mlr] $*
{
  "color": "yellow",
  "shape": "triangle",
  "flag": "true",
  "k": 1,
  "index": 11,
  "quantity": 43.6498,
  "rate": 9.8870
}

[mlr] :context
FILENAME="example.csv",FILENUM=1,NR=1,FNR=1

Using Miller without the REPL

Using put and filter, you can do the following as we've seen above:

  • Specify input format (e.g. --icsv), output format (e.g. --ojson), etc. using command-line flags.
  • Specify filenames on the command line.
  • Define begin {...} blocks which are executed before the first record is read.
  • Define end {...} blocks which are executed after the last record is read.
  • Define user-defined functions/subroutines using func and subr.
  • Specify statements to be executed on each record -- which are anything outside of begin/end/func/subr.
  • Example:
mlr --icsv --ojson --from example.csv head -n 2 \
  then put 'begin {print "HELLO"} $qr = $quantity / $rate; end {print "GOODBYE"}'

HELLO
[
{
  "color": "yellow",
  "shape": "triangle",
  "flag": "true",
  "k": 1,
  "index": 11,
  "quantity": 43.6498,
  "rate": 9.8870,
  "qr": 4.414868008496004
},
{
  "color": "red",
  "shape": "square",
  "flag": "true",
  "k": 2,
  "index": 15,
  "quantity": 79.2778,
  "rate": 0.0130,
  "qr": 6098.292307692308
}GOODBYE

]

Using Miller with the REPL

Using the REPL, by contrast, you get interactive control over those same steps:

  • Specify input format (e.g. --icsv), output format (e.g. --ojson), etc. using command-line flags.
  • REPL-only statements (non-DSL statements) start with :, such as :help or :quit or :open.
  • Specify filenames either on the command line or via :open at the Miller REPL.
  • Read records one at a time using :read.
  • Write the current record (maybe after you've modified it with things like $z = $x + $y) using :write. This goes to the terminal; you can use :> {filename} to make writes go to a file, or :>> {filename} to append.
  • You can type :reopen to go back to the start of the same file(s) you specified with :open.
  • Skip ahead using statements :skip 10 or :skip until NR == 100 or :skip until $status_code != 200.
  • Similarly, but processing records rather than skipping past them, using :process rather than :skip. Like :write, these go to the screen; use :> {filename} or :>> {filename} to log to a file instead.
  • Define begin {...} blocks; invoke them at will using :begin.
  • Define end {...} blocks; invoke them at will using :end.
  • Define user-defined functions/subroutines using func/subr; call them from other statements.
  • Interactively specify statements to be executed immediately on the current record.
  • Load any of the above from Miller-script files using :load.

The input "record" by default is the empty map but you can do things like $x=3, or unset $y, or $* = {"x": 3, "y": 4} to populate it. Or, :open foo.dat followed by :read to populate it from a data file.

Non-assignment expressions, such as 7 or true, operate as filter conditions in the put DSL: they can be used to specify whether a record will or won't be included in the output-record stream. But here in the REPL, they are simply printed to the terminal, e.g. if you type 1+2, you will see 3.

Entering multi-line statements

  • To enter multi-line statements, enter < on a line by itself, then the code (taking care for semicolons), then > on a line by itself. These will be executed immediately.
  • If you enter << on a line by itself, then the code, then >> on a line by itself, the statements will be remembered for executing on records with :main, as if you had done :load to load statements from a file.

Examples

Use the REPL to look at arithmetic:

mlr repl


[mlr] 6/3
2

[mlr] 6/5
1.2

[mlr] typeof(6/3)
int

[mlr] typeof(6/5)
float

Read the first record from a small file:

mlr repl


[mlr] :open foo.dat

[mlr] :read

[mlr] :context
FILENAME="foo.dat",FILENUM=1,NR=1,FNR=1

[mlr] $*
{
  "a": "eks",
  "b": "wye",
  "i": 4,
  "x": 0.38139939387114097,
  "y": 0.13418874328430463
}

[mlr] $z = $x + $i

[mlr] :write
a=eks,b=wye,i=4,x=0.38139939387114097,y=0.13418874328430463,z=4.381399393871141

Skip until deep into a larger file, then inspect a record:

mlr repl --csv


[mlr] :open data/colored-shapes.csv
[mlr] :skip until NR == 10000
[mlr] :r
[mlr] $*
{
  "color": "yellow",
  "shape": "circle",
  "flag": 1,
  "i": 496422,
  "u": 0.6530503199545348,
  "v": 0.23908588907834516,
  "w": 0.4799125551304738,
  "x": 6.379888206335166
}

Parsing and operator precedence

You can invoke mlr repl with the -v or -d flags to show parse trees for expressions you enter. (The -v and -d flags differ only in the format they use to present the parse trees.) For example, if you have any questions about operator precedence, you can check the operator-precedence section -- or, you can try it out and see for yourself:

mlr repl -v

Miller 6.0.0-rc1 REPL for darwin/amd64/go1.16.5
Docs: https://miller.readthedocs.io
Type ':h' or ':help' for online help; ':q' or ':quit' to quit.
[mlr] x = 1 * 2 + 3
* statement block
    * assignment "="
        * local variable "x"
        * operator "+"
            * operator "*"
                * int literal "1"
                * int literal "2"
            * int literal "3"
[mlr] x = 1 + 2 * 3
* statement block
    * assignment "="
        * local variable "x"
        * operator "+"
            * int literal "1"
            * operator "*"
                * int literal "2"
                * int literal "3"

mlr repl -d

$ mrpl -d
Miller 6.0.0-rc1 REPL for darwin/amd64/go1.16.5
Docs: https://miller.readthedocs.io
Type ':h' or ':help' for online help; ':q' or ':quit' to quit.
[mlr] x = 1 * 2 + 3
(statement-block
    (=
        x
        (+
            (* 1 2)
            3
        )
    )
)
[mlr] x = 1 + 2 * 3
(statement-block
    (=
        x
        (+
            1
            (* 2 3)
        )
    )
)

History-editing

No command-line-history-editing feature is built in but rlwrap mlr repl is a delight. You may need brew install rlwrap, sudo apt-get install rlwrap, etc. depending on your platform.

Suggestion: alias mrpl='rlwrap mlr repl' in your shell's startup file.

Customization

In your shell environment you can set the MLR_REPL_PS1 and MLR_REPL_PS2 environment variables if you like:

$ mlr repl -q
> 1+2
3
> <
... func f(x) {
...   return x**2
... }
... >
> f(99)
9801

$ export MLR_REPL_PS1='\nMLR REPL> '
$ export MLR_REPL_PS2='  '
$ mlr repl -q

MLR REPL> 1+2
3

MLR REPL> <
  func f(x) {
    return x**2
  }
  >

MLR REPL> f(99)
9801

See also MLR_REPL_PS1_COLOR on the output-colorization page.

Online help

After mlr repl, type :help to see more about your options. In particular, :help examples.

Back to top