[ planet-factor ]

Factor has had virtual sequences for a long time. A virtual sequence is typically lazy and can compute an element when asked for it, or look it up in another sequence, rather than build all the elements up front. It only needs to provide the sequence operations that describe its contents.

That lets us use normal sequence words like nth, each, map, and reduce with ranges, reversed views, windows into arrays, and even collections of combinations. I thought it would be fun to look at the different kinds we have accumulated over the years.

Note: In the examples below, >array materializes the elements into an array so the printed output shows their contents. >string does the same for character sequences. In some of the examples, the elements are themselves views like slices, we use [ >array ] map to materialize each one. These conversions are just for display; words such as nth, each, map, etc. can work directly with the virtual sequence.

Computed elements

We can start with sequences whose elements follow a simple formula:

USING: arrays prettyprint ranges sequences ;

IN: scratchpad 5 <iota> >array .
{ 0 1 2 3 4 }

IN: scratchpad 2 10 2 <range> >array .
{ 2 4 6 8 10 }

IN: scratchpad 4 "hello" <repetition> >array .
{ "hello" "hello" "hello" "hello" }

An iota stores its length, a range stores the information needed to compute its arithmetic progression, and a repetition stores a length and one element. Repeating a mutable object repeats the reference to that object.

The ranges vocabulary also has convenient constructors such as [a..b], [a..b), and [1..b] for inclusive and exclusive endpoints. And math.bits lets us treat an integer as a sequence of bits:

USE: math.bits

IN: scratchpad 0b10110 5 <bits> >array .
{ f t t f t }

Here the least significant bit comes first. <binary-bits> provides the same ordering using 0 and 1 instead of booleans.

Selecting and rearranging elements

Many virtual sequences are views of existing data. Two particularly useful ones, <slice> and <reversed>, are part of the core sequences vocabulary:

IN: scratchpad 1 4 { 10 20 30 40 50 } <slice> <reversed> >array .
{ 40 30 20 }

The slice selects indices 1 through 3, and the reversed view reads those indices in the opposite order. Neither view copies the elements. They also support writing through to a mutable backing sequence:

USE: kernel

IN: scratchpad { 10 20 30 } clone
               dup <reversed> 99 0 rot set-nth .
{ 10 20 99 }

Changing index zero of the reversed view changes the last element of the original array.

There are several other ways to select or rearrange existing elements:

For example, a step of two selects every other element (equivalent to [::2] in Python):

USE: sequences.extras

IN: scratchpad f f 2 { 10 20 30 40 50 } <step-slice> >array .
{ 10 30 50 }

The two f values select the default start and end. A column view picks an element from each row:

USE: columns

IN: scratchpad { { 1 2 3 } { 4 5 6 } } 1 <column> >array .
{ 2 5 }

The related <flipped> word builds a sequence of column views, giving us a transposed view of a rectangular matrix. It allocates the outer collection of views while sharing the row data.

Groups and windows

Sometimes the elements of a virtual sequence are themselves views. The grouping vocabulary provides non-overlapping groups and overlapping clumps:

USE: grouping

IN: scratchpad { 1 2 3 4 5 } 2 <groups> [ >array ] map .
{ { 1 2 } { 3 4 } { 5 } }

IN: scratchpad { 1 2 3 4 5 } 3 <clumps> [ >array ] map .
{ { 1 2 3 } { 2 3 4 } { 3 4 5 } }

A group can be shorter at the end; a clump has the requested size. Their elements are slices, which is why these examples convert each element to an array. The <circular-slice> and <circular-clumps> constructors extend the idea to windows that wrap around the end.

The sequences.windowed vocabulary has trailing windows, including shorter ones at the beginning:

USE: sequences.windowed

IN: scratchpad { 1 2 3 4 5 } 3 <windowed-sequence>
               [ >array ] map .
{ { 1 } { 1 2 } { 1 2 3 } { 2 3 4 } { 3 4 5 } }

And grouping.extras provides <prefixes> and <suffixes> for the nonempty initial and final slices of a sequence:

USE: grouping.extras

IN: scratchpad { 1 2 3 } <prefixes> [ >array ] map .
{ { 1 } { 1 2 } { 1 2 3 } }

IN: scratchpad { 1 2 3 } <suffixes> [ >array ] map .
{ { 1 2 3 } { 2 3 } { 3 } }

Repeating and combining sequences

We can make a sequence appear longer by reusing its elements. The sequences.repeating vocabulary offers two different arrangements:

USING: sequences.repeating strings ;

IN: scratchpad "abc" 8 <cycles> >string .
"abcabcab"

IN: scratchpad "abc" 3 <element-repeats> >string .
"aaabbbccc"

<cycles> takes the desired total length. <cycles-from> also accepts a starting offset. <element-repeats> takes the number of times to repeat each element.

Other views assemble data from multiple sources:

Here are a few examples:

USING: sequences.cords sequences.interleaved sequences.merged
sequences.padded sequences.zipped ;

IN: scratchpad { 1 2 } { 3 4 } cord-append >array .
{ 1 2 3 4 }

IN: scratchpad { 1 2 3 } { 10 20 30 40 } <2merged> >array .
{ 1 10 2 20 3 30 }

IN: scratchpad { 1 2 3 } { 10 20 } <zipped> >array .
{ { 1 10 } { 2 20 } }

IN: scratchpad "abc" CHAR: - <interleaved> >string .
"a-b-c"

IN: scratchpad { 1 2 3 } 5 0 <padded-head> >array .
{ 0 0 1 2 3 }

The assocs vocabulary also provides <enumerated>, a view of index/value pairs that works as both a sequence and an association. <zip-index> in sequences.extras provides a sequence of index/value pairs as well.

Transforming values

Views can transform values as they are read. The sequences.modified vocabulary provides scaling, offsets, and elementwise sums:

USE: sequences.modified

IN: scratchpad { 1 2 3 } 10 <scaled> 1 <offset> >array .
{ 11 21 31 }

IN: scratchpad { { 1 2 3 } { 10 20 } } <summed> >array .
{ 11 22 3 }

The summed view extends to the longest input, treating missing elements as zero. Scaled and offset views can also translate writes back to their underlying sequence; writing through a scaled view requires a nonzero scale factor.

For complex numbers, sequences.complex interprets adjacent real values as real and imaginary components, while sequences.complex-components exposes those components from a sequence of complex numbers.

Products and combinations

Some virtual sequences represent collections that could be much larger than their inputs. The sequences.product vocabulary provides a Cartesian product:

USE: sequences.product

IN: scratchpad { { "red" "blue" } { 1 2 3 } }
               <product-sequence> >array .
{
    { "red" 1 }
    { "red" 2 }
    { "red" 3 }
    { "blue" 1 }
    { "blue" 2 }
    { "blue" 3 }
}

Similarly, math.combinatorics provides <permutations>, <k-permutations>, and <combinations>:

USE: math.combinatorics

IN: scratchpad { "a" "b" "c" } 2 <combinations> >array .
{ { "a" "b" } { "a" "c" } { "b" "c" } }

IN: scratchpad { "a" "b" "c" } <permutations> length .
6

These can compute a selected result without constructing every result before it. Iterating over the entire sequence still takes work proportional to the number of results, and >array asks to store all of them.

Sharing data and materializing results

A view shares data with its source. Even sequences.frozen, which prevents writes through the view, can observe changes made through the original array:

USE: sequences.frozen

IN: scratchpad { 1 2 3 } clone dup <frozen>
               swap 99 0 rot set-nth >array .
{ 99 2 3 }

Keeping a small slice can therefore keep a much larger backing sequence alive. Nested views add indexing work, and some views allocate an element, such as a pair or a slice, when it is requested.

Implementing a virtual sequence

There is a useful implementation detail behind these examples. The core virtual-sequence mixin supports views that translate an index into an index and another sequence using virtual@. A wrapped-sequence already supplies a backing sequence and delegates its length. Reversing one takes very little code:

USING: accessors kernel math sequences ;

TUPLE: backwards < wrapped-sequence ;
C: <backwards> backwards

M: backwards virtual@
    seq>> [ length swap - 1 - ] keep ;

virtual-exemplar supplies the sequence used to choose result types for operations such as map. Computed sequences, such as ranges and combinations, can instead implement the sequence protocol directly. The broader idea of a virtual sequence covers both approaches.

The same machinery appears in specialized structures: composed quotations in quotations, strided storage in arrays.shaped, BLAS vectors in math.blas.vectors, gap buffers, and insertion views in sequences.inserters.

Have ideas for other virtual sequences that would be useful in Factor? Let us know!

I highly recommend reading the guided tour of Factor. It provides a great introduction to the language and libraries of Factor. Even still, I sometimes have also wanted to have more code-forward examples of everyday syntax, control flow, combinators, and some of the main libraries. This is that overview. It assumes you have programmed before, but have not necessarily used a stack-based language.

Hello, world

The simplest Hello, world is just:

"Hello, world!" print

You can run that from the listener:

IN: scratchpad "Hello, world!" print
Hello, world!

And you can run it from the command-line:

$ ./factor -e="\"Hello, world!\" print"
Hello, world!

Of course, you can also make this a file named hello.factor, which defines a hello vocabulary (something you learn about in the your first program tutorial).

USING: io ;
IN: hello

: main ( -- )
    "Hello, world!" print ;

MAIN: main

The syntax used above includes:

• USING: imports vocabularies (named collections of words)
• IN: selects the vocabulary for definitions, and
• MAIN: sets an entry point.

And then you can run it either as a script:

$ ./factor hello.factor
Hello, world!

Or, if this is available in the vocabulary roots search path, run the vocabulary’s main word:

$ ./factor -run=hello
Hello, world!

For the rest of this overview, try the examples in the listener, Factor’s interactive REPL. Start the terminal listener with ./factor -run=listener, or use the graphical listener in the development environment. Each example includes its imports; examples that build on a definition assume you have entered that definition too. IN: scratchpad puts experimental definitions in the listener’s usual working vocabulary. Feel free to paste the code directly, to see what it does. Comments beginning with ! are part of valid Factor source.

For a quick start, work through the stack, word definitions, quotations, control flow, and sequences. The later sections introduce objects, metaprogramming, and libraries that you can return to as you need them.

Values and the stack

Literals push values onto the data stack. Words consume inputs from the top of that stack and push their outputs. Code runs from left to right:

USING: math prettyprint ;

2 3 + .                         ! 5
10 4 - .                        ! 6
2 3 + 4 * .                     ! 20

. consumes and prints an object. print consumes and prints a string. The comments beside examples show the output. Because printing removes the value, these examples leave the stack empty unless stated otherwise. There are no parentheses around function arguments: put the arguments on the stack, then invoke the word. Below, the top of the stack is on the right:

Code       Stack
2          2
3          2 3
+          5
4          5 4
*          20
.          (empty)

Spaces matter. 2 3 + is three tokens; 2+3 is a single token, which would need to be the name of a word. Names like number>string, empty?, and set-at are ordinary word names. A trailing ? conventionally marks a predicate; > often appears in conversion names. Those characters are part of the name, not separate operators. A trailing ! often marks a mutating variant, such as append!; * usually marks an alternative form. There are some conventions useful for learning word and type naming.

Comments and literals

USING: math multiline prettyprint ;

! A comment runs to the end of the line.
/* A block comment can span
   several lines. */

42 .                            ! Integer
-17 .                           ! Negative integer
0xff .                          ! 255, hexadecimal
0b1010 .                        ! 10, binary
3/4 .                           ! Exact rational
1.25 .                          ! Floating point
C{ 2 3 } .                      ! Complex number: 2 + 3i

t .                             ! True
f .                             ! False
"hello\nworld" .                ! String with an escape
CHAR: A .                       ! 65, a character code point

{ 1 2 3 } .                     ! Array
V{ 1 2 3 } .                    ! Growable vector
B{ 0 127 255 } .                ! Byte array
H{
    { "name" "Ada" }
    { "age" 36 }
} .                             ! Hashtable
[ 1 + ] .                       ! Quotation: code as a value

Arrays and quotations contain objects without executing them. Collection literals are useful for fixed data; when mutating one inside a word, use clone to obtain a fresh copy rather than changing a shared literal. This is a shallow copy: objects inside the collection are still shared.

Block comments come from the multiline vocabulary. The literals vocabulary can also run code at parse time and insert its results into literal values:

USING: literals math prettyprint ;

{ 1 $[ 2 3 + ] 6 } .           ! { 1 5 6 }

CHAR: produces an integer code point; Factor has no separate character type. { ... } is an array, while [ ... ] is executable code held as a value called a quotation. Spaces separate the literal openers, their contents, and the closing delimiters, as in { 1 2 3 } and [ 1 + ].

Strings and escape characters

String literals use double quotes. A backslash introduces a character escape:

For example:

USING: io math.parser prettyprint sequences splitting unicode ;

"She said \"hello\"." print        ! She said "hello".
"C:\\Users\\Ada" print             ! C:\Users\Ada
"\x41\u000042\u{43}" print         ! ABC
"\u{greek-small-letter-pi}" print  ! π
"first\nsecond" print              ! Prints two lines
"\t" length .                      ! 1: the escape represents one character
"hello" length .                   ! 5
"hello" >upper .                   ! "HELLO"
"a,b,c" "," split .                ! { "a" "b" "c" }
{ "a" "b" "c" } ", " join .        ! "a, b, c"
"42" string>number .               ! 42
42 number>string .                 ! "42"
"oops" string>number .             ! f

The six-digit \u form differs from languages that use four digits; the braced form is often easier to read. Unknown escapes are errors. Strings can also span source lines directly: an actual newline becomes part of the string. A backslash immediately before a source newline continues the string without including that newline. A backslash followed by a literal space also represents a space, like \s.

Note: the length of a string is the number of code points, not the number of visible glyphs. You can learn a bit more by reading about Factor’s Unicode support.

Stack shuffling

Typical of concatenative languages, the stack is a data structure with it’s own access patterns that we often call stack shuffling.

USING: kernel prettyprint ;

10 dup . .                      ! Prints 10, then 10
10 20 swap . .                  ! Prints 10, then 20
10 20 over . . .                ! Prints 10, then 20, then 10
10 20 nip .                     ! 20: discard the second item
10 20 drop .                    ! 10: discard the top item

The usual stack shuffling words have these effects:

! dup   ( x -- x x )
! drop  ( x -- )
! swap  ( x y -- y x )
! over  ( x y -- x y x )
! nip   ( x y -- y )
! rot   ( x y z -- y z x )

Most Factor code uses short definitions and combinators to keep explicit shuffling to a minimum.

In a stack effect, inputs and outputs run from left to right, with the topmost value last. swap therefore changes a stack ending in x y into one ending in y x; values below those inputs are untouched. Repeated . calls print the topmost result first.

Defining words

You can create words that contain code that is executed when called:

USING: kernel math prettyprint ;
IN: scratchpad

: square ( n -- n-squared ) dup * ;
: neighbors ( n -- below above )
    dup 1 - swap 1 + ;

5 square .                      ! 25
5 neighbors . .                 ! Prints 6, then 4

CONSTANT: answer 42
answer .                        ! 42

: begins a definition and ; ends it. The stack effect ( inputs -- outputs ) documents how many values the word consumes and produces. Its names describe the values; they do not bind variables or specify types. The compiler checks stack effects, including compatible effects for branches. Words can return several values simply by leaving them on the stack. There is no explicit return: execution finishes at the end of the word.

ALIAS: new-name existing-word defines another name for a word.

Arithmetic and comparisons

Lots of arithmetic is available for computing with numbers:

USING: kernel math math.functions math.order prettyprint ;

7 2 / .                         ! 3+1/2, an exact rational
7 2 /i .                        ! 3, integer division
7 2 mod .                       ! 1
2 10 ^ .                        ! 1024
9 sqrt .                        ! 3.0
-5 abs .                        ! 5
3 8 min .                       ! 3
3 8 max .                       ! 8

2 3 < .                         ! t
2 3 >= .                        ! f
"hello" "hello" = .             ! t, value equality

Integers grow beyond machine size automatically, and division of integers can produce exact ratios. Use floating-point inputs when you want floating-point arithmetic.

Bitwise operations have their own names, separate from boolean logic:

USING: math prettyprint ;

0b1100 0b1010 bitand .          ! 8
0b1100 0b1010 bitor .           ! 14
0b1100 0b1010 bitxor .          ! 6
1 3 shift .                     ! 8: shift left
8 -1 shift .                    ! 4: shift right

Quotations

Square brackets produce a quotation. call executes it:

USING: kernel math prettyprint sequences ;

5 [ 1 + ] call .                ! 6
{ 1 2 3 } [ 2 * ] map .         ! { 2 4 6 }

Quotations can be passed to words, returned from words, and stored in collections. Words that take quotations are called combinators.

Booleans and conditionals

In boolean tests, only f is false. Zero, an empty string, and an empty array are all true.

USING: kernel math prettyprint ;

t f and .                        ! f
t f or .                         ! t
f not .                          ! t

3 2 > [ "yes" ] [ "no" ] if .    ! "yes"
0 [ "truthy" ] [ "false" ] if .  ! "truthy"

t [ "runs" . ] when              ! "runs"
f [ "runs too" . ] unless        ! "runs too"

if consumes a condition and two quotations. It calls the first quotation for a true condition and the second for f. when and unless take one quotation. These are words that operate on code values, just like + operates on numbers.

For several alternatives, use cond or case:

USING: combinators kernel math prettyprint ;
IN: scratchpad

: sign-name ( n -- string )
    {
        { [ dup 0 < ] [ drop "negative" ] }
        { [ dup 0 = ] [ drop "zero" ] }
        [ drop "positive" ]
    } cond ;

-3 sign-name .                  ! "negative"

: color-name ( color -- string )
    {
        { "r" [ "red" ] }
        { "g" [ "green" ] }
        [ drop "unknown" ]
    } case ;

"g" color-name .                ! "green"

cond tries predicate quotations in order. case compares an input with each key; a matching branch consumes the key automatically, while the default branch receives the unmatched input.

and and or combine values that have already been computed. For short-circuit evaluation, pass predicate quotations instead:

USING: combinators.short-circuit kernel math prettyprint ;

5 { [ 0 > ] [ 10 < ] } 1&& .    ! t: positive and less than ten
-5 { [ 0 < ] [ 10 > ] } 1|| .   ! t: negative or greater than ten

Each predicate receives the same input. 1&& stops at the first false result; 1|| stops at the first true result. The leading number is the number of inputs passed to each predicate.

Keeping and hiding values

The dip word temporarily hides a value while a quotation works on the stack below it. keep gives a quotation a value and also preserves that value:

USING: kernel math prettyprint ;

10 20 [ 2 * ] dip + .           ! 40: double 10, then restore 20 and add
5 [ 1 + ] keep . .              ! Prints 5, then 6

! dip   ( ..a x quot -- ..b x )
! keep  ( ..a x quot -- ..b x )

The overall shapes look alike, but dip hides x from the quotation and keep passes it in. 2dip hides two values; 2keep preserves two inputs.

Applying several quotations

The bi family covers several common ways to distribute inputs:

USING: kernel math prettyprint ;

! Apply two quotations to the same input.
5 [ 1 + ] [ 2 * ] bi . .        ! Prints 10, then 6

! Apply one quotation to each of two inputs.
3 4 [ 2 * ] bi@ . .             ! Prints 8, then 6

! Apply separate quotations to separate inputs.
3 4 [ 1 + ] [ 2 * ] bi* . .     ! Prints 8, then 4

The 2bi lets a word calculate two results from the same inputs:

USING: kernel math prettyprint ;
IN: scratchpad

: sum-and-product ( a b -- sum product )
    [ + ] [ * ] 2bi ;

3 4 sum-and-product . .         ! Prints 12, then 7

Then tri, tri@, and tri* extend these patterns to three quotations or inputs.

You can find cleave, napply and spread as the generalizations of those patterns.

Partial application and composition

The curry word binds a value to the beginning of a quotation. compose joins two quotations so that one runs after the other:

USING: kernel math prettyprint sequences ;

{ 1 2 3 } 10 [ + ] curry map .    ! { 11 12 13 }
5 [ 1 + ] [ 2 * ] compose call .  ! 12

10 [ + ] curry behaves like [ 10 + ]. This is a convenient way to build a quotation using a value computed at runtime.

The fry vocabulary provides quotation templates. _ inserts a value; @ inserts a call to a supplied quotation:

USING: fry kernel math prettyprint sequences ;

{ 1 2 3 } 10 '[ _ + ] map .     ! { 11 12 13 }
5 [ 1 + ] '[ @ 2 * ] call .     ! 12

The apostrophe in '[ ... ] makes this a template rather than an ordinary quotation. Its placeholders consume their values when the template is constructed, not when the resulting quotation is called.

Defining combinators

A combinator can be an ordinary word with quotation inputs. Give those inputs their own stack effects and declare the word inline so the compiler can infer the effects at its call sites:

USING: kernel math prettyprint ;
IN: scratchpad

: twice ( ... quot: ( ... -- ... ) -- ... )
    dup [ call ] dip call ; inline

3 [ 2 * ] twice .               ! 12

The ... represents values carried through the combinator. Here, the supplied quotation must preserve stack height, and twice calls it twice.

Loops and recursion

USING: kernel math prettyprint sequences ;

3 [ "hello" . ] times           ! Print three times
{ "Ada" "Grace" } [ . ] each    ! Visit each element
5 <iota> [ . ] each             ! Print 0 through 4

0 [ dup 3 < ] [ dup . 1 + ] while drop
! Print 0, 1, 2; keep the counter on the stack

The looping combinator while calls its predicate before each iteration. The predicate leaves a condition; the body updates the loop’s values. until reverses the condition. Often each, map, or reduce expresses the loop directly.

Recursion uses an ordinary call to the word being defined:

USING: kernel math prettyprint ;
IN: scratchpad

: factorial ( n -- n! )
    dup 1 <=
    [ drop 1 ]
    [ dup 1 - factorial * ] if ;

5 factorial .                   ! 120

Definitions are read in order: define helper words before words that use them. DEFER: declares a word before its implementation, allowing mutual recursion:

USING: kernel math prettyprint ;
IN: scratchpad

DEFER: odd-count?

: even-count? ( n -- ? )
    dup 0 = [ drop t ] [ 1 - odd-count? ] if ;

: odd-count? ( n -- ? )
    dup 0 = [ drop f ] [ 1 - even-count? ] if ;

6 even-count? .                 ! t
7 odd-count? .                  ! t

These examples accept nonnegative integers. Factor guarantees tail-call optimization, so a final call such as the one to odd-count? can continue without growing the call stack.

Local variables and closures

When names make an algorithm easier to read, import locals and define a word with ::. Inputs become lexical variables:

USING: kernel locals math prettyprint sequences ;
IN: scratchpad

:: rectangle-area ( width height -- area )
    width height * ;

:: add-offset ( seq offset -- newseq )
    seq [| n | n offset + ] map ;

3 4 rectangle-area .            ! 12
{ 1 2 3 } 10 add-offset .       ! { 11 12 13 }

:: hypotenuse-squared ( a b -- n )
    a a * :> a-squared
    b b * :> b-squared
    a-squared b-squared + ;

:> binds a computed value. [| n | ... ] names quotation inputs and can capture enclosing variables, as offset does above. Output names in :: still describe stack results; there is no implicit return variable.

Mutable locals have an exclamation point in their declaration and an associated setter:

USING: kernel locals math prettyprint ;

[let
    0 :> total!
    5 [ total 1 + total! ] times
    total .                     ! 5
]

[let ... ] establishes a lexical scope, including in the listener.

Sequences

Arrays, vectors, strings, and several other types share the sequence protocol. Most sequence words work across these types:

USING: kernel math prettyprint sequences sorting ;

{ 10 20 30 } length .               ! 3
{ 10 20 30 } first .                ! 10
1 { 10 20 30 } nth .                ! 20, zero-based indexing
{ 1 2 } { 3 4 } append .            ! { 1 2 3 4 }
{ 1 2 3 } reverse .                 ! { 3 2 1 }

{ 1 2 3 4 } [ dup * ] map .         ! { 1 4 9 16 }
{ 1 2 3 4 } [ 2 mod 0 = ] filter .  ! { 2 4 }
{ 1 2 3 4 } 0 [ + ] reduce .        ! 10
{ 1 2 3 } [ 0 > ] all? .            ! t
{ 1 2 3 } [ 2 = ] any? .            ! t
{ 3 1 2 } natural-sort .            ! { 1 2 3 }

V{ 1 2 } clone
3 over push .                       ! V{ 1 2 3 }

The sequence combinator map collects quotation results; each is for side effects. reduce threads an accumulator through the sequence. push mutates a growable sequence and consumes both the new element and the sequence.

For incremental construction, make collects values produced inside a quotation. , adds one element and % adds the elements of a sequence:

USING: make prettyprint ;

[ 1 , { 2 3 } % 4 , ] { } make .            ! { 1 2 3 4 }
[ "Hello" % CHAR: \s , "Ada" % ] "" make .  ! "Hello Ada"

The final exemplar ({ } or "") chooses the result type. Prefer map, filter, or append when one of those directly expresses the operation.

Specialized arrays store elements as C numeric types in contiguous memory while supporting the sequence protocol:

USING: alien.c-types prettyprint sequences specialized-arrays ;
SPECIALIZED-ARRAY: double

double-array{ 1.0 2.0 3.0 } length .  ! 3

Hashtables and sets

Associative collections use the assocs protocol:

USING: assocs kernel prettyprint ;

"Ada" H{ { "Ada" 36 } { "Grace" 85 } } at .  ! 36
"missing" H{ { "Ada" 36 } } at .             ! f
"enabled" H{ { "enabled" f } } at* . .       ! Prints t, then f

H{ { "Ada" 36 } } clone
37 "Ada" pick set-at
"Ada" swap at .                              ! 37

at* returns a presence flag as well as a value, distinguishing a missing key from a key whose value is f. set-at takes a value, key, and assoc.

Sets also have a protocol, with useful operations on ordinary sequences:

USING: prettyprint sets ;

{ 1 2 2 3 } members .           ! { 1 2 3 }
2 { 1 2 3 } in? .               ! t
{ 1 2 } { 2 3 } union .         ! { 1 2 3 }
{ 1 2 } { 2 3 } intersect .     ! { 2 }
{ 1 2 } { 2 3 } diff .          ! { 1 }

For repeated membership checks, use a hash set rather than scanning a sequence:

USING: hash-sets prettyprint sets ;

2 HS{ 1 2 3 } in? .             ! t

Tuples and accessors

Tuples define classes with named slots. boa constructs a tuple from slot values in declaration order:

USING: accessors kernel prettyprint ;
IN: scratchpad

TUPLE: person name age ;
C: <person> person

"Ada" 36 <person>
dup name>> .                    ! "Ada"
37 >>age
age>> .                         ! 37

C: defines a constructor using boa. name>> reads a slot; >>age writes a slot and returns the tuple, allowing chained updates. You can also construct an instance with person new and set its slots explicitly. Names such as <person> conventionally denote constructors; the angle brackets are part of the word’s name.

Tuple literals use T{ ... }. Slots can also declare a class, an initial value, or the read-only attribute:

USING: accessors kernel math prettyprint ;
IN: scratchpad

T{ person { name "Grace" } { age 85 } } name>> .  ! "Grace"

TUPLE: counter { value integer initial: 0 } ;

counter new
[ 1 + ] change-value
value>> .                                         ! 1

Slot declarations constrain stored values. { name string read-only }, for example, declares a string slot that is initialized at construction and has no generated setter. change-value applies a quotation to the current slot value, stores the result, and returns the tuple.

Structs and C layouts

STRUCT: defines a record backed by a C memory layout. Every field declares a C type, and the usual slot accessors work on struct instances:

USING: accessors alien.c-types classes.struct kernel prettyprint ;
IN: scratchpad

STRUCT: c-point
    { x double }
    { y double } ;

3.0 4.0 c-point boa
dup x>> .                       ! 3.0
y>> .                           ! 4.0

PACKED-STRUCT: packet-header
    { kind uint8_t }
    { length uint32_t } ;

packet-header heap-size .       ! 5

boa initializes fields from stack values; c-point <struct> creates an instance with its declared initial field values. These constructors use garbage-collected storage. STRUCT: includes alignment padding according to the platform’s C layout rules. PACKED-STRUCT: removes padding between fields and at the end, for layouts that explicitly require packed storage. It does not choose byte order.

UNION-STRUCT: defines overlapping C fields that share the same storage. It serves a different purpose from UNION:, which groups Factor classes. Use tuples for ordinary Factor records and structs when you need C-compatible memory or an explicitly specified binary layout.

Generic words and classes

A generic word chooses a method based on the class of its topmost input. This example reuses person and <person> from “Tuples and accessors”:

USING: accessors kernel math math.parser prettyprint ;
IN: scratchpad

GENERIC: description ( obj -- string )

M: person description name>> ;
M: integer description number>string ;

"Ada" 36 <person> description .  ! "Ada"
42 description .                 ! "42"

A tuple subclass inherits its parent’s slots and can add its own. An overriding method can reuse the next less-specific method with call-next-method:

USING: accessors kernel prettyprint sequences ;
IN: scratchpad

TUPLE: employee < person role ;
C: <employee> employee

M: employee description
    [ call-next-method ] [ role>> ] bi " - " glue ;

"Ada" 36 "programmer" <employee> description .
! "Ada - programmer"

The constructor takes inherited slots first (name, age), then role. Here call-next-method receives the employee, calls the person method, and returns "Ada"; the override combines that with the employee’s role. It must appear inside a method definition and receives its inputs from the stack, just like an ordinary call.

M: defines a method. This is how protocols such as sequences and assocs provide common operations for many concrete types. Classes also have predicate words, and you can define narrower predicate classes or unions:

USING: kernel math prettyprint strings ;
IN: scratchpad

PREDICATE: positive-integer < integer 0 > ;
UNION: text-or-integer string integer ;

3 positive-integer? .           ! t
-3 positive-integer? .          ! f
"hello" text-or-integer? .      ! t

Mixin classes are open groups of classes: INSTANCE: adds a member, including after the mixin was defined. They are useful for protocols spanning unrelated types:

USING: prettyprint ;
IN: scratchpad

MIXIN: named
INSTANCE: person named

"Ada" 36 <person> named? .      ! t

Singleton classes each have one stateless instance, useful as distinct states or options. Unlike a plain symbol, each can have its own generic methods:

USING: prettyprint ;
IN: scratchpad

SINGLETONS: pending running finished ;
UNION: job-state pending running finished ;

pending job-state? .            ! t

UNION: accepts instances of any listed class. INTERSECTION: requires membership in all listed classes. For named numeric values, ENUMERATION: is available in classes.enumeration:

USING: classes.enumeration prettyprint ;
IN: scratchpad

ENUMERATION: priority low medium high ;

priority.low .                  ! 0
priority.high .                 ! 2

Symbols and dynamic variables

Lexical locals are scoped by source structure. namespaces provides variables scoped dynamically around a quotation:

USING: namespaces prettyprint ;
IN: scratchpad

SYMBOL: current-user

"Ada" current-user [
    current-user get .          ! "Ada"
] with-variable

Called words inside the quotation see the binding too. with-variable restores the previous binding on exit. set changes a binding in the current namespace; set-global sets a global binding. A symbol is itself a value, so symbols also work as distinct markers and hashtable keys.

Errors and cleanup

USING: continuations kernel prettyprint ;
IN: scratchpad

ERROR: invalid-age age ;

[ -1 invalid-age ] [ drop "handled" ] recover .  ! "handled"

[ "work" . ] [ "cleanup" . ] finally
! Prints "work", then "cleanup"

The exception handling form ERROR: defines an error class and a word that throws an instance. recover calls a handler with the thrown object. The data stack is restored to its state before the protected quotation, then the error is pushed. finally runs cleanup on either normal completion or an error.

Factor also exposes continuations, which capture execution state and can later resume it. They underpin error handling and cooperative threads; most everyday code uses those higher-level facilities directly.

Resource disposal

Ordinary objects are garbage collected. Resources such as open streams also need deterministic disposal. dispose releases a resource explicitly. with-disposal passes a resource to a quotation and disposes it when the quotation finishes or throws:

USING: destructors io io.encodings.utf8 io.files prettyprint ;

"Hello!\n" "disposal.txt" utf8 set-file-contents

"disposal.txt" utf8 <file-reader>
[ stream-readln . ] with-disposal  ! "Hello!"

This example creates disposal.txt in the current directory. The reader is closed after reading the line. For several resources, use with-destructors and register each one for cleanup:

USING: destructors io io.encodings.utf8 io.files prettyprint ;

[
    "disposal.txt" utf8 <file-reader> &dispose
    stream-readln .             ! "Hello!"
] with-destructors

Both registration words leave the resource on the stack so you can use it:

&dispose is for resources used within a scope. |dispose is useful when building a result that owns resources: if construction fails, clean up; if it succeeds, return the resources to the caller. For example:

USING: destructors io.encodings.utf8 io.files kernel ;
IN: scratchpad

: open-two-readers ( path1 path2 -- reader1 reader2 )
    [ [ utf8 <file-reader> |dispose ] bi@ ] with-destructors ;

"disposal.txt" "disposal.txt" open-two-readers
[ dispose ] bi@                 ! Caller closes both readers

If opening the second reader throws, the first reader is disposed. On success, both readers remain open and the caller owns their cleanup. Within each registration group, destructors run in reverse registration order. The with-file-reader and with-file-writer combinators shown below manage stream cleanup automatically.

Vocabularies

A vocabulary is a namespace and a unit of source organization. A vocabulary named examples.greeting conventionally lives in examples/greeting/greeting.factor under a vocabulary root:

USING: io ;
IN: examples.greeting

<PRIVATE

: greeting ( -- string ) "Hello, world!" ;

PRIVATE>

: greet ( -- ) greeting print ;

MAIN: greet

<PRIVATE ... PRIVATE> places helper definitions in the vocabulary’s private namespace. Import public definitions with USE: examples.greeting or include it in a USING: list. Run the entry point with ./factor -run=examples.greeting once its directory is in a vocabulary root, such as your installation’s work directory. Dots organize vocabulary names; importing a parent does not automatically import its children.

Source files need explicit imports. If a word is missing, its documentation shows which vocabulary provides it. The listener may offer to import a word automatically; include that vocabulary in USING: when saving the code. For ambiguous names, use a vocabulary prefix or select a word with FROM::

USING: math prettyprint ;

2 3 math:+ .                    ! 5

FROM: math => + ;
2 3 + .                         ! 5

Editing and reloading

Factor’s listener runs in a live image containing loaded definitions and objects. You can redefine a word and try it again in the same session. For code saved in a vocabulary, load it once with USE:, then reload changes after editing its source:

USING: vocabs.loader vocabs.refresh ;
USE: examples.greeting

"examples.greeting" reload      ! Reload this vocabulary
refresh-all                     ! Reload changed files in loaded vocabularies

This assumes you saved examples.greeting in a vocabulary root as above. The scaffold tool can create source, documentation, and test files for a new vocabulary.

Code as data, macros, and parsing words

Words are objects too. A backslash obtains a word without executing it:

USING: accessors math prettyprint words ;

\ + name>> .                    ! "+"

Quotations are built out of objects and words. Macros compute quotations that the compiler expands at call sites:

USING: kernel macros math prettyprint ;
IN: scratchpad

MACRO: add-constant ( n -- quot ) [ + ] curry ;

5 10 add-constant .             ! 15

Here 10 is the macro input, and the expansion adds it to the runtime value 5. Macro inputs must be known at compile time.

Syntax is extensible through parsing words, which execute while source is being read. :, TUPLE:, and literal openers are examples. Libraries can add their own syntax, such as R/ ... / for regular expressions.

Memoization

MEMO: defines a word whose results are cached by its inputs:

USING: kernel math memoize prettyprint ;
IN: scratchpad

MEMO: fibonacci ( n -- m )
    dup 1 <= [ ] [
        [ 1 - fibonacci ] [ 2 - fibonacci ] bi +
    ] if ;

10 fibonacci .                  ! 55

This is useful for pure computations. Cached mutable results are shared objects, so memoization needs care when callers mutate those results.

Files and formatted output

USING: formatting io io.encodings.utf8 io.files prettyprint ;

"Ada" 36 "%s is %d years old.\n" printf

"Hello, world!\n" "hello.txt" utf8 set-file-contents
"hello.txt" utf8 file-contents print

"hello.txt" utf8 [
    readln .
] with-file-reader

The file examples create hello.txt in the current directory. formatting provides printf for formatted output. with-file-reader binds the current input stream and closes it after the quotation finishes. with-file-writer does the same for output.

JSON, regular expressions, and HTTP

The json vocabulary converts between JSON text and Factor objects:

USING: assocs json kernel prettyprint ;

"{\"name\":\"Ada\",\"age\":36}" json>
"name" swap at .                ! "Ada"

H{ { "name" "Ada" } } >json .   ! "{\"name\":\"Ada\"}"

Regular expressions use their own literal syntax:

USING: prettyprint regexp ;

"12345" R/ [0-9]+/ matches? .   ! t
"hello" R/ [0-9]+/ matches? .   ! f

The HTTP client returns both a response object and the downloaded content:

USING: http.client kernel ;

"https://factorcode.org" http-get
nip                             ! Leave only the content

Dates and calendars

calendar provides timestamps and durations and computations on them.

USING: calendar prettyprint ;

now .                              ! Current local timestamp
10 months duration>minutes         ! Lots of minutes
today next-monday                  ! The next monday after today

Random

random selects random numbers or collection elements:

USING: prettyprint random ;

10 random .                        ! Random integer from 0 through 9
{ "red" "green" "blue" } random .  ! Random element

We also have various random distributions available.

Threads

Factor threads are cooperatively scheduled. yield lets another runnable thread execute, and blocking I/O integrates with the scheduler:

USING: kernel math prettyprint threads ;

42 [ 1 + . ] curry "worker" spawn drop
yield                           ! Worker prints 43

The worker starts with an empty data stack; curry explicitly carries the input into its quotation. The concurrency vocabularies provide additional tools such as mailboxes and promises.

Calling C

The foreign function interface declares C functions as Factor words. For example, this binds strlen from the C library:

USING: alien.c-types alien.syntax prettyprint ;
IN: scratchpad

LIBRARY: libc
FUNCTION: size_t strlen ( c-string str )

"hello" strlen .                ! 5

The c-string argument converts a Factor string for the C call. The FFI also supports structures, pointers, callbacks, and arrays. Unlike the managed objects used above, foreign allocations can require explicit lifetime management.

Testing and exploring

tools.test expresses expected stack results as an array:

USING: kernel math tools.test ;

{ 5 } [ 2 3 + ] unit-test
{ 25 } [ 5 dup * ] unit-test
[ 1 0 / ] must-fail

Tests for a vocabulary conventionally live alongside its source in a *-tests.factor file. After saving tests for examples.greeting, run them with "examples.greeting" test in the listener. This also runs tests in its child vocabularies.

The development environment also lets you inspect definitions, look up documentation, and time quotations:

USING: help kernel math see sequences tools.time ;

\ map help                      ! Open documentation for map
\ + describe                    ! Describe the object ``+``
\ square see                    ! Show the earlier definition
[ 1000000 [ ] times ] time      ! Time a quotation

The Factor handbook is the next stop for more detail. For a project walkthrough, the first-program tutorial covers creating a vocabulary, editing and reloading it, and extending it with tests. The vocabulary index covers the libraries, and the source distribution includes documentation and tests next to the code. Start with small words, follow their stack effects, and use combinators to make the flow of values clear.

Translating an existing piece of code from one language to another is often a good way to learn a language. A great long time ago, Arkady Rost posted to the mailing list a question about translating into Factor.

The original code that was being translated looked a bit like this:

for i in rangeA {
    for j in rangeB {
        foo(param, i, j);
    }
    bar();
}

Version 0

The first solution he found seemed a little bit involved, and he wondered if it could be improved:

rangeA rangeB param [ foo ] curry
[ swapd [ call ] 2curry each bar ] 2curry each

Using these values, for example:

• rangeA: { "a" "b" "c" }
• rangeB: { "1" "2" "3" }
• param: " "
• foo: append append write
• bar: "" print

We can get this output:

IN: scratchpad { "a" "b" "c" } { "1" "2" "3" } " "
               [ append append write ] curry
               [ swapd [ call ] 2curry each "" print ] 2curry each
1a 2a 3a 
1b 2b 3b 
1c 2c 3c

Version 1

I suggested something like this:

IN: scratchpad { "1" "2" "3" } [
                   { "a" "b" "c" } [ append ] with map
                   " " join print
               ] each
1a 2a 3a 
1b 2b 3b 
1c 2c 3c

But then Arkady pointed out that “the original task is more complicated that’s why I’ve used foo and bar in definition of the problem”.

Version 2

I suggested an approach that takes two sequences, applies foo to create an intermediate sequence and then applies bar to each element:

: my-func ( a b foo: ( x y -- z ) bar: ( z -- ) -- )
    [ [ with map ] 2curry ] dip compose each ; inline

The readability of this can be improved quite a bit by using “fry quotations”:

: my-func ( a b foo: ( x y -- z ) bar: ( z -- ) -- )
    '[ _ with map @ ] curry each ; inline

Either way, using it is pretty easy:

IN: scratchpad { "1" "2" "3" } { "a" "b" "c" }
               [ append ] [ " " join print ] my-func
1a 2a 3a 
1b 2b 3b 
1c 2c 3c

More Ideas

Of course, we could also use local variables for a direct translation:

rangeA [| i |
    rangeB [| j |
        param i j foo
    ] each
    bar
] each

Or some inline fancy currying to get:

param rangeA [ rangeB [ foo ] 2with each bar ] with each

Some other ideas included a suggestion to use the sequences.product vocabulary.

And then there might be the potential need for row polymorphism, meaning that quotations are allowed to use a generalized amount of values present on the stack when called.

It’s fun to be reminded that there are many ways to solve problems, that expressibility of your programming language is important, and that ultimately being able to perform desired computations is the focus here.

Something I’ve always respected about Python is their infamous batteries included philosophy:

The Python source distribution has long maintained the philosophy of “batteries included” – having a rich and versatile standard library which is immediately available, without making the user download separate packages. This gives the Python language a head start in many projects.

We have a similar approach in Factor, which leads us to our extensive vocabulary index. Sometimes we see new things that we could add. One recent example is Python’s shlex module for splitting a command into arguments in a shell-like manner, keeping quoted strings together.

I’m happy to say that we now have a shlex vocabulary in the main repository.

We can use it to read a command with a quoted argument:

USING: io prettyprint shlex ;

IN: scratchpad "echo -n 'hello world'" parse-shlex .
{ "echo" "-n" "hello world" }

This is also useful for options containing spaces or an empty value:

IN: scratchpad "--title='My Project' --label ''" parse-shlex .
{ "--title=My Project" "--label" "" }

For command-like lines in a configuration file, we might want to allow comments. The two flags to shlex-split enable comments and POSIX mode (although perhaps instead of word arguments, maybe these should be dynamic variables…):

IN: scratchpad "copy 'annual report.txt' archive # keep a backup" t t shlex-split .
{ "copy" "annual report.txt" "archive" }

IN: scratchpad "echo '#hello' # a comment" t t shlex-split .
{ "echo" "#hello" }

Going in the other direction, we can quote a filename as one argument for a POSIX shell:

IN: scratchpad "annual report.txt" shlex-quote print
'annual report.txt'

Or turn a whole sequence of arguments into a command string, ready to copy into a terminal:

IN: scratchpad { "cp" "annual report.txt" "backup copy.txt" } shlex-join print
cp 'annual report.txt' 'backup copy.txt'

Joining and splitting preserves the original arguments, including empty strings:

IN: scratchpad { "echo" "" "hello world" } shlex-join .
"echo '' 'hello world'"

IN: scratchpad { "echo" "" "hello world" } shlex-join parse-shlex .
{ "echo" "" "hello world" }

These words work with strings; they do not execute commands or expand variables and wildcards. The quoting is intended for POSIX-style shells.

This is now available in the development version of Factor!

Sometimes a few characters are enough to show the shape of some data. Edward Tufte wrote about this concept in sparkline theory and practice, sharing how very small graphs can provide incredible insight.

I wrote a small sparkline vocabulary for Factor that can turn numbers into little Unicode sparklines:

IN: scratchpad { 0 30 55 80 33 150 } sparkline .
"▁▂▃▄▂█"

Each number becomes one of eight block characters, from ▁ to █. The smallest value uses the lowest block and the largest uses the highest. The result is an ordinary string that we can print in a terminal, include in a table, or put next to a number in a report.

There is also a method that supports comma-separated numbers:

IN: scratchpad "0,30,55,80,33,150" sparkline .
"▁▂▃▄▂█"

Automatic scaling makes a chart fill the available height, but that means two very different sets of numbers can have the same shape:

IN: scratchpad { 10 20 30 } sparkline .
"▁▄█"

IN: scratchpad { 100 200 300 } sparkline .
"▁▄█"

To compare them, we can use sparkline-range with the same minimum and maximum:

IN: scratchpad { 10 20 30 } 0 300 sparkline-range .
"▁▁▁"

IN: scratchpad { 100 200 300 } 0 300 sparkline-range .
"▃▅█"

Or use the sparklines word to automatically normalize several inputs that use a shared range:

IN: scratchpad { { 10 20 30 } { 100 200 300 } } sparklines .
{ "▁▁▁" "▃▅█" }

Most of the implementation fits in one word:

SYMBOL: ticks
"▁▂▃▄▅▆▇█" ticks set-global

:: sparkline-range ( seq min max -- str )
    max min - ticks get length 1 - / [ 1 ] when-zero :> unit
    seq [ min max clamp min - unit /i ticks get nth ] "" map-as ;

We divide the range into intervals, clamp each value, and use integer division to select its character.

Tiny charts, tiny implementation!

Factor includes support for runtime code reloading, something that I have demoed previously using the included tetris game vocabulary. It is one of my favorite features, and part of the magic that makes for fun development using the Factor programming language.

I previously wrote about using Raylib in Factor, and we now support Raylib 6.0. Over the years, some demos have been shared by the Factor community, and I recalled one in particular that was shared on the Factor Discord server that I thought was particularly neat and highlights hot code reloading.

This is a live coding demo by and Null:

Really neat!

To give it a try yourself, build Factor from the latest development branch with Raylib 6.0 installed, then open the UI Listener and USE: raylib.live-coding, running your code in a with-live-coding block.

I’ve had fun watching the Bun project over the years. It has been neat in particular to see the efforts they have made to improve the performance of the JavaScript ecosystem. One of the main voices in that effort has been Jarred Sumner, who shared a small JavaScript program a bit more than a year ago:

Jarred Sumner on spawning cat

This benchmark repeatedly starts cat in batches of 100 processes. Each child reads the JavaScript source file, with standard input, output, and error redirected to /dev/null. There is no shell involved, and each batch waits for all of its children to exit before starting the next one.

I was curious how Factor would perform and decided to benchmark this on an Apple Silicon Mac, taking advantage of our recently stabilized Native ARM64 builds.

Unfortunately, the answer was not good!

It took 5.445 seconds to run a Factor equivalent benchmark, compared to the 1.936 seconds it took to run the 10,000 process JavaScript benchmark in Bun 1.4.2.

Yikes!

I was, however, able to make a few improvements to Factor’s process launcher:

• Inherit the existing environment directly when there are no overrides.
• Have the discarded standard streams share one temporary /dev/null descriptor.
• Check PATH candidates before calling posix_spawn, with a fallback to posix_spawnp.
• Use kqueue’s SIGCHLD notifications to wake the waiting thread when children exit.

I’m excited to say that we now run the “spawning cat” benchmark at about the same speed, on an Apple Silicon Mac, launching and waiting for 10,000 processes takes about two seconds:

This is available in the development version of Factor.

The Factor UI framework is a tree of gadgets — labels, buttons, tracks, tables, panes, and others. While we have pretty good documentation in some places, sometimes it can be a little sparse in others. For learning, nothing beats seeing how a gadget actually looks and what the code that generates it looks like.

Recently, I wrote a ui-demo vocabulary. At the moment, it contains one section per gadget type. You can click around and see some examples.

Each section right now is one small word. For example, here is the whole packs section shown above:

: <three-boxes> ( pack -- pack )
    { 3 3 } >>gap
        COLOR: DodgerBlue { 60 25 } <color-box> add-gadget
        COLOR: MediumSeaGreen { 90 25 } <color-box> add-gadget
        COLOR: chocolate1 { 45 25 } <color-box> add-gadget ;

: <packs-section> ( -- gadget )
    <section>
        "A pack lays its children out along one axis, each at its preferred size." <prose> add-gadget
        "Shelf, horizontal:" <heading> add-gadget
        <shelf> <three-boxes> add-gadget
        "Pile, vertical:" <heading> add-gadget
        <pile> <three-boxes> add-gadget
        "Filled pile, children stretched across:" <heading> add-gadget
        <filled-pile> <three-boxes> add-gadget
    <page> ;

You can check it out from within a Factor listener:

IN: scratchpad "ui-demo" run

Or try ./factor -run=ui-demo from the command line.