[ planet-factor ]

There was an interesting question on the Unix & Linux StackExchange asking how to remove subdomains or existing domains. I thought it would be fun to show a few different approaches to solving this using Factor.

Our first step should be to understand what is a subdomain:

A subdomain is a prefix added to a domain name to separate a section of your website. Site owners primarily use subdomains to manage extensive sections that require their own content hierarchy, such as online stores, blogs, job boards or support platforms.

Common Subdomains

If we’re curious about what common subdomains are, we can turn to the SecLists project – described as a “security tester’s companion” – which maintains a list of common 5,000 subdomains, 20,000 subdomains, and 110,000 subdomains that were generated in 2015 as well as a combined subdomains list that has some additional ones added.

You can download the top 5,000 common subdomains using memoization to cache the result:

MEMO: top-5000-subdomains ( -- subdomains )
    "https://raw.githubusercontent.com/danielmiessler/SecLists/refs/heads/master/Discovery/DNS/subdomains-top1million-5000.txt"
    cache-directory download-once-into utf8 file-lines ;

And then see what the “top 10” are:

IN: scratchpad top-5000-subdomains 10 head .
{
    "www"
    "mail"
    "ftp"
    "localhost"
    "webmail"
    "smtp"
    "webdisk"
    "pop"
    "cpanel"
    "whm"
}

You could remove “common subdomains” – adding a dot to make sure we only strip a full subdomain – by recursively trying to clean the hostname until it stops changing.

: remove-subdomains ( host -- host' )
    top-5000-subdomains [ "." append ] map '[ _ [ ?head ] any? ] loop ;

And try it out:

IN: scratchpad "www.mail.ftp.localhost.factorcode.org" remove-subdomains .
"factorcode.org"

That works pretty well, but it’s reliant on a scraped list of subdomains that might not be exhaustive, and could become stale over time as the tools and techniques that developers use change.

Observed Subdomains

Similarly, another technique we could use would be to use our own observations about domains, and if we observe a domain being used and then subsequently see a subdomain of it, we can ignore the subdomain.

First, we write a word to remove any item that is prefixed by another, sorting to make sure we see the prefix before the item prefixed by it:

: remove-prefixed ( seq -- seq' )
    sort V{ } clone [
        dup '[
            [ _ [ head? ] with none? ] _ push-when
        ] each
    ] keep ;

Second, we can remove the subdomains by using a kind of Schwartzian transform:

1. reverse the domain names
2. remove the ones that are prefixed by another
3. un-reverse the domain names

: remove-subdomains ( hosts -- hosts' )
    [ "." prepend reverse ] map remove-prefixed [ reverse rest ] map ;

And then see it work:

IN: scratchpad { "a.b.c" "b.c" "c.d.e" "e.f" } remove-subdomains .
V{ "b.c" "c.d.e" "e.f" }

Resolving Domains

And, finally, another technique might be to use the Domain Name System to find the rootiest domain name.

First, we use our dns vocabulary to check that a host resolves to an IP address:

: valid-domain? ( host -- ? )
    {
        [ dns-A-query message>a-names empty? not ]
        [ dns-AAAA-query message>aaaa-names empty? not ]
    } 1|| ;

And try it out:

IN: scratchpad "re.factorcode.org" valid-domain? .
t

IN: scratchpad "not-valid.factorcode.org" valid-domain? .
f

Second, we write a word to split a domain into chunks to be tested:

: split-domain ( host -- hosts )
    "." split dup length 1 [-] <iota> [ tail "." join ] with map ;

And try it out:

IN: scratchpad "a.b.c.com" split-domain .
{ "a.b.c.com" "b.c.com" "c.com" }

Third, we find the rootiest domain that is valid:

: remove-subdomains ( host -- host' )
    split-domain [ valid-domain? ] find-last nip ;

And try it out:

IN: scratchpad "a.b.c.d.factorcode.org" remove-subdomains .
"factorcode.org"

IN: scratchpad "sorting.cr.yp.to" remove-subdomains .
"cr.yp.to"

It’s fun to explore these kinds of problems!

Andrew Shitov recently published a book called “A Language A Day”, which is a collection of brief overviews to 21 programming languages – including Factor!

This book provides a concise overview of 21 different programming languages. Each language is introduced using the same approach: solving several programming problems to showcase its features and capabilities. Languages covered in the book: C++, Clojure, Crystal, D, Dart, Elixir, Factor, Go, Hack, Hy, Io, Julia, Kotlin, Lua, Mercury, Nim, OCaml, Raku, Rust, Scala, and TypeScript.

Each chapter covers the essentials of a different programming language. To make the content more consistent and comparable, I use the same structure for each language, focusing on the following mini projects:

1. Creating a ‘Hello, World!’ program.
2. Implementing a Factorial function using recursion or a functional-style approach.
3. Creating a polymorphic array of objects (a ‘zoo’ of cats and dogs) and calling methods on them.
4. Implementing the Sleep Sort algorithm—while impractical for real-word use, it’s a playful demonstration of language’s concurrency capabilities.

Each language description follows—where applicable—this pattern:

1. Installing a command-line compiler and running a program.
2. Creating and using variables.
3. Defining and using functions.
4. Exploring object-oriented features.
5. Handling exception.
6. Introducing basic concurrency and parallelism.

You can find all the code examples in this book on GitHub: https://github.com/ash/a-language-a-day.

You can buy it on Amazon or LeanPub as an electronic or Kindle edition, or as a paper hardcover or paperback version. More information with the links to the shops.

Check it out!

Factor has programmable syntax, a feature that allows for concise source code, reducing repetition and allowing the programmer to express forms and intent with minimal tokens. As an example of this, today I want to discuss constants.

You can define a word with a constant value, using syntax like this:

CONSTANT: three 3

Someone on our Factor Discord server asked if it was possible to define multiple constants in one syntax expression, to avoid the line noise of defining them one-by-one.

So, instead of these four definitions:

CONSTANT: foo 1
CONSTANT: bar $[ 2 sqrt ]
CONSTANT: baz $ bar
CONSTANT: qux \ foo

We could instead make this syntax:

SYNTAX: CONSTANTS:
    ";" [
        create-word-in
        [ reset-generic ]
        [ scan-object define-constant ] bi
    ] each-token ;

Breaking that down into steps:

1. SYNTAX: indicates we’re defining new syntax
2. CONSTANTS: is the name of our new syntax word
3. ";" defines the terminator that will end our constant definitions
4. each-token will process each token until it hits the terminator

For each constant definition, it performs these steps:

• create-word-in creates a new word in the current vocabulary
• reset-generic clears any generic word properties
• scan-object reads and parses the next value
• define-constant makes it a constant with the parsed value

And now this expression works, reducing the visual noise in our source code:

CONSTANTS:
   foo 1
   bar $[ 2 sqrt ]
   baz $ bar
   qux \ foo
;

As an aside, the different syntaxes used above are:

• 1 is just a token parsed as a number literal
• $[ ... ] evaluates the code inside at parse time
• $ gets the value of another constant
• \ gets the word object itself rather than its value

Factor’s syntax parsing words allow a great deal of flexibility in making custom DSL-style syntax forms work nicely to reduce repetition, and generate code with less effort.

I’m not sure if this is worth adding to the standard library or not, but it’s neat!

Over a decade ago, Chris Kempson created the Base16 theme framework for creating color palettes of 16 colors that can be used to provide theming of user interfaces. These have been commonly supported by many text editors, with some developers gravitating toward setting their favorite theme in every user interface that supports it.

A few years ago, this framework and the many themes that became popular in it were forked into the Tinted Theming project described in a post called Base16 Project Lives On. You can view their gallery of Base16 themes which gives a good sense of the variety and utility of these color schemes having commonly recognizable names such as dracula, mocha, solarized, and more.

I was reminded of this recently in a discussion around a recent contribution to change the scrollbar and button implementations to not use images, but to draw the scrollbars using the colors configured in the user’s theme.

Since 2021, the ui.theme.base16 vocabulary has allowed theming the Factor user interface by choosing a base16-theme-name and setting base16-mode. We have just improved our support for Base16 theme support by adding all the current styles from the Tinted Theming schemes list.

So, now you can try solarized-dark:

IN: scratchpad "solarized-dark" base16-theme-name set-global

IN: scratchpad base16-mode

Or perhaps greenscreen:

IN: scratchpad "greenscreen" base16-theme-name set-global

IN: scratchpad base16-mode

Or any of the other 270 named color schemes now available!

Enjoy!

One of the interesting aspects of a concatenative language like Factor is that blocks of logic can be easily extracted and easily reused since they apply logic to objects on the stack.

For example, if this was a word that operated on stack values:

: do-things ( a b -- c d )
    [ sqrt * ] [ swap sqrt + ] 2bi ;

One change we could easily make is to extract and name the two pieces of logic:

: calc-c ( a b -- c ) sqrt * ;
: calc-d ( a b -- d ) swap sqrt + ;

: do-things ( a b -- c d )
    [ calc-c ] [ calc-d ] 2bi ;

We could also convert it to operate on local variables:

:: do-things ( a b -- c d )
    a b sqrt * a sqrt b + ;

And extract those same two pieces of logic:

:: calc-c ( a b -- c ) a b sqrt * ;
:: calc-d ( a b -- d ) a sqrt b + ;

:: do-things ( a b -- c d )
    a b calc-c a b calc-d ;

But, notice that we have to specify that the local variable a and b have to be put back on the stack before we can call our extracted words that make the computations.

Hypothetical Syntax

Today, someone on the Factor Discord server asked about this very issue, wanting to have extractable pieces of logic that would effectively be operating on nested local variables, wherever they are used. Inspired by the goal of don’t repeat yourself and the convenience of extracting logic that operates on the data stack.

Specifically, they wanted to be able to take blocks of logic that operate on named variables, and extract them in a similar manner to the logic blocks that operate on the stack – offering this hypothetical syntax as the goal:

EMIT: calc-c ( a b -- c ) a b sqrt * ;
EMIT: calc-d ( a b -- d ) a sqrt b + ;

:: do-things ( a b -- c d )
    calc-c calc-d ;

Let’s try and build real syntax that allows this hypothetical syntax to work.

Building the Syntax

First, we make a tuple to hold a lazy variable binding:

TUPLE: lazy token ;
C: <lazy> lazy

Then, we need a way to generate temporary syntax words in a similar manner to temporary words:

: define-temp-syntax ( quot -- word )
    [ gensym dup ] dip define-syntax ;

We create temporary syntax words to convert each named references to lazy variables:

: make-lazy-vars ( names -- words )
    [ dup '[ _ <lazy> suffix! ] define-temp-syntax ] H{ } map>assoc ;

Given a quotation that we have parsed in an emit description, we can build a word to replace all these lazy variables by looking them up in the current vocabulary manifest:

: replace-lazy-vars ( quot -- quot' )
    [ dup lazy? [ token>> parse-word ] when ] deep-map ;

And, finally, create our emit syntax word that parses a definition, making lazy variables that are then replaced when the emit word is called in the nested scope:

SYNTAX: EMIT:
    scan-new-word scan-effect in>>
    [ make-lazy-vars ] with-compilation-unit
    [ parse-definition ] with-words
    '[ _ replace-lazy-vars append! ] define-syntax ;

Using the Syntax

Now, let’s go back to our original example:

EMIT: calc-c ( a b -- c ) a b sqrt * ;
EMIT: calc-d ( a b -- d ) a sqrt b + ;

:: do-things ( a b -- c d )
    calc-c calc-d ;

Does it work?

IN: scratchpad 1 2 do-things

--- Data stack:
1.4142135623730951
3.0

Yep! That’s kind of a neat thing to build.

I have added this syntax in the locals.lazy vocabulary, if you want to try it out.

I’m not sure how useful it will be in general, but it is always fun to build something new with Factor!

Battlesnake is “a competitive game where your code is the controller”. In particular, in answering the question “What is Battlesnake?”, the documentation says:

In this game, each Battlesnake is controlled in real-time by a live web server, responding to the Battlesnake API. It navigates the game board based on your algorithm, trying to find food, avoid other Battlesnakes, and survive as long as possible. Battlesnakes can be built using any tech stack you’d like, and we encourage you to step outside of your comfort zone.

It is also a very neat set of episodes of “Coding Badly” from almost two years ago that talks about building battlesnakes using Factor. In particular, they use a live-coding style to explore the development environment, build web servers using the furnace web framework, and learn how to use and deploy their program!

I did not know about these videos until today, but I thought it makes a nice series to share with the world. I love it when people build things using Factor and am always glad to hear about it!

More information is also available on the @BattlesnakeOfficial GitHub organization, as well as an archive of the Coding Badly implementation and a different Factor battlesnake library by another contributor.

Episode 1

Episode 2

Episode 3

“Life can only be understood backwards; but it must be lived forwards.” — Kierkegaard

I’m very pleased to announce the release of Factor 0.100!

Source code: 0.100

This release is brought to you with over 1400 commits by the following individuals:

Aditya Aryaman Das, Alex null Maestas, Alexander Ilin, Andy Kluger, Bhargav Shirin Nalamati, Charlie Weismann, Dave Carlton, David Enders, Doug Coleman, Evgenii Petrov, Giftpflanze, Ikko Eltociear Ashimine, J. Ryan Stinnett, Jean-Marc Lugrin, John Benediktsson, Keldan Chapman, Limnanthes Serafini, Marc Michael, Michael Raitzam, Michael Thies, Pragya Pant, Raghu Ranganathan, Rebecca Kelly, Rudi Grinberg, Sandesh Pyakurel, Sebastian Strobl, Shruti Sen, Surav Shrestha, Val Packett, @Capital-EX, @Smoothieewastaken, @TheWitheredStriker, @TryAngle, @chunes3, @inivekin, @nomennescio, @olus2000.

Besides some bug fixes and library improvements, I want to highlight the following changes:

• Upgraded to Unicode 15.1
• Fix some xmlns that were accidentally changed to https
• Improved the printing of shortest decimal representation of floating-point numbers
• Some early support for ARM64 in the non-optimizing compiler, more to do for full support
• Automatic light/dark theme detection works on Microsoft Windows
• Support for compressed images, useful when reducing file size is important

Some possible backwards compatibility issues:

• ui: focusable-child* now returns f to indicate parent should be focused
• peg: change to compile-time PEG: and PARTIAL-PEG: forms, not delay to first invocation
• system: renamed macosx to macos
• math.trig: moved deg>rad and rad>deg to math.functions vocabulary
• math.functions: fix divisor? to support mixed numbers (floats and integers)
• math.functions.integer-logs: moved integer-log10 and integer-log2 to math.functions vocabulary
• ranges: fixed exclusive range to be more correct for non-integer types
• http.client: moved some download words to http.download vocabulary
• rosetta-code: moved solutions to the factor-rosetta-code git repository
• json: read-json returns a single object, use read-jsons to read multiple
• base32: now contains all of the words from the base32-crockford and base32hex vocabularies

I would also like to bring particular recognition to Raghu Ranganathan, also known as @razetime, who was an incredible developer with an incredibly good attitude and contributing member to many technical communities including code golfing and various programming languages including Factor. We are very sad that he passed away a couple of months ago and would like to have this moment dedicated in his memory.

What is Factor

Factor is a concatenative, stack-based programming language with high-level features including dynamic types, extensible syntax, macros, and garbage collection. On a practical side, Factor has a full-featured library, supports many different platforms, and has been extensively documented.

The implementation is fully compiled for performance, while still supporting interactive development. Factor applications are portable between all common platforms. Factor can deploy stand-alone applications on all platforms. Full source code for the Factor project is available under a BSD license.

New libraries:

• base45: adding support for Base45 Data Encoding (RFC 9285)
• bend: features from the Bend programming language
• checksums.khash: implementation of k-hash
• command-line.parser: experimental command-line argument parser
• containers: experimental high-level container words across sequences, sets, dlists, and assocs.
• drunken-bishop: implements OpenSSH Drunken Bishop algorithm
• editors.chime: support Chime editor
• editors.focus: support Focus Editor
• editors.notepadnext: support NotepadNext editor
• gilded-rose: adds the Gilded Rose Refactoring Kata
• golden-section: recovers an old UI demo that uses processing for rendering
• hex-strings: words for changing bytes to hex strings and back
• http.download: collection of various download words
• io.streams.tee: adding a “tee’ing” stream utility
• images.viewer.scaling: adding a “scaling image” gadget
• kaggle: wrapper for the Kaggle API
• lazy: renamed from promises
• leb128: implements LEB128 or Little Endian Base 128 encoding
• math.statistics.running: implement “running statistics” and “running regression” tuples
• math.vectors.ranges: more efficient vector operations on range objects
• models.combinators: some extensions to models
• models.combinators.templates: a functor for generating model objects
• persistency: simple wrapper for db.tuples, possibly a little redundant
• project-euler.061: solution for Cyclical Figurate Numbers
• project-euler.098: solution for Anagramic Squares
• quoted-printable.rfc2047: implementation of RFC2047 strings
• random.xorshift: simple Xorshift random number generator
• recipes: recovers an old UI demo of a recipe browser
• scryfall: extensive library for working with MtG cards on Scryfall
• sequences.prefixed: virtual “prefixed” sequence
• sequences.suffixed: virtual “suffixed” sequence
• stomp: client library for the STOMP protocol
• stomp.cli: command-line interface for STOMP
• sudokus: a graphical Sudoku game and solver
• tools.image: common code for working with Factor image files
• tools.image.analyzer: renamed tools.image-analyzer
• tools.image.compressor: working with compressed Factor image files
• tools.image.uncompressor: working with compressed Factor image files
• ui.gadgets.alert: UI elements to prompt user and popup alerts
• ui.gadgets.cartesian: three-axis coordinate system
• ui.gadgets.comboboxes: UI elements for a combo box
• ui.gadgets.controls: recovers some old UI support code
• ui.gadgets.layout: recovers some old UI support code
• ui.gadgets.slate: recovers some old UI support code
• msgpack.rpc: implementation of MsgPack-RPC

Improved libraries:

• alien.c-types: added ssize_t
• alien.data: adding stream-read-c-ptr and read-c-ptr
• assocs: merged some useful words from assocs.extras, removed with-assoc
• assocs.extras: demoted set-of from assocs
• base85: added Ascii85, Adobe85, and Z85 variants
• calendar: added sunrise, sunset, and solar-noon
• classes.algebra: fix class<= for anonymous-predicate
• cocoa.statusbar: some cleanup, additional words
• codebase-analyzer: handle owner and version files, print clickable links
• colors.contrast: adding contrast-text-color to select white/black text on dark/light backgrounds
• combinators.extras: adding sequence-case, fix 3tri*
• command-line: adding command-line-options for easy options parsing
• contributors: added contributors. and make the changelog respect .mailmap file
• crontab: adding support for “~” randomization
• curses: added nodelay
• db.tuples: adding LIKE" column" syntax
• debugger: fixed error description for bad-escape
• discord: some improvements to the Discord bot library
• dns: adding some more DNS types
• formatting: split out a format-directive EBNF word
• game-of-life: small performance improvements
• geo-ip: fix database download link
• github: implement more of the GitHub REST API
• globs: added rglob for recursive glob
• gpu.shaders: change two tuples used in errors to be error classes
• html.templates.chloe: improve <t:meta> tag to be able to specific any meta attributes
• http.server.responses: adding all the real and joke response codes
• http2.hpack.huffman: simplify implementation using nths
• interpolate: adding I" interpolated string syntax, allow format directives to be used
• inverse: adding under
• io.directories: added ?move-file
• io.directories.windows: fixed move-file to properly replace existing files
• io.files: added if-file-exists combinators and (file-writer-secure)
• io.files.temp.macosx: allow default-cache-directory to work in MacPorts environment
• io.files.unique: added safe-replace-file and safe-modify-file
• io.pipes: added <connected-pair>
• io.streams.256color: formatting ANSI string tables properly
• io.streams.ansi: formatting ANSI string tables properly
• io.streams.escape-codes: added strip-ansi-escapes and format-ansi-tables
• json: renamed read-json to read-jsons, added read-json that reads a single object
• ldcache: fix to work on platforms that don’t have /etc/ld.so.cache (such as NixOS)
• libc: added some setlocale support
• libclang: working on header parsing to factor source
• machine-learning.functions: added gelu, stable-softmax, and stable-log-softmax
• mason.release.archive: build tar file without xattrs
• math.bits: adding a binary-bits tuple
• math.combinatorics: fix <k-permutations> for k=0
• math.distances: added squared-euclidian-distance and normalized-squared-euclidian-distance, aliases for taxicab-distance and chessboard-distance
• math.extras: added weighted-randoms-as
• math.functions: moved math.trig words in (deg>rad and rad>deg), math.functions.integer-logs, and integer-sqrt, added fma (fused-multiply-add)
• math.matrices: fixed stack effect for <matrix-by>
• math.parser: implement Dragonbox algorithm, added >digits and digits>
• mirrors: changed to display all assocs like hashtables, all sets like hash-sets.
• modern.html: added some XML traversal works
• msgpack: added ?read-msgpack and read-msgpacks
• oauth2: simplify using the json.http vocabulary
• openai: adding <cheapest-chat-completion> for ease-of-use with “gpt-4o-mini”, add timestamps to the list-models api
• opengl: adding OpenGL 4+ extensions and demos, better support for multi-texture scaling
• peg.ebnf: reset ebnf words properly
• pong: fix issue when pressing N for new game a lot
• prettyprint.config: adding qualified-names? to allow word names to be prettyprinted as fully-qualified
• quotations: added compose-all
• random: rename random-bits* to random-bits-exact, rename the *-random-float distributions to *-random, add *-distribution types, added more of them, defined a base-random that allows a better not-a-random-generator error to be produced in some cases
• random.c: fixed to guarantee rand() is used to generate full range of 32-bit numbers
• ranges: support some set operations more efficiently
• raylib: added extensive documentation
• reddit: fix domain-stats
• sequences.extras: adding count=, faster longest-subseq
• sequences.generalizations: adding lastn, ?lastn, set-lastn
• sequences.parser: rename get+increment to consume, and change next to return the next element
• sequences.product: making product-each, product-map, and product-find significantly faster
• serialize: added deep-clone
• shuffle: added dupdd
• splitting: break on more line separators \r\n\v\f\x1c\x1d\x1e\x85\u002028\u002029
• system: renamed macosx to macos
• syntax: adding VOCAB: syntax
• toml: fix issue with sub-tables being defined first
• tools.completion: added completion for VOCAB: syntax
• tools.disassembler.capstone: support Capstone 4 and Capstone 5
• ui: adding close-all-windows
• ui.backend.gtk2: check for non-empty DISPLAY os-var for graphical capability
• ui.backend.windows: support horizontal mouse wheel, detect light/dark theme on startup
• ui.backend.x11: check for non-empty DISPLAY os-var for graphical capability
• ui.commands: adding update-command-map
• ui.gadgets: fix f focusable-child busy loop
• ui.tools.listener.history: saving and restoring UI listener history to a ~/.factor-history file
• units.imperial: add area units, change “fingerbreadth” unit
• units.si: adding km^2 and more aliases
• vocabs.loader: make vocab-exists? no longer throw bad-vocab-name
• vocabs.refresh.monitor: fix warnings for non-vocab-like files
• xml: fix xml namespace
• webapps.mason: improve build farm dashboard
• websites.concatenative: allow use of <t:meta> in child templates
• words: adding uninterned-word predicate, undefined-word error class
• xmode.catalog: adding qdoc and sparql modes
• zeromq: change zmq-error to be an error class
• zoneinfo: updated to tzdata version 2024b

VM Improvements:

• Improved ARM64 bootstrap assembly to allow small forms to execute successfully and natively in the non-optimizing compiler. This continues to be a work-in-progress to fully support ARM64.
• Tentative support for compressed images, allowing Factor images to be as much as 8x smaller in size with run-time uncompression overhead.
• Improved console I/O on windows to work in environments such as cygwin and GitHub Actions.