[ planet-factor ]

John Benediktsson: Removing Subdomains

There was an interesting question on the Unix & Linux StackExchange asking how to remove subdomains or existing domains. I thought it would be fun to show a few different approaches to solving this using Factor.

Our first step should be to understand what is a subdomain:

A subdomain is a prefix added to a domain name to separate a section of your website. Site owners primarily use subdomains to manage extensive sections that require their own content hierarchy, such as online stores, blogs, job boards or support platforms.

Common Subdomains

If we’re curious about what common subdomains are, we can turn to the SecLists project – described as a “security tester’s companion” – which maintains a list of common 5,000 subdomains, 20,000 subdomains, and 110,000 subdomains that were generated in 2015 as well as a combined subdomains list that has some additional ones added.

You can download the top 5,000 common subdomains using memoization to cache the result:

MEMO: top-5000-subdomains ( -- subdomains )
    "https://raw.githubusercontent.com/danielmiessler/SecLists/refs/heads/master/Discovery/DNS/subdomains-top1million-5000.txt"
    cache-directory download-once-into utf8 file-lines ;

And then see what the “top 10” are:

IN: scratchpad top-5000-subdomains 10 head .
{
    "www"
    "mail"
    "ftp"
    "localhost"
    "webmail"
    "smtp"
    "webdisk"
    "pop"
    "cpanel"
    "whm"
}

You could remove “common subdomains” – adding a dot to make sure we only strip a full subdomain – by recursively trying to clean the hostname until it stops changing.

: remove-subdomains ( host -- host' )
    top-5000-subdomains [ "." append ] map '[ _ [ ?head ] any? ] loop ;

And try it out:

IN: scratchpad "www.mail.ftp.localhost.factorcode.org" remove-subdomains .
"factorcode.org"

That works pretty well, but it’s reliant on a scraped list of subdomains that might not be exhaustive, and could become stale over time as the tools and techniques that developers use change.

Observed Subdomains

Similarly, another technique we could use would be to use our own observations about domains, and if we observe a domain being used and then subsequently see a subdomain of it, we can ignore the subdomain.

First, we write a word to remove any item that is prefixed by another, sorting to make sure we see the prefix before the item prefixed by it:

: remove-prefixed ( seq -- seq' )
    sort V{ } clone [
        dup '[
            [ _ [ head? ] with none? ] _ push-when
        ] each
    ] keep ;

Second, we can remove the subdomains by using a kind of Schwartzian transform:

  1. reverse the domain names
  2. remove the ones that are prefixed by another
  3. un-reverse the domain names
: remove-subdomains ( hosts -- hosts' )
    [ "." prepend reverse ] map remove-prefixed [ reverse rest ] map ;

And then see it work:

IN: scratchpad { "a.b.c" "b.c" "c.d.e" "e.f" } remove-subdomains .
V{ "b.c" "c.d.e" "e.f" }

Resolving Domains

And, finally, another technique might be to use the Domain Name System to find the rootiest domain name.

First, we use our dns vocabulary to check that a host resolves to an IP address:

: valid-domain? ( host -- ? )
    {
        [ dns-A-query message>a-names empty? not ]
        [ dns-AAAA-query message>aaaa-names empty? not ]
    } 1|| ;

And try it out:

IN: scratchpad "re.factorcode.org" valid-domain? .
t

IN: scratchpad "not-valid.factorcode.org" valid-domain? .
f

Second, we write a word to split a domain into chunks to be tested:

: split-domain ( host -- hosts )
    "." split dup length 1 [-] <iota> [ tail "." join ] with map ;

And try it out:

IN: scratchpad "a.b.c.com" split-domain .
{ "a.b.c.com" "b.c.com" "c.com" }

Third, we find the rootiest domain that is valid:

: remove-subdomains ( host -- host' )
    split-domain [ valid-domain? ] find-last nip ;

And try it out:

IN: scratchpad "a.b.c.d.factorcode.org" remove-subdomains .
"factorcode.org"

IN: scratchpad "sorting.cr.yp.to" remove-subdomains .
"cr.yp.to"

It’s fun to explore these kinds of problems!

Tue, 5 Nov 2024 15:00:00

John Benediktsson: A Language A Day

Andrew Shitov recently published a book called “A Language A Day”, which is a collection of brief overviews to 21 programming languages – including Factor!

This book provides a concise overview of 21 different programming languages. Each language is introduced using the same approach: solving several programming problems to showcase its features and capabilities. Languages covered in the book: C++, Clojure, Crystal, D, Dart, Elixir, Factor, Go, Hack, Hy, Io, Julia, Kotlin, Lua, Mercury, Nim, OCaml, Raku, Rust, Scala, and TypeScript.

Each chapter covers the essentials of a different programming language. To make the content more consistent and comparable, I use the same structure for each language, focusing on the following mini projects:

  1. Creating a ‘Hello, World!’ program.
  2. Implementing a Factorial function using recursion or a functional-style approach.
  3. Creating a polymorphic array of objects (a ‘zoo’ of cats and dogs) and calling methods on them.
  4. Implementing the Sleep Sort algorithm—while impractical for real-word use, it’s a playful demonstration of language’s concurrency capabilities.

Each language description follows—where applicable—this pattern:

  1. Installing a command-line compiler and running a program.
  2. Creating and using variables.
  3. Defining and using functions.
  4. Exploring object-oriented features.
  5. Handling exception.
  6. Introducing basic concurrency and parallelism.

You can find all the code examples in this book on GitHub: https://github.com/ash/a-language-a-day.

You can buy it on Amazon or LeanPub as an electronic or Kindle edition, or as a paper hardcover or paperback version. More information with the links to the shops.

Check it out!

Mon, 4 Nov 2024 15:00:00

John Benediktsson: Constants

Factor has programmable syntax, a feature that allows for concise source code, reducing repetition and allowing the programmer to express forms and intent with minimal tokens. As an example of this, today I want to discuss constants.

You can define a word with a constant value, using syntax like this:

CONSTANT: three 3

Someone on our Factor Discord server asked if it was possible to define multiple constants in one syntax expression, to avoid the line noise of defining them one-by-one.

So, instead of these four definitions:

CONSTANT: foo 1
CONSTANT: bar $[ 2 sqrt ]
CONSTANT: baz $ bar
CONSTANT: qux \ foo

We could instead make this syntax:

SYNTAX: CONSTANTS:
    ";" [
        create-word-in
        [ reset-generic ]
        [ scan-object define-constant ] bi
    ] each-token ;

Breaking that down into steps:

  1. SYNTAX: indicates we’re defining new syntax
  2. CONSTANTS: is the name of our new syntax word
  3. ";" defines the terminator that will end our constant definitions
  4. each-token will process each token until it hits the terminator

For each constant definition, it performs these steps:

  • create-word-in creates a new word in the current vocabulary
  • reset-generic clears any generic word properties
  • scan-object reads and parses the next value
  • define-constant makes it a constant with the parsed value

And now this expression works, reducing the visual noise in our source code:

CONSTANTS:
   foo 1
   bar $[ 2 sqrt ]
   baz $ bar
   qux \ foo
;

As an aside, the different syntaxes used above are:

  • 1 is just a token parsed as a number literal
  • $[ ... ] evaluates the code inside at parse time
  • $ gets the value of another constant
  • \ gets the word object itself rather than its value

Factor’s syntax parsing words allow a great deal of flexibility in making custom DSL-style syntax forms work nicely to reduce repetition, and generate code with less effort.

I’m not sure if this is worth adding to the standard library or not, but it’s neat!

Tue, 29 Oct 2024 15:00:00

John Benediktsson: Base16 Themes

Over a decade ago, Chris Kempson created the Base16 theme framework for creating color palettes of 16 colors that can be used to provide theming of user interfaces. These have been commonly supported by many text editors, with some developers gravitating toward setting their favorite theme in every user interface that supports it.

A few years ago, this framework and the many themes that became popular in it were forked into the Tinted Theming project described in a post called Base16 Project Lives On. You can view their gallery of Base16 themes which gives a good sense of the variety and utility of these color schemes having commonly recognizable names such as dracula, mocha, solarized, and more.

I was reminded of this recently in a discussion around a recent contribution to change the scrollbar and button implementations to not use images, but to draw the scrollbars using the colors configured in the user’s theme.

Since 2021, the ui.theme.base16 vocabulary has allowed theming the Factor user interface by choosing a base16-theme-name and setting base16-mode. We have just improved our support for Base16 theme support by adding all the current styles from the Tinted Theming schemes list.

So, now you can try solarized-dark:

IN: scratchpad "solarized-dark" base16-theme-name set-global

IN: scratchpad base16-mode

Or perhaps greenscreen:

IN: scratchpad "greenscreen" base16-theme-name set-global

IN: scratchpad base16-mode

Or any of the other 270 named color schemes now available!

Enjoy!

Thu, 17 Oct 2024 15:00:00

John Benediktsson: Emit

One of the interesting aspects of a concatenative language like Factor is that blocks of logic can be easily extracted and easily reused since they apply logic to objects on the stack.

For example, if this was a word that operated on stack values:

: do-things ( a b -- c d )
    [ sqrt * ] [ swap sqrt + ] 2bi ;

One change we could easily make is to extract and name the two pieces of logic:

: calc-c ( a b -- c ) sqrt * ;
: calc-d ( a b -- d ) swap sqrt + ;

: do-things ( a b -- c d )
    [ calc-c ] [ calc-d ] 2bi ;

We could also convert it to operate on local variables:

:: do-things ( a b -- c d )
    a b sqrt * a sqrt b + ;

And extract those same two pieces of logic:

:: calc-c ( a b -- c ) a b sqrt * ;
:: calc-d ( a b -- d ) a sqrt b + ;

:: do-things ( a b -- c d )
    a b calc-c a b calc-d ;

But, notice that we have to specify that the local variable a and b have to be put back on the stack before we can call our extracted words that make the computations.

Hypothetical Syntax

Today, someone on the Factor Discord server asked about this very issue, wanting to have extractable pieces of logic that would effectively be operating on nested local variables, wherever they are used. Inspired by the goal of don’t repeat yourself and the convenience of extracting logic that operates on the data stack.

Specifically, they wanted to be able to take blocks of logic that operate on named variables, and extract them in a similar manner to the logic blocks that operate on the stack – offering this hypothetical syntax as the goal:

EMIT: calc-c ( a b -- c ) a b sqrt * ;
EMIT: calc-d ( a b -- d ) a sqrt b + ;

:: do-things ( a b -- c d )
    calc-c calc-d ;

Let’s try and build real syntax that allows this hypothetical syntax to work.

Building the Syntax

First, we make a tuple to hold a lazy variable binding:

TUPLE: lazy token ;
C: <lazy> lazy

Then, we need a way to generate temporary syntax words in a similar manner to temporary words:

: define-temp-syntax ( quot -- word )
    [ gensym dup ] dip define-syntax ;

We create temporary syntax words to convert each named references to lazy variables:

: make-lazy-vars ( names -- words )
    [ dup '[ _ <lazy> suffix! ] define-temp-syntax ] H{ } map>assoc ;

Given a quotation that we have parsed in an emit description, we can build a word to replace all these lazy variables by looking them up in the current vocabulary manifest:

: replace-lazy-vars ( quot -- quot' )
    [ dup lazy? [ token>> parse-word ] when ] deep-map ;

And, finally, create our emit syntax word that parses a definition, making lazy variables that are then replaced when the emit word is called in the nested scope:

SYNTAX: EMIT:
    scan-new-word scan-effect in>>
    [ make-lazy-vars ] with-compilation-unit
    [ parse-definition ] with-words
    '[ _ replace-lazy-vars append! ] define-syntax ;

Using the Syntax

Now, let’s go back to our original example:

EMIT: calc-c ( a b -- c ) a b sqrt * ;
EMIT: calc-d ( a b -- d ) a sqrt b + ;

:: do-things ( a b -- c d )
    calc-c calc-d ;

Does it work?

IN: scratchpad 1 2 do-things

--- Data stack:
1.4142135623730951
3.0

Yep! That’s kind of a neat thing to build.

I have added this syntax in the locals.lazy vocabulary, if you want to try it out.

I’m not sure how useful it will be in general, but it is always fun to build something new with Factor!

Sun, 13 Oct 2024 15:00:00

Joe Groff: Type-erased generic functions for C: A modest non-proposal

Earlier this year, I read Martin Uecker's proposal N3212 to add parametric polymorphism to C. It's easy to scoff at the idea of adding generic programming to plain C, when C++ already has templates, a…

Mon, 30 Sep 2024 00:40:00

John Benediktsson: Battlesnake

Battlesnake is “a competitive game where your code is the controller”. In particular, in answering the question “What is Battlesnake?”, the documentation says:

In this game, each Battlesnake is controlled in real-time by a live web server, responding to the Battlesnake API. It navigates the game board based on your algorithm, trying to find food, avoid other Battlesnakes, and survive as long as possible. Battlesnakes can be built using any tech stack you’d like, and we encourage you to step outside of your comfort zone.

It is also a very neat set of episodes of “Coding Badly” from almost two years ago that talks about building battlesnakes using Factor. In particular, they use a live-coding style to explore the development environment, build web servers using the furnace web framework, and learn how to use and deploy their program!

I did not know about these videos until today, but I thought it makes a nice series to share with the world. I love it when people build things using Factor and am always glad to hear about it!

More information is also available on the @BattlesnakeOfficial GitHub organization, as well as an archive of the Coding Badly implementation and a different Factor battlesnake library by another contributor.

Episode 1

Episode 2

Episode 3

Sun, 15 Sep 2024 15:00:00

John Benediktsson: Factor 0.100 now available

“Life can only be understood backwards; but it must be lived forwards.” — Kierkegaard

I’m very pleased to announce the release of Factor 0.100!

OS/CPU Windows Mac OS Linux
x86 0.100 0.100
x86-64 0.100 0.100 0.100

Source code: 0.100

This release is brought to you with over 1400 commits by the following individuals:

Aditya Aryaman Das, Alex null Maestas, Alexander Ilin, Andy Kluger, Bhargav Shirin Nalamati, Charlie Weismann, Dave Carlton, David Enders, Doug Coleman, Evgenii Petrov, Giftpflanze, Ikko Eltociear Ashimine, J. Ryan Stinnett, Jean-Marc Lugrin, John Benediktsson, Keldan Chapman, Limnanthes Serafini, Marc Michael, Michael Raitzam, Michael Thies, Pragya Pant, Raghu Ranganathan, Rebecca Kelly, Rudi Grinberg, Sandesh Pyakurel, Sebastian Strobl, Shruti Sen, Surav Shrestha, Val Packett, @Capital-EX, @Smoothieewastaken, @TheWitheredStriker, @TryAngle, @chunes3, @inivekin, @nomennescio, @olus2000.

Besides some bug fixes and library improvements, I want to highlight the following changes:

  • Upgraded to Unicode 15.1
  • Fix some xmlns that were accidentally changed to https
  • Improved the printing of shortest decimal representation of floating-point numbers
  • Some early support for ARM64 in the non-optimizing compiler, more to do for full support
  • Automatic light/dark theme detection works on Microsoft Windows
  • Support for compressed images, useful when reducing file size is important

Some possible backwards compatibility issues:

  • ui: focusable-child* now returns f to indicate parent should be focused
  • peg: change to compile-time PEG: and PARTIAL-PEG: forms, not delay to first invocation
  • system: renamed macosx to macos
  • math.trig: moved deg>rad and rad>deg to math.functions vocabulary
  • math.functions: fix divisor? to support mixed numbers (floats and integers)
  • math.functions.integer-logs: moved integer-log10 and integer-log2 to math.functions vocabulary
  • ranges: fixed exclusive range to be more correct for non-integer types
  • http.client: moved some download words to http.download vocabulary
  • rosetta-code: moved solutions to the factor-rosetta-code git repository
  • json: read-json returns a single object, use read-jsons to read multiple
  • base32: now contains all of the words from the base32-crockford and base32hex vocabularies

I would also like to bring particular recognition to Raghu Ranganathan, also known as @razetime, who was an incredible developer with an incredibly good attitude and contributing member to many technical communities including code golfing and various programming languages including Factor. We are very sad that he passed away a couple of months ago and would like to have this moment dedicated in his memory.

What is Factor

Factor is a concatenative, stack-based programming language with high-level features including dynamic types, extensible syntax, macros, and garbage collection. On a practical side, Factor has a full-featured library, supports many different platforms, and has been extensively documented.

The implementation is fully compiled for performance, while still supporting interactive development. Factor applications are portable between all common platforms. Factor can deploy stand-alone applications on all platforms. Full source code for the Factor project is available under a BSD license.

New libraries:

Improved libraries:

VM Improvements:

  • Improved ARM64 bootstrap assembly to allow small forms to execute successfully and natively in the non-optimizing compiler. This continues to be a work-in-progress to fully support ARM64.
  • Tentative support for compressed images, allowing Factor images to be as much as 8x smaller in size with run-time uncompression overhead.
  • Improved console I/O on windows to work in environments such as cygwin and GitHub Actions.

Wed, 11 Sep 2024 15:00:00

Blogroll


planet-factor is an Atom/RSS aggregator that collects the contents of Factor-related blogs. It is inspired by Planet Lisp.

Syndicate