Does Betteridge's Law of Headlines ever NOT apply?

No.

BuckleScript Gradual Typing: Incremental Change

IN THE previous post, I showed a basic example of overlaying an OCaml type over a dynamically-typed JavaScript function, using BuckleScript's FFI feature. In this post, I will show a slightly more complex example: binding to the Web API's Intl.DateTimeFormat constructor. As usual, you can follow along by typing and pasting code into the BuckleScript playground.

In JavaScript, we would use the constructor like:

const format =
  new Intl.DateTimeFormat("en-CA", { hour12: false });

We do need to note, though, that both of the arguments to Intl.DateTimeFormat are optional, and will be given default values if they are not passed in. So really in our BuckleScript binding we want to somehow make them optional as well.

Fortunately, OCaml and BuckleScript allow for optional function parameters, as long as we make the final parameter non-optional:

type dateTimeFormat

external dateTimeFormat_make :
  ?locales:'locales -> ?options:'options -> unit -> dateTimeFormat =
  "Intl.DateTimeFormat" [@@bs.new]

let format =
  dateTimeFormat_make
    ~locales:"en-CA" ~options:[%obj { hour12 = false }] ()


We introduce several new concepts here:

type dateTimeFormat is an abstract type that, for now, is defined only by the fact that we create values of the type with:

external dateTimeFormat_make ... is a binding to a constructor (thanks to the [@@bs.new] attribute) which takes optional locales and options arguments, a required unit argument, and returns an instance of dateTimeFormat. The optional parameters are marked by the '?' symbol.

let format = ... is a usage of the dateTimeFormat_make FFI binding to force BuckleScript to actually generate a call to the Intl.DateTimeFormat constructor. By default BuckleScript only generates JavaScript output lazily.

In the spirit of gradual typing, I've also been lazy about working out the actual types of the parameters to the dateTimeFormat_make binding. Partly it's because the free type variables 'locales and 'options allow me to pass in any type of argument into those slots. In the previous snippet, I passed in the correct types (a string and a JavaScript object with a valid keyword) intentionally, but there is no type-level guarantee that I have to do so. I could have passed in values of invalid types, and BuckleScript would merrily generate JavaScript which would only fail at runtime.

So how can I force some more measure of type safety into the bindings here? Well, for the locales parameter, the Web API reference for Intl.DateTimeFormat says its type has to be either a string or an array of strings. In BuckleScript, we can model this 'either-or' scenario using the [@bs.unwrap] attribute:

external dateTimeFormat_make :
  ?locales:([ `one of string | `many of string array ] [@bs.unwrap]) ->
  ?options:'options ->
  unit ->
  dateTimeFormat =
  "Intl.DateTimeFormat" [@@bs.new]

let formatOne = dateTimeFormat_make ~locales:(`one "en-CA") ()

let formatMany = dateTimeFormat_make ~locales:(`many [|"en-US"; "en-CA"|]) ()

Now, we can pass locales only as a single string or as an array of strings. Anything else is a type error. The compiler does need some help though, to statically tell the difference; hence we have to use polymorphic variants to differentiate one case from the other.

Note that I left out the options argument in these cases to make them simpler. If you paste this into the playground, you'll see BuckleScript generates undefined arguments in their place.

This post is getting rather long now, so I'll explore how to make the options argument more type-safe in the next one. It's going to be quite fun, because there's quite a lot we can do in BuckleScript to enforce the business rules of the API at the type level!

Gradual Typing of JavaScript Using BuckleScript

USUALLY when we say 'gradual typing', we mean gradually adding type annotations to a dynamically typed codebase like JavaScript or Python, and then running it through a typechecker like Flow/TypeScript/mypy. However, an equivalent approach is to leave the existing codebase as it is, write typed bindings to it using the FFI capabilities of a statically-typed language, and use those typed bindings to interact with it.

This of course implies that we have no intention of retiring the original codebase; we simply want to interoperate with it as safely as possible. This is an especially ideal scenario when we're dealing with a massively successful and active body of code like that of the JavaScript community.

In fact, TypeScript and mypy themselves adopt this approach--using typed bindings. Their communities maintain massive libraries of library bindings ready for consumption. In this post, I'm going to demonstrate how BuckleScript actually allows us to gradually introduce type-safe bindings to existing JavaScript APIs.

console.log

We'll start with a simple example. From using JavaScript before, we will know that the built-in JavaScript Web API function console.log prints out an object that we give it on the browser's JavaScript log:

console.log(obj);

So in BuckleScript terms, we want to bind to a global value console.log that takes a single value and returns ... nothing. However, in BuckleScript (i.e., OCaml), all functions have to return some value. What do we return when we return nothing?

Well, since we're practising gradual typing, we won't worry about that right now, we'll just say that we return 'whatever':

external consoleLog : 'obj -> _ = "console.log" [@@bs.val]

According to the BuckleScript manual, the [@@bs.val] attribute binds a given name to a given global value. In this case, the value is the console.log function. The 'obj type means 'any type, let's call it "obj" ', and the _ 'wildcard' type means 'I don't care what the type is, let the typechecker infer it'.

If you bind and then call this function in the BuckleScript playground:

external consoleLog : 'obj -> _ = "console.log" [@@bs.val]

let () = consoleLog "Hello, World!"

You get the following JavaScript output:

// Generated by BUCKLESCRIPT VERSION 1.9.2, PLEASE EDIT WITH CARE
'use strict';


console.log("a");

/*  Not a pure module */

So, a binding with almost no concrete type information (except that it's a function of one argument) compiles down to JavaScript that just works! Admittedly, console.log is a simple function which accepts pretty much any input. In the next installment (this one is getting a little long) I'll cover another slightly more complex function and show how we can gradually type it in BuckleScript.

Idiomatic F# Design

HERE are the basic points of idiomatic (according to me) F# design. The idioms I present below are not the ones generally used in the F# community; they are fairly controversial. But they are similar to idioms used in the wider ML languages, and I genuinely believe they offer value in terms of easy-to-read code.

Prefer modules, records, and functions

Try to avoid classes, because they are somewhat higher-ceremony than plain F# records. For example: records can be destructured effortlessly and updated immutably by replacing the values of named fields with new values.

Avoid member methods and properties

They don't mesh well with the functional style because they can't be passed around directly. Module member values and functions can.

Put everything inside modules

Ideally, dedicate a module to a (primary) type and name the type just t to avoid repetition; refer to the type (and other module members) prefixed by the module name for maximum clarity.

In fact, put everything (types, exceptions, and values) inside modules; they are an excellent organisational tool and clarify your thinking about what things go together.

Design top-down using interface (.fsi) files

F# is fantastic for top-down design because you can write the signatures of your modules, data, and operations separately as syntactically valid code in .fsi (F# interface) files and then fill in the blanks later with the actual implementation (.fs) files. Even if you don't consider yourself a top-down thinker, give it a try and you may be surprised by F#'s expressive power here.

Interface files are like C/C++ header files, and let you hide any module members you don't want to expose. Crucially, they also let you hide data definitions and give your caller abstract types to work with, enabling information hiding and decoupling. Quick example:

(* person.fsi *)
namespace MyProject

module Person =
  type t

  val make : string -> t
  val id : t -> int64
  val name : t -> string
  val with_name : string -> t -> t

(* person.fs *)
namespace MyProject

module Person =
  type t = { id : int64; name : string }

  let make name = { id = Id_service.get (); name = name }
  let id { id = i } = i
  let name { name = n } = n
  let with_name new_name t = { t with name = new_name }

In the above example, we hide the Person.t type's implementation details so that callers can't create values of the type themselves bypassing our API. Nor can they operate on values of the type in any way except the ones we provide as part of the Person module.

Also, we take advantage of the above-mentioned F# destructuring pattern matching and immutable record update--benefits we don't get from OOP style--to quickly access and change exactly the parts of the data we're interested in.

Use classic ML-style type parameter application

The recommended style of type parameter application in .NET is with angle brackets after the type constructor. But this is noisy, and just gets noisier as your types get more complex. Use ML-style, which reverses the order of application to postfix, but gets rid of (most of) the noise. Let's compare:

type a1 = Async<option<Person.t>>
type a2 = Person.t option Async

type b1 = Choice<string, Person.t>
type b2 = (string, Person.t) Choice

type c1<'a> = Map<string, 'a>
type 'a c2 = (string, 'a) Map

ML-style type application gives you the biggest wins for sequences of single-parameter applications, but it still spaces out the code in general, which helps with reading.

Use type parameters to control allowed operations

When your type has operations that are only valid when the type is in a certain state, then you can express it as a discriminated union (a sum type) so that operations work in different ways for different cases of the DU. But the problem with exposing the cases of a DU is that you lose decoupling--you can't easily control future refactoring of the type because user code will depend on existing DU cases.

Another problem is, maybe certain operations don't work for all states of a type. Let's try an example:

namespace MyProject

module Bulb =
  type t = On | Off

  exception Already_on
  exception Already_off

  let turn_on = function
    | Off -> On
    | On -> raise Already_on

  let turn_off = function
    | On -> Off
    | Off -> raise Already_off

Both of our critical operations on the Bulb.t type are raising exceptions if called wrongly. Sure, we could have returned a None : Bulb.t option instead if something went wrong; but that just passes the checking to the caller somewhere else. There is a better way: phantom types. Here's the above example converted:

(* bulb.fsi *)
namespace MyProject

module Bulb =
  type on
  type off
  type 'a t

  val on : on t
  val off : off t
  val turn_on : off t -> on t
  val turn_off : on t -> off t

(* bulb.fs *)
namespace MyProject

module Bulb =
  type on = interface end
  type off = interface end

  (* Phantom type--left-hand type parameter isn't used on right hand. *)
  type 'a t = On | Off

  let on = On
  let off = Off

  (*
  Compile warnings are OK here because we're controlling allowed inputs.
  *)
  let turn_on Off = On
  let turn_off On = Off

Note that the compiler warns us here about incomplete pattern matches on the Bulb.t type, but we ignore them in this case because we're explicitly controlling allowable function inputs at the type level. We can turn off this warning with a #nowarn "25" compiler directive in the bulb.fs file, but in general turning off warnings is not a good idea.

Prefer records of functions to interfaces

Here is a good write-up about this, but let me reiterate: interface methods can't be easily passed around, unlike record member functions (which can be closures, remember). An example:

(* api.fs *)
namespace MyProject

module Api =
  exception Invalid_id

  type 'a t =
    { get_exn : int64 -> 'a Async
      add : 'a -> unit Async
      remove_exn : int64 -> unit Async
      list : unit -> 'a seq Async }

(* person.fs *)
namespace MyProject

module Person =
  type t = { id : int64; name : string }

  let make id name = { id = id; name = name }
  let id { id = i } = i
  let name { name = n } = n

  (* val api : t Api.t *)
  let api =
    { get_exn = fun id -> ...
      add = fun person -> ...
      remove_exn = fun id -> ...
      list = fun () -> ... }

Above we implement a generic data store API and a domain type that implements the API. Note that as a convention we add an _exn suffix to code which might raise an exception. This helps the reader prepare to reason about exception handling.

Wrap OOP-style APIs in modular F#

When dealing with traditional OOP-style APIs, e.g. Windows Forms, try to wrap them up in modular F# and expose only the F# abstraction. E.g., suppose you're writing a simple GUI calculator app in WinForms. You need to (1) write out the logic and make it testable; and (2) render the GUI in a form. Solution--put these things in a module:

(* calculator.fsi *)
namespace CalculatorApp

module Calculator =
  type number = Zero | One | Two | ... | Nine
  type op = Plus | Minus | ... | Sqrt

  type t

  val init : t

  (*
  We will implement these operations as mutating to allow a more fluent,
  pipeline-based API like:

  let result =
    calc
      |> clear
      |> press_number Five
      |> press_op Plus
      |> press_number Six
      |> calculate

  At this point, `calc` is in the state that it holds a calculation
  result. We can `clear` it to do more calculations.
  *)

  val clear : t -> t
  val press_number : number -> t -> t
  val press_op : op -> t -> t
  val press_decimal : t -> t
  val calculate : t -> double

  (*
  Draws the app and hooks up event handlers to the above operations.
  *)
  val render : t -> System.Windows.Forms.Control

Now in the implementation, follow the types to implement the business logic and the GUI. Note how the logic is testable because the main operations are exposed, but the messy OOP GUI rendering is not--the caller just gets back a Control to embed in their main window. This is composable and readable.

General philosophy

To conclude, we aim for these design points:

• Information hiding with interface files
• Simpler, less noisy syntax is better
• Modular is better--ideally everything is in a module
• Logic encoded in the types is better--you get compile-time guarantees and have to do less runtime error handling.

Can F# be liberated from the .NET architecture?

Recently I've been thinking about the delicate situation that the F# language designers find themselves in whenever they want to introduce new features (especially semantics) into the language, or whenever the C# team want to introduce features that may affect that language or the common language runtime.

The problem has clearly existed for a while; F# hasn't gotten any significant new features since its last major release. The latest example to pop up is probably the uncertainty over adding typeclasses, especially now that it looks like C# is thinking about adding them too.

So how can F# be freed from its ties to C# and the .NET runtime? Well, one way would be to split the F# compiler into two parts: the frontend that compiles F# code to its own intermediate language (let's call it F# Intermediate Language, or FSIL); and the backend that compiles FSIL to some other target language like MSIL (or JavaScript like Fable, JVM bytecode, native, ...).

Compiling F# to FSIL

This presents the main advantage that the F# language can evolve independently of C# and the CLR. The compiler frontend need only worry about doing a great job of implementing F# syntax and semantics in terms of a single target representation that it controls fully. The language can rapidly experiment with new syntax and semantics, or in fact with the traditional ML language family semantics, just by defining a general FSIL to target.

Some examples of this would be implementing typeclasses, or ML-style parameterised modules (functors), or Scala-style implicit evidences, or an easy-to-use call-by-name function parameter declaration syntax for easier control-flow DSLs, or higher-kinded types, or ... any of a myriad of things. All you have to do is teach the frontend to understand your new syntax and compile it to FSIL.

The nice thing about FSIL is that it would preserve nice concepts like units of measure, typeclasses, ML modules and functors, etc., because it was specifically built to understand them. F# modules would be compiled down to FSIL files on a one-to-one basis and F#-specific libraries could be distributed as FSIL packages. Users could link against them to get the nice higher-level functionality that MSIL doesn't offer.

Compiling FSIL to MSIL (and others)

This step would be concerned with the best way to encode FSIL in terms of the target language or runtime. E.g., currently F# modules are encoded as static MSIL classes. This is obviously a great fit, and others may be found for higher-level F# features.

But no doubt some FSIL functionality would be erased--e.g. functors would be defunctorised and compiled away to nothing. Any concrete instantiations of functors would survive as normal modules encoded as static classes as usual. Or, typeclasses would be compiled down to pass instances in explicitly in the MSIL encoding.

With this organisation, other backends could spring up. Fable could become a backend from FSIL to ES5/6/.... Someone could write a backend for FSIL to JVM, or the Erlang runtime (BEAM), or the OCaml runtime. There are quite a few possibilities opened up by decoupling the two ends from each other.

Optimisation

With FSIL we would have the opportunity to do separate optimisation passes on the higher-level, typed, F# focused FSIL, and on the lower-level CLR (or other target) outputs. We would be able to take advantage of the information that we would have at those points in the process while not worrying about the previous or following stages.

F#ixing up

There are many advantages to a decoupled frontend/backend design for the F# toolchain. Possibly the biggest one is that it places F#'s future as a powerful language squarely in the hands of the F# language designers, and lifts from them the burden of having to worry about compatibility with the rest of .NET.

BuckleScript: a significant new OCaml to JavaScript compiler

RECENTLY I've been paying attention to the BuckleScript compiler from OCaml to JavaScript (ES5, to be exact). I'm seeing some significant advances that it brings to the state of the art, and thought I would share.

A little background: I use Scala.js at work, and it is rock-solid. Once you're set up with a project, you are pretty much good to go. Bindings to significant JS libraries are high-quality; I use some of the biggest names in JS frontend dev libraries today, and all in a nice Scala-style object-functional layer.

That aside, I still periodically look around the ecosystem and try to evaluate the latest developments. I've been hearing about BuckleScript for a while, but only recently decided to try it out for a side project after trying and failing to understand how this works in JavaScript.

So, without further ceremony, let me present my findings.

BuckleScript's compiler is insanely fast, because Bob Zhang (the project lead) has taken to heart the OCaml obsession with performance and routinely tries to optimise away millisecond-level delays. Once you get a taste of that speed (entire project compiled faster than you can blink), you'll find it difficult to go back to something slower. It's like getting to use git after you've used svn all your life.

It compiles to idiomatic, readable ES5, with nice indentation, (almost) no name mangling, and a one-to-one mapping from OCaml modules to ES modules (whichever kind you prefer: Require, AMD, Google).

It targets and integrates with the existing npm ecosystem; it doesn't try to introduce yet another package manager. It makes writing bindings (types) for existing JS libraries reasonably easy, and the documentation (the manual especially) is fantastic at guiding you through that.

OCaml is a bit of an odd duck syntax-wise, even among the functional programming languages. There are nuances to get used to. But once you get used to them, it is a pleasure to program in. And if you just can't get used to them, you can always try out Facebook's Reason, which is an alternative, JavaScript-lookalike syntax for OCaml.

This focus on integration and ease of reuse of the JavaScript ecosystem means it's feasible to leverage the npm package collection in your pure OCaml project. You can deploy a backend server which performs core functions as a statically compiled, native binary (i.e. not nodejs); deploy ES5 nodejs services which take advantage of specialised npm packages for MSSQL querying, or SOAP clients, or what have you; and you can deploy ES5 in your frontend webapp scripts, all written in pure OCaml.

So, why OCaml specifically? After all, there are plenty of nice languages out there.

As it turns out, that OCaml obsession with speed and type-safety together serve it well here. It's a pragmatic, simple, and matter-of-fact language, and its runtime model maps very well to the JavaScript runtime model, while also preserving important compile-time safety guarantees.

I should emphasise that it's pragmatic: you're not forced to deal with the added mental load of monads and other type system rabbit holes--but they're all available if you need them! Personally, I feel that laziness, purity, and monads have driven away more people than they've attracted. I think that OCaml gets the balance right. Others obviously feel differently. But in concrete terms, BuckleScript is a significant contribution that shouldn't be missed.

If you've developed in a compiled language for any length of time and like type-safety guarantees, after trying BuckleScript you'll be asking yourself how much time you've wasted over the years waiting for your compiler to finish so you can continue your edit-compile cycle. Maybe it's best not to think too much about that.

Nana

NANA is what I called him, but to the world he is the late Lutful Quadir Chowdhury, a well-loved husband, father, grandfather, prominent in the respect he earned from his large extended family, and a highly-regarded elder statesman of the South Asian banking community. For a time he settled down and raised his family in then-West Pakistan, but when the time came he chose to migrate to the newly-formed Bangladesh.

My Nana was a foundational rock of my childhood in Dhaka. To me his spacious home compound, with its large bungalow, lush green back lawn, and giant driveway with two garages at opposite ends, were my personal domain to explore and play in. I have many happy hours of running, jumping, cycling, cricket, soccer, badminton, carrom-board, Monopoly ... and every other childhood activity there. I remember monsoon seasons when we would bring the storm shutters down and the family would gather for tea and watch the rain pour down outside through the wall-to-wall balcony windows. Hours and hours of poring through his rooms and shelves filled to bursting with books. To this day I have dreams that are set in that house.

Nana was a collector of knick-knacks that any hipster today would give his right arm for. Hand-powered ice-cream makers, spotless Swiss food processor sets, a video cassette recorder that he would bring out to keep me entertained, a slide rule he gave me when I showed signs of interest in math ... but books got the pride of place in his home. They were proudly displayed on his shelves, kept in storage in room after room, kept near his side of the bed for easy access. I would spend hours entranced by those books. Every subject you can think of, from Greek mythology to Somerset Maugham, O. Henry (The Gift of the Magi) to Khushwant Singh. I, a small child, absorbed as much of them as he would lend me, like a sponge. They awakened my interest in chess, algebra, calculus, mythology, history, essays, poetry, ... the world.

He would take me for fun-filled trips to the video cassette rental store, treat us out to fast food, order in food whenever we visited (we accidentally got cuttlefish once instead of his order of cutlets), show me how he recorded his expenses in his ledger, and all the while carry out his responsibilities as a busy man even after retirement.

He would receive many visitors, whether friends,  family, or others; to some he would give the help or advice they sought; with others he would enjoy a few rounds of chess; and from others he would procure various services: palm tree tappers, snake charmers, gardeners, hairdressers....

He would make the rounds of his home every evening, one of his well-loved flashlights in hand, checking that all windows were securely fastened, all doors were locked and bolted shut, and that all of his household were accounted for. He would discourage anyone from leaving the house after nightfall, and be up again in the early morning, making sure everyone was awake. I've never been a morning person, and he always had to spend quite a lot of time convincing me to wake up.

I think Nana, coming from a zamindari (landowning) family that lost much of its wealth to the turbulent times, had an idea in his mind of what his life should be as a patriarch of his family, and he made that a reality through sheer hard work and persistence. He created an evergreen world, a perfect world for a childhood.

In the years after that, I was away from Bangladesh most of the time. I didn't get the chance or take the opportunity to see Nana as much as I should have. For years I kept telling myself I would meet him again and we would have another game of chess. Earlier this year I finally saw him and we had our game. I told him I would come back and we would have a rematch, and he agreed. I told him my younger brother would visit him soon, and he was glad to hear it. It gave him something to look forward to. I'm so grateful I was able to keep at least one promise.

After his passing, my overwhelming feeling is gratitude that I knew him.