Breaking referential transparency in Racket

Created: 2020-08-07

I recently purchased a copy of SICP to spend time productively during the quarantine. I decided to read it all the way from the beginning, cover-to-cover, solving all exercises along the way. I decided to use Racket for this, instead of MIT/GNU Scheme, the traditional choice. There's a very nice Racket package for SICP, containing #lang sicp, a simple extension of #lang r5rs. But it turned out to be too inadequate for my... rather peculiar needs.

You see, I decided to use this opportunity to make a toy #lang. It started out dead simple:

#lang racket/base
(provide #%app #%datum #%module-begin #%top-interaction #%top
         λ define let quote
         cond if and or not else
         + - / * = > <)

The first line of exported identifiers is necessary for the reader and the module system. λ is an alias of lambda (because I'm a spoilt brat like that), else needs to be exported explicitly because it's a syntax parameter for cond (something of a Racketism) and the rest is the usual.

Okay, so far so good. I'll need to add more later on but this is quite enough for the first bits of the first chapter. This is where my hare-brained scheme (heh heh) comes in. I wanted to do something like this: Put procedures and constants defined in the first chapter into 1.scm, then load it from, say, 1.7.scm, containing my solution to Exercise 1.7, so that I can overload some procedures with the improved ones as part of the exercise.

1.scm is as follows:

;;; Procedures defined in Chapter 1
(define abs
  (λ (x)
    (if (< x 0) (- x) x)))

(define square
  (λ (x)
    (* x x)))

(define improve
  (λ (guess x)
    (average guess (/ x guess))))

(define average
  (λ (x y)
    (/ (+ x y) 2)))

(define good-enough?
  (λ (guess x)
    (< (abs (- (square guess) x)) 0.001)))

(define sqrt-iter
  (λ (guess x)
    (if (good-enough? guess x)
        guess
        (sqrt-iter (improve guess x) x))))

(define sqrt
  (λ (x)
    (sqrt-iter 1.0 x)))

They're all given in the book as-is. Exercise 1.7 asks you to redefine good-enough? so that it continuously compares the iteration of the guess. All right, we can do that. This isn't the complicated part yet.


(define good-enough?
  (λ (guess x)
    (= (improve guess x) guess)))

At first, I used Racket's default module system: I declared 1.scm a module that exports every defined identifier through (provide (all-defined-out)). But Racket is very adamant about preventing redefinition of imported identifiers. In fact, this is a subject Racket deviates from the Lisp tradition. So we reach into the guts of the module system and set the parameter (compile-enforce-module-constants #f) to break that. Great! Now we can redefine imported identifiers!

It turns out it (quite reasonably) lets you shadow the identifiers, but naturally the contents of the closures remain the same, as the sqrt-iter continues to refer to the original good-enough?. I should've expected that... But I remain undaunted because I want my stupid hack to work, dammit!

What if... we circumvent this hard-nosed module system altogether and directly include the file at compile-time? That's how most Schemes separated code into files prior to R6RS — with a mere load. After some cursory search, I discover the module racket/include. It does exactly what it says on the tin: It exports an include macro that directly inlines the given file. Great! And voilà, 1.7.scm now shares the same namespace as the prepended 1.scm, which now contains... two separate definitions... of the same identifier... which is unacceptable for most Schemes... Okay, we're on the wrong track.

Hey, wait a minute... How does the REPL do it? I can continuously call define on the same identifiers in the REPL without having to wipe the namespace or calling set! on everything like a barbarian! Then I remember REPL doesn't use require (nor include for that matter), but load for importing files. Could this be what I'm looking for? There it is: racket/load! The guide says says it musn't be used outside the REPL except for experimenting with the namespace. What we're doing counts as experimentation, I suppose. A huge price to pay for this hack is that it breaks macros in inexplicable ways. Oh well.

Besides exporting the load macro, racket/load overrides the #%module-begin and #%top-interaction forms so that it treats every line of code as though it's run in the REPL. Sounds good! So here it is:

#lang racket/base
(provide #%app #%datum #%module-begin #%top-interaction #%top
         λ define let quote load
         cond if and or not else
         + - / * = > <)
(require racket/load)

(compile-enforce-module-constants #f)
(print-as-expression #f)
(load-on-demand-enabled #f)
(print-mpair-curly-braces #f)

Does it work? Let's see. Since macros no longer work, I can't write assertion macros, so I just wrap all assertions in an and expression and see if it returns #t.

#lang s-exp "sicp.rkt"
;; Excercise 1.7

(load "1.scm")

(define old-value (sqrt 900))

(define good-enough?
  (λ (guess x)
    (= (improve guess x) guess)))

(define new-value (sqrt 900))

(< (- new-value 30) (- old-value 30))

When run, 1.7.scm returns #t. Splendid! Our new good-enough? is indeed better!

So we managed to create an abomination of a #lang to break referential transparency, just to be lazy. We probably could've done this with some crafty syntax-parse macros around module language primitives but I'm not smart enough for that. What we've got works and that's all that matters, right? I don't mind building up the module language piece-by-piece in provide but I don't want to deal with compatibility/mlist (or its less scary alias racket/mlist) to get mutable lists so eventually I'm just going to provide the whole #lang r5rs. (Be careful not to let it export identifiers like #%module-begin.)