I’m in love with the Crafting Interpreters book. In it, Bob Nystrom teach us how to writer an interpreter by implementing a little programming language called Lox. It was a long time since I had so much fun programming! Besides being well-written, the book is funny and teach way more than I would expect. But I have a problem.
The snippets in the bug are written in a way we can copy and paste them. However, the book has challenges at the end of each chapter, these challenges have no source code and sometime they force us to change the interpreter a lot. I do every one of these exercises and as a result my interpreter diverges too much from the source in the book. Consequently, I often break some part of my interpreter.
How to solve that?
Unity tests would be brittle since the code structure changes frequently. End-to-end tests seem more practical in this case. So, for each new feature of the language, I wrote a little program. For example, my interpreter should create closures, and to ensure that I copied the Lox program below to the file counter.lox:
return count;
}
var counter = makeCounter();
counter(); // “1”.
counter(); // “2”.</code></pre>
<p>
This program result should be the numbers 1 and 2 printed in different lines. So I put these values in a file called counter.lox.out. The program cannot fail either, so I created an empty file called counter.lox.err. (In some cases, it is necessary to ensure the Lox program will fail. In these cases, the file .lox.err should have content.)
Well, I wrote programs and output files for various examples; now I need to compare the programs’ results to the expected outputs. I decided to use the tool that helps me the most in urgent times: shell script. I did a Bash script with a for iterating over all examples:
done</code></pre>
<p>
For each example, I executed the Lox program, redirecting the outputs to temporary files:
Now, we compare the real output with the expected output through diff. When it compares two files, diff returns 0 if there is no difference, 1 if there exists a difference or 2 in case of error. Since in Bash the conditional if considers 0 as true, we just check the negation of diff‘s exit code.
If the program prints something in standard output that is different from what is in its .lox.out file, we have a failure:
if ! diff $l.out $out
then
FAIL=1
fi
done</code></pre>
<p>
We also check the standard error and the .lox.err file:
if ! diff $l.out $out
then
FAIL=1
fi
if ! diff $l.err $err
then
FAIL=1
fi
done</code></pre>
<p>
Finally, I check if there was some failure and report the result:
if ! diff $l.out $out
then
FAIL=1
fi
if ! diff $l.err $err
then
FAIL=1
fi
if [ "$FAIL" = "1" ]
then
echo "FAIL" $l
else
echo "PASS" $l
fi
done</code></pre>
<p>
Not all of my Lox programs can be checked, though. For example, there is a program which times loop executions, it is impossible to anticipate the value it will print. Because of that, I added the possibility to jump some programs: we need just to create a file with the .lox.skip extension:
out=$(mktemp)
err=$(mktemp)
java -classpath target/classes/ br.com.brandizzi.adam.myjlox.Lox $l > $out 2> $err
if ! diff $l.out $out
then
FAIL=1
fi
if ! diff $l.err $err
then
FAIL=1
fi
if [ "$FAIL" = "1" ]
then
echo "FAIL" $l
else
echo "PASS" $l
fi
done</code></pre>
<p>
If, however, I have a Lox example and it does not have expected output files (nor the .lox.skip file) then I have a problem and the entire script fails:
out=$(mktemp)
err=$(mktemp)
java -classpath target/classes/ br.com.brandizzi.adam.myjlox.Lox $l > $out 2> $err
if ! diff $l.out $out
then
FAIL=1
fi
if ! diff $l.err $err
then
FAIL=1
fi
if [ "$FAIL" = "1" ]
then
echo "FAIL" $l
else
echo "PASS" $l
fi
done</code></pre>
<p>
With that, my test script is done. Let us see how it behaves:
$ ./lcheck.sh
PASS examples/attr.lox
PASS examples/bacon.lox
PASS examples/badfun.lox
PASS examples/badret.lox
PASS examples/bagel.lox
PASS examples/bostoncream.lox
PASS examples/cake.lox
PASS examples/checkuse.lox
PASS examples/circle2.lox
PASS examples/circle.lox
1d0
< 3
1c1
<
---
> [line 1] Error at ',': Expect ')' after expression.
FAIL examples/comma.lox
PASS examples/counter.lox
PASS examples/devonshinecream.lox
PASS examples/eclair.lox
PASS examples/fibonacci2.lox
PASS examples/fibonacci.lox
PASS examples/func.lox
PASS examples/funexprstmt.lox
PASS examples/hello2.lox
PASS examples/hello3.lox
PASS examples/hello.lox
PASS examples/math.lox
PASS examples/notaclass.lox
PASS examples/noteveninaclass.lox
PASS examples/point.lox
PASS examples/retthis.lox
PASS examples/scope1.lox
PASS examples/scope.lox
PASS examples/supersuper.lox
PASS examples/thisout.lox
PASS examples/thrice.lox
SKIP examples/timeit.lox
PASS examples/twovars.lox
PASS examples/usethis.lox
PASS examples/varparam.lox
Oops, apparently I removed the support for the comma operator by accident. Good thing I wrote this script, right?
I hope this post was minimally interesting! Now, I am going to repair my comma operator and keep reading this wonderful book.
(This post is a translation of Não me Interprete Mal: Improvisando Testes para um Interpretador.)