5th February 2019 - Presentations , Nim
This article is intended as a companion to the lightning talk I held at FOSDEM 2019. If you haven't seen the talk yet the official recording from the FOSDEM site is embeded below (if you want you can also grab the slides for this presentation on that site).
After all the fun we had at FOSDEM last year I decided to go again this year, but this time I wanted to contribute something. So I decided to submit for a lightning talk, a short distilled presentation that might get more people interested in trying out Nim. But instead of doing yet another introduction to Nim talk that would only scratch the surface and not get into anything that was particularly thrilling I decided to do a more specific presentation on a feature of Nim. As you probably now know the feature I landed on was meta-programming. It was certainly one of the aspects of Nim that caught my attention when I first got into using the language, and a feature that can be used across all kinds of different projects for all targets.
My talk was mostly based off of this very good article on metaprogramming in Nim. But I wanted to fill in some of the gaps that I didn't have time to go over in my talk with this post. Since the talk is intended to mostly be about the meta-programming aspects I didn't really have all that much time to go through the actual introduction to what Nim is in any more detail than a couple of bullet points. And this list of bullet points tend to raise some questions, I certainly was sceptical of what Nim promised the first time I read about it.
Deeper look at Nim
More about meta-programming
With that out of the way it's time to get to the topic of meta-programming. Now meta-programming is not unique to Nim, but it's fairly uncommon for compiled, imperative languages. This is partially caused by the fact that in order to do it Nim has itself as a target in a sense. Since macros are written in Nim the compiler needs to be able to execute arbitrary Nim code. This means that the compiler is actually running a Nim VM that runs your macro code. This does have some limitations, for example it's not possible to call C libraries like you normally can in Nim from inside a macro. It is however possible to do things like read and write files, run external scripts and programs, etc. Alongside the AST-based templates and macros Nim is also able to use this VM for regular computation, passing around complex compile-time object structures and everything else your normal program can do. This means that you are able to write simple pre-compute macros to e.g. embed a sine values table in a micro-controller program, or parse the interesting parts out of a third party format, all without having to copy-paste things or make complex custom pipelines. I've even implemented a Protobuf parser directly as a macro so you always know that your compiled Nim program is using the latest version of your Protobuf specification.
Now for a word of caution. I mention in my talk that meta-programming can be used to make things safer, faster, and more read- and maintainable. But as with any abstraction it also has its pitfalls. If you do something wrong in you macro it might be very hard for the person using it to understand why it doesn't do what it's supposed to. Take this simple example:
template checkIt(it: int) = if it == 3: echo "It is 3: ", it var it = 0 proc addToIt(): int = result = it it += 1 for _ in 0..5: checkIt(addToIt()) echo "it: ", it
You might expect this to output something like this:
it: 1 it: 2 it: 3 It is 3: 3 it: 4 it: 5 it: 6
But what we would actually see is that when we use
it to output the value after the check, the
addToIt procedure gets called again and the output becomes:
it: 1 it: 2 it: 3 It is 3: 4 it: 5 it: 6 it: 7
If we wrap line 11 in a
expandMacros debug call we can see what our template actually does:
if addToIt() == 3: echo ["It is 3: ", addToIt()]
This is because it's the AST that we pass in that
it expands to, and not the value. The same reason is why our optimised logger worked, so it's a practical feature once one is aware of it. But it's important to make sure that your macros and templates have a very well-defined behaviour so that the user doesn't get confused. The wxWidgets macro I demo'ed at the end of the presentation tries to solve this by being a very simple mapping from DSL to wxWidgets code. If you know how to write wxWidgets code and you have read the simple list of substitutions it should be rather obvious what the output of the macro is going to be. This also means that it is easy to read the official wxWidgets C++ documentation and adapt that into the macro. Generally speaking you want to do as little magic as possible with your macros.
If you are intrigued by what you've seen so far feel free to read up on the official Nim site, peruse this web-site for some more in-depth posts, or head over to the community page to find your favourite way to ask us questions.