RExArt

The RExArt Project

My son-in-law, Martijn, showed me a cool project he has been working with on and off since his college days.  The basic idea is this: build an expression by randomly combining a set of built-in functions, and then use that expression to generate a color for each pixel on an arbitrarily sized canvas.  I've called my implementation of this idea "RExArt", for Random Expression Art.

To cut to the chase, here's an example of a RExArt image:

It uses three randomly generated expressions to produce an HSV (hue/saturation/value) color at each pixel in a 1000 x 800 grid.  The randomly generated equations (ready for this?) are:

• (Hue)   Mod(Length(Cos(Sin(Times(Add(Tri(Round2(Sin(Z())),Times(Div(X(),Z()),Cos(Z())),Sin(Tri(X(),Z(),X()))),Length(Div(Div(X(),Z()),Round2(X())),Round2(Mod(Z(),Y())))),Round2(Round2(Cos(Div(Y(),Z()))))))),Add(Length(Div(Div(Sin(Tri(Tri(Z(),X(),Z()),Round2(X()),Times(Y(),Z()))),Tri(Cos(Cos(X())),Tri(Round2(Z()),Mod(Z(),Z()),Cos(X())),Cos(Sin(Y())))),Mod(X(),Div(Sin(Tri(Z(),Y(),X())),Round2(Div(Y(),X()))))),Times(Times(Round2(Mod(Sin(Y()),Round2(Z()))),Mod(Add(Add(Y(),X()),Cos(Z())),Length(Div(Y(),Z()),Sin(X())))),Sin(Length(Round2(Tri(X(),Y(),X())),Length(Add(Y(),Z()),Round2(Y())))))),Cos(Sin(Div(Mod(Mod(Round2(Y()),X()),Tri(Mod(Z(),Z()),Mod(Z(),Z()),Cos(Z()))),Add(Mod(Round2(Z()),Add(Y(),Z())),Mod(Round2(Z()),Mod(X(),Y())))))))),Round2(Div(X(),Tri(Tri(Div(Cos(Times(Add(X(),X()),Div(Z(),Z()))),Sin(Div(Round2(Z()),Div(X(),X())))),Mod(Length(Tri(Mod(X(),X()),Div(Z(),Z()),Round2(X())),Sin(Mod(Z(),Y()))),Sin(Mod(Times(Y(),Z()),Tri(X(),X(),Y())))),Round2(Div(Round2(Div(X(),Z())),Div(Cos(Z()),Cos(X()))))),Tri(Add(Length(Times(Add(Z(),Z()),Add(Y(),X())),Round2(Div(X(),X()))),Round2(Add(Times(Z(),X()),Div(Y(),X())))),Mod(Times(Times(Sin(Z()),Sin(Z())),Length(Tri(Z(),X(),Z()),Sin(Z()))),Cos(Z())),Mod(Add(Cos(Tri(Y(),X(),X())),Tri(Mod(X(),Z()),Tri(Y(),Y(),Y()),Times(Z(),Z()))),Mod(Round2(Add(Y(),Z())),Add(Cos(X()),Div(Z(),X()))))),Div(Div(Tri(X(),Times(Cos(X()),Cos(Y())),Mod(Div(Z(),Z()),Cos(Z()))),Add(Add(Tri(Y(),X(),Y()),Y()),Y())),Cos(Round2(Y())))))))
• (Saturation) Cos(Cos(Add(Times(Length(Z(),Add(Tri(Div(Tri(Z(),Z(),Y()),Cos(Y())),Sin(Div(X(),X())),Sin(Cos(X()))),Cos(Add(Add(Y(),Z()),Tri(Y(),X(),Y()))))),Sin(Length(Tri(Add(Add(X(),Y()),Cos(X())),Times(Tri(Z(),Z(),X()),Cos(Y())),Div(Round2(Z()),Sin(Z()))),Tri(Tri(Times(Z(),X()),Z(),Times(Z(),X())),Cos(Length(Z(),Y())),Cos(Mod(Z(),X())))))),Length(Mod(Sin(Cos(Tri(Tri(X(),X(),Y()),Times(Z(),X()),Mod(X(),Z())))),Round2(Round2(Sin(Round2(X()))))),Add(Y(),Cos(Times(Cos(Add(Y(),Y())),Add(Div(X(),Z()),Sin(X())))))))))
• (Value)  Cos(Div(Add(Round2(Add(Tri(Round2(Sin(Sin(X()))),Add(Y(),Sin(Tri(Y(),X(),Z()))),Length(Tri(Sin(X()),Length(X(),Z()),X()),Tri(Length(Z(),X()),Cos(Y()),Add(Y(),Y())))),Add(Mod(Round2(Add(Z(),Y())),Length(Div(X(),Z()),Times(X(),X()))),Round2(Tri(Sin(X()),Sin(Z()),Add(X(),Z())))))),Z()),Cos(Round2(Cos(Mod(Tri(Mod(Round2(Y()),Tri(Y(),Y(),Z())),Cos(Sin(Z())),Sin(Div(X(),Z()))),Cos(Length(Add(Z(),Y()),Times(Z(),Y())))))))))

It was surprisingly easy to put together the basics for the project.

First, you need to generate a random expression (or, in my case, three of them).  To do this, pick a root function randomly from a list of known functions.  (In the RExArt above, the root functions are Mod() (for hue) and Cos() (for both saturation and value).  If a function requires arguments, generate additional random functions for those arguments.  If those functions need arguments, generate more functions for them.  Keep at it until you reach some desired maximum depth.

(Note, incidentally, that all functions at the maximum depth need to be zero-argument functions; otherwise, the expression would just keep growing and growing and growing....)

Once you have the random expression(s), all you need to do is to evaluate the expression(s) at each point (0,0), (0,1), (0,2), ..., on the canvas (up to (999,799) for 1000 x 800 pixel canvas) and use the result(s) to set the color at that point.

I discovered that RExArt's can be animated by adding a third dimension.  Starting with the RExArt image above, I added a Z dimension that ranges from 0 to 200, then generated a separate image for each Z value and used the collection of images as frames for the animation.  Here's the video of the animated RExArt:

The Function Library

The key to good RExArt is the function library that is used to build the random expressions.  In RExArt, each library function requires zero or more arguments, and each argument (when needed) is always another function from the library.  Here are some (but not all) of the functions I use:

• Zero-argument functions:  X(), Y(), and Z() - The current X, Y, or Z position at which evaluation is being evaluated.
• One-argument functions: COS(Arg), SIN(Arg) - Compute the cosine or sine of Arg.
• Two-argument functions: ADD(Arg1, Arg2), MULT(Arg1, Arg2), DIV(Arg1, Arg2), and MOD(Arg2, Arg2) - Compute Arg1 + Arg2, Arg1 * Arg2, Arg1 / Arg2, or Arg1 mod Arg2.

It is really important to have at least one zero-argument function that can be selected at the maximum depth of your expression, or it would keep growing forever.  (Without a zero-argument function, each function would require at least one argument which would require at least one additional function which would require at least one more argument and so on ad infinitum).

It's easy to add new functions to the RExArt library; there is a section that discusses this later in the blog.

Building A Random Expression

Here's how a random expression is built.

Start by selecting a maximum depth for your expression; that is, how many levels of nested functions you want.  More depth yields more complexity, but it also adds (exponentially) to execution time.

maximum-depth = (whatever you want)

Once you've picked the maximum depth, the algorithm to generate the random expression is pretty straightforward if done recursively:

generate-random-expression(depth):

  if (depth = maximum-depth) 

    function = pick-a-zero-argument-function

  else 

    function = pick-any-function;

  print function + "(";

  for each argument of function

    generate-random-expression(depth + 1);

  print ")";

Note the two different methods: "pick-a-zero-argument-function", used at the deepest level of the formula, that needs randomly to select a library function that requires no arguments; and "pick-any-function" that randomly picks any function in the library.

The functions in the library are weighted so that some of them are more likely to be selected than others.  If, for instance, I want only a "little" SIN() but a "lot of" MULT(), I could weight SIN() by 1 and MULT() by 10 (making MULT() ten times more likely than SIN()).

Function Domains and Ranges

Okay, so it's not quite that easy.  Did you think it would be?

Suppose I add a SQRT(Arg) function that computes the square root of Arg.  Arg had better not be negative!  As it turns out, most functions have some expectations about the values of their arguments (domain restrictions) and, since function values are used as arguments for other functions, it means that the function results need to be restricted as well (range restrictions).

In RExArt, I require each function to generate a value in the range -1.0 to +1.0 (the range restriction).  Since the arguments for any function are other functions, the domain restriction for each argument is automatically the same range of -1.0 to +1.0.

Making the functions behave correctly can be a little tricky.  For instance:

• SQRT(Arg1) is implemented as "take the square root of the absolute value of Arg1".
• DIV(Arg1, Arg2) is implemented as "if Arg1 and Arg2 are both 0 return 0; if the absolute value of Arg1 is larger than the absolute value of Arg2, return Arg2 / Arg1; otherwise return Arg1 / Arg2".  (This both keeps the range between -1.0 and +1.0, and avoids division by 0.)
• COS(Arg) is implemented as "compute the cosine of PI * Arg".  Cosine will always return a value between -1.0 and +1.0, regardless of the argument value, but limiting the argument to -1.0 to +1.0 means that only part of the possible cosine values will ever be generated, so I multiply the argument by PI to get the full -PI to +PI argument range.)

"But," I hear you cry, "you said you evaluate the random functions at points (0, 0), (1, 0), ..., (999, 799) so doesn't this mean the X() and Y() functions will not be in the -1.0 to +1.0 range?"  Ah!  Good catch.  Well, what I really do is to map the X and Y coordinates of the canvas, which are indeed (0, 0), (1, 0), ..., to the range -1.0 to 1.0.  Thus, the coordinate (0, 0) is mapped to X() = -1.0 and Y = -1.0; the coordinate (999, 799) is mapped to X() = +1.0 and Y = +1.0; and so on.  (I do the same thing for the Z axis, by the way.)

Mapping Expression Results to Colors

A computer generally specifies colors using one of two models: RGB (red/green/blue), where each R, G, B component varies from 0 (none of the color) to 255 (all of the color); or HSV (hue/saturation/value), where the H component varies from 0 to 360, and S and V each vary from 0 to 1.

You can think of each model as a control panel with three knobs.  For RGB, there is a "red" knob (to make red brighter or dimmer), a "green" knob, and a "blue" knob; for HSV the knobs are "hue" (to select a a color), "saturation" to pick the amount of the color, and "value" to pick brightness.

I chose HSV for RExArt because the HSV "knobs" more closely modeled the way I wanted to tweak the generated images.

RExArt uses separate random expressions for the H, S, and V components.  Remember, though, that the expressions all return values from -1.0 to +1.0, so it is necessary to map the expression values to valid component values:

• H = (expression-1-result + 1.0) / 2.0 * 360.0
• S = (expression-2-result + 1.0) / 2.0
• V = (expression-3-result + 1.0) / 2.0

I added an enhancement that allows a subset of valid values to be specified for H, S, or V: maybe I only wants hues from 200 to 300, or only saturations from 0.5 to 0.9.  "Wrapping" is allowed, so you can specify a hue range from 300 to 60, meaning 300, 301, ..., 360, 0, 1, ..., 60 are possible, but 61, 62, .., 359 are not.

The Control Panel

RExArt has a control panel so that a user can change things on the fly without having to tweak program code.  Here's what is looks like:

(It's tough to read, I know.) Using the control panel, you can:

Change the Configuration

• Size.  Set the canvas size, in number of pixels wide x tall.
• Z-Axis.  Set a minimum value, a maximum value, and a step size for the Z value.  Each Z value is a separate frame in an animation.
• Colors.  Set the minimum and maximum values for hue, saturation, and value.
• Expressions.  Show, end, and create the random expressions used for generating hue, saturation, and value components.
• Expression Generation.  Control random expression generation.  You can set the maximum depth of a formula, optionally require the X(), Y(), and Z() formulas to be included somewhere in the expression, and set weights for each function in the library.
• Other.  Set the number of threads used for parallel calculation, and set the frame speed (frames per second) for animations.

Control the Display

• "New" to generate new random expressions and generate the first frame of the image.
• "Start" to generate the first frame of the image using the current settings.
• "Play" to generate an animation and display it.
• "Replay" to replay the entire animation once generated.
• "Previous" to move to the previous animation frame.
• "Next" to move to the next animation frame.
• "Pause" to play animation.
• "Load" to load the configuration from a file.
• "Save" to save this RExArt.  You can chose to:
• Save the configuration (as a JSON file)
• Save the first frame (as a BMP image)
• Save the animation (as an MP4 file)

Performance and Function Library Expansion

RExArt does a lot of computing.  For a single 1000 x 800 image with an expression maximum depth of 10, the number of functions that must be evaluated can be as large as 3 ^ 9 * 1000 * 800 = 15.7 billion functions.  (On the other hand, it can be as few as 800,000 functions for the simplest random expressions.)  Early versions of RExArt were taking a minute or so per image frame.

Fortunately, I found two ways to significant improve performance:

• Compile (at run-time) the expressions to be calculated.  This is easily an order of magnitude faster than interpreting the expressions.
• Use multiple processors if they are available.  Two processors are about twice as fast as one processor; three processors about three times as fast; etc.

With these optimizations, a frame can be generated O(10) times faster that before.

A nifty side-effect of the run-time compilation of functions is that it makes it very easy for a user (with some programming ability) to add new functions.  All of the functions, built-in and custom, are stored in a file that is loaded and compiled at run-time to build the function library.  Here, for instance, is the code for the ADD() function:

        public static double Add(double v1, double v2) {

            return (v1 + v2) / 2.0;

        }

Just remember, if you build a new function, that: a) you can assume that each argument has a value of -1.0 to +1.0 coming in, and b) that you have to return a value in the range -1.0 to +1.0 when you're done.

Loading and Saving RExArt

RExArt has the ability to save an image, its animation, and it's configuration; as well as the ability to load the configuration so that you can recreate (or change) a saved image.

The configuration, which contains all of the Control Panel configuration items, is stored in a JSON- formatted file.

The image (the first frame) is stored as a bitmap (BMP) file.

The animation is stored as a movie (MP4) file.

When you save a RExArt image, you specify a filename (and directory, if desired), but not an extension.  You then indicate which files (JSON, BMP, MP4) to save, then click the "Save" button.  RExArt saves the file(s) with the correct extensions.  (An external program, "ffmpeg.exe" is used to build the MP4 file.)

When you load a RExArt image, you actually only load the configuration (JSON) file.  The configuration values in the control panel are all set from the configuration file, and then you are ready to go.