Lens Flare — A Developer’s Diary pt 1

A lens flare is an artifact caused by reflections of a bright light source inside the complex lens assembly of a camera. There are thousands of permutations of lens flare appearance, but they can be broken down into a few component elements that are possible to parameterize.

Most of the lens flare tools I’ve used so far involved tracking a light source and using that data to determine the origin point of the flare. All of the flare’s properties are fully controlled by the artist, and occlusion has to be managed with roto mattes or a key.

Some 3d renderers, like Redshift, have a post-fx filter to create lens flares. These depend only on the intensity of rendered pixels and a few parameters, so the motion, occlusion, and evolution of the flare is achieved automatically.

In this exploration, I’d like to look at both approaches on my way to creating one or more lens flare tools for Houdini’s new image processing system, Copernicus (COPs). I’ll look into existing CG lens flare implementations and also real photographed flares and attempt to reproduce them in the OpenCL COP. First up is a public domain OpenGL flare posted at ShaderToy by mu6k:

https://www.shadertoy.com/view/4sX3Rs

This is an example of the kind of flare I’ve used in the past. It generates the light source, some god-rays, a couple of “orbs” and a partial halo. The source is quite dramatic and probably not suitable for the kind of application I have in mind, but the reflections are subtler and might be useful. At the very least, it gives me a place to start with my own. At only 120 lines of OpenGL, this should be relatively easy to reproduce.

Setting Up the Frame Buffer

When analyzing code, it’s tempting to just start at the top and try to understand everything in the order presented. That’s not often the easiest way to do it, as most programs declare a lot of constants and functions at the top that don’t make any sense until you’ve looked at the place where execution begins. So let’s start with the mainImage() function. I’ll go slowly and carefully because it’s very easy to get the OpenCL COP to crash. It’s probably also a good idea to keep Houdini’s Cache Manager open in order to purge the COP OpenCL Buffer Cache occasionally to prevent VRAM overflows. For the sake of clarity, I’m going to refer to the OpenGL ShaderToy code as ShaderToy and Houdini’s OpenCL code as OpenCL. That’s less likely to cause confusion due to the similarity in the shading language names.

We get some advantages from the OpenCL COP: We don’t need to calculate the normalized uv coordinates or correct for image aspect. All of that is handled by the built-in binding @P. So lines 106 & 107  of the ShaderToy are completely unnecessary.

The next bit of the ShaderToy sets up an odd variable: mouse. This is the location of the user’s mouse in the ShaderToy window, and the z component indicates whether or not a button is being clicked. If a mouse button is not being clicked, then the location of the flare comes from some sin waves and the time. When a button is held down, the flare moves to the mouse location. In the version of the tool where the light source location is set with a parameter, I’ll make a float2 parameter flaresrc to replace mouse, and I’ll replace iMouse.z with a parameter called animate, which will have a simple toggle in the UI.

There’s also some strangeness with the sin() functions in the animated version of the ShaderToy, so I’m going to simply use cos() for the x axis and multiply each by 0.5 to get a nice circle going when it’s animated. I can add a little chaos by tweaking @Time later, if I feel like it, but it seems likely that I’ll eventually dump the animated mode in favor of animating the flaresrc parameter itself.

The next segment of ShaderToy code includes calls to some functions that haven’t been set up yet: lensflare(), noise(), and cc(). I’m not going to tackle those immediately, but I still want to test my existing code, so I’ll render the image coordinates and a distance field for now.

OpenCL conveniently has a distance() function that does exactly what you’d expect: it returns the distance between two position vectors. I want a bright spot as my flare source, though, so I’ll invert the results, giving a radial gradient centered at flaresrc. And for good measure, I’ll add a radius parameter so I can control the size of the gradient.

It took some experimentation and a long, slow walk around the office to work out how to get the radial gradient to work the way I wanted. I knew that my radius control was supposed to scale the gradient, which means multiplication, but figuring out when to multiply fooled me for a bit. Since I knew I wanted a bright spot rather than a hole, I’d inverted dist. But when I multiplied that by radius, all I did was adjust the brightness of the spot instead of its size. Instead of multiplying the bright spot, I needed to divide the distance field by radius before inverting it. That way, the gradient shrinks as the radius control is increased, and when inverted it behaves like I want it to. But dividing by a variable is always dangerous, so I first clamped the low end to a very small value, ensuring I’d never divide by 0.

#bind layer src? val=0
#bind layer !&dst
#bind layer noise? val=0
#bind parm flaresrc float2
#bind parm animate int val=0
#bind parm radius float val = 0.01

@KERNEL
{
    float2 flaresrc = @flaresrc;
    
    if(@animate > 0)
    {
        flaresrc.x = cos(@Time)*0.5;
        flaresrc.y = sin(@Time)*0.5;
    }
    
    float radius = max(0.001f, @radius);
    
    float dist = 1.f-(distance(@P, flaresrc))/radius;
    
    float4 color = (float4)(@P.x, @P.y, dist, 1.0f);
    
    @dst.set(color);
}

This appears to be working as expected. I have a horizontal gradient in the red channel that visualizes imgcoords.x, a vertical gradient in green to visualize imgcoords.y, and a radial gradient in blue visualizing flaresrc. Of course, it is in image coordinate space, so there are negative pixel values across much of the image, but inspecting the values in the Composite Viewer shows that they’re what I expect.

The lensflare() Function

On line 116, the hotspot’s color is multiplied by the output of a function called lensflare(), which takes two vectors as input: uv and mouse.xy. As discussed above, we can use @P for uv, and mouse.xy has been replaced with flaresrc. The function is declared on line 52 of the ShaderToy.

This is a pretty long bit of code with quite a few cryptic variables. To best make sense of something like this, I like to look first at its output, which is a vec3 variable called c, declared on line 89. I presume that’s meant to be an RGB color. Each channel gets the sum of several of those variables, the whole thing is given a 30% gain up, and then what looks like the distance from the center of the screen is subtracted. The variable uvd looks like it’s doing a lot of work here, so I think my next step should be to confirm my intuition about what it is.

The nice thing about a function is that it doesn’t care about the variable names in your main program—it will change them to its own input names, so I’ll go ahead and match the function declaration from the ShaderToy, even though the variables I’m feeding into it have been renamed. I’ll go ahead and set c to black and call for the function just like the ShaderToy does. As expected, my image turns completely black because I am now multiplying RGB by 0.0f. My new color is:

float4 color = (float4)((float3)(@P.x, @P.y, dist)*lensflare(@P, flaresrc), 1.0f);

And the initial lensflare() function is:

float3 lensflare(float2 uv, float2 pos)
{
    float3 c = (float3)(0.0f);
    return c;
}

It will be easier to inspect the results of my code if I remove the gradients I’d set up before. In case I want to come back to them, though, I’ll make a copy of the line, modify it, and comment out the original. That way I can easily see what c is doing in isolation:

    //float4 color = (float4)((float3)(@P.x, @P.y, dist)*lensflare(@P, flaresrc), 1.0f);
    float4 color = (float4)((float3)lensflare(@P, flaresrc), 1.0f);   

I can test to be sure this works by changing the value of c. As expected, the output color changes, so I know all is well. Now I can start testing uvd. It’s a vec2 in ShaderToy, so I’ll need to cast float2 to float3 in order to store it in c.

    float2 uvd = uv*(length(uv));    
    float3 c = (float3)length(uvd, 0.0f);

If I put this output directly into the final color, I get a radial gradient around the center. Given that this is subtracted from c, I suspect that it’s intended to serve as a vignette, but it would be better to invert it and multiply it by c so as to avoid the possibility negative pixels. But after multiplying by 0.05, as is done in ShaderToy’s line 92, it’s effect is negligible. I’ll replace that constant with another parameter to make it more pronounced.

The very last thing that happens before we return to the main kernel is the variable f0 is added to c. That one appears to be fairly straightforward, and I suspect it’s going to turn out to be the hotspot itself. Let’s give it a try and see.

    float f0 = 1.0f/(length(uv-pos)*16.0f+1.0f);    
    float3 c = (float3)(0.0f);
    c += (float3)f0;

It’s not quite what I’d expected. Looking at the next line, I see that it gets modified by a noise() function, so I’ll bet this turns out to be the god rays. To verify that, I’ll need to start up yet another new function, and it looks like I’ll need to provide a noise field as a second input.

I ran into trouble at this point because the OpenCL COP doesn’t permit the bound inputs to be called from functions, and when you pass the bound layer, all you’re actually sending is the sampled value at the current fragment. The simplest, and probably best, solution would be to dismantle lensflare() and put it inside the kernel. And when I asked the developers for advice, that’s what they told me. But one of them did suggest looking at the generated code to see if I could get some hints, and being stubborn about my process, that’s what I did.

To see aaallll the OpenCL code, switch over to the Generated Code tab and click the Display Code button. For convenience, it’s a good idea to open this code in a separate window. You can do that with the Alt+E hotkey. Or you could copy it all into the programmer’s text editor of your choice. I prefer Sublime. It might take some long study of what’s going on inside here, but you can often get some hints from the error messages the COP provides, which will frequently tell you what the actual macro names are, or which hidden functions are failing. Sometimes it even offers advice on syntax or points out misspelled variable names.

I didn’t record everything I did while I was hacking my way through the next few revisions, so I will cut to the chase: If you hunt around in there long enough, you can find some code that defines “A structure containing metadata for a layer” and “A structure encapsulating a layer“. The layer struct is called IMX_Layer, and it contains another struct called IMX_Stat. You might also turn up some functions that look like this:

static float4
dCdyF4(const IMX_Layer* layer, float2 ixy,
       BorderType border, StorageType storage, int channels,
       const IMX_Layer* dst)
{
    float2 xy = imageToBuffer(layer->stat, ixy);
    float d = (dst->stat->buffer_to_image.y * layer->stat->image_to_buffer.y) / 2;
    float4 a = bufferSampleF4(layer, (float2)(xy.x, xy.y + d), border, storage, channels);
    float4 b = bufferSampleF4(layer, (float2)(xy.x, xy.y - d), border, storage, channels);
    return a - b;
}

This has some interesting clues. First, it told me how to send a layer to a function, giving me the clue to the data types I’d need. Second, that * indicates a pointer, which is a variable that only references a location in memory rather than containing data itself. This is useful because it means that instead of copying the layer every time it’s called in a function, we hold it in memory only once and just pass around references to its location. Since the OpenCL will run in parallel for every single pixel in the image, copying the whole layer in the kernel would be disastrous! I hadn’t really given it any thought when I started trying to do this, but if the COP had worked the way I originally wrote it, I would definitely have crashed my GPU.

Third, it gave me the -> syntax used to get information from inside the IMX_Layer struct. However, as soon as I revealed my findings to the devs, one said “I’d rather no one ever did that…. ” The only solution on offer was to inline the function in the kernel, though, so perhaps we’ll see a way of getting this done that’s a little friendlier in the future. For now, though, I can write a noise() function that can be accessed from inside lensflare():

float noise(float2 t, const IMX_Layer* layer) {
    float2 xy = imageToBuffer(layer->stat, t);
    return bufferSampleF4(layer, xy, layer->stat->border, layer->stat->storage, layer->stat->channels).x;
}

It will be a bit more complex than the ShaderToy because I’ll need to pass the noise layer into lensflare() so that it’s available to be passed to noise(). And during all of this experimentation, I wound up refining how I want to proceed with orbs and halos. Therefore, lensflare() has just become rays() and will handle only the god-rays and the hotspot. Since all of the other elements are purely additive, I think it makes a lot of sense to split each of the artifacts into its own function, and they can be added together in the kernel. It will make the code more modular and easier to understand. Here’s where I now stand:

#bind layer src? val=0
#bind layer !&dst
#bind layer noise val=0
#bind parm flaresrc float2
#bind parm godrays float val = 1.0
#bind parm hotspotbright float val = 32.0
#bind parm tint float3
#bind parm vignettestrength float val=0

float noise(float2 t, const IMX_Layer* layer) {
    float2 xy = imageToBuffer(layer->stat, t);
    return bufferSampleF4(layer, xy, layer->stat->border, layer->stat->storage, layer->stat->channels).x;
}

float3 rays(float2 uv, float2 pos, float hotspotbright, float godrays, const IMX_Layer* noiselayer ) {
    float2 main = uv-pos;
    
    float ang = atan2(main.x, main.y);
    float dist = length(main);
    dist = pow(dist, 0.1f);
    
    // f0 is the god-rays element
    float f0 = 1.0f/(length(uv-pos)*16.0f+1.0f);
     
    f0 = f0*hotspotbright + f0*(sin(noise(sin(ang*2.0f+pos.x)*4.0f - cos(ang*3.0f+pos.y), noiselayer)*16.0f)*0.1f+dist*0.1f+0.8f) * godrays;
     
    return (float3)(f0*1.3f);
}

float3 halo(float2 uv, float2 pos, float halobright, float haloradius, float halodisplace, const IMX_Layer* noiselayer) {
    float dist = length(-uv - pos);
    float3 halo = (float3)(dist);
    return max(0.f, 1.0f - halo);
}

@KERNEL
{
    float2 flaresrc = @flaresrc;

    float3 tint = @tint;
    float3 rayselement = rays(@P, flaresrc, @hotspotbright, @godrays, @noise.layer);
       
    float vignette = max(0.0f, 1.0f - length(@P*(length(@P)) * @vignettestrength));

    float4 color = (float4)((float3)(tint*vignette*(rayselement)), 1.0f);
        
    @dst.set(color);
}

That’s it for this article. In the next, I’ll tackle orbs and halos.

On a Man of Valour

This weekend, my brother was wed. As is always the case with weddings, it was a stressful time, and not everything went as planned. I won’t get into the specifics, but there were some disappointments and hard feelings. Among many families, there would been outright strife. I am prouder than I can say that such was not the case in mine. Anger, misunderstandings, and even an inability to reach consensus on essential (and I use that word in its full meaning) ideals, shall not break my family. Thank you, my loves, for your kindness to one another, even when you cannot see eye-to-eye.

But now I want to talk about Timothy. Although he’s my brother, I am sad to admit we do not know one another well. I was 18 years old and nearly ready to spread my own wings when he was born. It was long and long before we began to build a relationship upon common interests. Even now we have limited contact with one another and so haven’t been able to form the kind of bond I share with our sister. Nevertheless, I do know some things about him, and I think perhaps they’re the most important things.

Like me, Tim enjoys heroic fiction—science fiction, fantasy, roleplaying games, and comic books figure large in both of our lives. There are some who look down on such things as childish or escapist. Many others regard them as mere entertainment. It is rare to find someone who internalizes the lessons they teach and transforms them into guiding principles. My brother is one such. It is true that he is sometimes foolish and very often silly. I imagine it is easy for some to dismiss him or think poorly of him, but anyone who looks a little deeper will, I think, discover something astonishing: He really believes in kindness and in compassion. That these things are of value for their own sake, and not just for what they bring us.

Certainly, many of us think that it’s better to be kind. Some even manage to appear to be kind, as long as we’re not inconvenienced. For most of us, it’s a superficial feature of a public persona. For Timothy, however, a regard for others is a core element of his psyche. That the strong should stand up for the weak isn’t mere philosophy, but a principle he will act upon, even to the breaking of his own body. I cannot think of another person that I would describe as valorous. But I cannot hesitate to apply the word to Tim.

I know that both of my siblings look up to me. I do my best to at least appear worthy of their regard, although I know that the reality fails to measure up to what they believe about me. But this is the truth: Danica and Tim, you’re both heroes in my eyes. I admire you more than I can say.

Annic Nova, a Traveller deckplan

It’s been a while since I dropped any gaming-related content here. Anyone who’s familiar with Classic Traveller may remember the Annic Nova, a very early scenario involving a strange derelict spaceship. While it’s still possible to find the adventure in pdf format, the scan is atrocious. I’ve redrawn the deckplan as best I can in the minimalist Traveller style. And because I’m a crazy person, I did it in Fusion.

The key is available in the module, available from Drive Thru RPG (only $4), and it’s still reasonably legible. I believe sharing the deckplan alone falls inside FFE’s Fair Use Policy.

This map is Copyright © 1977 – 2022, Far Future Enterprises All Rights Reserved. Licensed for  individual use.

click for full resolution

Karma, Cryptomatte and Fusion

If all you need to know is how to render Cryptomatte from Houdini’s Karma renderer so that it can be used in Fusion, here’s the secret sauce:

In your USD Render ROP, add the flag --exrmode 0 to the end of the husk Render Command. In 19.5 and later, there’s a checkbox for Enable Legacy EXR Mode that adds the flag for you, but in earlier versions you have to put it there yourself.

Add the flag --exrmode 0 to the end of the husk command in the USD Render ROP's Render Command

If you’re using the Karma HDA, you’ll need to unlock it and dive inside to find the ROP. Right-click the Karma node and choose Allow Editing of Contents. Then double-click to dive in. The Render Command in this version is long and complex, but you can still just add the flag to the end of it:

After diving inside the Karma HDA, add the flag --exrmode 0 to the end of the husk command in the rop_usdrender's Render Command

Continue reading →

New Job with SideFX Software!

 

I’ve announced this in several places, but it occurs to me that I never updated the occasional reader of my blog. I’m now a Houdini Technical Consultant, working directly for SideFX. In a nutshell, I’ll be doing for Houdini the same thing I have been doing for Fusion, except I’ll be paid for it. I’ll therefore probably have a lot less to say about Fusion and production in general and a bit more to say about Houdini.

On the other hand, most of my teaching opportunities are going to be directly through SideFX, so I’m not sure how much I’ll be keeping up with it here. If I do anything cool on my own time, it’ll definitely show up here.

I do have a few bits and bobs left over from working in Fusion, and hopefully I’ll be able to find the time in the coming weeks to organize it, finish up some projects that I’d been procrastinating, and put it all into Reactor. I’m afraid the book is probably dead in the water at this point. Sorry. I’ll leave the draft up here.

Blackmagic Fusion’s Merge and Booleans Mathematics

This is an unfinished draft of one of the appendices to the unfinished draft of the book. I don’t remember how far I got, but I believe everything currently stated in here is at least true, if perhaps not complete.


Now children, when a Foreground and a Background love each other very much, the Foreground inverts its Alpha channel and multiplies it by the Background. Then they add their color channels together and a new composite image is born.

You can take your compositing skills up a notch with a thorough understanding of the math behind Fusion’s Merge and Channel Booleans tools. This article explains the various Operations available in the Channel Booleans node and the Apply Modes available in the Merge.

Continue reading →

New Reels for 2020

bryan-ray_compositing-reel_2020_v02 from Bryan Ray on Vimeo.

This Compositing and Lighting reel shows work from Legion, From Dusk Till Dawn, Charmed, Teen Wolf and Dog With a Blog. All clips are courtesy of Muse VFX and property of the respective productions.

The work was largely completed with Fusion, 3DS Max, Redshift, PFTrack and Houdini. There’s a little V-Ray in there, too.

The Music is “Tension 2” from Audio by The Blue Man Group.

Tapered Bezier curve for Fusion

I want a better Lightning tool for Fusion. The existing one, a fuse that was available in VFXPedia prior to the Blackmagic takeover, was never really good enough for production. As a result, any time I’ve wanted electricity effects, I’ve fallen back on After Effects. I’ve had some ideas about how to improve the Lightning Fuse, but when it came time to actually open it up and make some changes, I ran into two problems: Fusion can’t draw a tapered spline, and the BezierTo() method for the Fuse drawing API is broken. Solving one of those problems offered me the opportunity to solve both at once, so that’s what I did.

In this developer’s diary, I’ll walk through the creation of a tapered Bezier curve Fuse. All of my testing was done in Fusion 9. I have no reason to expect that the fuse won’t work in v16 or Resolve, but I haven’t tried it yet. I hope that I’ll be able to extend what I’ve done here into that Lightning Fuse, but that’s a few steps down the road. I’ll be certain to document it when I get there.Continue reading →

Fusion start-up troubleshooting

Let me lead by saying that I have no intention of being Fusion tech support. The information in this post is the only help I will offer, so don’t send me emails asking how to get Fusion running. I will steadfastly ignore them (so don’t take it personally when I don’t reply; it’s just a policy I’m using to maintain my sanity). I am interested in new solutions, though, so if you manage to overcome a start-up or crash problem that isn’t detailed here, do let me know your solution so I can disseminate it. Or better yet, post it at both the official forum and We Suck Less. You probably already know that I frequent both of those sites, so I’m almost guaranteed to see it.

Continue reading →

Toolbox UI Script for Fusion

 

Management of custom tools and scripts in Fusion can begin to become difficult as the number of assets increases. Not only does finding the specific script you need to run get harder when you have to sort through dozens of others, but discoverability becomes a problem. To counteract these issues, I built this Toolbox script to give easy access to the most useful of our assets. For TDs interested in the nuts-and-bolts, there is a detailed description of the code down below with the original version of this article. To be clear, this script doesn’t actually do anything on its own. It only provides an interface for easy access to whatever you decide to add to it.

To install the Toolbox, download this file: MuseToolBox.zip

Unzip it into your Fusion Scripts folder. It should place the file MuseToolBox.lua in the Comp subfolder, and three files in a new folder named Support: Cog.png, MuseToolBox_configurator.lua, and Toolbox_buttons.cfg

Run the script from the Script menu in Fusion.

Continue reading →