That Voronoi Fuse I documented recently is really slow, and there are quite a few features that I would like to add to it. Unfortunately, each new feature will only make it slower, so I have decided to try once again to tackle OpenCL. This time, though, I’m going to do it a little more formally by converting the algorithms detailed in Patricio Gonzalez Vivo and Jen Lowe’s The Book of Shaders (BoS).
The information in We Suck Less’ recovered VFXPedia wiki is good, but there are some holes in it, and it assumes a certain level of expertise that I, frankly, do not have. Therefore, this series of diaries is intended to be a little more rigorous for the benefit of dabblers like myself. Enough with the introduction; let’s get working!
There are quite a few bits of a Fuse that will always be present, and it’s tedious to have to set them up every time. I have therefore made a template that I can use to (relatively) quickly set up a new Fuse. For your convenience, you may download it here. So far, all of my Fuses have been Creator type tools—they make pixels from scratch instead of operating on an input image—so the template is pre-configured for that style of tool. The commented section called CONTROL SHELLS has input control templates for each control type ready to copy-and-paste into the Create function. Once the control panel has been set up to my satisfaction, I delete that section.
The template also has some commented-out lines that are used to intialize OpenCL. Those lines are located between the control shells and the Process function. If you’re planning to make an OCL tool (as I am), uncomment those lines.
With that template in hand, let’s walk through it in order to understand what all the pieces do and how they work together. This, by the way, will be very similar to information already presented in the Voronoi article, but I am going to try to be more thorough here
The first three lines:
--[[
]]--
This is a Lua comment block. Everything between these lines will be comments—not executable code—even if there are line breaks. I put a description of the Fuse here, along with authorship and version information.
Registering the tool in Fusion
FuRegisterClass("Fusename", CT_SourceTool, {
This section of the Fuse gives Fusion information about the tool. “Fusename” is the name that Fusion uses for the tool internally. CT_SourceTool tells Fusion that this tool creates its own pixels, which unlocks a few default controls (see below). Note the curly brace at the end of the line, which indicates the beginning of a Lua table. Everything between the braces is a table, with multiple key/value pairs.
REGS_Category = "",
The Category is the folder where the tool will appear in the Add Tool dialog or the Tools menu.
REGS_Name = "Fusename",
If for some reason you don’t want the displayed name to be the same as Fusion’s internal name, you can set a different one here. This line is optional.
REGS_OpIconString = "",
The three-letter code that can be used to search for the tool in the alt+space dialog. Although more than one tool can have the same code, try to select a unique one to reduce collisions with other tools.
REGS_OpDescription = "Description of tool",
Should be fairly self-explanatory.
REGS_Company = "Muse VFX",
Obviously, if you’re using this template for your own purposes, you should change or delete this line. It’s not required, and I’m not sure if it has any actual function other than identifying the tool to people who look at the code.
REGS_URL = "http://www.musevfx.com",
Again, I’m not sure if this URL actually does anything. It doesn’t call the help page; I know that much. And unless you happen to work for Muse, you should probably change it for your own Fuses.
Now we get to those default controls I mentioned earlier. If you want access to the standard Image tab of a SourceTool, this is where you’d turn it on.
REG_Source_GlobalCtrls = true,
Shows a Range control that determines which frames the tool is valid for. Also lets you set the Process Mode to output fields or full frame images. I am not sure the purpose of this line, since setting it to false or deleting it will cause Fusion to crash as soon as you try to view the tool.
REG_Source_SizeCtrls = true,
Activates the Frame Format controls in the Image tab and creates the global variables Width and Height.
REG_Source_AspectCtrls = true,
Adds the Pixel Aspect controls on the Image tab. I assume it also creates some global variables to go along with them.
REG_Source_DepthCtrls = true,
Adds the color depth, color space, and gamma space controls in the Image tab.
REG_TimeVariant = true,
Allows iterative development of the image. I’m not 100% sure I know what’s going on with this parameter, but I know that if it’s turned off, my random walk algorithm stops working. I imagine that if you’re doing any kind of procedural animation with the Fuse, you’ll want this to be on. I suspect, though, that leaving it on unnecessarily will make the Fuse slower.
}) -- End of RegisterClass
The end curly brace tells us that we’ve reached the end of the table. The end parenthesis closes out the FuRegisterClass. Also, I should point out the commas at the end of each line in that table. You can declare a table on a single line, like this:
table = { 1, 6, 12, apples, oranges, red }
In that format, it’s easy to see that the commas delimit each entry. When you’re using a columnar format like we did above, it’s easy to forget them. They’re still necessary, though.
OCL kernel declarations
The next section is:
clsource = [[
__kernel
void clfunction()
{
}
]]
This is where any OpenCL code will go. In Lua, anything between [[ ]] will be treated as a string, but it will ignore escape sequences and newline codes. This makes it possible to place program code inside the braces without having to worry about special characters, except for “]]”. The entire OCL code is packaged as a string into the variable clsource, which makes it easy to transfer into the OCL manager elsewhere. If the OCL manager isn’t invoked, then this code will do nothing except take up a tiny bit of memory. I usually leave it as it is, but if you want to streamline a Fuse that isn’t using OCL, you could delete it or comment it out. We’ll come back to this in the next lesson when we start building our first OCL Fuse. You could also put the OCL code into a separate file, in which case this section would not be needed.
The Create() Function
function Create()
When the tool is added to the Flow, Fusion will call the Create() function. This is where you set up your image inputs and outputs in addition to the all of the inputs in the control panel. We’ll go through a few of the different types of controls you can put here in a bit, but let’s start with the one that’s pre-configured:
OutImage = self:AddOutput("Output", "Output", {
LINKID_DataType = "Image",
LINK_Main = 1,
})
OutImage is a global variable that can be used to pass information into and out of the main part of the Fuse. “self” is a special object that points at the Fuse itself. The colon “:” indicates that a method carried by the object class is being invoked. A method is a function that is stored in an object class. Since the object’s properties are known, calling a method like this is easier and faster than passing properties of an object into a standalone function. In this case, the AddOutput() method requires only three arguments: The scriptable name of the output, the human-readable name, and a list of parameters contained within curly braces.
LINKID_DataType tells Fusion what kind of data the output will send, and LINK_Main = 1 tells Fusion that this is the primary output of the node. The last line again closes with a curly brace, indicating the end of the list, and a parenthesis, indicating the end of the arguments.
end -- end of Create
Lua’s control structures do not use brackets like C or indentation like Python. Instead, the end keyword is used to close a block. Since it’s not uncommon to have several “end” lines in a row closing nested if and for statements, and those lines may be far removed from whatever statement they’re closing, I like to give myself a little comment to remind myself of what exactly is ending. If I’ve missed one, it makes it easier to find where it should go.
Inputs and Controls
Okay, the next bit is pretty long, so I won’t bother quoting all of it. The Control Shells are code that you can copy-and-paste into the Create() function in order to add sliders and widgets to your Fuse. Most of them can be used as-is, but there are a few lines you will need to adjust, and a couple more you may need to adjust.
inSlider = self:AddInput("Slider Label", "SliderName", {
LINKID_DataType = "Number",
-- INP_Integer = true,
INPID_InputControl = "SliderControl",
-- IC_ControlPage = 0,
LINKS_Name = "Slider",
INP_MinScale = 0,
INP_MaxScale = 1,
-- ICD_Center = 1,
-- IC_Steps = 21,
INP_Default = 1,
})
The structure is the same as the Output we examined before, but this input has many more parameters in the third argument. You will certainly want to change the global variable name “inSlider” to something more descriptive. “inBrightness”, for instance, if it’s meant to be a Brightness control. The global variable will be linked to a local variable later on, and it’s customary to give the local the same name without the “in” prefix. We’ll get to that.
The Output had a DataType of “Image”, but most controls will use “Number” instead. A few can take other types, and the LabelControl doesn’t have a DataType at all (it doesn’t contain any data—it’s purely a display item). You will almost never need to change the DataType; I honestly can’t think of an example where you would. “INP_Integer” is commented out by default. This line will set the control to accept only whole numbers. “INPID_InputControl” determines how the control appears in the panel. The Slider is the most common.
“IC_ControlPage” determines which tab the control appears on. 0 puts it on the default “Controls” tab. If you want to name the tab, use ICS_ControlPage = “Tab Name”, instead (notice the “S” that indicates a string). If you want the control to appear above the tabs list, use IC_ControlPage = -1,. “LINKS_Name” is the label that appears on the control. This is distinct from the name that appears in the status bar when you mouse-over the control.
INP_MinScale and INP_MaxScale set the highest and lowest values the slider will reach by dragging it. These values can be overridden by typing out-of-range values in the numeric entry box. If you need to enforce the range, you can add INP_MinAllowed and INP_MaxAllowed. You can have both Scale and Allowed parameters on the same control.
ICD_Center can make a control non-linear. For instance, if your range goes from 0-1 and you set ICD_Center = 0.1, then the left half of the slider will cover only 1/10 of the numeric range, and the right-half will cover the remaining 9/10. The control will be linear to either side of this threshold, though; It won’t make an actual logarithmic control. This feature is not often useful, so I have commented it out by default.
IC_Steps lets you control the behavior of the left- and right-arrows when adjusting a control. The value determines the number of arrow presses it takes to traverse the entire control from end-to-end. It doesn’t care about the resulting value, only about the actual distance covered by the slider. Once again, this uncommon feature is commented out.
INP_Default sets the default value of the control as well as where the reset dot appears.
Most of the controls have parameters similar to these. From here on, I’ll just make a note of any control type that differs or adds something new.
Next up is the MixSlider. It is functionally identical to the SliderControl, adding only an extra label to the right end of its range.
The RangeControl comes in two parts, and it has a parameter IC_ControlGroup to connect the two pieces together. Every grouped control needs a unique ControlGroup number so that Fusion can associate the different pieces with the same on-screen widget. The IC_ControlID determines which part of the control you’re dealing with: 0 for the left end and 1 for the right. RangeControls can also have a label in the middle, set with the RNGCS_Midname parameter. Finally, you can add a little green bar at each end of the range with RNGCD_LowOuterLength and RNGCD_HighOuterLength. I am not sure if this feature is working as intended; I can’t fathom what it might be for.
The CheckboxControl has a parameter CBC_TriState that configures it to produce three possible outputs: 0, 1, and -1. If this option is not enabled it outputs only 0 or 1. It also shows the optional parameter ICD_Width, which can be set to a percentage of the control panel width that the control will occupy. In fact, any control can use this parameter, but it’s most useful with checkboxes because they don’t take up much space to begin with.
MultiButton has an option to turn off its label with MBTNC_ShowName. You might want to do this if the buttons themselves adequately describe the control’s function. Each button added to a MultiButtonControl goes in a list with two items: the button’s name and an optional width. { MBTNC_AddButton = “Go!”, } will place a button with the label “Go!” in the control panel. { MBTNC_AddButton = “Stop?”, MBTNCD_ButtonWidth = 0.25 } adds one labelled “Stop?” that occupies 25% of the panel’s width. Every button added after a button with the ButtonWidth attribute will get the same width. MBTNC_StretchToFit causes the buttons to adjust their size to fill the available horizontal width. If this is set to false, then the buttons will be very small, probably too small for their labels. Multibuttons output an integer value from 0 to one less than the total number of buttons. That is, a control with four buttons can output values of 0, 1, 2, or 3.
ColorControls, like RangeControls, are grouped. In this case, there are four controls, and one of them can have the parameter CLRC_ShowWheel, which determines whether or not the HSV color wheel is revealed by default. ColorControls can be used for more than just the RGB channels. I am not sure which ControlID values correspond with which channels other than 0, 1, 2, and 3 being associated with R, G, B, and A, respectively.
The ColorWheelControl doesn’t introduce anything new, so I’ll limit myself to pointing out that it’s the same color wheel as is available in a ColorCorrect node.
The TextEditControl takes, unsurprisingly, the “Text” DataType. It has an optional parameter TEC_ReadOnly, which can be used to prevent the user from changing the text. This is useful if you want to display instructions for use right in the tool. TEC_Lines makes the text box bigger, and INPS_DefaultText pre-loads the box with some text.
The LabelControl has no actual function—it is purely an organizational aide for the control panel. LBLC_DropDownButton = true converts the label into the header for nested controls—you’ll get a little twirl-down button, and the next few controls will appear in that nest. The number of controls to be included is determined by the LBLC_NumInputs value.
Finally, we get to the Flow inputs. The only things remaining to note are that giving an input the name EffectMask will give it a blue input arrow, and uncommenting INPID_InputControl will provide a text field in the Control Panel where the input node can be linked, like Wireless Link node has.
The OpenCL Environment
-- function OnAddToFlow()
-- mgr = OCLManager()
-- if mgr then
-- local path = string.sub(debug.getinfo(1).source, 2)
-- path = string.gsub(path, "%.[fF][uU][sS][eE]$", "%.cl")
-- prog = mgr:BuildCachedProgram("Fuse.BoSOCL", path, clsource)
-- end
-- end
The OnAddToFlow() function, as might be expected, executes when the Fuse is added to the Flow. The variable mgr becomes a handle to the OpenCL Manager process, which is responsible for the communication between Fusion and OCL. The path variable gets a path to wherever Fusion stores its debugging data, and prog becomes a handle for the compiled OCL code. Notice that it gets the name of the Fuse, the path, and the clsource string, in which all of the OCL functions were stored.
-- function OnRemoveFromFlow()
-- prog = nil
-- mgr = nil
-- end
When the Fuse is removed from the Flow, the OCL program is cleared from memory, and then the Manager itself is destroyed.
The Process() Function
At this point, we start in on the actual working code for the Fuse. Aside from the clsource section, everything up to now has just been setting the table. Now we start serving up the meal.
function Process(req)
Once the tool has been created, Fusion calls for the Process() function, passing a single argument: the Request object, which is a table containing all kinds of information about the environment in Fusion, such as whether Proxy mode is on, the current frame, the Region of Interest, and so forth.
Most everything that the Fuse actually does happens in the confines of Process(). Additional functions can be declared outside of Process(), including the aforementioned clsource functions, but the main program logic and flow happens here.
-- Standard for Creator tools.
local realwidth = Width;
local realheight = Height;
-- We'll handle proxy ourselves
Width = Width / Scale
Height = Height / Scale
Scale = 1
-- Attributes for new images
local imgattrs = {
IMG_Document = self.Comp,
{ IMG_Channel = "Red", },
{ IMG_Channel = "Green", },
{ IMG_Channel = "Blue", },
{ IMG_Channel = "Alpha", },
IMG_Width = Width,
IMG_Height = Height,
IMG_XScale = XAspect,
IMG_YScale = YAspect,
IMAT_OriginalWidth = realwidth,
IMAT_OriginalHeight = realheight,
IMG_Quality = not req:IsQuick(),
IMG_MotionBlurQuality = not req:IsNoMotionBlur(), }
if not req:IsStampOnly() then
imgattrs.IMG_ProxyScale = 1
end
if SourceDepth ~= 0 then
imgattrs.IMG_Depth = SourceDepth
end
This long block sets up the attributes of the images we’ll be dealing with. The only thing that you might want to change from time to time is the channels available to images. This template includes only RGBA, but it’s a simple enough matter to add more.
local img = Image(imgattrs)
The img variable is declared, holding an image with standard attributes as defined above. This is the initial working image. In a regular tool, it would probably receive whatever is in the inImage input. In that case, the imgattrs declaration wouldn’t be necessary since the input would already have all those attributes, and you could declare a new image using IMG_Like. That would look like this:
local newimg = Image({IMG_Like = img})
local out = Image(imgattrs)
OutImage:Set(req, out)
We make a variable out to hold the output image. Once we’re completely done with whatever processing we’re performing, the result goes into this image. Then we use the Set() method on OutImage (which we declared way back in the Create() function), passing it the Request and the output image.
end
And this final end statement closes the Process() function.
I know, it’s a lot of information, and we haven’t even written a functioning Fuse yet. There are also more pieces that could be added. A PreCalc() or NotifyChanged() function might be necessary for some tools. In the next article, we’ll look at the simplest of OpenCL programs and learn how to build the interface between OCL and the rest of the Fuse.
