Intro to Shaders
Written in November 2017 by Alejandro Hitti, a video game programmer and designer from Venezuela.
Shaders are often used to create beautiful graphical effects in games. They are also among the most advanced features offered by GameMaker Studio 2, but I will try to explain everything as simply as possible in this post. To follow along, you do not need any previous knowledge of these features, but it is necessary that you have a basic understanding of programming and know your way around GameMaker Studio 2.
- What Is a Shader?
- Variable Qualifiers in Shaders: Attribute, Varying, and Uniform
- Vectors in GLSL
- Swizzling in Vectors
- De-Constructing the Default Pass-Through Shader
What Is a Shader?
Initially created to provide shading for lighting (hence the name), they are now used to produce a huge variety of effects. Shader code is similar to regular code, but it is (almost always) run by the GPU (your graphics card), not the CPU (your processor). This difference comes with its own set of rules and limitations, but we'll cover those later.
Each shader is made up of two separate scripts: a vertex shader and a fragment shader (also referred to as pixel shader). When I first started learning about shaders, I had a tough time getting my head around how they work. However, I came up with an analogy that made it much simpler for me to think about them.
Let's start with the vertex shader. Each sprite is formed by a rectangle, but computers like to draw triangles, so those rectangles are split into two triangles. This leaves us with six vertices per sprite, but two of those are the same one, so we should only worry about four. Now, imagine we have a for loop that goes over every vertex and runs the code inside the vertex shader for each. This allows us to change the vertex position and color before passing it over to the fragment shader since the vertex shader is run earlier.
Here's how that would look:
For the fragment shader, you can imagine the same loop as before, but this time it goes over every single pixel in your sprite, giving you information such as location and color of that pixel. In your fragment shader code, you perform operations and calculations to determine the color of that pixel to get the effect you want. For example, if you want a shader to make your sprite be black and white, then you'd calculate which shade of grey each pixel needs to be to create the effect.
It would look something like this:
Shader code is usually run by the GPU thanks to its efficiency. Modern CPUs typically have between two to eight cores. Each core can perform one task at a time, so by taking advantage of multiple cores, we can perform that many tasks simultaneously. In contrast, modern GPUs can perform thousands, and even tens of thousands, of tasks running at the same time. This is helpful for shaders because we can run the shader code of thousands of pixels concurrently. The limitation is that we only have access to the initial state of the sprite, so we don't know about any modifications done to other pixels since we can't know for sure the code has run on them yet.
GameMaker Studio 2 allows users to write shaders in GLSL (OpenGL Shader Language), HLSL (High-level Shader Language, used when working with DirectX), and GLSL ES (a subset of GLSL which is common in mobile devices). In this tutorial, I will use GLSL ES as my shader language since it's the one that provides the best portability across systems. The math and techniques should be similar between all three languages, save for a few syntax differences here and there.
Variable Qualifiers in Shaders: Attribute, Varying, and Uniform
If you have created a shader in GameMaker Studio 2, you might have noticed these words in the default pass-through shader. These qualifiers help the shader understand the purpose and scope of each variable.
-
Attribute: Variables passed in by OpenGL to the vertex shader. These can change per vertex and are read-only. These include information such as vertex position, texture coordinates, vertex color, and vertex normal.
-
Varying: Variables used to pass data between the vertex and fragment shaders. These are available for writing in the vertex shader, but are read-only in the fragment shader.
-
Uniform: Variables that change per object and are passed by the user to the shader. These can be used in both the vertex and fragment shaders, but are read-only.
Vectors in GLSL
Vectors are very important when working with shaders. That is why they are implemented as a base type in GLSL. If you are unfamiliar with them, they are a mathematical term represented as a matrix with only one column. In programming, we usually represent them as an array where the number of components corresponds to the dimension. Two and three-dimensional vectors are often used for positions, texture coordinates, or colors without an alpha channel, while four-dimensional ones are used for colors with an alpha channel. We can also specify if they hold booleans, integers, or floating point values. The syntax to declare a vector is this:
vec2 firstVec; // Two-dimensional vector of floats
vec3 secondVec; // Three-dimensional vector of floats
vec4 thirdVec; // Four-dimensional vector of floats
bvec3 boolVec; // Three-dimensional vector of booleans
ivec4 intVec; // Four-dimensional vector of booleans
To initialize them, we can use the constructor to create the vector. You need to provide the same number of values as the length of the vector, but you can mix and match scalars and smaller vectors to reach the target length. Here are some examples of this:
vec2 firstVec = vec2(2.0, 1.0);
vec4 secondVec = vec4(1.0, firstVec, 0.0); // 2 scalars and a vec2 form a vec4
vec3 thirdVec = vec3(secondVec.x, firstVec); // 1 component of a vec4 plus a vec2 form a vec3
We can also assign them another vector of the same length (or swizzle the vector until it has the proper length, more on that in the next section):
vec3 firstVec;
vec3 secondVec = firstVec;
vec4 thirdVec = secondVec.xyz;
vec2 fourthVec = thirdVec.zx;
Swizzling in Vectors
When accessing vector components in GLSL, we have a few options. The most basic one is to treat the vector as an array and access the components using square brackets, like this:
vec4 myVec;
myVec[0] = 1.0;
myVec[1] = 0.0;
myVec[2] = 2.0;
myVec[3] = 1.0;
However, there is another way to access the components with the following syntax:
vec4 myVec;
myVec.x = 1.0;
myVec.y = 2.0;
This uses the component names inside the vector to access them. You can use x, y, z, or w, to get the first, second, third, or fourth components, respectively. We refer to this method as swizzling because the following syntax is also valid:
vec4 firstVec;
vec3 secondVec = firstVec.xyz;
vec2 thirdVec = secondVec.zy;
vec4 fourthVec = thirdVec.yxxy;
As you can see, we can use any combination of up to four letters to create a vector of that length. We cannot attempt to access a component that would be out of bounds (for example, trying to access w in secondVec or thirdVec, since they don't have a fourth component). Also, we can repeat letters and use them in any order, as long as the variable it's being assigned to is the same size as the number of letters used.
Swizzling also works on l-values:
vec4 myVec;
myVec.wxyz = vec4(1.0, 2.0, 3.0, 4.0); // Adds the values in reverse order
myVec.zx = vec3(3.0, 5.0); // Sets the 3rd component to be 3.0 and the first to be 5.0
For obvious reasons, when using swizzle to set component values, you can't use the same component twice. For example, the below is not valid as you are trying to set the same component to two different values:
myVec.xx = vec2(2.0, 3.0);
Lastly, we have been using xyzw as our swizzle mask, which is usually the case when dealing with positions. There are two more sets of masks you can use: rgba (used for colors), or stpq (used for texture coordinates). There is no difference between this, and we only use them to make our code clearer to what the vector represents in that instance. Also, we can't combine swizzle masks in the same operation, so this is invalid:
myVec = otherVec.ybp;
Those were a lot of definitions and information, but knowing these things is necessary to understand shaders themselves. Now, let's move forward with our first shader.
De-Constructing the Default Pass-Through Shader
When you create a shader in GameMaker Studio 2, it will open two files for you: a vertex shader (.vsh) and a fragment shader (.fsh). This is the most basic shader you can make, which takes a sprite, reads the texture, and colors each pixel with that color. If you specify vertex colors when drawing, those colors will blend with the texture.
Let's go through the code and analyze it, starting with the vertex shader.
// Passthrough Vertex Shader
attribute vec3 in_Position; // (x,y,z)
//attribute vec3 in_Normal; // (x,y,z) unused in this shader.
attribute vec4 in_Colour; // (r,g,b,a)
attribute vec2 in_TextureCoord; // (u,v)
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
void main()
{
vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
v_vColour = in_Colour;
v_vTexcoord = in_TextureCoord;
}
Outside of the main function, we see some variable declarations and their qualifiers. The attributes are given to us by GM. The varying ones are created by the user to pass that information over to the fragment shader. Inside the main function, we have the calculations to find the screen position of the vertex.
First, we create a vec4 and initialize it with the components of the position, adding one as the fourth component. In linear algebra, the convention is that we add a one to the fourth component if the vector is representing a point, or a zero if it represents an actual vector. We need to add this fourth component to multiply it with the MATRIX_WORLD_VIEW_PROJECTION matrix, which is a 4x4 matrix. This multiplication will project the world position of the vertex into screen coordinates. Then, we pass the vertex color and texture coordinate to the fragment shader through our varying variables. This shader should be left alone if you are not planning to play with vertex positions. For this tutorial, I will use it by default, because our effects are created using the fragment shader.
Let's take a quick look at the fragment shader:
// Passthrough Fragment Shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
void main()
{
gl_FragColor = v_vColour * texture2D( gm_BaseTexture, v_vTexcoord );
}
As explained before, the idea behind a fragment shader is to return the color of the current pixel. This is done by assigning the variable gl_FragColor the final color value. The texture2D function takes a texture and a vec2 with the UV coordinates you want to check in that texture, which returns a vec4 with the color. In the pass through shader, all we are doing is grabbing the color of the texture in the coordinate of this pixel and multiplying it by the color of the vertex associated with this pixel.
Now that we have our first shader, all we have to do to test it is create an object and assign it a sprite. In the Draw Event, you set the shader like this:
// Draw Event
shader_set(shdrColorOverlay);
draw_self();
shader_reset();
Every draw call we make between shader_set and shader_reset will have the shader applied to it. Here, we are drawing the object sprite with our passthrough shader.
As you might have guessed, we are not changing anything.
Let's try something a bit more interesting now.
Color Overlay Shader
Create a new shader and leave the vertex shader as-is. However, on the fragment shader, let's set the gl_FragColor to be red. The code should look like this:
// Color Overlay Fragment Shader
void main()
{
gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
}
Not exactly what we expected. What we need to remember is that every sprite is ultimately a square, so unless we consider transparency, this is the result we'll get. Remember that texture2D function? We will use it to grab the color at the pixel we are working on. The return value is a vec4, where the components are the red, green, blue, and alpha channels, in that order. We can access the alpha channel either by putting a period followed by an a or a w after the variable name. This corresponds to RGBA and XYZW, respectively.
Here's the updated code:
// Color Overlay Fragment Shader
varying vec2 v_vTexcoord;
void main()
{
vec4 texColor = texture2D(gm_BaseTexture, v_vTexcoord);
gl_FragColor = vec4(1.0, 0.0, 0.0, texColor.a);
}
We are now assigning a new vec4 to gl_FragColor, where the red channel is maxed, the green, and blue channels are zero, and the alpha channel is the same as the original texture. The output looks like this:
Now that's what we were after! We have replaced the color of every pixel with red, but have kept the alpha channel intact.
Having to change the shader each time we want to use a different color isn't a good idea, especially since we'd need to have a separate shader for each color we want. Instead, we will pass the color information to the shader using a uniform. To do this, we first need to get a pointer to the uniform. We will do this in the create event of our object that has the sprite by adding:
// Create Event
_uniColor = shader_get_uniform(shdrColorOverlay, "u_color");
_color = [1.0, 1.0, 0.0, 1.0];
All we need to do is call shader_get_uniform to get a pointer to the uniform. The parameters we need to pass are the shader name (without quotation because we want to pass the ID that GameMaker generates for us) and the name of the uniform variable inside of the shader, this time as a string. This name needs to match exactly the one inside the shader code for it to work. I also added a color variable so we can change it at runtime and have it remember our changes.
Now the code in our draw event will change slightly to pass the uniform variable.
// Draw Event
shader_set(shdrColorOverlay);
shader_set_uniform_f_array(_uniColor, _color);
draw_self();
shader_reset();
It's the same code as before, but before we draw anything, we need to pass all the uniforms values to the shader. In this case, we are passing the color as an array of floats. As for the shader, we will change it to include the shader and use it, so it becomes:
// Color Overlay Fragment Shader
varying vec2 v_vTexcoord;
uniform vec4 u_color;
void main()
{
vec4 texColor = texture2D(gm_BaseTexture, v_vTexcoord);
gl_FragColor = vec4(u_color.rgb, texColor.a);
}
We declare a variable with the same name as in the create shader (u_color) and we pass it as the first three components of the gl_FragColor vector, taking advantage of swizzling. If we compile again, we should see this:
Now the shader is much more useful and reusable. It's up to you to add more functionality if you need it to set the color (using the variable _color) during runtime.
Let's move onto another shader, which was actually the first shader I ever made.
Black and White Shader
This was a fun shader to figure out when I was still learning. The concept is simple: get every pixel and assign it a shade of gray. When using RGB, if all three components are the same value, then we have a tone of gray. The naive approach I took initially was to add all three color channels (red, green, and blue) and divide it by three. Then, I assigned that value to all three channels, creating a tone of gray. Here's what that fragment shader looks like:
// Black and white fragment shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
void main()
{
vec4 texColor = texture2D(gm_BaseTexture, v_vTexcoord);
float gray = (texColor.r + texColor.g + texColor.b) / 3.0;
gl_FragColor = v_vColour * vec4(gray, gray, gray, texColor.a);
}
One other thing you might have noticed is that in the gl_FragColor I'm multiplying my vec4 with something called v_vColour. This is a variable passed by the vertex shader which tells us the color of the vertex associated with this pixel. It's always a good idea to multiply your final calculated color with the vertex color. In most cases, it won't do anything, but if you changed the vertex color in GML, this will reflect that (by using functions such as draw_sprite_ext or draw_sprite_general).
As for the draw event, it's quite simple since we don't have a uniform to pass in:
// Draw Event
shader_set(shdrBlackAndWhite);
draw_self();
shader_reset();
Let's compile and see what we get.
Later on, I found a solution that is more "correct." Instead of adding the components and dividing by three, we multiply each component by the standard NTSC values for black and white. Here's the modified fragment shader code:
// Black and white fragment shader
varying vec2 v_vTexcoord;
void main()
{
vec4 texColor = texture2D(gm_BaseTexture, v_vTexcoord);
float gray = dot(texColor.rgb, vec3(0.299, 0.587, 0.114));
gl_FragColor = vec4(gray, gray, gray, texColor.a);
}
We use the dot product as a shorthand for multiplying each component of texColor with the correct weights and then add them together. If you are unfamiliar with the dot product, this is essentially what's happening:
float gray = (texColor.r * 0.299) + (texColor.g * 0.587) + (texColor.b * 0.114);
In the end, it looks very similar, but it's technically more correct.
Rainbow Shader
I will start with the basics and add functionality gradually since this shader is highly customizable. I may also go a bit faster on this one, since there's a lot to cover, so feel free to go back to previous sections if you feel lost.
The first thing we want to do is color pixels with every hue, depending on the pixel's horizontal position. The way to do this is to set the x position to be the hue and then convert from HSV format (hue, saturation, brightness) to RGB format (red, green, and blue). For this, we will need to write a helper function in our fragment shader that takes HSV values and returns an RGB vector. I will use a function I found online which does this without the need for any if-statements. Whenever possible, avoid using conditionals in shader code, as branching makes shaders very slow.
Here's what the shader looks like at this stage:
// Fragment Shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
vec3 hsv2rgb(vec3 c)
{
vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}
void main()
{
vec3 col = vec3(v_vTexcoord.x, 1.0, 1.0);
float alpha = texture2D(gm_BaseTexture, v_vTexcoord).a;
gl_FragColor = v_vColour * vec4(hsv2rgb(col), alpha);
}
First, there's our hsv2rgb function, which takes a vec3 with our HSV color and returns another vec3 with our RGB conversion. In the main function, we start by creating our HSV color, where the hue is our x position, and we'll leave the saturation and brightness as 1.0 for now. Then, we get the alpha from the texture so it only colors our sprite character and not the entire sprite rectangle (as we did before). Lastly, we set our Fragment color to be our HSV color converted to RGB with the alpha, multiplied by the vertex color (good practice to do this always).
As for our draw code, it is trivial at the moment:
// Draw Event
shader_set(shdrRainbow);
draw_self();
shader_reset();
Let's check out what we get:
We are close to what we want, but there's an issue: we are not seeing all the colors at once in every sprite of the animation, but the colors seem to change. The reason is that we assumed that v_vTexcoord gave us the coordinates of the sprite, starting at the top-left corner (0,0) and ending in the bottom right corner (1,1), which is standard in shaders. This is due to how GameMaker works internally. For optimization, GameMaker stuffs as many textures together as it can fit in what is called a texture page. By default, texture pages are 2048x2048, but this can be changed.
Because of that, this is how our texture actually looks like:
As explained above, v_vTexcoord gives us the absolute coordinates of the sprite within this entire texture page, but what we want is a value from 0.0 to 1.0 that only covers our current sprite. This process is called normalizing (getting a value and translating it to a 0 to 1 range). To normalize our horizontal values, we need to know the values of x0 and x1 in the picture above. Luckily, GameMaker has a function that gives us the location of every corner in our sprite within the texture page. First, we need to go to the create event and create a uniform to pass this data over to the shader:
// Create Event
_uniUV = shader_get_uniform(shdrRainbow, "u_uv");
And we modify the draw event to get the values and then pass them to the shader:
// Draw Event
shader_set(shdrRainbow);
var uv = sprite_get_uvs(sprite_index, image_index);
shader_set_uniform_f(_uniUV, uv[0], uv[2]);
draw_self();
shader_reset();
The function sprite_get_uvs takes a sprite and an index, and it returns an array with tons of information, such as the coordinates for each corner, how many pixels were cropped to optimize it, etc. We are interested in two of those values: the left and right coordinates of the sprite, which are stores in uv[0] and uv[2] respectively. We will pass those two values to the shader and draw ourselves. In the fragment shader, we will use those values now to calculate the normalized horizontal position like this:
// Fragment Shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform vec2 u_uv;
vec3 hsv2rgb(vec3 c)
{
vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}
void main()
{
float pos = (v_vTexcoord.x - u_uv[0]) / (u_uv[1] - u_uv[0]);
vec3 col = vec3(pos, 1.0, 1.0);
float alpha = texture2D(gm_BaseTexture, v_vTexcoord).a;
gl_FragColor = v_vColour * vec4(hsv2rgb(col), alpha);
}
Remember to add the uniform variable at the top of the file with the same name we used in the create event. Next, we will calculate the normalized horizontal position by translating our current x coordinate to the origin (v_vTexcoord.x - u_uv[0]) and then dividing it by the width of the sprite to make the range from 0 to 1 (u_uv[1] - u_uv[0]).
The result is:
There we go! This is exactly what we wanted. We can see every color in the spectrum inside our sprite.
Let's have more fun with this shader. What if we added an offset to the colors based around time to produce movement? To do this, we will need two extra variables: speed and time. We also need two more uniforms, one for each of the new variables, so the create event becomes:
// Create Event
_uniUV = shader_get_uniform(shdrRainbow, "u_uv");
_uniTime = shader_get_uniform(shdrRainbow, "u_time");
_uniSpeed = shader_get_uniform(shdrRainbow, "u_speed");
_time = 0;
_speed = 1.0;
We also need to increase the time every frame, so in the step event we add:
// Step Event
_time += 1 / room_speed;
Excellent! Let's go to the draw event to send these uniforms:
// Draw Event
shader_set(shdrRainbow);
var uv = sprite_get_uvs(sprite_index, image_index);
shader_set_uniform_f(_uniUV, uv[0], uv[2]);
shader_set_uniform_f(_uniSpeed, _speed);
shader_set_uniform_f(_uniTime, _time);
draw_self();
shader_reset();
Let's get back to our shader to actually use these variables now. What we will do is multiply the speed with the time and add it to the position, like so:
// Fragment Shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform vec2 u_uv;
uniform float u_speed;
uniform float u_time;
vec3 hsv2rgb(vec3 c)
{
vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}
void main()
{
float pos = (v_vTexcoord.x - u_uv[0]) / (u_uv[1] - u_uv[0]);
vec3 col = vec3((u_time * u_speed) + pos, 1.0, 1.0);
float alpha = texture2D(gm_BaseTexture, v_vTexcoord).a;
gl_FragColor = v_vColour * vec4(hsv2rgb(col), alpha);
}
If you did everything correctly, you should be seeing something like this:
To finish this shader, we will add a few more uniforms to customize it even further. The first two are trivial, which are saturation and brightness. The next one I called section and its function is to allow the user to pass a number between zero and one to determine what percentage of the entire spectrum we see at a time. Lastly, I will add a variable called mix, which will specify how much we want to mix our shader color with the original texture color (1.0 is all rainbow, 0.0 is all texture). As always, let's start by adding the variables to the create event:
// Create Event
_uniUV = shader_get_uniform(shdrRainbow, "u_uv");
_uniTime = shader_get_uniform(shdrRainbow, "u_time");
_uniSpeed = shader_get_uniform(shdrRainbow, "u_speed");
_uniSection = shader_get_uniform(shdrRainbow, "u_section");
_uniSaturation = shader_get_uniform(shdrRainbow, "u_saturation");
_uniBrightness = shader_get_uniform(shdrRainbow, "u_brightness");
_uniMix = shader_get_uniform(shdrRainbow, "u_mix");
_time = 0;
_speed = 1.0;
_section = 0.5;
_saturation = 0.7;
_brightness = 0.8;
_mix = 0.5;
Our draw event changes to include these uniforms like this:
// Draw Event
shader_set(shdrRainbow);
var uv = sprite_get_uvs(sprite_index, image_index);
shader_set_uniform_f(_uniUV, uv[0], uv[2]);
shader_set_uniform_f(_uniSpeed, _speed);
shader_set_uniform_f(_uniTime, _time);
shader_set_uniform_f(_uniSaturation, _saturation);
shader_set_uniform_f(_uniBrightness, _brightness);
shader_set_uniform_f(_uniSection, _section);
shader_set_uniform_f(_uniMix, _mix);
draw_self();
shader_reset();
As for the shader, we need to pass the saturation and brightness to the color, which will affect the color generated by our helper function. The section needs to be multiplied by our position to reduce the range. I also grabbed the entire texture color, so I can calculate our final color by mixing the texture color with the RGB conversion of our color. The last parameter of the mix function determines how much of the second color we want to add. This is our final shader code:
// Fragment Shader
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
uniform vec2 u_uv;
uniform float u_speed;
uniform float u_time;
uniform float u_saturation;
uniform float u_brightness;
uniform float u_section;
Uniform float u_mix;
vec3 hsv2rgb(vec3 c)
{
vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}
void main()
{
float pos = (v_vTexcoord.x - u_uv[0]) / (u_uv[1] - u_uv[0]);
vec4 texColor = texture2D(gm_BaseTexture, v_vTexcoord);
vec3 col = vec3(u_section * ((u_time * u_speed) + pos), u_saturation, u_brightness);
vec4 finalCol = mix(texColor, vec4(hsv2rgb(col), texColor.a), u_mix);
gl_FragColor = v_vColour * finalCol;
}
And our final result is this!
Demo
I have made a demo with all these shaders included, and some extra functionality. If you want to experiment with it, or need to check why something isn't working on your side, click here.