« Back to tutorial listing

— An old-school isometric game —
Part 6: Occlusion testing

Note: this tutorial assumes knowledge of C, as well as prior tutorials.

Introduction

We now have a small map that we can navigate, as well as other objects littering the ground. However, the walls and trees that we've added can cause us to lose sight of Purple Guy as he stands behind them. In this part, we're going to look into how we can make these objects transparent, so that we can still see him.

Extract the archive, run cmake CMakeLists.txt, followed by make, and then use ./isometric06 to run the code. You will see a window open like the one above. Click on ground squares to have Purple Guy move around. Notice how as Purple Guy moves behind larger objects that might otherwise obstruct our view of him that they become transparent, allowing us to still see him. As he moves away, they return to their solid colour. Once you're finished, close the window to exit.

Inspecting the code

Making our objects become transparent as Purple Guy moves behind them isn't nearly as tricky as one might think. It involves quite a few tweaks, but is otherwise simple.

Let's start with defs.h:

#define IF_NONE                   0
#define IF_TEST_OCCLUSION         (2 << 0)

We've added in two new defines. IF stands for Iso Flags, and will be the flags that we'll pass over to our addISObject function. IF_TEST_OCCLUSION means that we want the ISOObject to test whether it is obstructing our view of Purple Guy, as we'll see later.

Now over to structs.h, where we've made a number of updates:

typedef struct {
	int x;
	int y;
	int sx;
	int sy;
	int layer;
	AtlasImage *texture;
	struct {
		int r;
		int g;
		int b;
		int a;
	} color;
} ISOObject;

To ISOObject, we've added an anonymous struct called `colour`, holding `r`, `g`, `b`, and `a` values. These are our red, green, blue, and alpha values, and will allow us to control the colour and alpha value of the ISOObject as a whole.

We've also updated Entity:

struct Entity {
	char name[MAX_NAME_LENGTH];
	int x;
	int z;
	int base;
	unsigned long flags;
	unsigned long isoFlags;
	AtlasImage *texture;
	Entity *next;
};

We've added a field called isoFlags, that will contain the isometric flags that this entity will use when being turned into an ISOObject.

Last, we've updated World:

typedef struct {
	MapTile map[MAP_SIZE][MAP_SIZE];
	Entity entityHead, *entityTail;
	Node routeHead;
	Entity *player;
	SDL_Rect playerISORect;
	struct {
		int x;
		int z;
	} cursor;
} World;

We have a new field called playerISORect. This is a rectangular area that will represent Purple Guy on screen. We'll be using this to determine where he is when we add other ISOObjects, and allow us to test if they are overlapping him.

Now over to tree.c:

void initTree(Entity *e)
{
	char filename[MAX_FILENAME_LENGTH];

	sprintf(filename, "gfx/entities/tree%d.png", 1 + rand() % 3);

	e->base = -1;
	e->flags = EF_SOLID;
	e->isoFlags = IF_TEST_OCCLUSION;
	e->texture = getAtlasImage(filename, 1);
}

We're setting the tree's isoFlags as IF_TEST_OCCLUSION.

That's all there is for tree.c, so we can now move onto entities.c, where we can see the major updates put to work. Turning now to drawEntities:

void drawEntities(void)
{
	Entity *e;
	int sx, sy;

	for (e = world.entityHead.next ; e != NULL ; e = e->next)
	{
		sx = TILE_WIDTH / 2;
		sx -= e->texture->rect.w / 2;

		sy = TILE_HEIGHT / 2;
		sy -= e->texture->rect.h;
		sy -= e->base;

		addISOObject(e->x, e->z, sx, sy, e->texture, LAYER_FOREGROUND, e->isoFlags);

		if (e == world.player)
		{
			world.playerISORect.w = world.player->texture->rect.w;
			world.playerISORect.h = world.player->texture->rect.h - 8;

			toISO(e->x, e->z, &world.playerISORect.x, &world.playerISORect.y);
			world.playerISORect.x += sx;
			world.playerISORect.y += sy;
		}

		// add the shadow

		sx = TILE_WIDTH / 2;
		sx -= shadowTexture->rect.w / 2;

		sy = -shadowTexture->rect.h;
		sy += TILE_HEIGHT;

		addISOObject(e->x, e->z, sx, sy, shadowTexture, LAYER_MID, IF_NONE);
	}
}

Our addISOObject function now takes one extra argument - the `flags` to use. When calling addISOObject for our entity (the first call), we're passing over the entity's isoFlags. In the case of the tree, this will be IF_TEST_OCCLUSION. All other entities will be passing 0 (IF_NONE) as their default. Note that the addISOObject call for the shadow also passes over IF_NONE, as the shadow won't be occluding our view.

We've also added in a new test, to check if the current entity (`e`) is the player. If so, we're setting the values of World's playerISORect `w` (width) and `h` (height) to the player's texture's `w` and `h`. Notice how we're subtracting 8 from the height. This is simply to allow for a small leeway in the coverage; we're not going to worry if a single pixel is overlapping Purple Guy, since he's still quite visible. Next, we're calling toISO, and passing over the player's `x` and `z` values, outputting them into playerISORect's `x` and `y`. Finally, we're adding the previously calculated `sx` and `sy` adjustments. In short, we're calculating where Purple Guy will be rendered on screen, and storing those dimensions into World's playerISORect. There is room for optimisation here, but for now this approach is working fine.

Onwards now to map.c, where we've updated drawMap:

void drawMap(void)
{
	int x, z, n;

	for (x = 0 ; x < MAP_RENDER_SIZE ; x++)
	{
		for (z = 0 ; z < MAP_RENDER_SIZE ; z++)
		{
			n = world.map[x][z].tile;

			if (n >= 0)
			{
				if (n >= TILE_WALL)
				{
					addISOObject(x, z, 0, -TILE_HEIGHT, tiles[n], LAYER_FOREGROUND, IF_TEST_OCCLUSION);

					addISOObject(x, z, 0, 0, baseTileTexture, LAYER_BACKGROUND, IF_NONE);
				}
				else
				{
					addISOObject(x, z, 0, 0, tiles[n], LAYER_BACKGROUND, IF_NONE);
				}
			}
		}
	}
}

We've updated our calls to addISOObject. For any tile that is a wall, we're now passing over IF_TEST_OCCLUSION. Otherwise, we're passing over IF_NONE. The ground tiles won't be obstructing our view of Purple Guy, so there's no need to test them. Note that we've removed the check for the selected tile (and associated graphic loading), as we're not using it any more.

Now onto iso.c, where the major work for handling our transparency has taken place. Starting with drawISOObjects:

void drawISOObjects(void)
{
	int i;
	ISOObject *o;

	qsort(isoObjects, numISOObjects, sizeof(ISOObject), drawComparator);

	for (i = 0 ; i < numISOObjects ; i++)
	{
		if (drawTimer >= i)
		{
			o = &isoObjects[i];

			SDL_SetTextureAlphaMod(o->texture->texture, o->color.a);
			SDL_SetTextureColorMod(o->texture->texture, o->color.r, o->color.g, o->color.b);

			blitAtlasImage(o->texture, o->sx, o->sy, 0, SDL_FLIP_NONE);
		}
	}

	SDL_SetTextureAlphaMod(o->texture->texture, 255);
	SDL_SetTextureColorMod(o->texture->texture, 255, 255, 255);
}

Now, when drawing our ISOObjects, we're applying their alpha and colours, via calls to SDL_SetTextureAlphaMod and SDL_SetTextureColorMod. Notice how we're passing over the ISOObject's (`o`'s) color's `a` value to SDL_SetTextureAlphaMod, and the `r`, `g`, `b` values to SDL_SetTextureColorMod, as one would expect. At the end of the loop, we're resetting the texture's colours back to 255, since we're working with a texture atlas and don't want to permanently mess with the colors.

Easy. Now for something a bit more complicated. Onto addISOObject:

void addISOObject(int x, int z, int sx, int sy, AtlasImage *texture, int layer, unsigned long flags)
{
	ISOObject *o;

	if (numISOObjects < MAX_ISO_OBJECTS)
	{
		o = &isoObjects[numISOObjects++];

		toISO(x, z, &o->x, &o->y);

		o->sx = o->x + sx;
		o->sy = o->y + sy;
		o->layer = layer;
		o->texture = texture;
		o->color.r = o->color.g = o->color.b = o->color.a = 255;

		if (flags & IF_TEST_OCCLUSION &&
			world.player->x >= x && world.player->z <= z &&
			collision(world.playerISORect.x, world.playerISORect.y, world.playerISORect.w, world.playerISORect.h, o->sx, o->sy, texture->rect.w, texture->rect.h))
		{
			o->color.a *= 0.35;
		}
	}
}

As we saw before, the function has been updated to accept a new parameter: `flags`, the IF values. To begin with, we're setting the ISOObject's (`o`'s) rgba values to 255, as a default. We're then testing the flags that were passed over, to see if they contain IF_TEST_OCCLUSION. If so, we'll be checking to see if this ISOObject we're adding is overlapping Purple Guy.

First up, we want to check if World's playerISORect is behind the current ISOObject. We can do this by simply testing the `x` and `z` values passed into the function against the player's `x` and `z` values. If the player's `x` value is greater or equals to the function's `x` and the player's `z` is less than or equal to the function's `z`, then it means the current ISOObject is being drawn after the player, and could therefore be obstructing them. We then perform a simple collision check, using the player's screen dimensions against the ISOObject's screen dimensions. If they overlap, it means the ISOObject is obstructing Purple Guy! Again, there is space for optimisation here.

Having discovered that Purple Guy is being occluded by the new object, we set the new object's alpha value (color's `a`) to 35%. If we wanted it to vanish entirely, we could set it to 0. 35% is good for us.

Great! Now we don't have to worry about losing track of the player when they move behind large objects. One other thing we need to take care of is the tile select. Before, we were repurposing the zone exit tile (the yellow one!). Now, we're displaying a wireframe sprite, that is easier to track and find when it moves behind walls.

If we turn now to world.c, we can see how this is being done. Starting with initWorld:

void initWorld(void)
{
	initMap();

	initEntities();

	initISORender();

	initPlayer();

	tileSelect[0] = getAtlasImage("gfx/misc/tileSelectBack.png", 1);
	tileSelect[1] = getAtlasImage("gfx/misc/tileSelectFront.png", 1);

	app.delegate.logic = logic;
	app.delegate.draw = draw;
}

We're loading two new textures, into an array called tileSelect. These will form the front and back of our select box, and give it a 3D look.

Next, we've updated `draw`:

static void draw(void)
{
	clearISOObjects();

	drawMap();

	drawEntities();

	drawCursor();

	drawISOObjects();

	drawHud();
}

Here, we're calling a new function drawCursor:

static void drawCursor(void)
{
	if (world.routeHead.next == NULL && world.cursor.x >= 0 && world.cursor.z >= 0 && world.cursor.x < MAP_RENDER_SIZE && world.cursor.z < MAP_RENDER_SIZE && world.map[world.cursor.x][world.cursor.z].tile >= 10)
	{
		addISOObject(world.cursor.x, world.cursor.z, 0, -(TILE_HEIGHT * 2) - 1, tileSelect[0], LAYER_MID, IF_NONE);
		addISOObject(world.cursor.x, world.cursor.z, 0, -(TILE_HEIGHT * 2) - 1, tileSelect[1], LAYER_FOREGROUND, IF_NONE);
	}
}

This function is responsible for drawing our wireframe box. Before we begin drawing, we check that Purple Guy isn't currently walking, that the cursor is within map bounds, and that the cursor is over a ground tile or a wall tile.

To actually draw our wireframe select box, we're calling addISOObject twice (after performing a check that our cursor is over a valid tile). We're passing over the relevant `x`, `z`, `sx`, and `sy` values, and then each of the tileSelect textures, the first one being drawn on the middle layer (LAYER_MID), and the second being drawn in the foreground layer (LAYER_FOREGROUND). This gives the impression of the wireframe being 3D, and mostly works when hovering over entities.

I say mostly, because here is an example of where the sprite-based isometric sorting starts to break down a bit. If one positions the wireframe down from walls, it can be seen that it doesn't quite work the way we expect. This is but a minor error, and one that doesn't detract too much from our game. Most importantly, we don't lose sight of our select box when it goes behind walls.

Another part done! The foundations of our game are pretty much there. What we need to do next is load our map data, so that we can start to explore more. We've been standing by that river a bit too long.

Purchase

The source code for all parts of this tutorial (including assets) is available for purchase:

From itch.io

It is also available as part of the SDL2 tutorial bundle: