« Back to tutorial listing

— Working with TTF fonts —
Part 4: Unicode support

Note: this tutorial assumes knowledge of C, as well as prior tutorials.

Introduction

SDL2 TTF natively supports unicode; passing a unicode string to TTF_RenderUTF8_Blended will render all the glyphs as expected, so long as the font you're using supports them. Calling getTextTexture and passing over the text string and font we want to use will do jus thtat. However, when it comes to using a glyph atlas, so we can do things like the typewriter effect, things become somewhat more complicated. In this tutorial, we'll look at how to handle unicode in our text atlas.

Extract the archive, run cmake CMakeLists.txt, followed by make, and then use ./ttf04 to run the code. You will see a window open like the one above, showing left and right aligned paragraphs of unicode text (Google Translated French from the text in the previous demo). This demo contains a number of different scenes. Pressing the space bar will cycle through them. Close the window to exit.

Inspecting the code

As you know, unicode differs from standard ASCII by using multiple bytes to represent a single character. Calling strlen on an ASCII character will return 1. Doing the same on the Ö character will return 2. And doing so on the Pancake emoji (🥞) will return 4. Already, you can probably see that this will be problematic for our glyph atlas, as we now can't index against a single number. However, we actually can, as we'll see in a bit.

A quick note: the description of some of these functions will be shorter than usual, we are just making tweaks to existing functions that were already covered in the last tutorial.

Starting with text.h, we've added two new enums to hold our glyphs:

#define MAX_GLYPHS           400
#define MAX_GLYPH_SIZE       8

We'll see these used through the code. Turning now to text.c, we'll see a good number of tweaks and changes, including to our static variables:

static SDL_Rect glyphs[FONT_MAX][MAX_GLYPHS];
...
static char *characters = "Ö&|_# POfileorTBFS:handWCpygt2015-6,JwsbuGNUL3.Emj@c/\"IV\\RMD8+v?x;=%!AYq()'kH[]KzQX4Z79*àéí¡Çóè·úïçüºòÉÒÍÀ°æåøÆÅØ<>öÄäßÜá¿ñÁÊûâîôÈêùœÙìëęąłćżńśźŻŚŁĆÖ";

Our glyphs array now uses MAX_GLYPHS instead of NUM_GLYPHS (in fact, NUM_GLYPHS have been removed entirely). Beneath this, we can see a large string called characters, containing a great number of ASCCI and unicode characters. This will act as our source glyph pool when we come to generate our text atlas. Note that this could quite happily have resided in a file, and loaded and removed as needed.

Now onto our initFont function. We've made a few tweaks, but quite a lot of it remains the same:

static void initFont(int fontType, char *filename)
{
	SDL_Surface *surface, *text;
	SDL_Rect dest;
	int i, n;
	char glyphBuffer[MAX_GLYPH_SIZE];

	memset(&glyphs[fontType], 0, sizeof(SDL_Rect) * MAX_GLYPHS);

	fonts[fontType] = TTF_OpenFont(filename, FONT_SIZE);

	surface = SDL_CreateRGBSurface(0, FONT_TEXTURE_SIZE, FONT_TEXTURE_SIZE, 32, 0, 0, 0, 0xff);

	SDL_SetColorKey(surface, SDL_TRUE, SDL_MapRGBA(surface->format, 0, 0, 0, 0));

	dest.x = dest.y = 0;

	i = 0;

	while ((n = nextGlyph(characters, &i, glyphBuffer)) != 0)
	{
		if (n >= MAX_GLYPHS)
		{
			printf("Glyph '%s' index exceeds array size (%d >= %d)\n", glyphBuffer, n, MAX_GLYPHS);
			exit(1);
		}

		text = TTF_RenderUTF8_Blended(fonts[fontType], glyphBuffer, white);

		TTF_SizeText(fonts[fontType], glyphBuffer, &dest.w, &dest.h);

		if (dest.x + dest.w >= FONT_TEXTURE_SIZE)
		{
			dest.x = 0;

			dest.y += dest.h + 1;

			if (dest.y + dest.h >= FONT_TEXTURE_SIZE)
			{
				SDL_LogMessage(SDL_LOG_CATEGORY_APPLICATION, SDL_LOG_PRIORITY_CRITICAL, "Out of glyph space in %dx%d font atlas texture map.", FONT_TEXTURE_SIZE, FONT_TEXTURE_SIZE);
				exit(1);
			}
		}

		SDL_BlitSurface(text, NULL, surface, &dest);

		glyphs[fontType][n] = dest;

		SDL_FreeSurface(text);

		dest.x += dest.w;
	}

	fontTextures[fontType] = toTexture(surface, 1);
}

The first change is that we're creating a char array called glyphBuffer, of MAX_GLYPH_SIZE length. This is to hold our unicode glyph data. Our for loop has been replace with a while loop, that calls a function called nextGlyph. To this function, we pass over the array of characters we want to work with, a pointer to the current index within the string (which here is starting at 0), and our glyphBuffer. The function returns a number, which we're assigning to n. So long as this number isn't 0, we continue the loop (we'll also exit if the number returned exceeds MAX_GLYPH_SIZE, since we won't be able to use it). We'll go into the detail of what nextGlyph does in a while. For now, know that it will return in index of the glyph array we want to work with, and that glyphBuffer will contain the character data (whether that be a 1 byte ASCII character, or a multi-byte unicode one).

We'll then use glyphBuffer with TTF_RenderUTF8_Blended and TTF_SizeText to get our image data and dimensions, and then use the glyph's atlas entry data to the appropriate glyphs entry for the font as before. As you can see, just some tweaks here and there, and not a major overhaul. The same is true of the other functions, starting with drawTextWrapped:

static int drawTextWrapped(char *text, int x, int y, int r, int g, int b, int fontType, int align, int maxWidth, int doDraw)
{
	char word[MAX_WORD_LENGTH], line[MAX_LINE_LENGTH], glyphBuffer[MAX_GLYPH_SIZE];
	int i, n, wordWidth, lineWidth, len;

	i = 0;

	memset(word, 0, MAX_WORD_LENGTH);
	memset(line, 0, MAX_LINE_LENGTH);

	n = 0;

	lineWidth = wordWidth = 0;

	len = strlen(text);

	while ((n = nextGlyph(text, &i, glyphBuffer)) != 0)
	{
		wordWidth += glyphs[fontType][n].w;

		if (n != ' ')
		{
			strcat(word, glyphBuffer);
		}

		if (n == ' ' || i == len)
		{
			if (lineWidth + wordWidth >= maxWidth)
			{
				if (doDraw)
				{
					drawTextLine(line, x, y, r, g, b, fontType, align);
				}

				memset(line, 0, MAX_LINE_LENGTH);

				y += glyphs[fontType][' '].h;

				lineWidth = 0;
			}
			else if (lineWidth != 0)
			{
				strcat(line, " ");
			}

			strcat(line, word);

			lineWidth += wordWidth;

			memset(word, 0, MAX_WORD_LENGTH);

			wordWidth = 0;
		}
	}

	if (doDraw)
	{
		drawTextLine(line, x, y, r, g, b, fontType, align);
	}

	return y + glyphs[fontType][' '].h;
}

We're once again creating a glyphBuffer char array of MAX_GLYPH_SIZE length, and calling upon nextGlyph, this time passing over the text that has come into the drawTextWrapped function. One thing that is different is that we're using strcat to copy characters over, rather than simply replacing an index in the word array. This is, of course, because a unicode character will be more than one byte, and so using strcat makes more sense here. Otherwise, this function remains the same.

drawTextLine has similarly seen some minor changes:

static void drawTextLine(char *text, int x, int y, int r, int g, int b, int fontType, int align)
{
	int i, w, h, n;
	SDL_Rect *glyph, dest;

	if (align != TEXT_ALIGN_LEFT)
	{
		calcTextDimensions(text, fontType, &w, &h);

		if (align == TEXT_ALIGN_CENTER)
		{
			x -= (w / 2);
		}
		else if (align == TEXT_ALIGN_RIGHT)
		{
			x -= w;
		}
	}

	SDL_SetTextureColorMod(fontTextures[fontType], r, g, b);

	i = 0;

	while ((n = nextGlyph(text, &i, NULL)) != 0)
	{
		glyph = &glyphs[fontType][n];

		dest.x = x;
		dest.y = y;
		dest.w = glyph->w;
		dest.h = glyph->h;

		SDL_RenderCopy(app.renderer, fontTextures[fontType], glyph, &dest);

		x += glyph->w;
	}
}

We're simply calling nextGlyph here instead of using a for-loop. Notice here how our call to nextGlyph does not include a buffer to hold the character itself. This is because we don't need it here, just the index of the glyph in the array. The same is true of calcTextDimensions:

void calcTextDimensions(char *text, int fontType, int *w, int *h)
{
	int i, n;
	SDL_Rect *g;

	*w = *h = 0;

	i = 0;

	while ((n = nextGlyph(text, &i, NULL)) != 0)
	{
		g = &glyphs[fontType][n];

		*w += g->w;
		*h = MAX(g->h, *h);
	}
}

So, finally we come to nextGlyph, the principle function behind getting this all to work. At first glance, this will look quite complicated, but once we work our way through it, we'll see it's rather straight forward:

static int nextGlyph(const char *str, int *i, char *glyphBuffer)
{
	int len;
	unsigned int bit;
	const char *p;

	bit = (unsigned char) str[*i];

	if (bit < ' ')
	{
		return 0;
	}

	len = 1;

	if (bit >= 0xF0)
	{
		bit  = (int)(str[*i]     & 0x07) << 18;
		bit |= (int)(str[*i + 1] & 0x3F) << 12;
		bit |= (int)(str[*i + 2] & 0x3F) << 6;
		bit |= (int)(str[*i + 3] & 0x3F);

		len = 4;
	}
	else if (bit >= 0xE0)
	{
		bit  = (int)(str[*i]     & 0x0F) << 12;
		bit |= (int)(str[*i + 1] & 0x3F) << 6;
		bit |= (int)(str[*i + 2] & 0x3F);

		len = 3;
	}
	else if (bit >= 0xC0)
	{
		bit  = (int)(str[*i]     & 0x1F) << 6;
		bit |= (int)(str[*i + 1] & 0x3F);

		len = 2;
	}

	/* only fill the buffer if it's been supplied */
	if (glyphBuffer != NULL)
	{
		p = str + *i;

		memset(glyphBuffer, 0, MAX_GLYPH_SIZE);

		memcpy(glyphBuffer, p, len);
	}

	*i = *i + len;

	return bit;
}

We're passing in the string of text we want to use, *str, the index within the string we want to work with (as a pointer, this is important), and a buffer to contain the resulting characters (glyphBuffer).

The first thing we do is grab the character at index i, and convert it to an unsigned char, assigning to result to an unsigned int called bit. We then test to see if the bit is less than a space in the ASCII table, returning 0 immediately if so; we don't support anything less than a space (such as a NULL terminator), so this allows us to fail fast. Otherwise, we'll want to test to bit to see what we can learn about it. The first byte in a unicode character will tell us all about it. This is information that we can use to decode the character.

Consider the first if check. This tests if the bit is greater than or equal to 0xF0. If it is, then we know this a 4 byte unicode character. We would therefore want to read the four bytes that follow, as they represent the character. You'll notice we're doing some extra things: we're performing several bitwise operations and shifts on the bytes that follow. Doing this will give us the actual value of the character itself, meaning that we know what index to assign it in our glyph index. Very useful indeed.

At the end of the bit tests, we'll have the length of the character (len, defaulting to 1) and the index within our glyph array it will reside (bit). We next want to read the character, if the glyphBuffer has been supplied (in some cases, we only want the index of the glyph). If so, we want to read the bytes that represent our glyph. We do so using memcpy, copying from the index within the string the character starts at, to the length it finishes; so, either 1, 2, 3, or 4 bytes.

The last thing we do in the function before returning the value of bit is to advance the value of i by the number of bytes the character we read contained. This is an important step, as we if reading a 2 byte unicode character, we need to advance correctly to the next part of the string. If we were to only increment our pointer by 1, we would end up in the middle of a unicode character and our subsequent reads would be incorrect.

That's our changes to text.c done! Hurrah, we can now create and use a glyph atlas that can handle unicode characters. How about we see it in action? We'll turn to demo.c to do so, where we've made a few alterations.

To begin with, our initDemo function now includes a typeWriterTimer variable to handle the speed at which our typewriter will print characters.

void initDemo(void)
{
	scene = SCENE_BOXED;

	typeWriterPos = 0;
	typeWriterTimer = TYPING_SPEED;

	app.delegate.logic = logic;
	app.delegate.draw = draw;
}

The typeWriterPos variable will now take on a new purpose, as we can see in our logic function:

static void logic(void)
{
	if (scene == SCENE_TYPEWRITER)
	{
		if (--typeWriterTimer <= 0)
		{
			typeWriterPos++;
			typeWriterTimer = TYPING_SPEED;
		}
	}

	if (app.keyboard[SDL_SCANCODE_SPACE])
	{
		scene = (scene + 1) % SCENE_MAX;

		if (scene == SCENE_TYPEWRITER)
		{
			typeWriterTimer = TYPING_SPEED;
			typeWriterPos = 0;
		}

		app.keyboard[SDL_SCANCODE_SPACE] = 0;
	}
}

If we're using on the typewriter scene, we want to decrement the value of our typeWriterTimer. If that reaches zero or less, we increment the value of typeWriterPos before then reseting the value of typeWriterTimer. This means that typeWriterPos will increment at a set rate. Why we changed this from the previous incarnation will become clear in a while (although you may already have realised why this is..!).

Our main draw function remains the same, except for the addition of a new call to drawAllCharacters().

static void draw(void)
{
	switch (scene)
	{
		case SCENE_ALIGN:
			drawAlignedText();
			break;

		case SCENE_WRAPPED:
			drawWrappedText();
			break;

		case SCENE_BOXED:
			drawBoxedTexts();
			break;

		case SCENE_TYPEWRITER:
			drawTypeWriter();
			break;

		case SCENE_ALL_CHARACTERS:
			drawAllCharacters();
			break;

		default:
			break;
	}

	drawScenePrompt();
}

In fact, many of our functions here are unchanged, other than now being in French, to help demonstrate the use of Unicode:

static void drawAlignedText(void)
{
	drawText("Texte aligné à gauche.", 25, 25, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_LEFT, 0);

	drawText("Texte aligné au centre.", SCREEN_WIDTH / 2, 100, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_CENTER, 0);

	drawText("Texte aligné à droite.", SCREEN_WIDTH - 25, 25, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_RIGHT, 0);
}

And:

static void drawWrappedText(void)
{
	char *text;

	text = "Une très longue ligne de texte qui est trop large pour l'écran, mais qui s'adapte désormais parce que nous l'enveloppons et peut donc être lue correctement.";

	drawText(text, 25, 25, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_LEFT, SCREEN_WIDTH / 2);

	drawText(text, SCREEN_WIDTH - 25, 350, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_RIGHT, SCREEN_WIDTH / 2);
}

As well as:

static void drawBoxedTexts(void)
{
	drawBoxedText("Une chaîne de texte affichée dans une boîte, un peu comme une scène de dialogue dans un RPG.", 100, 65, 128, 160, 192);

	drawBoxedText("Un autre chargement de texte encadré, mais celui-ci a un fond rouge foncé.", 550, 250, 128, 64, 64);

	drawBoxedText("Regardez, ma! Je suis vert et maigre! Je t'ai dit que je serais célèbre un jour. Maintenant, je suis dans une démo SDL2 TTF!", 50, 400, 32, 160, 64);
}

Now we come to the drawTypeWriter function. Other than the French text, you'll notice we're testing the bit value once again:

static void drawTypeWriter(void)
{
	char textBuffer[1024], *text;
	int n, len;
	unsigned bit;

	text = "Les plus grandes œuvres de William Shakespeare, telles qu'écrites par un tutoriel SDL TTF. Un million de singes utilisant un million de machines à écrire pourraient gérer la même chose en quelques années. Peut-être?";

	len = strlen(text);

	n = MIN(typeWriterPos, len);

	if (n > 0)
	{
		bit = (unsigned char) text[n - 1];

		if (bit >= 0xF0)
		{
			n += 3;
			typeWriterPos += 3;
		}
		else if (bit >= 0xE0)
		{
			n += 2;
			typeWriterPos += 2;
		}
		else if (bit >= 0xC0)
		{
			n += 1;
			typeWriterPos += 1;
		}

		STRNCPY(textBuffer, text, n + 1);

		if (n < len)
		{
			strcat(textBuffer, "_");
		}

		drawText(textBuffer, 10, 100, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_LEFT, 600);
	}
}

The reason for this, as mentioned earlier on, is because we need to correctly advance within our string when reaching a unicode character. If we didn't do this, and only read one character at a time, we would experience some strange rendering effects as we read and displayed an incomplete character. We therefore need to ensure that the correct number of bytes is copied into textBuffer when we read the bits. Note that as well as increasing the value of n, we want to increase the value of typeWriterPos, so that we are typing at the correct speed. If we didn't do this, the typing effect would take a little longer when processing a unicode character.

Our final function is one that renders all the character we support in our glyph map:

static void drawAllCharacters(void)
{
	char *characters = "Ö&|_# POfileo rTBFS:handW Cpygt2015-6,J wsbuGNUL3.Emj@ c/\"IV\\RMD8+ v?x;=%!AYq()'kH []KzQX4Z79*àé í¡Çóè·úï çüºòÉÒÍÀ°æåøÆ ÅØ<>öÄäß Üá¿ñÁÊûâîôÈ êùœÙìëęąłć żńśźŻŚŁĆÖ";

	drawText(characters, 10, 100, 255, 255, 255, FONT_LINUX, TEXT_ALIGN_LEFT, SCREEN_WIDTH);
}

All this function does is displays a wrapped string of unicode characters. Simple, but a good test to check the support. Note that we've introduced some spaces here and there to break up the long sequence, since our text wrapping function breaks on white space.

That's it for the SDL2 TTF tutorial. Hopefully, this will have taught you a great deal about TTF and it can be used to wrap text, work with unicode, and much more.

Purchase

The source code for all parts of this tutorial (including assets) is available for purchase:

From itch.io

It is also available as part of the SDL2 tutorial bundle: