GSoC 2019: Adding support for OpenType-SVG fonts to FreeType

A Project Report

Project Information

FieldValue
Project Linkhttps://summerofcode.withgoogle.com/projects/#6096036961976320
OrganizationThe FreeType Project
MentorsWerner Lemberg and Suzuki Toshiya
Languages/ToolsANSI C, Unix Build Tools
My Emailmoazinkhatri@gmail.com

Project Discussion

In this project I successfully added OpenType-SVG font support to FreeType. This involved:

  1. Writing code to read the SVG table and load SVG documents.

  2. Researching on SVG rendering libraries and using them to render the documents. This involved:

    • Closely observing the rendering behavior of popular SVG rendering libraries (librsvg, resvg and svgnative) with SVG documents.
    • Studying the specification to learn about glyph placement in documents and getting them rendered properly.

  3. Writing the necessary code to connect FreeType to the external library and render the glyphs. This was done by creating a rendering module named ot-svg.

  4. Modifying the FreeType demo tools and the FreeType cache system to work with OpenType-SVG glyphs.

  5. Modifying the build system of FreeType to incorporate OT-SVG support.

All of the goals have been achieved. The only thing remaining is to decide on a default SVG rendering library. There is no hurry for that. We need features that are either absent or under development in most libraries. Thus this decision will be taken quite late.

Final Work Product

  1. Development code in FreeType repository: All of my code lives in the branch GSoC-2019-moazin. Since it's a development branch, the commit history doesn't look clean. I made many mistakes and later fixed them. The work done within the GSoC period is from commit 0729a6516 to f943af649. See the diff here.

  2. Writing rendering ports: FreeType doesn't link directly with SVG rendering libraries. Instead, a callback hook mechanism was created to allow users to plug-in their own libraries. Later on, once we decide on a default rendering library, we will set default hooks so that FreeType supports SVG fonts out-of-the-box. Currently, librsvg has been set as the default one. This is just temporary. I call these hooks rendering ports. I have written three ports for three popular SVG rendering libraries:

    Out of these, librsvg port is the most up-to-date and I plan to update the other ones soon.

Outline

This report is a large document. Please jump to the relevant section depending on what you seek.

Usage

The project code hasn't been merged to master yet. Thus, it only lives in the branch GSoC-2019-moazin. OT-SVG feature can be compiled with or without default hooks.

Procedure to test OT-SVG fonts without default hooks

Firstly, I should mention that this has been only tested on Ubuntu 19.04. To be able to use this project, you must have the latest in-development version of librsvg compiled and installed in your system. At the time of writing this, I am using tag 2.45.91. Please follow their instructions to compile and install the library. Once you've done that, follow these steps:

  1. Clone the FreeType repository and checkout GSoC-2019-moazin.

  2. Clone the port repository and checkout to the current commit.

  3. Copy the rsvg_port.c and rsvg_port.h files to the FreeType source.

  4. Build FreeType.

  5. Build the port and run the test program.

In case you're getting some errors, please make sure you have all the dependencies installed. Check the Makefile in librsvg-port-freetype-otsvg/tester to see the dependencies. In case you need help, feel free to reach me out at moazinkhatri@gmail.com.

A brief summary of how OT-SVG fonts work

SVG is a popular vector graphics format which is being extensively used in web pages these days. The SVG table is a feature addition to OpenType that allows SVG documents to be used to describe glyphs. The table contains a list of SVG documents where each SVG document can contain glyph descriptions for one or a whole range of glyph IDs. If an SVG document contains the glyph description of only one glyph ID, the whole document is supposed to be rendered. If on the other hand, the document contains multiple glyph descriptions for a range of glyph IDs, only the element with the ID 'glyph<ID>' is to be rendered. Read the section Glyph Identifiers to read more about this.

SVG coordinate system and glyph placement

SVG, being a vector graphics format, ultimately relies on Bézier curves. These curves consist of points which exist at certain coordinates in a coordinate system that we shall call the SVG coordinate system. This coordinate system should be analogous to the coordinate system of TTF/CFF outlines. However, unlike the TTF/CFF coordinate system, this one has an inverted y axis. So, positive x is to the right while positive y points down. Just like the TTF/CFF coordinate system has an EM square which font designers use, same is true with the SVG coordinate system, though, unlike TTF/CFF outlines, the EM square's size can vary. The viewBox.width and viewBox.height or width/height attribute on the root SVG node can indicate the size of the EM square. If such attributes are missing, it should be assumed that the EM square is of the same number of units as the corresponding TTF/CFF outlines' EM square. The viewBox attribute describes a rectangle. Imagine that the whole SVG coordinate system is covered by an opaque sheet with one transparent window and that transparent window is the viewBox. Any SVG rendering library will output what can be seen through this window. We shall define a new coordinate system SVG' as the coordinate system formed by translating the SVG coordinate system to the point viewBox.x and viewBox.y. In case the viewBox attribute is missing, SVG' coordinate system is totally identical to the SVG coordinate system. The glyph is placed in such a way that the baseline is at y = 0 of the SVG' coordinate system. The advance.x is the same as that for the corresponding TTF/CFF glyph. For properly rendering the glyph, the origin of SVG' must be placed on top of the origin of the current pen point. If that's done properly, the glyph will just land up right where it's supposed to.

OT-SVG project in clean steps

In FreeType, there are two basic steps involved in rendering a glyph that has a particular ID.

  1. Loading the glyph: The glyph outlines (or the vector data) is loaded into the glyphslot.
  2. Rendering the glyph: The glyph outlines that have been loaded are rendered into a pixmap.

For OpenType SVG glyphs, we devise a similar plan. In the loading part, the appropriate SVG document will be fetched and stored in the glyph-slot. In the rendering part, the outlines will be sent to an external SVG rendering library, which will return a pixmap.

Making OT-SVG an optional feature

FT_CONFIG_OPTION_SVG is added to allow toggling OT-SVG support. This was done so that smaller builds of FreeType can disable OT-SVG support to reduce build size.

Adding code to load the SVG table

In FreeType, for fonts that have an sfnt header, loaded tables are usually stored in TT_FaceRec. The manner in which they are stored differs between tables. For SVG a strategy similar to CPAL and SVG has been adopted. An internal module specific structure Svg is created to hold basic information extracted from the SVG table. TT_FaceRec gets a new field of type void * named svg which would hold a pointer to this structure. A flag FT_FACE_FLAG_SVG is added, this will be set whenever an SVG table is present and has been loaded. sfnt related functions are exposed by the sfnt module. Thus, SFNT_Interface was modified to add two more fields load_svg and free_svg. The actual functions to fill these fields were created with the names tt_face_load_svg and tt_face_free_svg. The function sfnt_load_face loads all the available tables of interest and sfnt_free_face frees all the allocations performed by sfnt_load_face. Both of these were modified to load and free the Svg structure respectively.

Adding code to load SVG glyphs

To store the SVG document in the glyphslot, a structure named FT_SVG_DocumentRec is created which contains the SVG document along with other necessary details to accompany it. Please see the in-file documentation of otsvg.h for more details of this structure. This document is referenced by other field of FT_GlyphSlotRec. ft_glyphslot_init, ft_glyphslot_clear and ft_glyphslot_done were modified to take care of allocating and freeing memory for this structure.

To fetch the correct SVG document, one more function is added to SFNT_Interface named load_svg_doc. The actual function that populates this place is tt_face_load_svg_doc.

Ultimately, cff_slot_load and TT_Load_Glyph were modified to add special code that'll call load_svg_doc and fetch advance information (and set it) if an OT-SVG glyph is found, otherwise the control flow remains totally the same.

Adding code to render SVG glyphs

To render OT-SVG glyphs, an external library is to be used. For this purpose, a new Renderer Module is created named ot-svg. However, instead of directly linking an external library to FreeType, hooks are used. Thus, the client application can plug in any SVG rendering library of its choice by writing the hook functions. Later, we will decide on a default rendering library and have it linked by default. Thus, FreeType will support OT-SVG glyphs out-of-box and will also give users the ability to plug in other SVG rendering libraries if they want to.

We could just create a renderer module like the other ones out there but we need to store the hooks somewhere. For this reason, we create a new type SVG_RendererRec that inherits from FT_RendererRec and adds some more fields

  1. loaded is a flag that allows us to lazily load the SVG rendering library at the point when the first SVG glyph is rendered.

  2. hooks_set is a flag that simply indicates whether hooks have been set or not. In case they haven't been set, the module isn't really functional.

  3. hooks is a structure that contains the hooks. There are a total of four hooks.

    1. Init Hook

    This hook is called at the time when the first OT-SVG glyph is to be rendered. Once it has been called loaded will be set to TRUE. The job of this hook is to let the SVG rendering library perform any sort of initializations that it wants to. We would want the SVG rendering library to have a state or persistence between different function calls. The library can create a state structure of its own, allocate it upon initialization and set svg_renderer_state of library to reference it.

    2. Free Hook

    To let the SVG rendering library perform any cleanups. For example, the library should free the state structure if it created one.

    3. Preset Hook

    The preset hook does the job of presetting the bitmap slot. However, for an OT-SVG glyph, presetting the slot needs many calculations which are useful at the rendering stage too. The preset hook is called from two places only.

    1. At the loading stage from ft_glyphslot_preset_bitmap.
    2. Right before rendering the glyph in ft_svg_render.

    When it is the latter, we want the function to store those useful calculations in the state of the library so that they can be later used by the render hook. The calls are right next to each other so we can be sure that the state will remain intact. However, when it's the former, we do not want the function to modify state in anyway. That's the reason we have the cache argument. When it is TRUE the calculations can saved in state, when it's FALSE the calculations can't be stored in state. The reason why the preset hook is called again in ft_svg_render is because we want to make sure that the glyphslot has been preset, so we can use its width/height to allocate enough memory for the pixmap buffer.

    4. Render Hook

    The render hook simply renders the loaded SVG document and places the resultant pixmap in slot->bitmap.buffer. Memory for the buffer is already allocated so it shouldn't be allocated from inside. The following values must be set:

    Adding support of OT-SVG glyphs in Glyph Management API

    A crucial part of FreeType is the Glyph Management API. Thus, it's necessary to add OT-SVG support to it. This is done by creating new handle FT_SvgGlyph, its structure FT_SvgGlyphRec and creating a new glyph class, named ft_svg_glyph_class. We also need to modify some functions to appropriately handle OT-SVG glyphs but the changes are small.

    Adding support for transforms to OT-SVG glyphs

    For traditional outlines, transformations are directly applied on the actual outlines itself. For OT-SVG glyphs, FreeType cannot reach any of the actual graphics data, since, only the SVG document string is held. Thus, we can't apply any transforms ourselves. The only thing we can do is store the transformation matrix and later have the external library apply it for us. For this purpose, we add two fields, transform and delta to FT_SVG_Document and FT_SvgGlyphRec. If multiple transformations are applied one after another, some maths is done to compute an equivalent transformation matrix. The external library should pick up these fields and apply the transforms while presetting and rendering the document. So far, this approach has worked pretty well.

Design choices and challenges I faced

For any project, I think one of the hardest part is making choices. Same was the case with this project. I list the major challenges I faced below:

  1. Deciding where to store the SVG document and other relevant information: I needed to store the SVG document itself and some information about it in the glyphslot but couldn't really figure out where to store it. Firstly, I tried adding new fields to TT_GlyphSlotRec and CFF_GlyphSlotRec. The approach worked but soon I realized there were some problems with this approach. [See the thread: msg00092] So, upon the suggestion of FreeType developers, I created a new structure FT_SVG_DocumentRec and referenced it in the other field of glyphslot. FT_SVG_DocumentRec contains all the fields needed. This approach has turned out to work pretty well.
  2. Nature of the SVG rendering module: We wanted to give client users the ability to plug in different SVG rendering libraries if they want to. Thus, it was decided not to link the external library directly. Instead, we created a callback hook mechanism. The users can set their own hooks to plug-in another library. There was a contrary idea under consideration too, which was to create different SVG rendering modules for different libraries. The matter was under discussion in the mailing list but ultimately it was decided that the callback hooks idea is preferable. [See these threads: msg00042, msg00074 and msg00051]
  3. Setting the hooks: How do we get the hooks from the external user? At first, we were using an API function, FT_Set_Svg_Hooks, but later we dropped it. Instead, we use module properties to set the hooks. [See these threads: msg00077, msg00098 and msg00051]
  4. Storing the hooks: There was a confusion about where to store the function pointers for the callback hooks. The solution I came up with, was, instead of creating an FT_RendererRec as the rendering module, I created a subclass of it named SVG_RendererRec and I added a hooks field to this structure as well as some flags. This solution has worked well and looks fine from a design point of view.
  5. Rendering the SVG glyphs: Read the section SVG coordinate system and glyph placement to read about how SVG glyphs are placed within the SVG coordinate system. The behavior of popular SVG rendering libraries is that they just render whatever is in the viewBox to a pixmap. To properly render a glyph its bounding box must be known so that we can shift the coordinate system to get a proper rendering. How to get this bounding box is a challenging problem. Different SVG rendering libraries do provide ways of getting the bounding box but they are not always accurate (can be bigger than the actual bounding box). Firstly, we decided to use the bounding box of TTF/CFF outlines and from that predict the bounding box in SVG coordinates. [See the thread: msg00050] This worked perfectly for many fonts but failed for one. I brought up the issue in the OpenType-SVG list and ultimately it was realized that the bounding boxes don't have to be the same at all. [See the thread: 0000] Thus, we resort to using the the SVG rendering library specific methods to get the bounding boxes. [See the thread: msg00010] The bounding box is almost always accurate and when it's not, bitmap_left and bitmap_top make sure that the glyph is positioned correctly after the rendering. Everything works great! :-)
  6. Transformation support for OT-SVG glyphs: For traditional glyphs, FreeType can manipulate the outlines by itself, and thus it can apply the transformations directly to the outlines. For SVG glyphs, that isn't true, all we store is just the document string. To support transforms, we store a transformation matrix and a translate vector in FT_SVG_DocumentRec and FT_SvgGlyphRec and later the SVG rendering library picks it up and applies the transformation. There is a problem though. FreeType users input a transformation matrix that has been designed for the TTF/CFF coordinate system and it turns out that the SVG coordinate system is quite different. Thus, directly applying the transformation should give unexpected results. The solution I came up with is, we let the user pretend that the coordinate system is just a traditional one, then we convert the transformation to an equivalent one in SVG coordinates and then apply it. Another problem is, how do we support cascaded transformations? The solution I figured out was to use do some maths to compute an equivalent transformation matrix that has the effect of all the transformations applied. This has worked pretty well and the results are good. [See the thread: msg00037]