Øyvind Kolås    pippin@gimp·org twitter github bitcoin patreon  

ctx compact 2d vector drawing context and rasterizer.

Single header HTML5 canvas 2D context|cairo|skia|postscript like drawing context API. Providing a compact vector renderstream representation, and a rasterization engine. Configurability and optional renderstream abstraction makes the code components provided scale from running on microcontrollers like ESP32 and ARM-cortex-M4 to bandwidth constrained network rendering to easy threaded software rendering on laptop or smartphone cpus.

compact binary size

The whole engine, when building a minimal RGBA8 capable rasterizer, dropping all other formats apart from GRAY8 which is needed for the glyph cache, the following numbers apply (2019-oct-24), rendering a couple of lines of text, built for x86_64 with gcc -Os, on other architectures the binary size can be smaller.

font data size:  11.9 kb  (DejaVuSans - scalable subsetted to only ASCII)
code size:       42.3 kb  (x86_64 - 32bit archs probably smaller)
peak heap use:   14.0 kb  (with a font-cache tuned for)
bss:               60 kb  (optional - not optimized for size, glyph cache of 60 up to 28x28x1byte)
~1kb for a glyph index can be moved from heap to font data.

microcontroller demo

In this video both microcontrollers, the arm-cortex-m4 at 97mhz in the card10 badge and on the TTGO ESP32 both with immediate rendering, the card10 has a 180x80 display with really high visual quality, while the TFT LCD on the ESP32 has slightly higher pixel resolution at 160x128. Both of these microcontrollers are more powerful than PCs I have worked with in the past.

The video is a debug tool, the one above was made when glyph caching - when the first adding of glyph caching started working, when rendering with text at sizes <=28px, the code here uses a glyph cache. The glyph cache currently only works without rotations, which is the reason the card10 is upside down. In the part with the red unicode text bits; slowness in other code paths is apparent where glyphs later in the font take noticably longer to get rendered than the ones appearing early.

Development funding

The library is still under development in terms of performance and capabilities. ctx is possible due to my financial supporters. If you want to encourage continued development of the concepts involved in ctx – do consider becoming one.

features

  • Can use both stb_truetype for dynamically loading fonts, as well as fonts reusing the renderstream representation that gets directly included.
  • ctx renderstream a binary vector drawing data protocol, see paragraph below.
  • API currently covers almost all of HTML5 Canvas 2d context, though setting the clipping path is ignored. moveto/lineto/curveto/arc/quad/rel_../fill/stroke/set_line_width/set_global_alpha/etc
  • linear and radial gradients, with opacity
  • compiletime configurable level of antialiasing
  • Pixel-formats, including dithering of gradients for RGB565 and <8bpc grayscale formats.
    RGBA8
    like html5 canvas, a todo item is doing compositing in linear light temporarily in 16bit instead.
    RGB565
    16bit RGB, 65536 colors - gradients get dithered other colors are quantized.
    RGB565_BYTESWAPPED
    same as RGB565 but with the bytes swapped.
  • RGBAF
    The floating point code is less squeezed for performance than the 8bit code, SIMD and other forms of vectorization are also more readily available to the compiler on platforms where this is enabled, so not very relevant, the code is structured to in the future also work for CMYK and other types of multi-component formats.
    RGB8
    BGRA8
    Format with red and blue components swapped around, compatible with cairo (pending endianness).
    GRAY1
    black and white, gradients get dithered, other colors are thresholded.
    GRAY2
    4 level grayscale, gradients get dithered, other colors are quantized
    GRAY4
    16 level grayscale, gradients get dithered, ...
    GRAY8
    256 level grayscale
    GRAYF
    floating point, grayscale
    GRAYA8
    256 level grayscale, with alpha component
  • All code, including much that is not built for embedded use is less than 8000 lines of C, including internal implementation of needed 32bit float trigonometric functions; the code also builds with C++ compilers.

ctx renderstream

The renderstream is built up during rendering, it has many potential uses and is not yet frozen but evolving with the many uses in mind. It is currently used for the internal font representation, as well as communicating the data to render between micropython on one microcontroller core and feeding it to the rasterization engine running free rtos on the other. This usage mimics one of the intended uses which is networked vector graphics (perhaps as a modern replacement for REGIS).

During rendering the renderstream memory is not touched by the rasterizer, this permits multiple threads to render different tiles/parts of a framebuffer without locking.

On smaller memory controllers than the ones I've used, or possibly to drive larger displays than I have. One can trade off framerate and cpu usage for RAM needed for storing the framebuffer, by serial blitting chunks instead of doing it in parallell and feeding an SPI connected framebuffer each chunk in sequence.

Transforms of the renderstream for compression is configurable in code, and some features are currently turned off because they conflict with features that already work without those transforms active. There are two main types of compression bitpacking, which encodes multiple vector drawing commands with signed 8bit coordinates in the room taken by a single move-to using floating point precision. To be able to do this bitpacking the coordinates are first all transformed to display/screen coordinates, and absolute curve construction commands are replaced with relative ones. Encoding in relative and transformed coordinates also enables us to do compression, by replacing already seen parts of the stream with a reference to the past (this is planned extended to also be able to refer to the preceding rendered frames renderstream).

At the moment the render-stream is traversed back to front, the datasstructure should with some small adaptations also work for rendering front-to back, and avoid both rasterizing vector geometry and computing sources for already opaque pixels.

Rendering tests

aa-test tests of AA rasterization settings impact on rasterizing text.

Downloads

It is early stages for ctx, more instructions on integration and use; with arduino and other environments can be expected later.

Library header: ctx.h, ctx-font-regular.h font used in examples, card10 test l0dable: ctx.elf, ctx is under heavy development.

test-size.c compiled size test-case


#include <stdint.h>

#include "ctx-font-regular.h"
#define CTX_LIMIT_FORMATS
#define CTX_ENABLE_RGBA8                1

#define CTX_IMPLEMENTATION

#define CTX_GLYPH_CACHE 0

#include "ctx.h"

#define WIDTH    72
#define HEIGHT   24

uint8_t pixels[WIDTH*HEIGHT*4];

int main (int argc, char **argv)
{
  Ctx *ctx = ctx_new_for_framebuffer (
    pixels, WIDTH, HEIGHT, WIDTH*4,
    CTX_FORMAT_RGBA8);

  for (int i = 0; i < WIDTH*HEIGHT*4; i++)
    pixels[i] = 0;

  ctx_set_rgba (ctx, 0.5, 0.5, 0.5, 1);
  ctx_rectangle (ctx, 0, 0, 80, 24);
  ctx_fill (ctx);

  char *utf8 = "tinytest\necho foobaz\n";
  ctx_move_to (ctx, 10, 9);
  ctx_set_font_size (ctx, 12);
  ctx_set_line_width (ctx, 2);
  ctx_set_rgba (ctx, 0, 0, 0, 1);
  ctx_text_stroke (ctx, utf8);
  ctx_set_rgba_u8 (ctx, 255, 255, 255, 255);
  ctx_move_to (ctx, 10, 9);
  ctx_text (ctx, utf8);

  static char *utf8_gray_scale[]={" ","░","▒","▓","█","█", NULL};
  int no=0;
  for (int y= 0; y < HEIGHT; y++)
  {
    for (int x = 0; x < WIDTH; x++)
    {
      printf ("%s", utf8_gray_scale[5-(int)CTX_CLAMP(pixels[no+1]/255.0*6.0, 0, 5)]);
      no+=4;
    }
    printf ("\n");
  }
  ctx_free (ctx);

  return 0;
}
which outputs (the rightmost column of pixels is a bug):
~/src/ctx/tests$ ./test-size 
▓▓▓▓▓▓▓▓▓▓▓█▓▓▓█▓▓▓▓▓▓▓▓▓▓▓▓▓▓██▓▓▓▓▓▓▓▓▓▓▓▓▓▓█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓██▓▓█ █▓▓▓▓▓▓▓▓▓▓▓▓███▓▓▓▓▓▓▓▓▓▓▓▓▓███▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓██ ██████████▓██▓▓▓██▓▓██▓████▓▓█████▒███▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓█░  ░█ ██░░ ░██ █▓█ █   ▓█▒  ███   ▓▓   █▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓██ ███ ██ ▓█░██░███░█▓▓██▓░██░██░████▒███▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓█ █▓█ ██░██▓▓█▓▒█▒▓█▓▒██░▓██░▓█ ▓███▒██▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓█ █▓█ ██░██▓▓██ █ ██▓▓██░▒▓▓▓███▒░▒█▒█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓█ ███ ██░██▓▓██▒░▒██▓▓██▒▒████████░█▒███▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓█▒░▒█ ██░██▓▓██▓ █▓██░░██░░▒░█▓░▒░▓██░░█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓█████▓██▓▓██▓██ █▓▓████▓██████████▓████▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█▓ █▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓████▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓███▓▓▓▓▓▓▓▓▓▓▓▓▓▓███▓▓▓▓▓▓▓▓▓▓▓▓▓██▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█ █▓▓▓▓▓▓▓▓▓▓▓▓▓██░░█▓▓▓▓▓▓▓▓▓▓▓█▒▓█▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓████▓▓▓█████ ███▓▓▓████▓▓▓▓█░███████▓▓████▓█▒▓███▓▓████▓▓████
▓▓▓▓▓▓▓▓▓▓██▒▒▓███   ▓█ ▒ ░█▓█▒  ██▓▓█▒  ░█▓  ████  ▒██▒░  ███   ▓██   █
▓▓▓▓▓▓▓▓▓██░██▒██▒▓████ ▓█▒▓█▓▒██░█▓▓██░███░██░██░██▒▓█▒░██░█████░██████
▓▓▓▓▓▓▓▓▓█▓▒██▓▒█ █▓▓▓█ ███▒█░███░█▓▓▓█░██▓▓██▓▓█ ███░█▒▓██▒███▓▓░▓███░█
▓▓▓▓▓▓▓▓▓█▓▒█████ █▓▓▓█ ███▒█░█▓█░█▓▓▓█░██▒▓██▓▓█░███░█▒▓██▒▓█ ██▒▓██░▓█
▓▓▓▓▓▓▓▓▓█▓░█████░█████ ███▒█░▓██ █▓▓▓█░██▓░██░██ ██▓░█▒▒██░██░██░▓█▒▒██
▓▓▓▓▓▓▓▓▓▓█▓░▒░▓██ ▒░██ ███▒██░░ ▓█▓▓▓█░█▓█▒  ▒██▓ ░░██▒░▒░▒██░░░▒▓█ ▒▒█
▓▓▓▓▓▓▓▓▓▓▓█████▓███████▓▓██▓▓████▓▓▓▓▓██▓▓████▓▓████▓██████▓▓██████████
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█