Trying Zig with C web servers

Tags:
code
zig
nix
h2o
facil.io
lwan

I wanted to try using Zig to build an application server.

The C integration should, in theory, allow me to use an existing web server library. Here are my results.

I tried the following web servers:

H2O is usable with some workarounds.
facil.io works in principle but is unusable in practice.
LWAN was a complete failure.

TL;DR: C translation is obviously not production ready, but I was able to get it to work with H2O.

H2O

This is the only one I had success with.

I ported the simple.c example to Zig, following this intro to libh2o.

Here you can download the entire project.

A few problems came up which I want to briefly discuss below.

Zig cannot translate C bitfields

H2O makes use of C bitfields throughout its codebase. Unfortunately, Zig does not (yet) understand them. There was an effort to fix this but it seems to have stagnated:

issue #1499 "translate-c: support C bitfields" (open)
PR #4165 "Add support for C bitfields" (closed)

Consequently we have to remove all bitfields from H2O. Fortunately using Nix it is trivial to apply a patch and use the recompiled H2O:

pkgs.h2o.overrideAttrs (oldAttrs: {
  patches = [ ./h2o-no-bitfields.patch ];
});

libuv

H2O has its own event loop implementation that is faster than libuv, so for a simple demo I did not want to link against libuv. So without libuv in the build environment obviously the header file cannot be found.

./src/c.zig:1:20: error: C import failed
pub usingnamespace @cImport({
                   ^
/nix/store/…-h2o-2.3.0-beta2-dev/include/h2o/socket/uv-binding.h:27:10: note: 'uv.h' file not found
#include <uv.h>

We can fix that be setting the H2O_USE_LIBUV preprocessor variable.

pub usingnamespace @cImport({
    @cDefine("H2O_USE_LIBUV", "0");
    @cInclude("h2o.h");
});

If you want to compile H2O without libuv support you need to pass that to H2O’s CMakeLists.txt. Using Nix that looks like this:

pkgs.h2o.overrideAttrs (oldAttrs: {
  cmakeFlags = [ "-DDISABLE_LIBUV=1" ];
});

`#define H2O_STRLIT`

H2O passes strings around with an additional length parameter using the preprocessor.

h2o_iovec_init(H2O_STRLIT("default");

That expands to:

h2o_iovec_init("default", 7)

That H2O_STRLIT macro cannot be used in Zig code, so we will have to type the length ourselves.

For common use cases you can write little wrapper functions to make it more comfortable:

pub fn iovec_init(data: var) h2o_iovec_t {
    return h2o_iovec_init(data, data.len);
}

Zig stdlib bug when linking libc

Setting up the listening socket is easier than in C. Zig’s standard library handles everything for us:

var server = std.net.StreamServer.init(.{});
defer server.deinit();
try server.listen(try std.net.Address.parseIp4("127.0.0.1", 12345));

Here I hit issue #4797 in the standard library, but that was quickly solved by a Zig contributor.

In case you are using a version before commit 778dbc17acce77588a46a27074ecb8b418786d94, there is a workaround.

Make a copy of the stdlib.

Replace the setsockopt() signature in std/os.zig:

- pub fn setsockopt(fd: fd_t, level: u32, optname: u32, opt: []const u8) SetSockOptError!void {
+ pub fn setsockopt(fd: fd_t, level: u32, optname: u32, opt: []u8) SetSockOptError!void {

Use a compiler binary that is at a path relative to your stdlib:

Zig locates lib files relative to executable path by searching up the filesystem tree for a sub-path of lib/zig/std/std.zig or lib/std/std.zig. Typically the former is an install and the latter a git working tree which contains the build directory.

— Contribution Guidelines

H2O_TOKEN_…

Similarly Zig can not parse the H2O_TOKEN_… macros. There is a h2o_lookup_token() function that we can use though.

To avoid looking up tokens all the time, we can assign all the tokens on startup of our program to our own constants.

// lookup_token() cannot run comptime so these have to be vars...
pub var TOKEN_CONTENT_LENGTH: *const h2o_token_t;
pub var TOKEN_CONTENT_LOCATION: *const h2o_token_t;
pub var TOKEN_CONTENT_RANGE: *const h2o_token_t;
pub var TOKEN_CONTENT_TYPE: *const h2o_token_t;
// …

pub fn _init() void {
    TOKEN_CONTENT_LENGTH = lookup_token("content-length");
    TOKEN_CONTENT_LOCATION = lookup_token("content-location");
    TOKEN_CONTENT_RANGE = lookup_token("content-range");
    TOKEN_CONTENT_TYPE = lookup_token("content-type");
    // …
}

pub fn lookup_token(name: []const u8) *const h2o_token_t {
    return h2o_lookup_token(&name[0], name.len);
}

Admittedly that is not very nice. Maybe we could build the H2O_TOKEN_… structs directly and assign them to actual consts.

Converting `h2o_iovec_t` to slices

We can simplify here with a little helper.

pub fn _from_iovec(data: h2o_iovec_t) []u8 {
    return data.base[0..data.len];
}

That allows us to use std.mem.eql() instead of h2o_ismem(), if desired.

if (!std.mem.eql(u8, c.h2o._from_iovec(req.*.method), c.h2o.METHOD_GET)) return -1;

Conclusion

Since this was the only web server I could get to work, there is still work to do in the C translation. But that is to be expected from such an early version of the language.

I find it remarkable that C translation works so well already. Very promising!

facil.io

I was able to start the server and answer a request.

Unfortunately Zig is unable to translate fiobj_free(). Therefore I did not think it was worth investigating further.

Download all files.

const std = @import("std");

const c = @cImport({
    @cInclude("fio.h");
    @cInclude("http.h");
});

pub fn main() !void {
    const HTTP_HEADER_X_DATA: c.FIOBJ = c.fiobj_str_new("X-Data", "X-Data".len);
    defer c.fiobj_free(HTTP_HEADER_X_DATA); // FIXME https://github.com/ziglang/zig/blob/8ea0a00f406bb04c08a8fa4471c3a3895f82b24a/src-self-hosted/translate_c.zig

    const socket_uuid = c.http_listen("3000", null, .{
        .on_request = on_request,
        .log = 1,
        .on_upgrade = null,
        .on_response = null,
        .on_finish = null,
        .udata = null,
        .public_folder = null,
        .public_folder_length = 0,
        .max_header_size = 0,
        .max_body_size = 0,
        .max_clients = 0,
        .tls = null,
        .reserved1 = 0,
        .reserved2 = 0,
        .reserved3 = 0,
        .ws_max_msg_size = 0,
        .timeout = 0,
        .ws_timeout = 0,
        .is_client = 0,
    });
    if (socket_uuid == -1) return error.Listen;

    c.fio_start(.{ .threads = 1, .workers = 1, });
}

fn on_request(request: [*c]c.http_s) callconv(.C) void {
    const body_ = "Hello World!\r\n";
    var body: [body_.len]u8 = undefined;
    std.mem.copy(u8, body[0..], body_[0..]);

    _ = c.http_send_body(request, body[0..], body.len);
}

./zig-cache/o/…/cimport.zig:2918:24: error: unable to translate function
pub const fiobj_free = @compileError("unable to translate function");
                       ^
./src/main.zig:8:12: note: referenced here
    defer c.fiobj_free(HTTP_HEADER_X_DATA);
           ^

LWAN

Those macros can clearly not easily be translated to Zig, so the journey ends abruptly.

#define LWAN_HANDLER_REF(name_) lwan_handler_##name_
#define LWAN_HANDLER(name_)                                                    \
    static enum lwan_http_status lwan_handler_##name_(                         \
        struct lwan_request *, struct lwan_response *, void *);                \
    static const struct lwan_handler_info                                      \
        __attribute__((used, section(LWAN_SECTION_NAME(lwan_handler))))        \
            lwan_handler_info_##name_ = {.name = #name_,                       \
                                         .handler = lwan_handler_##name_};     \
    static enum lwan_http_status lwan_handler_##name_(                         \
        struct lwan_request *request __attribute__((unused)),                  \
        struct lwan_response *response __attribute__((unused)),                \
        void *data __attribute__((unused)))

H2O