← Back to AILP Home

Async file I/O

The runtime's async file I/O surface — npk_write_file_async, npk_read_file_async, npk_file_exists_async, npk_delete_file_async — runs on a separate 4-thread I/O pool that is independent of the user-level coroutine executor. In v0.31.x there is no await bridge between the two; observation is via spin-poll on npk_future_is_ready plus the v0.31.0.7 disambiguator npk_future_is_error.

This page documents the user-surface idiom. The fixtures it references — bug354 / bug355 / bug356 — are the regression coverage.

Why spin-poll, not await

The two future types are different:

The Phase 1 user-level await resolves only the first kind. Bridging the second kind into the coroutine awaiter is a real engineering project (it needs a custom llvm.coro.suspend / resume integration, a wakeup callback from the I/O pool into the executor's run queue, and a thread-safety story for the shared Future state). It is explicitly deferred out of v0.31.x.

Until that bridge lands, file I/O is observed from .npk by declaring the C-API as extern and spin-polling.

The C-API surface (extern declarations)

Use the standalone extern func: form, not the extern "libc" { ... } block — the block form rejects wild int8* return types as of v0.31.x.

extern func:npk_write_file_async  = wild int8*(string:path, string:content);
extern func:npk_read_file_async   = wild int8*(string:path);
extern func:npk_file_exists_async = wild int8*(string:path);
extern func:npk_delete_file_async = wild int8*(string:path);

extern func:npk_future_is_ready   = bool(wild int8*:fut);
extern func:npk_future_is_error   = bool(wild int8*:fut);
extern func:npk_future_get_value  = wild int8*(wild int8*:fut);
extern func:npk_future_destroy    = NIL(wild int8*:fut);

Each npk_*_async call returns an opaque wild int8* (NitpickFutureHandle) that the user must eventually pass to npk_future_destroy.

Pointer-syntax gotchas

Three syntax rules that bit during fixture work:

  1. Inside extern declarations: pointers use * (the C ABI form). wild int8* is the correct extern spelling.
  2. Inside .npk function bodies: pointers use -> (the fat-pointer form). wild int8->:fut is the correct spelling for a local variable that holds an NitpickFutureHandle.
  3. string literals decay to const char* across extern boundaries. Declaring path/content arguments as string:name is the path of least resistance — it saves you from having to materialise a wild int8-> from a string literal at the call site.
// CORRECT:
wild int8->:fut = npk_read_file_async("/tmp/x.txt");

// WRONG — `*` in body:
wild int8*:fut = npk_read_file_async("/tmp/x.txt");
// "Use '->' for pointers in Nitpick code, not '*'..."

// WRONG — string literal where extern wants wild int8*:
extern func:npk_read_file_async = wild int8*(wild int8*:path);
let fut = npk_read_file_async("/tmp/x.txt"); // "expects 'int8@'"

The is_ready + is_error disambiguation idiom

The runtime Future has three states: PENDING, READY, ERROR. The accessor npk_future_is_ready returns true for both READY and ERROR. To distinguish success from failure after the future settles, the v0.31.0.7 slice added npk_future_is_error:

is_ready is_error Meaning
false false Still pending.
true false Settled successfully.
true true Settled with error.

The polling shape:

wild int8->:fut = npk_read_file_async("/tmp/x.txt");

int32:spins = 0i32;
while (spins < 100000i32 && npk_future_is_ready(fut) == false) {
    spins = spins + 1i32;
}

if (npk_future_is_ready(fut) == true) {
    if (npk_future_is_error(fut) == false) {
        // success — read the value
    } else {
        // I/O failed (e.g. missing file, permission denied)
    }
}
npk_future_destroy(fut);

The spin bound (100000 in the fixtures) is a stopgap. In production code, prefer a bounded wait with a sleep — or better, wait for the awaiter bridge to land in a later cycle.

Example: write a file (bug354)

extern func:npk_write_file_async = wild int8*(string:path, string:content);
extern func:npk_future_is_ready  = bool(wild int8*:fut);
extern func:npk_future_is_error  = bool(wild int8*:fut);
extern func:npk_future_destroy   = NIL(wild int8*:fut);

func:failsafe = int32(tbb32:err) { exit 1; };

func:main = int32() {
    wild int8->:wfut = npk_write_file_async("/tmp/nitpick_demo.txt", "hello");

    int32:spins = 0i32;
    while (spins < 100000i32 && npk_future_is_ready(wfut) == false) {
        spins = spins + 1i32;
    }

    if (npk_future_is_error(wfut) == false) {
        println("WRITE-OK");
    }
    npk_future_destroy(wfut);
    exit 0;
};

Example: error path (bug355)

The user-facing contract for a missing file is "is_ready goes true and is_error is also true". The fixture polls until ready then asserts both halves:

wild int8->:rfut = npk_read_file_async("/tmp/does_not_exist.txt");

int32:spins = 0i32;
while (spins < 100000i32 && npk_future_is_ready(rfut) == false) {
    spins = spins + 1i32;
}

if (npk_future_is_ready(rfut) == true) { println("READ-READY"); }
if (npk_future_is_error(rfut) == true) { println("READ-ERROR"); }
npk_future_destroy(rfut);

Without the npk_future_is_error accessor (i.e. on every pre-v0.31.0.7 build), there is no way from .npk to tell a successful read of an empty file from a failed read of a missing file. That gap is exactly what v0.31.0.7 closed.

Example: write → read → compare → delete roundtrip (bug356)

The full roundtrip uses an extra extern to byte-compare the read-back content. npk_future_get_value is declared as returning string so the comparison can use libc strcmp directly:

extern func:npk_future_get_value = string(wild int8*:fut);
extern func:strcmp               = int32(string:a, string:b);

The compare:

string:buf = npk_future_get_value(rfut);
int32:cmp  = strcmp(buf, "nitpick-async-d9-roundtrip-payload");
if (cmp == 0i32) { println("CONTENT-MATCH"); }

The full fixture (tests/bugs/bug356_async_file_roundtrip_pass.npk) chains write → spin → read → spin → compare → delete → spin, with explicit markers (WRITE-OK, READ-OK, CONTENT-MATCH, DELETE-OK, END) verified in order by the runner's expect_run_lines_in_order awk-based assertion.

A note on helper functions

A naive refactor would push the spin-poll into a helper:

func:wait_future = NIL(wild int8*:fut) {        // does NOT compile
    while (npk_future_is_ready(fut) == false) { /* ... */ }
};

This currently fails for two reasons:

  1. The wild int8* extern-pointer shape is not valid in a .npk function-body parameter (use wild int8->).
  2. Even with the body type corrected, calling wait_future(fut) from main emits an "Unused result value" error — the NIL-returning sync helper is being flagged as a non-drop'd call.

Both are surface-syntax gaps. Until they are fixed, the spin-poll has to be inlined at each call site. The three v0.31.0.7 fixtures all do this.

Known limitations of the file I/O surface

See also