buffer — Raw Mutable Byte Buffer

v0.54.3 — Introduced as part of the Uncovered Types Hardening series (UTH-006..010).

1. Overview

buffer is Nitpick's raw mutable byte region type. It gives you a fixed-capacity block of memory that you can read and write at any byte offset, with bounds checking on every access.

buffer is not a serializer. It does not perform any encoding, endian conversion, or structured layout. For structured binary encoding, use binary.

When to use buffer

2. Creating a Buffer

Import the standard library module and call buf_new:

use "buffer.npk".*;

int64:buf = raw buf_new(256i64);   // 256-byte heap buffer

buf_new returns an opaque int64 handle. A return value of 0 means allocation failed. Always check before use.

if (buf == 0i64) { exit 1; }

Stack-backed buffers (future)

buf_new_stack is provided as a named alias that documents intent, but it currently allocates on the heap. True stack-backed buffers are deferred pending compiler support.

int64:scratch = raw buf_new_stack(64i64);   // heap for now

3. Writing Bytes

Single byte

_ = raw buf_write_byte(buf, 0x42i32);    // write one byte; returns 0 on success, -1 if full

The write cursor advances by 1 on each successful write. If the buffer is full, returns -1 and the cursor does not advance.

Fill a region

_ = raw buf_fill(buf, 0i32, 64i64);    // zero-fill 64 bytes at current cursor
_ = raw buf_fill(buf, 0xFFi32, 16i64); // fill 16 bytes with 0xFF

buf_fill writes count copies of the byte value at the current cursor and advances by count. Returns bytes written, or -1 on overflow.

Copy bytes from a string

_ = raw buf_write_bytes(buf, "hello", 5i64);    // writes 5 bytes from "hello"

Copies len bytes of src into the buffer at the current cursor. Returns bytes written, or -1 if not enough capacity remains.

4. Reading Bytes

Read uses an absolute offset (not cursor-relative):

int32:b = raw buf_read_byte(buf, 0i64);    // read byte at offset 0
int32:b = raw buf_read_byte(buf, 7i64);    // read byte at offset 7

Returns the byte value [0..255] on success, or -1 if the offset is out of bounds (negative, or ≥ buf_len).

5. Buffer State

int64:len  = raw buf_len(buf);           // bytes written so far (write cursor)
int64:cap  = raw buf_capacity(buf);      // total capacity
bool:fits  = raw buf_fits(buf, 100i64);  // true if 100 more bytes fit

Clearing

drop raw buf_clear(buf);    // reset write cursor to 0 (does NOT zero memory)

buf_clear resets the internal length counter to 0, making the buffer logically empty again. It does not zero the underlying memory — subsequent reads of unwritten offsets will see stale bytes.

6. Copying Between Buffers

buf_copy copies bytes from src's cursor position into dst at dst's cursor:

int64:src = raw buf_new(256i64);
int64:dst = raw buf_new(256i64);

// ... write to src ...

int64:copied = raw buf_copy(dst, src, 64i64);   // copy 64 bytes src → dst

Both cursors advance by the number of bytes copied. Returns bytes copied, or -1 if: - src has fewer than len bytes available - dst does not have room for len bytes

7. Viewing as String

string:s = raw buf_as_string(buf, 5i64);   // view first 5 bytes as a Nitpick string

Returns a string backed by the buffer's internal memory. The string is only valid while the buffer is alive — do not free the buffer and then use the string.

Returns "" if: - h is 0 - len <= 0 - len > buf_len(h) (you cannot view more bytes than have been written)

Zero-Copy Receive Pattern

A common networking pattern is receiving raw bytes into a buffer, then interpreting them as a string without a secondary allocation:

int64:recv_buf = raw buf_new(4096i64);
// ... FFI call fills recv_buf with 1024 bytes ...
string:payload = raw buf_as_string(recv_buf, 1024i64);
// Evaluate payload directly. Do NOT free recv_buf until done with payload.

8. Performance Notes

• Pre-allocation: Buffers have a fixed capacity and do not auto-resize. Pre-allocate large buffers upfront rather than frequently creating/destroying smaller buffers, especially in tight loops.
• Avoid Frequent Reallocation: If you require dynamic resizing, it must be implemented manually via buf_new, buf_copy, and buf_free. This is expensive. Consider allocating the maximum expected size initially.
• Copy Costs: While buf_copy is optimized at the libc level, memory moves still incur latency. Avoid unnecessary intermediate buffer copies.

8. Memory Management

Always free the buffer when done:

drop raw buf_free(buf);

Use defer for automatic cleanup in scoped code:

int64:buf = raw buf_new(1024i64);
defer drop raw buf_free(buf);

// ... use buf freely ...
// buf is freed when scope exits

buf_free releases both the 24-byte header block and the data region. Passing 0 to buf_free is a no-op (safe).

9. Bounds Safety

Every read and write operation is bounds-checked:

No operation will write past the allocated region. Buffer corruption and out-of-bounds writes are prevented at the API level.

10. FFI Usage

To pass a buffer's raw data pointer to a C function, read the data pointer from the header at offset 16:

extern "nitpick_libc_mem" {
    func:nitpick_libc_mem_read_i64 = int64(int64:ptr, int64:offset);
}

int64:data_ptr = nitpick_libc_mem_read_i64(buf, 16i64);
// data_ptr is now a raw uint8_t* — pass to C extern

[!WARNING] The data pointer is only valid while the buffer is alive. Do not store it past a buf_free call.

11. Complete API Table

12. binary vs buffer Comparison

13. Known Limitations

• No stack-backed buffers — buf_new_stack is currently identical to buf_new. True stack allocation requires compiler support (deferred).
• No typed views — there is no buf_read_int32 or similar. Use binary for typed reads, or manually reconstruct values from individual bytes.
• Fixed capacity — buffer does not auto-resize. If you need a growing buffer, allocate generously, track utilization with buf_len/buf_capacity, and reallocate manually if needed.
• No K-semantics — buffer is not formally modeled in k-semantics/nitpick.k. It is an opaque int64 handle; semantics are entirely defined by stdlib/buffer.npk and nitpick-libc.