Multi-Client Automation & Flat Architecture

#

This is the fourth post in the development story of WindRPC, a lightweight framework designed to support Remote Procedure Calls (RPC) between microcontrollers and host applications.

https://github.com/micro-artwork/windrpc

Introduction

#

Work resumed after a hiatus of nearly eight months. During this period, as AI agents became significantly more capable, I leveraged their assistance to optimize the C server logic of WindRPC and build comprehensive unit tests step by step.

Logic Optimization & Multi-Client SDK Generation

#

Initially, the desktop client was developed primarily for C# (WinForms). The focus was straightforward: using the C# message classes generated by the Protobuf compiler, sending them over a serial port, and receiving responses.

As I began utilizing AI agents more actively, UI/UX development accelerated dramatically. This prompted a serious re-evaluation: shifting the primary desktop client target from C# to Electron (JS/TS).

To support the Electron environment, a JavaScript/TypeScript client communication library was essential. Transmitting Protobuf messages generated from .proto files over serial and receiving responses from the MCU was conceptually simple.

However, expanding from C# only to supporting JS/TS revealed a long-term challenge: maintaining separate client implementations across multiple languages manually would quickly become error-prone and hard to maintain.

Therefore, I decided that the code generator should produce not only the C server code, but also the full client SDKs for both C# and JS/TS. The generator script (windrpc_gen.py) was extended so that a single specification descriptor (user_spec.yml) automatically outputs client communication SDKs (WindRpcClient.cs, WindRpcClient.js) for both language ecosystems.

// Example usage of the generated Electron JS/TS Client SDK
import { WindRpcClient, decodePowerManagerPowerStatus } from './windrpc/WindRpcClient.js';

const client = new WindRpcClient();

// 1. Bind binary serial RX handler
serialPort.on('data', (chunk) => {
    client.receiveBytes(chunk, (notification) => {
        // Handle server push notifications (events)
        if (notification.rpcId === 0x0882) {
            const alert = decodePowerManagerPowerStatus(notification.payload);
            console.log('Server Push Alert Received:', alert);
        }
    });
});

// 2. Perform Request-Response RPC call (async/await)
async function fetchPowerStatus() {
    const responseFrame = await client.sendRequest(
        0x0801, // (Service ID 8, RPC ID 1)
        new Uint8Array(0),
        (frame) => serialPort.write(frame),
        3000 // Timeout in ms
    );
    return decodePowerManagerPowerStatus(responseFrame.payload);
}

Full-Code Generation & Architectural Re-evaluation

#

With the automated generation of C server engines alongside C# and Electron JS/TS client SDKs, complete Full-Code Generation across the entire communication pipeline was achieved.

Once full-code generation became a reality, a fundamental architectural question emerged:

“If code generation tools handle all serialization and transport plumbing automatically, is there any reason to maintain the complex, nested Protobuf message structure that was originally designed for human readability?”

Initially, the message schema used nested oneof fields under ClientMessage to make .proto files easy for humans to read and inspect. The .proto schema was intended to act as the specification itself. However, due to its limitations, a YAML specification descriptor was introduced, making YAML the true specification while .proto files served merely as data containers.

Given that all code is auto-generated, maintaining human-friendly nested message structures offered no remaining benefit.

Ultimately, as long as the internal implementation remains a clean black box, application developers only need to invoke generated functions/methods with expected inputs and wait for responses.

Hybrid Packet Format (6-Byte Header + Protobuf Payload)

#

To optimize performance and eliminate NanoPB decoding callbacks on MCU servers, a simpler packet structure with static memory bounds was designed.

This led to the creation of the ‘Hybrid Packet Format’ and the Flat Architecture:

[ WindRPC 6-Byte Binary Header + Protobuf Payload Structure ]

+-------------------+-------------------+-------------------+-----------------------+
|  RPC ID (2 Byte)  |  Seq ID (2 Byte)  | Payload Len (2B)  | Protobuf Payload      |
+-------------------+-------------------+-------------------+-----------------------+
| (Service << 8)    | Transaction ID    | Raw Payload Bytes | Serialized Data Bytes |
|   | RPC ID        |                   | Length            |                       |
+-------------------+-------------------+-------------------+-----------------------+

Instead of wrapping the entire packet inside a top-level Protobuf message, the new hybrid approach prefixes each packet with a fixed 6-byte raw binary header, followed directly by a flat Protobuf payload.

This hybrid model combines the simplicity of raw binary headers with the reliability of Protobuf serialization for payload fields, reducing custom protocol parsing while keeping message definitions minimal.

• RPC ID (2 Bytes): A 16-bit identifier combining Service ID and RPC ID ((service_id << 8) | rpc_id)
• Sequence ID (2 Bytes): Transaction sequence number for matching asynchronous requests and responses
• Payload Length (2 Bytes): Byte length of the trailing Protobuf payload
• Protobuf Payload: A simple, flat Protobuf message containing only procedure parameters or return values

Advantages of Flat Architecture & Unification

#

The Hybrid Flat Architecture provided major performance advantages over the previous nested approach:

1. O(1) Lookup Table Dispatching

   • Eliminates tree-traversal callbacks and tag comparisons when routing incoming requests.
   • The MCU reads the 2-byte RPC ID from the 6-byte header and immediately dispatches the handler function via a C lookup table with O(1) complexity.

2. Elimination of NanoPB Callbacks & Guaranteed Zero-Heap Allocation

   • Enforces static memory constraints (nanopb: max_count / max_size) in the YAML specification.
   • NanoPB decodes and encodes payloads in a single pass into fixed C structures without dynamic memory allocation (malloc/free) or pb_callback_t handlers.

Initially, both Nested and Flat modes were supported as generator options.

However, after deploying and running the Flat architecture in the Bitnari project, it proved to be significantly more stable, efficient, and easier to maintain.

Consequently, the legacy Nested mode was fully deprecated and removed from the codebase, solidifying the Flat Hybrid Architecture as the single standard for WindRPC v1.0.

Conclusion

#

Comprehensive test scripts were added to verify generated code integrity across C, JS/TS, and C# targets. While balancing daily life and parenting leaves limited time, WindRPC will continue to evolve alongside the Bitnari project and future hardware endeavors.

This concludes the architectural journey of WindRPC. Future posts will cover API usage guides, integration tutorials, and lessons learned.