An Example Production Scenario Using R.A.I.N. (Part 1)
Memory Safety & Finance with AI??!
There's a lot of hand-waving out there in vibe coding blogs and videos, a lot of people talking the talk but never walking the walk. Likewise, there's a lot of people saying that they'd never trust vibe coded software with their financial data.
Strap intight, because today, we're going to start walking through an example of how to build real application functionality with ChatGPT. We're going to build tested, performant, memory-safe double entry ledger functionality for money tracking in C. That's pretty much everything the doubters will tell you AI can't do.
This is going to be a mini-series here on Nurture the Vibe, it'll take a few posts to get it done because we're going to go through all the thought process as well as the action.
Getting Set Up
If you want to follow along, you'll need a few tools:
A Decent Reasoning Model
Any decent reasoning model with 'Deep Research' type functionality is fine. ChatGPT, Grok 3, Gemini 2.5 Pro, or Claude 3.7 Sonnet are all perfectly capable.
You may need a premium subscription or some credits for the API to access 'Deep Research'.
C-specific tooling
I'm on a Mac, so I'll be using XCode's built in tools (install XCode from the App Store). If you're using Windows, you'll want Build Tools for Visual Studio, and on Linux, your distribution's build-essential
package.
You'll also need check, a common testing library for C.
An Editor
You'll also need a text editor, ideally one with syntax highlighting. Even better but completely optional, one that can hook into your LLM of choice's API so that you don't have to keep flicking between your editor and your browser.
But what about {X Hype Tool}?
You don't need any expensive subscriptions or expensive tooling to get started building with AI. If you want to use Cursor, v0, replit, Co-Pilot or any other tools, they'll make your life easier but they're absolutely optional.
Let's Get Started
So, we're building a piece of money tracking software for a financial institution. It can process a transaction between account A and account B, and keep track of the balances. Put the kettle on and we'll build out a simple spec.
Functionality
- It needs to be auditable, that probably means double-entry bookkeeping -- if it sounds scary, it just means that every transaction has to have a debitor and a creditor so that if we reconcile every account, it'll add up to zero
- We're building in C, so it has to be memory safe, we need to make sure that every
malloc
(memory allocation) has a correspondingfree
- We will have internal accounts & external accounts, we don't know the actual balance of external accounts, but we need them in the system so that we can perform our reconciliation
- We should reject any transaction that would make an internal account's balance drop below zero (this constraint does not apply to external accounts)
Constraints
To keep this task from ballooning and keep things moving fast, we're going to make some assumptions:
- For now we're only interested in getting the logic right, persistence can come later, so everything will be stored as structs in memory
- We're not going to worry about transaction timestamps, we'll assume that ingress order is the correct order to process in
- We're going to process every transaction in a single thread
- We're not going to add overdrafts etc at this stage
- We're not going to use checkpointing, we'll just process every transaction to get an account balance
We will ask the AI to consider how we'd manage some of these things later, but they're out of scope for this task
Architect-Level View
Introduction
When I start a new ticket, I like to start with Deep Research to set up an architect-level view of the functionality with an eye to breaking it down.
Speed tip: I'll often do a bunch of these in parallel, as soon as I have the tickets specified, because they take a while to process.
Initial Prompt & Answers
Here's the prompt I started with (ChatGPT o3 with Deep Research on) based on our above specification & constraints:
## Role
You are a software architect with experience in writing robust, memory-safe and performant software.
## Assignment
I need you to plan out a performant C function which takes a transaction struct with these fields:
- Origination Account ID (u32)
- Destination Account ID (u32)
- Amount (u32, in cents)
It should generate double entry ledger outputs allowing the movement of money to be reconciled. These can be structs for now (you should design the struct), we will deal with persisting them later.
You should also plan a performant C function which, given an array of every transaction struct, can generate the balance for a given u32 account ID.
I'd also like you to provide a complete testing strategy so that we can ensure that this functionality is robust and reliable, as well as memory-safe and performant.
You should consider whether there is information that you need that I haven't provided, and ask me for it rather than making assumptions.
## Information
For now, we will call the functions and stick all of the transactions in memory, consider the best way to handle this dynamically resizing array.
There are two types of account, you should plan how these will be represented in memory:
- External accounts are outside of our domain, we do not know the balances for these accounts so we should process any transaction from them regardless of balance
- Internal accounts are inside of our domain, we will not process a transaction that would take an internal account's balance below zero
## Necessities
The functionality must:
- be memory-safe
- error gracefully if we cannot allocate enough memory (assume our application has a 256MB maximum allocation)
- be written in a way that it's easy to refactor to use checkpointing so that we can avoid re-processing the entire event log every time
- be written in a way that it's easy to add overdrafts to accounts
- be testable, including testing OOM errors, invalid transactions, reconciliation, auditability
- be auditable (we should be able to see exactly how an account got in the state it was in)
ChatGPT came back with some questions. I've put my answers beneath the questions to make it readable, but these were two separate messages:
Thanks for the detailed requirements. To plan the C functions and memory layout effectively, could you clarify a few things:
Do we have a predefined list of internal vs external accounts, or should the system determine this dynamically? If it's predefined, how is that information stored?
We do not, you should think about how to store them in memory.
Should we support multiple currencies or just a single currency (e.g., USD)?
We can support USD only for now, but it would be good to consider how we could support multiple currencies (and exchanges between currencies) in future.
Is transaction ordering guaranteed (e.g., based on ingestion order or a timestamp)?
We can assume ingestion order is correct for now.
Should the system allow duplicate transactions (same origination, destination, and amount)?
Perhaps every transaction should have an ID and we should refuse any transaction whose ID has already been processed. How can we do this in a performant way?
Should double-entry records have additional metadata (e.g., timestamp, transaction ID, memo)?
Let's add timestamp & transaction ID.
Will account IDs always be unique globally, or is there any scoping (e.g., by organization)?
Account IDs are globally unique.
A Quick Note on These Decisions
I'm relentlessly de-scoping here because I don't want this 'ticket' to get out of control. The trick to building resilient software when you're using AI is to build in small increments.
There are a lot of things here which should really be handled upstream of transaction processing: ordering of transactions, account ID management and even currency swaps are out of scope.
I threw the AI a curveball on the currency swap question. The correct way to do this looks something like this:
Pete's USD account -> Bank USD account ($200)
Bank GBP account -> Pete's GBP account (£150)
With the swapping & exchange rate calculations done in another service. This isn't a responsibility of transaction processing, and including it here will make the service brittle and slow if it has to do currency lookups. The only way currencies would interact with this service in a production service is that the service would probably check that the transaction currency matches the origin & destination account currencies.
AI Response
This is what ChatGPT came back with (it's a lengthy read):
Click to expand
Thanks for the clarification. I will design a performant and memory-safe C system that:- Processes transaction structs with IDs, timestamps, and account info.
- Produces double-entry ledger structs that are auditable and reconcilable.
- Maintains account balances while differentiating between internal and external accounts.
- Prevents duplicate transactions using an efficient lookup mechanism.
- Handles dynamic memory allocation safely, within a 256MB cap.
- Is structured for easy extension to overdrafts, currency support, and checkpointing.
I'll also include a robust testing strategy to cover correctness, performance, memory use, and auditability. I’ll get back to you with a complete plan shortly.
Designing a Performant and Memory-Safe Transaction Ledger System in C
Overview and Objectives
We aim to design a C-based transaction processing system that is both performant and memory-safe. The system will ingest a list of financial transactions and maintain account balances using a double-entry ledger model. Key features include:
- Double-Entry Recording: Every transaction yields two equal and opposite ledger entries (debit and credit) to ensure consistency. This supports full auditability, as the sum of debits equals the sum of credits for internal accounts, preventing "lost" money.
- Internal vs External Accounts: Internal accounts are tracked within our system and must never have a negative balance (unless overdraft is allowed). External accounts represent outside entities; we do not strictly track their running balances, and they can show negative net positions (since a negative balance simply means the external party has received net funds from the system).
- Balance Computation: Functions will derive account balances from the transaction history (ledger) rather than relying on stored balances, ensuring consistency with the source of truth. This derived approach eases audits and avoids update anomalies that can occur with duplicated stored balances.
- Duplicate Prevention: Each transaction carries a unique ID. The system will efficiently detect and reject duplicate transaction IDs to prevent double-processing (e.g. double spending).
- Memory Management: We will use dynamic memory carefully to stay within a hard cap of 256 MB. Data structures and algorithms are chosen to be memory-efficient and safe (preventing buffer overruns, leaks, etc.), and all allocations are checked for success to handle out-of-memory (OOM) conditions gracefully.
- Extensibility: The design anticipates future requirements such as overdraft limits on internal accounts, multi-currency transactions with exchange rates, and checkpointing of balances to avoid replaying the entire transaction log for each query. We choose flexible data structures and code organization to accommodate these enhancements with minimal refactoring.
The following sections detail the data structures, functions, memory layout, error handling, and testing strategy for this system. Code snippets (in C99) are provided to illustrate key structures and logic. Diagrams and examples are used where appropriate to clarify the double-entry bookkeeping process and data flow.
Data Structures
Transaction Structure (Transaction
)
Each incoming transaction is represented by a Transaction
struct. This struct captures all core information about a financial transfer:
#include <stdint.h>
typedef struct {
uint64_t id; // Unique Transaction ID
uint64_t timestamp; // Timestamp (seconds since Unix epoch)
uint32_t src_account; // Originating Account ID
uint32_t dest_account; // Destination Account ID
uint32_t amount; // Amount in cents (unsigned)
} Transaction;
id
: A unique identifier for the transaction (64-bit). Using a 64-bit ID accommodates a very large number of transactions over the system’s lifetime and minimizes the risk of collisions. This ID will be used to detect duplicates efficiently.timestamp
: The time of the transaction (64-bit to handle dates far in the future). Storing the timestamp allows sorting or ordering of transactions and supports time-based queries or checkpointing.src_account
anddest_account
: 32-bit account identifiers for the source (originator) and destination of funds. We useuint32_t
which supports over 4 billion distinct accounts, balancing range and memory footprint (4 bytes each).amount
: The transfer amount in cents, stored as a 32-bit unsigned integer. Using cents (integer) avoids floating-point rounding issues for currency. The maximum of ~$42 million (for 32-bit) per transaction is acceptable for typical use; this can be extended by using 64-bit if needed for very large transfers.
Memory Layout: The Transaction
struct is likely 32 bytes due to alignment (8-byte alignment for the 64-bit fields). In an array, each Transaction
will occupy 32 bytes (including padding). This predictable, compact layout allows iterating large transaction arrays efficiently in memory.
Ledger Entry Structure (LedgerEntry
) – Double-Entry Records
To implement double-entry bookkeeping, each Transaction
is converted into two LedgerEntry
records: one for the debit (outflow from the source account) and one for the credit (inflow to the destination account). The LedgerEntry
struct is designed for auditability and future extensibility:
typedef struct {
uint64_t txn_id; // Associated Transaction ID (links back to original transaction)
uint32_t account_id; // Account affected (could be internal or external)
int64_t change; // Change in balance for this account (positive = credit, negative = debit)
uint64_t timestamp; // Timestamp of the transaction (for sorting/audit)
uint16_t flags; // Flags/metadata (e.g., bit0: internal/external, bit1: debit/credit if needed)
uint16_t currency; // Currency code or ID (for future multi-currency support)
} LedgerEntry;
Key considerations for LedgerEntry
:
- Double-Entry Fields: Each entry stores the
txn_id
to tie it back to the parent transaction for reconciliation. By querying all ledger entries with a giventxn_id
, an auditor can retrieve the debit-credit pair and verify they net to zero (in the same currency). Theaccount_id
identifies which account’s ledger (internal or external) this entry belongs to. - Amount Change (
change
): We use a signed 64-bitchange
value to record the balance impact on the account. A positive value represents a credit (money added to the account), and a negative value represents a debit (money removed). Using a signed field directly encodes debit vs. credit and simplifies summation of balances. The 64-bit width accommodates large running totals and future currency conversions (e.g. summing many 32-bit transactions could exceed 32-bit range). - Timestamp: Copying the transaction timestamp into each entry simplifies ledger sorting and querying by time, without needing to dereference the original transaction. This is useful for generating account statements in chronological order or implementing checkpoints (since each entry knows when it occurred).
- Flags: A 16-bit field is reserved for metadata about the entry. We can use bit flags to mark characteristics without extra memory overhead: e.g., a bit to distinguish internal vs. external account entries, a bit for credit/debit if we want an explicit marker aside from the sign of
change
, or a bit to indicate if an overdraft was utilized. In the current design, we can derive internal/external by looking up the account in our internal set, so these flags are optional. However, reserving this space ensures future features can be added without changing the struct size (maintaining binary compatibility). - Currency: A field reserved for currency information. For now, if we operate in a single currency, this can be a constant (or an index for “USD” etc.). In the future, this can store an identifier for the currency of the transaction (or ledger entry). Keeping it as a 16-bit value (which could index into a currency table or enum) is memory-efficient. This anticipates multi-currency support where ledger entries might be recorded in different currencies and later converted (we ensure debits equal credits per currency or via a conversion account).
Auditability & Reconciliation: With this design, the ledger forms a complete audit trail. An auditor or debugging tool can iterate over all LedgerEntry
records: for each unique txn_id
, there should be two entries of equal and opposite change
(assuming a single-currency transaction). Any discrepancy (e.g., a missing entry or mismatched amount) indicates data corruption or a bug. This satisfies the fundamental accounting control that “every entry to an account requires a corresponding and opposite entry to a different account”. Moreover, value is not created or destroyed – it only moves between accounts, which our double-entry records enforce. Summing all change
values for all internal accounts will net to zero (since every debit has a matching credit) – a useful invariant to check system-wide integrity.
Memory Footprint: The LedgerEntry
is 8+4+8+8+2+2 = 32 bytes (likely padded to 32 or 40 bytes depending on alignment). We expect to store two LedgerEntry
per Transaction
. For N transactions, that’s 2N entries. This is memory-heavy but within our 256 MB cap for typical values of N (for example, ~8 million transactions would produce ~16 million entries, requiring roughly 16M * 32 = 512 MB, which is over our cap – so in practice we might not retain all entries in memory at once, see Checkpointing in Extensibility). If memory becomes a concern, we might opt to not store all ledger entries permanently in RAM, or to compress older entries after processing. Alternatively, we could store only internal-account entries and treat external accounts as a sink/source that doesn’t need a stored ledger (since we don’t track their balance). For now, we assume moderate transaction volumes or that checkpointing will alleviate memory use over time.
Internal Account Table and Balance Tracking
We maintain an in-memory table of known internal accounts to enforce rules and quickly compute balances. This table distinguishes internal vs. external accounts and stores running balances for internal accounts (for fast lookup), while external accounts are not stored (they are implicitly any account not in the internal set). We design a structure and mechanism as follows:
-
Internal Accounts Set: We use a hash set or hash map data structure keyed by
account_id
for all internal accounts. This allows O(1) average-time lookup to check if an account is internal. A boolean flag per account could suffice, but since account IDs are 32-bit and sparsely used, a hash-based structure is more memory-efficient than a giant boolean array of size 2^32. We can implement this in C by using an open-addressing hash table (to avoid pointer overhead) or a balanced binary search tree (O(log N) lookups) if N is small. Given performance goals, a hash table is preferred for internal account lookup.- Memory Safety: We allocate the hash table with a size proportional to the expected number of internal accounts (e.g., if expecting M internal accounts, allocate table size ~2M for low collision rate). We will monitor memory usage to ensure this stays within budget. All insertions check for available space; if we must expand the table, we do so carefully (rehashing) and check for OOM on reallocation.
- Dynamic Updates: Since the set of internal accounts might not be known ahead of time, the system should allow marking an account as internal at runtime. For example, if a new internal account is introduced (e.g., a new user in the system), we add its ID to this set (and possibly initialize its balance to 0). This insertion operation will handle resizing the hash if needed and is done in a thread-safe manner if concurrent (single-threaded assumption for now means no concurrent writes, but extension to multi-thread would require a lock or atomic updates around this structure).
- Alternatives: If the number of internal accounts is very large and contiguous (or has an upper bound), a fixed-size bit-array could mark internal accounts (bit set = internal). For instance, if account IDs range 0–1e6, a bit array of 1e6 bits (~125 KB) would be extremely efficient. However, for a general solution with 32-bit IDs, a dynamic set is the safer approach.
-
Account Balance Map: In addition to the set, we maintain a mapping from account ID to its current balance (for internal accounts only). This can be combined with the internal set (e.g., a hash map from account ID to an
Account
struct that includes a balance field). For example:typedef struct { uint32_t id; int64_t balance; // current balance in cents int64_t overdraft_limit; // allowed overdraft (negative limit) if any uint16_t currency; // currency of account (if multi-currency, else 0 or default) uint8_t is_internal; // boolean flag (1 for internal account) } Account;
We can maintain an array or map of
Account
. In a simple approach, we only store internal accounts in this map (because we don’t track externals’ balance). Theis_internal
flag in each entry would thus always be 1 in this map, but we include it for completeness or in case we ever store external accounts with limited info. Theoverdraft_limit
andcurrency
fields anticipate future extension (overdraft and multi-currency). Initially,overdraft_limit
can be 0 for all accounts (meaning no overdraft allowed beyond zero balance).The map allows quick updates to balances as transactions are processed, enabling us to enforce the no-negative rule in real-time: before debiting an internal account, we check its current
balance
against theamount
. Also, it provides O(1) retrieval of an account’s latest balance without scanning the entire transaction list. This is crucial for performance if balances are frequently queried (e.g., showing a user’s balance after each transaction).Memory considerations: Each
Account
is small (let’s say 24 bytes, if we pack fields efficiently). If there are, for example, 100,000 internal accounts, that’s ~2.4 MB – negligible. Even 1 million internal accounts would be ~24 MB, under our cap. The hash map overhead (pointers, etc.) will add some cost, but it's manageable. We will ensure the total of all such structures stays within 256MB. If memory gets tight, we could opt to not store balances for dormant accounts (i.e., remove or swap to disk if needed, though this is unlikely necessary unless accounts number in the millions and memory is critical).
Duplicate Transaction ID Set
To prevent processing the same transaction twice, we keep a deduplication set of seen transaction IDs. We need a fast membership check for each new transaction’s id
. The chosen data structure should handle up to the maximum number of transactions (since in worst case all are unique IDs). Options:
- A hash set of 64-bit keys (transaction IDs) – average O(1) insertion and lookup. This is effective if we have a rough idea of volume to size the table appropriately. We can implement this similarly to the internal accounts hash (open addressing or chaining). Each entry stores an 8-byte ID, and possibly a byte flag for occupancy. For millions of entries, this is memory-heavy but feasible. For example, 8 million IDs at 8 bytes each is ~64 MB just for the raw IDs, perhaps ~128 MB with overhead – acceptable within 256 MB. We must tune the load factor to avoid slow lookups.
- A sorted dynamic array of seen IDs – we could insert IDs in sorted order and use binary search for lookup (O(log N) per check). Insertion in sorted array is O(N) worst-case, making this less ideal for large N. A binary search tree (like a balanced BST or skip list) would give O(log N) insertion and lookup, but with pointer overhead and less cache-friendly layout. Given performance needs, hashing is preferred over O(log N) structures.
Implementation note: We will likely implement the dedup set as an open-addressed hash table of uint64_t
keys. Each new transaction ID is looked up; if found, the transaction is a duplicate and will be rejected (or skipped). If not found, it is inserted. Removal isn’t needed unless we choose to purge old IDs after checkpointing (see later), but generally we keep all seen IDs to prevent any duplication over the entire lifetime (assuming IDs are never reused). If memory usage of the ID set grows too large, one strategy is to use a Bloom filter to approximate duplicates with less memory – but that risks false positives (dropping valid transactions), which is unacceptable for financial data. Therefore, an exact set is required despite memory cost.
Memory Safety: All operations on the dedup set will check bounds. We allocate initial memory for the table with a size larger than the expected count to minimize rehashing. If we do need to grow the table, we allocate a new larger buffer and re-insert existing IDs, checking for OOM. We also guard against integer overflow in size calculations (e.g., when doubling the table size, ensure it doesn’t overflow 256MB or size_t
). After processing is complete (or on program shutdown), we free the memory for this set.
Functions and Algorithms
1. Processing a Transaction – Double-Entry Creation (process_transaction
)
We design a function to intake a single Transaction
and update the system state accordingly. This involves creating the two ledger entries, updating in-memory balances, and performing safety checks (duplicate detection and balance validation). Pseudocode for process_transaction
:
// Returns 0 on success, or an error code (e.g., 1 for duplicate, 2 for insufficient funds, 3 for OOM)
int process_transaction(const Transaction* tx) {
// Duplicate check
if (id_set_contains(tx->id)) {
return 1; // Duplicate transaction ID detected
}
// Prepare ledger entries
LedgerEntry debitEntry, creditEntry;
debitEntry.txn_id = tx->id;
debitEntry.account_id = tx->src_account;
debitEntry.change = -(int64_t) tx->amount; // money out of source
debitEntry.timestamp= tx->timestamp;
debitEntry.flags = 0; // set flags if needed (e.g., mark debit, internal/external)
debitEntry.currency = 0; // assuming single currency (e.g., index 0 for USD)
creditEntry.txn_id = tx->id;
creditEntry.account_id = tx->dest_account;
creditEntry.change = (int64_t) tx->amount; // money into destination
creditEntry.timestamp= tx->timestamp;
creditEntry.flags = 0; // set flags if needed (mark credit)
creditEntry.currency = 0;
// Internal account rules enforcement
bool srcInternal = is_internal_account(tx->src_account);
bool destInternal = is_internal_account(tx->dest_account);
if (srcInternal) {
Account *srcAcc = get_account(tx->src_account);
if (!srcAcc) {
// If a source internal account is not yet in our map (should not happen if all internal accounts are pre-registered)
// handle accordingly (e.g., create it with balance 0).
}
int64_t newBalance = srcAcc->balance + debitEntry.change; // change is negative
// Enforce no negative balance (or overdraft limit)
if (newBalance < -srcAcc->overdraft_limit) {
return 2; // Insufficient funds (would go beyond overdraft limit)
}
srcAcc->balance = newBalance;
}
if (destInternal) {
Account *destAcc = get_account(tx->dest_account);
if (!destAcc) {
// Possibly create new internal account record if it wasn't present (initial balance 0)
destAcc = create_internal_account(tx->dest_account);
}
int64_t newBalance = destAcc->balance + creditEntry.change; // change is positive
destAcc->balance = newBalance;
// (No need to check dest for negative, credit can only increase balance)
}
// Append ledger entries to ledger storage (if maintaining a ledger list/array)
if (!ledger_append(debitEntry) || !ledger_append(creditEntry)) {
return 3; // OOM or capacity error when storing ledger entry
}
// Mark transaction ID as seen
id_set_insert(tx->id);
return 0; // success
}
Let’s break down the logic:
-
Duplicate Check: We call
id_set_contains(tx->id)
on our global (or passed-in) ID set. If the ID is already present, we reject this transaction immediately with an error code. This prevents any further processing of duplicates. The check is O(1) average. -
Prepare Ledger Entries: We populate two
LedgerEntry
structs, one for the source (debit) and one for the destination (credit). The debit entry’schange
is-amount
(cast toint64_t
to avoid overflow ifamount
is near 2^31), and the credit’schange
is+amount
. We copy over thetxn_id
andtimestamp
so that these entries are fully self-contained records of the event. We also set any relevant flags:- We might set a flag bit to indicate the debit vs credit nature (though the sign of
change
already indicates that). - We might set a flag if the account is internal or external. However, since we can determine that via lookup when needed, we may skip storing it. Storing it could save a lookup later at the cost of using a bit. For extensibility, we could do:
if (srcInternal) debitEntry.flags |= 0x1; else debitEntry.flags &= ~0x1;
(for example, bit0 = 1 meaning internal account entry). This could be useful if we need to filter or treat entries differently in reporting.
- We might set a flag bit to indicate the debit vs credit nature (though the sign of
-
Internal Account Rules Enforcement: Before finalizing the transaction, we must ensure that no internal account’s balance would drop below 0 (or below its overdraft limit).
- If the source account is internal, we lookup its current balance (
srcAcc->balance
). We then computenewBalance = balance + debitEntry.change
. NotedebitEntry.change
is negative, so this subtracts the amount. We checknewBalance
against the allowed floor. Normally, the floor is 0 (no negative allowed). Ifoverdraft_limit
is zero (default), the condition isif (newBalance < 0) { reject; }
. If we plan for overdrafts,overdraft_limit
would be a positive number indicating how far below zero we can go (e.g., 10000 cents overdraft means balance can go to -10000). We ensurenewBalance >= -overdraft_limit
. If this fails, we return an error code (insufficient funds) and do not apply the transaction. This prevents the internal account from exceeding its allowed negative balance. - If the destination account is internal, we update its balance by adding the credit. Credits only increase balance, so there’s no risk of violating the non-negative rule here. (In a future overdraft scenario, a credit might bring a negative balance back towards zero, which is fine). If the internal dest account did not exist in our records (for instance, a new internal account receiving its first deposit), we create it on the fly via
create_internal_account
with initial balance 0, then apply the credit. This ensures we don’t miss tracking any account that becomes active.
- If the source account is internal, we lookup its current balance (
-
Ledger Storage: We assume we maintain a global ledger (e.g., an array or list of
LedgerEntry
) for audit and possibly for recomputing balances from scratch. The functionledger_append()
appends a new entry to this ledger. Under the hood, this might push the entry into a dynamic array, reallocating if necessary. We must handle the possibility of reallocation failure here. We check each append; if it fails (returns false or NULL due to OOM), we return an error code. If we cannot store the ledger entry, we also should rollback any state changes (balance updates, ID insertion) to keep the system consistent. In practice, to simplify, we might allocate the ledger array in large chunks in advance to minimize mid-processing failures. -
Duplicate ID Insertion: If all above steps succeed, we insert the new transaction ID into the deduplication set (
id_set_insert
). This marks the ID as processed. We do this at the very end only after we have successfully appended the ledger entries and updated balances, to avoid a case where an ID is marked seen but the transaction wasn’t fully applied (e.g., if ledger append failed after marking ID, that would block a retry – so order matters). -
Return Value: We return 0 on success, or a code indicating the type of failure. This allows the caller (or higher-level system) to log errors or handle them (e.g., notify that a transaction was rejected). We could also design this to
abort()
on critical failures like OOM, but returning an error is safer for graceful degradation.
Performance: This process_transaction
function runs in constant time for each transaction: O(1) for duplicate check, O(1) for balance lookups/updates, and amortized O(1) for ledger append. Thus, processing N transactions is O(N). The critical path for performance is likely the memory accesses (hash lookups, array appends) which are optimized by using contiguous storage and proper sizing. We avoid any heavy computations per transaction beyond basic arithmetic and hash index calculations. This design can handle high throughputs (millions of transactions) as long as the data structures are well-tuned in size to avoid excessive collisions or reallocations.
Example: Suppose Transaction #1001 transfers $50 (5000 cents) from internal Account 200 to internal Account 300. Before processing, account 200 has balance $100, account 300 has $20. The steps would be:
- Check ID 1001 in set (not present, continue).
- Create ledger entries: debit (acct 200, change = -5000), credit (acct 300, change = +5000).
- Account 200 internal: newBalance = 10000 - 5000 = 5000 cents (>=0, okay). Update balance to $50.
- Account 300 internal: newBalance = 2000 + 5000 = 7000 cents. Update balance to $70.
- Append both ledger entries to ledger array.
- Insert ID 1001 into seen set.
- Return success.
If a second transaction #1002 tries to withdraw $60 from account 200 (which now has $50):
- Not a duplicate, prepare entries.
- Account 200 internal: newBalance = 5000 - 6000 = -1000 cents, which is below 0 and overdraft not allowed. The function returns error (insufficient funds) and aborts this transaction – no ledger entries written, no balances changed, ID not recorded. The system state remains consistent (account 200 stays at $50).
2. Computing an Account Balance (compute_balance
)
We provide a function to compute the balance for a given account ID on demand, based on an array of all transactions or the ledger entries. If we have maintained running balances (as above), one could argue this function is not needed (we can directly return the stored balance for internal accounts). However, deriving the balance from the transaction history is a reliable method to verify correctness (especially for audit) and is useful if we did not continuously track the balance. It ensures derived consistency: “the account balance can always be derived by simply summing up the transaction values”. We’ll implement it for completeness and integrity checks.
// Compute balance by iterating over all transactions (or ledger entries)
int64_t compute_balance(uint32_t account_id, const Transaction *tx_array, size_t tx_count) {
int64_t balance = 0;
for (size_t i = 0; i < tx_count; ++i) {
const Transaction *tx = &tx_array[i];
if (tx->src_account == account_id) {
// Money going out of this account
balance -= tx->amount;
}
if (tx->dest_account == account_id) {
// Money coming into this account
balance += tx->amount;
}
}
return balance;
}
This function scans through all transactions and sums the net effect on the specified account. We treat every occurrence of the account as a source (subtract amount) or destination (add amount). This naturally incorporates double-entry: each transaction that involves the account contributes either a debit or credit.
Internal vs External Logic: The summation logic itself doesn’t need to treat internal vs external differently – it’s simply adding and subtracting. The distinction is in interpretation and enforcement:
- For an internal account, if the transactions list is complete and valid (no unauthorized overdrafts), the resulting
balance
will be >= 0 (or >= -overdraft_limit if that feature is in use). We expect the system never allowed it to dip below allowed thresholds, so the derived balance should reflect that. If this computed balance is negative (below allowed), it indicates an anomaly (a bug or a missing initial credit). In normal operation, this shouldn’t happen becauseprocess_transaction
prevents it. So this function can be used as a sanity check as well. - For an external account, the computed balance is simply the net flow from the system’s perspective. It could be positive (meaning that external account has net received funds from internal accounts) or negative (meaning the external account sent in more funds than it received, i.e., the system owes money to that external party or has their funds). We do not restrict external balances. In fact, we might not even call
compute_balance
for external accounts in normal operation, since we “don’t track their balances” in the same sense. If needed (for reporting), this function will still give the net position of an external account. For example, an external account that only received $100 from the system will show balance +$100 (from the system’s perspective, that is how much was paid out to it). An external account that only sent $50 into the system would show -$50 (the system received $50 from it, so external’s net position relative to system is negative $50). These numbers are informational; no enforcement is done on them.
Performance: This computation is O(N) over transactions. If we call this frequently for different accounts, it’s inefficient. In a live system, we would instead use the stored balances (for internal accounts) for instant results. However, for audit or recovery, this function is vital: we can recompute any account’s balance from scratch to verify the stored balance is correct. If we suspect a bug or corruption, running compute_balance(account)
for all internal accounts and comparing to our stored balances can detect inconsistencies. This is essentially performing a full ledger reconciliation.
Memory Consideration: compute_balance
as written uses the transaction array, which we assume is in memory. If the transaction history is huge and doesn't fit in memory, this approach would be problematic. In such cases, we might stream transactions from disk or use the checkpoint mechanism (explained later) to avoid reading everything. Given our memory cap (256MB), a full array of transactions that fits is likely on the order of a few million transactions maximum (since each is ~32 bytes, 8 million transactions ~ 256MB). So this function is viable within that limit. With checkpointing, we could incorporate initial balances from the last checkpoint and only sum recent transactions.
Alternative Implementation: We could compute the balance from the ledger entries instead of the raw transactions. For instance, we could maintain a data structure mapping account->list of ledger entries (so we don’t iterate over unrelated transactions). If we had such an index, computing balance would involve summing the change
of all entries for that account. This could be more efficient if an account has far fewer entries than the total transaction count (e.g., computing the balance of one account out of millions by scanning only its specific entries). However, maintaining per-account lists requires more memory (storing pointers or splitting the ledger) and adds complexity. Our current design doesn’t index ledger by account to save memory, but it’s a possible extension. Instead, we rely on either scanning the array or using the stored Account.balance
.
Usage: In practice, we might use compute_balance
primarily for verification or on external accounts if needed. For internal accounts, we trust our live balance tracking, but compute_balance
gives the same result (and can double-check it). We could integrate this in tests (see Testing section) to ensure our incremental updates match a full recompute.
3. Helper Functions for Internal Account Management
We referenced functions like is_internal_account(id)
and get_account(id)
in the pseudocode. These are part of the internal account management module:
-
is_internal_account(uint32_t id) -> bool
: Checks if the given account ID is in the internal accounts set/map. This would typically do a hash set lookup. If we use a unified map of Account structs, this can simply check ifid
exists in the map. Complexity O(1) average. This is used throughout the processing logic to branch on rules. -
get_account(uint32_t id) -> Account*
: Retrieves a pointer/reference to the Account struct for the given ID, if it exists (internal). If using a hash map, this lookup gives us the stored balance and overdraft info for updates. If the account is not found, returns NULL (meaning either it’s external or not yet created internal). -
create_internal_account(uint32_t id) -> Account*
: Allocates a new Account entry for a new internal account. This involves inserting into the internal account map. We set initial balance to 0 (unless there's an initial deposit transaction, which would be processed immediately after). Also set overdraft_limit (initially 0 unless configured otherwise) and currency (default). If insertion fails due to memory, we handle that as a critical error (transaction processing might abort).
These helper functions encapsulate the data structure operations so that the main logic (process_transaction
) remains clean. They also make it easier to later change how we store accounts (for example, if we switch to a different structure, we update these functions accordingly without changing business logic).
4. Ledger Storage and Query Functions
We treat the ledger (the list/array of LedgerEntry
) as append-only. We might define:
-
ledger_append(LedgerEntry entry) -> bool
: Appends an entry to the in-memory ledger. If the ledger is stored in a dynamic array, this function checks if there is capacity; if not, it will attempt to grow the array (e.g., double its size usingrealloc
). It must checkrealloc
result for NULL (OOM) and handle errors (possibly by returning false, which triggers the caller to handle it). We ensure that even if the ledger fails to record an entry due to memory, the system does not lose track of the transaction – in such a scenario, we might log an error and stop processing further transactions (since audit trail is compromised if we cannot record). An alternative in low-memory conditions is to write ledger entries to disk (spillover), but that’s beyond current scope. -
get_ledger_entries_by_account(uint32_t account_id)
: (Optional) If we want to support queries like "list all transactions for account X", we might implement a function that scans the ledger and filters entries matching the account. If this is needed frequently, maintaining an index per account is better (as mentioned earlier). Without an index, this is O(total entries), which might be slow. For the design, we note this as a possible extension rather than implement now. -
Integrity: The ledger, as a sequential log of all entries, could be periodically verified by summing all entries and checking the total is zero (for internal accounts in base currency), which is a strong integrity test. We might provide an admin function to do this check, or at least mention that an auditor can do so using the data.
5. Memory Management and Safety
Handling dynamic memory carefully is crucial given our 256MB cap and the need for reliability:
-
Pre-allocation: When possible, allocate upfront buffers for large structures to avoid fragmentation and check usage against the cap. For example, if we know or can estimate the number of transactions, we allocate the transaction array and ledger array once. If not, we grow these arrays geometrically (doubling) to balance between reallocation frequency and wasted space. Each reallocation is checked to ensure we don’t exceed 256MB. We also consider alignment (use
realloc
which usually aligns to at least 8 or 16 bytes). -
Custom Allocator / Memory Pool: We could manage a dedicated memory pool of 256MB for the program. This pool could be a single
malloc(256*1024*1024)
and then we sub-allocate to our structures. For instance, carve out a portion for the ID set, a portion for accounts, etc. This approach can eliminate overhead from many small allocations and ensure we never exceed the cap (since we never allocate outside the pool). It also makes deallocation simple (free the whole pool on exit). However, implementing a robust custom allocator is complex and prone to its own bugs, so we might instead rely on standardmalloc
but keep careful track of our usage. -
Tracking Memory Usage: We maintain counters for memory used by each major component:
- Transaction array (if loaded in memory) – size =
num_transactions * sizeof(Transaction)
. - Ledger entry array – size =
num_entries * sizeof(LedgerEntry)
. - Internal accounts map – size ≈
num_internal_accounts * (sizeof(Account) + overhead)
. - ID hash set – size ≈
capacity * (size of bucket)
. If open-addressing, each bucket might be 8 bytes + overhead for state. We can calculate roughly or measure at runtime if needed.
By summing these, we can enforce the 256MB limit. For instance, if adding a new transaction would exceed memory (because ledger + id set growth pushes us over), we could refuse it or trigger a checkpoint/flush to disk (depending on system requirements). In practice, hitting exactly 256MB is an extreme scenario; we aim to stay below it with a comfortable margin.
- Transaction array (if loaded in memory) – size =
-
Out-of-Memory (OOM) Handling: All allocations (
malloc
,calloc
,realloc
) are checked for NULL. If an allocation fails, the system has a few strategies:- If it’s during processing (e.g., trying to expand a ledger or ID set), we stop processing new transactions and return an error. The system could then output an error message like "Memory limit reached – cannot process more transactions." Ideally, we would have graceful degradation (maybe move some data to disk or compress it), but that’s beyond this design.
- If a critical allocation fails (like creating an internal account record), we also return an error from
process_transaction
. The calling application might log it and possibly abort, because continuing after failing to allocate an account could lead to inconsistencies (transaction not applied fully). - No memory leaks: For every allocation, we ensure there’s a corresponding free when the data is no longer needed (or at program termination). For long-running processes, we would free memory for accounts or IDs only if those entities are truly obsolete (which usually they are not, unless we implement removal after checkpointing). Tools like Valgrind or AddressSanitizer would be used in testing to confirm no leaks or out-of-bounds accesses occur.
-
Pointer Safety: We avoid unsafe pointer arithmetic. When resizing arrays, we use the reallocated pointer and adjust our stored pointer. We be careful not to use any pointer after freeing its memory. Buffer overflow is prevented by always doing bounds checks: e.g., ledger append checks that
current_count < capacity
before writing, and duplicate set checkscount < capacity
before inserting (or triggers a resize). Also, when computing indices for open-addressing, we modulo by capacity to ensure we don’t go out of range. -
Thread Safety: The current design assumes a single-threaded processing of transactions or that the caller handles synchronization if multiple threads feed transactions. If we later extend to multi-threading for performance, we will need to lock the shared structures (balances map, ID set, ledger) or use atomic operations carefully. For now, ensuring correctness in one thread is the focus, but our data structures (especially using atomic builtins for e.g. ID check-insert) can be made thread-safe if needed.
-
256MB Hard Cap Enforcement: We can implement a global counter of allocated memory. For each
malloc/realloc
, increment the counter by the new allocation size (for realloc, consider difference of new vs old). If at any point a requested allocation would push us over 256MB, we can fail that allocation on purpose (simulate OOM even if the system might have more). This ensures we never cross the intended cap. The exact cap might also consider stack usage and code size, but those are relatively small; the main use is in heap for our data.
In summary, careful allocation strategies and checks at every step will maintain memory safety. We will also use defensive programming techniques (assertions, and initializing memory to 0 when appropriate) to avoid undefined behavior. For example, after malloc
, immediately set new memory regions to 0 via calloc
or memset
to avoid any uninitialized reads. Each array will track its length and capacity, and we’ll pass those lengths to any function iterating to avoid overruns.
Extensibility for Future Features
One design goal is to make the system easy to extend with new features without massive changes to the core. Here’s how we structure for three anticipated features:
Overdraft Limits for Internal Accounts
Currently, internal accounts cannot go below 0 balance. To allow overdrafts (a limited negative balance), we introduced an overdraft_limit
field in the Account
struct. To activate overdrafts:
- Account Setup: Set
acc.overdraft_limit
to a positive amount (in cents) for accounts that should be allowed to go negative up to that limit. This could be a per-account setting (some accounts may have different overdraft allowances). For example, if an account hasoverdraft_limit = 10000
, it can go down to -$100.00 at most. - Processing Logic Changes: In
process_transaction
, the checkif (newBalance < 0)
becomesif (newBalance < -srcAcc->overdraft_limit)
. This is a trivial code change since we already coded it in a generic way. So an internal account can be debited into negative as long as it doesn’t exceed its overdraft cap. If it would, we reject the transaction (or possibly could allow it but mark it as an overdraft violation – but typically we prevent it). - Ledger Entries: We might want to mark when an overdraft is utilized. For audit, it could be useful to flag an entry that caused an account to go negative. We could use the
flags
field inLedgerEntry
for this purpose (e.g., set a bit if post-transaction balance is below 0). Alternatively, we could generate a separate “overdraft fee” transaction if required by business logic, but that’s outside core ledger mechanism. - Account Struct: The
Account
struct already hasoverdraft_limit
. We ensure that any new internal account defaults to the standard limit (likely 0, meaning no overdraft unless explicitly allowed). - Testing Overdraft: We will add test cases where an internal account with an overdraft limit can successfully go negative within the limit, and another test where it exceeds the limit and is correctly blocked (see Testing section).
This extension does not require changing the Transaction
or LedgerEntry
formats, and only a minor tweak in the processing logic, thanks to forethought in our design.
Multi-Currency Support with Exchange Rates
Supporting multiple currencies means a transaction could be denominated in, say, EUR or GBP instead of USD (base currency). This complicates double-entry accounting because debits and credits must balance in each currency or via conversion entries. Our design mitigates complexity as follows:
-
Currency Field: We included a
currency
field inTransaction
(not explicitly in the struct above, but we can add it) or we can interpret certain account IDs as tied to currencies. A better approach is to extendTransaction
with a currency code, e.g.:char currency[4]; // Currency code like "USD", "EUR" (3 letters + null terminator)
or use a numeric code (as we did in
LedgerEntry.currency
). EachLedgerEntry
already has acurrency
field to mark the currency of that entry. Initially, this might all be 0 (meaning USD or base currency). -
Double-Entry in Multi-currency: In real accounting, a cross-currency transaction is handled by splitting into multiple entries: e.g., if transferring money from a USD account to a EUR account, one approach is to create an extra ledger entry for the currency conversion. For instance: debit USD from one account, credit an intermediary “Forex” account in USD; then debit that Forex account in EUR (converted amount) and credit the destination in EUR. This ensures each currency ledger balances out. Implementing this automatically is complex. We might simplify by only allowing transactions where
src_account
anddest_account
use the same currency (and let external systems handle conversion), or by requiring the transaction provide both amounts (source amount and destination amount after conversion) and we book the difference to a currency gain/loss internal account. -
System Changes: At minimum, we store a currency code per account (so each internal account could be tagged as a USD account, EUR account, etc.). The
Account.currency
field in our struct serves this. We must then enforce that transactions have consistent currency or handle conversion:- If
tx.currency
exists, we ensureAccount[src].currency == tx.currency
andAccount[dest].currency == tx.currency
if the transaction is not doing conversion. If it’s meant to do conversion, thentx
might need two currency fields (one for source amount/currency, one for dest amount/currency). A simpler design is to restrict each transaction to a single currency and treat currency exchange as two transactions (one converting money between two internal currency holding accounts, and another transferring in the target currency).
- If
-
Ledger Entries: Each entry carries a
currency
ID. Balancing is done per currency. An auditor would check that for each currency, sum of debits equals sum of credits (plus any known exchange gain/loss accounts). Our system can be extended to maintain separate subtotals per currency for integrity checks. -
Exchange Rates: The system may need access to exchange rates to know how to split a cross-currency transaction. We might maintain a table of rates (e.g., a function
convert(amount, from_currency, to_currency)
that uses a rate chart). This would be used when creating multi-currency transactions and additional ledger entries. For example, if 1 USD = 0.85 EUR at the time, a $100 withdrawal from a USD account to a EUR account might be recorded as -$100 in the USD account, +$100 in an internal USD->EUR clearing account, then -€85 from that clearing account, +€85 in the EUR destination account (with possibly slight rounding considerations). This is complex but doable. -
Code Structure: We would likely introduce a currency conversion module. But importantly, our existing code can remain mostly untouched for single-currency transactions. For multi-currency,
process_transaction
would detect currency differences and invoke conversion logic (issuing additional ledger entries accordingly). TheLedgerEntry
struct can already handle different currencies by design. -
Memory Impact: The currency strings or codes add slight overhead. We might use a small enum or index for currencies to keep memory down (as we did with
uint16_t currency
). The exchange rate table would be tiny. So memory is not a big issue here.
Planning for multi-currency at design time (as we did by reserving fields) will save significant refactoring later. We ensure all functions (like balance computation) are currency-aware if needed. For example, compute_balance
in a multi-currency world should probably sum only transactions of the relevant currency for that account, or convert all to base currency if we want a unified balance (which is not straightforward). More likely, an account is tied to a currency, and you only compute balance in that currency. So we’d ensure Account.currency
is considered or require callers to specify currency if needed.
Checkpointing and Snapshotting
As the number of transactions grows, recomputing balances from genesis (the first transaction) becomes inefficient. Checkpointing means creating a snapshot of all internal account balances at a certain point in time (or transaction ID), so that we don’t need to replay the entire log from the beginning for future computations. Our design can incorporate this as follows:
-
Periodic Snapshots: For example, every 100,000 transactions, the system can take a snapshot of all internal accounts’ balances and persist it (either to a file or to a separate in-memory structure). This snapshot would include the snapshot ID (maybe the last transaction ID or timestamp included in it) and a map of account balances at that point. We could use a compact format to store only non-zero balances or all internal accounts.
-
Memory Representation: In-memory, we could have a struct like:
typedef struct { uint64_t last_txn_id; size_t num_accounts; AccountBalanceSnapshot *accounts; // array of {id, balance} pairs } BalanceSnapshot;
where
accounts
is a list of account IDs and their balances at the snapshot. This can be saved to disk or kept in memory if space allows (it’s much smaller than full ledger since it’s one record per account, not per transaction). For a thousand internal accounts, this is negligible; for a million accounts, it’s larger but still less than replaying millions of transactions. -
Loading a Snapshot: On system startup, we could load the latest snapshot from disk, pre-populate the accounts map with those balances, set an internal pointer to the next transaction to process in the log, and only replay transactions after the snapshot point. This drastically reduces recovery time and memory usage (since we don’t need the entire history loaded).
-
Computing Balance with Snapshot: Modify
compute_balance(account_id)
to use a snapshot as a baseline if available. For example, find the latest snapshot before the end of our log, take that balance, then only sum transactions occurring after the snapshot. If snapshots are taken regularly, this saves a lot of looping. We need to ensure we only include transactions after the snapshot’slast_txn_id
. This implies transactions are processed in order. (If out-of-order processing is possible, snapshot by timestamp or an explicit sequence number is necessary to know what’s included.) -
Ledger Truncation: With snapshots in place, we can truncate or archive older ledger entries safely. For instance, once we have a snapshot at tx #100000, we no longer need to keep ledger entries older than that in memory for balance computation. We could write them to an archive (file) and free that portion of the array (or just keep them on disk for audit). This frees memory. Our ledger array could be managed in chunks (each chunk corresponding to a range of transactions). After snapshot, the first chunk could be freed. This way, even if millions of transactions occur, we only keep recent ones in memory plus snapshots. This strategy ensures we stay within our memory cap indefinitely by trading off older history to disk or summary form.
-
Concurrency and Atomicity: When taking a snapshot during live processing, we need to pause new transactions briefly to capture a consistent state (or use atomic operations per balance, but easier is to quiesce input). This can be managed by locks or by doing it during a maintenance window.
-
Extensibility Impact: Our existing data structures already hold all needed info for a snapshot (account map has balances). We just need to serialize that out. The introduction of snapshots doesn’t disrupt
process_transaction
logic except perhaps to occasionally call a snapshot function. OurAccount
struct might get an additional field likelast_snapshot_balance
or not needed if we externalize it. TheLedgerEntry
struct might get a flag if it's a special "checkpoint entry" (some systems log a checkpoint as a pseudo-transaction). That’s optional; we can manage snapshots outside of the normal ledger.
By designing with checkpointing in mind, we ensure the system can scale and recover quickly. Instead of always doing full replays (which are O(N) for each restart or heavy query), we reduce it dramatically. Checkpointing plus our fast incremental updates means the system combines the safety of derived balances with the speed of stored balances.
Testing Strategy
Comprehensive testing is essential for a financial system. We outline a multi-faceted testing approach covering correctness, performance, and safety:
-
Unit Tests for Core Functions: We will write focused tests for smaller units:
- Transaction Processing (
process_transaction
): Test that given a single transaction, the function creates correct ledger entries and updates balances properly. For example, a basic test deposits money into an account (src external, dest internal) and checks that the internal account’s balance increased and two ledger entries were recorded (debit external, credit internal). Another test transfers between two internal accounts and verifies both balances and ledger entries (and that sum of entries is zero). We also test edge scenarios like transferring 0 amount (should effectively do nothing but perhaps still record entries with 0 change), though 0-amount transactions might be disallowed by business logic. - Balance Computation (
compute_balance
): Construct a small array of transactions manually and verify thatcompute_balance
returns the expected result for various accounts. For example, transactions: A->B $100, B->C $50. Then testcompute_balance(A) = -100
,compute_balance(B) = +50
(B got 100 then sent 50),compute_balance(C) = +50
. Also verify that for accounts not involved, balance is 0. This ensures the summing logic is correct. - Internal Account Enforcement: Create a scenario where an internal account would go negative. For instance, internal account X with balance $50 tries to send $60. The test expects
process_transaction
to return an error (insufficient funds) and no changes to X’s balance or ledger. Also test that if X had an overdraft limit of $20 (i.e., can go to -$20), the same transaction of $60 (which would result in -$10) now succeeds, and the final balance is -$10. This covers overdraft logic. - Duplicate Detection: Feed two transactions with the same ID. For example, tx1 (ID=500, A->B $10), tx2 (ID=500, A->B $20). Process tx1 (should succeed). Then process tx2 – should be flagged as duplicate (function returns duplicate error code). Verify that the second transaction had no effect on balances or ledger. Additionally, test that the duplicate check is truly catching duplicates: e.g., if we process millions of unique IDs, ensure none erroneously collide or that the hash set logic works for various cases (maybe test edge ID values like 0 and UINT64_MAX).
- Memory Safety Simulation: We can simulate out-of-memory by injecting a custom allocator that fails after a certain number of allocations or above a threshold. For instance, test the case where ledger array growth fails: fill the ledger near capacity and then force
realloc
to return NULL – ensureprocess_transaction
returns an error and that no partial state is committed (balance remains unchanged, ID not inserted). Also test creating an internal account when memory is exhausted to see it fails gracefully. While these are artificial scenarios, they help confirm that error paths are handled (no crashes, no corrupt state).
- Transaction Processing (
-
Integration Tests (End-to-End Scenarios): Create realistic sequences of transactions and ensure the system handles them correctly:
- Simple Deposit/Withdrawal: Start with an empty system, process a deposit from an external account E into an internal account I (E->I $100). Check that I’s balance is $100. Then process a withdrawal from I to external E (I->E $30). Now I’s balance should be $70. Ensure ledger has 4 entries (two per transaction) and that summing those entries for I yields $70, for external E yields net $-70 (since E gave $100 and received $30). Internal account never went negative at any point.
- Transfer Between Internal Accounts: Two internal users A and B. A has $50, B has $20 initially (we can simulate initial balance by either directly setting it or via a prior external deposit). Process A->B $30 transfer. After, A should have $20, B should have $50. Verify ledger entries, and that the total internal money remains $70 (just redistributed). Try an invalid transfer: B->A $100 (B only has $50, no overdraft). That should be rejected, balances unchanged.
- High Volume Processing: Generate a large batch of random transactions within constraints (ensuring no account tries to overspend beyond what we’ll deposit to it). For example, 100,000 transactions with random internal accounts (choose, say, 100 internal accounts, randomly pick source and dest among them or an external, and random amounts within available balances). Process them all sequentially. Then verify a few properties: All internal balances computed by summing transactions equal the stored balances in our map. Also, the sum of all internal balances plus net external outflow is zero (conservation of money) for single currency. This is a kind of stress test for performance and memory as well.
- Multi-currency Test (if implemented): Create a scenario with two currencies. For example, internal account X (USD), internal account Y (EUR). Process a cross-currency transaction if supported (X->Y some USD to EUR). Check that ledger entries include a currency conversion (perhaps an intermediate account or proper flags). Ensure balances in each currency are updated correctly. This would require the conversion logic in place, and we’d verify no currency ledger is unbalanced: sum of USD debits = sum of USD credits, likewise for EUR, aside from the conversion account which should net zero across both currencies when combined.
- Overdraft and Recovery: Give an account an overdraft of $100. Perform a series of transactions that dip into that overdraft and then out of it. E.g., account P has $50, overdraft $100. Withdraw $120 (allowed, final balance -$70). Deposit $100 back (final balance $30). Ensure at each step the internal rules worked (first transaction succeeded only because overdraft made -$70 acceptable, second transaction brings it positive). Also, simulate an overdraft breach: if P tried to withdraw another $100 from -$70 (which would go to -$170, exceeding $100 overdraft), ensure rejection.
-
Performance Testing: We should test that the system meets performance expectations:
- Time how long it takes to process a large number of transactions (say 1 million) and ensure it’s within acceptable limits (since our operations are O(N), 1e6 operations should be fine in seconds in C). Profile any hot spots. The expectation is linear scaling. The use of efficient structures (hash maps, contiguous arrays) should keep throughput high (memory accesses are the main cost). We also test memory usage doesn’t explode beyond expected.
- If possible, test under memory pressure to see if fragmentation or slowdowns occur. For example, fill near 256MB and continue to operate and see if lookups slow down due to high load factors. Adjust data structure sizing if needed.
-
Memory Safety Testing: Use tools:
- AddressSanitizer or Valgrind: Run the test suite under these to catch out-of-bounds or use-after-free errors. For instance, intentionally run with debugging macros that fill freed memory with a pattern to ensure we never access it.
- Leak check: Ensure that after all tests, all allocated memory was freed or still reachable if intended. For a long-running service, we ensure no growth in memory usage over time (except with data, which should plateau or respect the cap).
- Concurrency (if applicable): If we later add multi-threading, test with multiple threads submitting transactions concurrently to ensure locks or atomic operations keep data consistent (no lost updates, no data races causing corruption). This might involve using thread sanitizers or heavy stress testing with random concurrent operations.
-
Audit and Consistency Tests: These verify the correctness of the double-entry logic:
- After a series of transactions, compute the total sum of all
LedgerEntry.change
for internal accounts – it should be zero. This confirms that every debit had a credit. If not zero, something is wrong in entry creation. (If external accounts are considered, their net effect should equal the negation of internal net effect; effectively internal sum + external sum = 0 if you treat external as part of the equation, since money leaving internal = money entering external and vice versa.) - Verify that each transaction ID appears exactly twice in the ledger (once for debit, once for credit). No ID should appear once or more than twice. This can be tested by scanning the ledger entry list. Our duplicate prevention helps ensure “no ID more than twice”; if one entry were missing (which should not happen if process_transaction is atomic), that’d be serious, and if duplicate prevention failed, an ID might appear 4 times, etc.
- If we implemented snapshots, test that starting from a snapshot state plus replaying incremental transactions yields the same balances as starting from scratch. Also, test that after loading a snapshot, the system can still detect duplicates correctly (meaning the duplicate set might need to persist or be rebuilt – perhaps we’d store seen IDs for post-snapshot transactions or assume IDs are never reused anyway).
- After a series of transactions, compute the total sum of all
-
Edge Cases: We also test unusual or extreme values:
- Transaction with
amount
= 0 (if allowed): should ideally be a no-op (balance unchanged, though ledger entries of 0 change might be recorded). Check nothing odd happens (like division by zero or skipping ledger creation, etc.). - Maximum values:
amount = 0xFFFFFFFF
(the max 32-bit) to ensure our 64-bit calculations handle it and no overflow in summing etc. Also testtransaction.id
at extremes (0 and 2^64-1) to ensure our handling (especially in hash set) works for those (some hash functions might have edge issues with such patterns). - Many small accounts: ensure the internal account map can handle a large number of accounts and that lookups still work as expected (this tests hash collisions resolution).
- If any input format quirks (not mentioned here, but e.g., out-of-order timestamps), ensure system doesn’t assume strict ordering unless documented. If a transaction from the past arrives late, our logic still processes it; however, this could temporarily allow a negative “historical” balance if we strictly chronological track. For our design, we assumed transactions are processed roughly in timestamp order. We can note that processing out-of-order could violate the no-negative rule in retrospect (since an earlier deposit might come after a withdrawal in processing). To be safe, we document that either transactions should be sorted by time or the system should recompute balances if out-of-order inputs are possible (this is a complex scenario best avoided by input pre-sorting).
- Transaction with
-
Documentation of Tests: We’ll document each test scenario and expected outcome, effectively creating a spec that mirrors the requirements. This ensures traceability: each requirement (e.g., “internal accounts cannot go negative”) has one or more tests validating it.
By executing this test plan, we ensure the system meets the criteria of correctness (accurate balances, proper double-entry, no duplicates), robustness (handles edge cases and errors safely), performance (stays within memory/time bounds), and auditability (ledgers balance out, etc.). Using both automated tests and code analysis tools provides confidence in the memory safety of the C implementation.
Conclusion
In this design, we combined principles of double-entry accounting with low-level C programming practices to create a reliable transaction processing module. We defined clear data structures (for transactions, ledger entries, accounts) that capture necessary data while remaining flexible for future needs. The approach of deriving balances from transactions upholds financial integrity and auditability, since every balance is backed by a chain of ledger entries. At the same time, we maintain in-memory summaries for performance, ensuring we can enforce rules (like no overdrafts) in real time and answer balance queries quickly.
The system uses hash tables for quick lookups (account classification and duplicate detection) and careful dynamic memory management to honor the 256MB limit. By planning for features like overdrafts, multi-currency, and checkpointing, we ensure that the architecture can evolve without drastic changes. Each extension can be layered on by utilizing reserved fields and modular functions in the code.
Finally, a rigorous testing regimen guarantees that every aspect — from basic functionality to edge-case behavior and failure modes — is validated. Financial systems demand a high degree of correctness and resilience; thus, we emphasized audit trails, consistency checks, and safe failure handling throughout the design. With this foundation, the implementation in C can proceed with confidence that the resulting system will be robust, efficient, and maintainable for large-scale use.
How Do We Build From Here?
In the next installment of this mini-series, we'll break down the response and how we use it to vibe code a production-ready money tracking system. Until then, read over it and think about how you might respond.