DEXBot2 Architecture

This document provides a high-level overview of the DEXBot2 architecture, module relationships, and key data flows.

For practical development guidance, see developer_guide.md for quick start, glossary, module deep dive, and common development tasks.

System Overview

DEXBot2 is a grid trading bot for the BitShares blockchain. It maintains a geometric grid of limit orders that automatically rebalance as the market moves, capturing profit from price oscillations.

Core Concepts

• Grid: A geometric array of price levels with orders placed at each level
• Spread Zone: A buffer of empty slots between buy and sell orders (constant width)
• Order States: VIRTUAL (planned) → ACTIVE (on-chain) → PARTIAL (partially filled)
• Fund Tracking: Atomic accounting system preventing race conditions and overdrafts

Design Philosophy

DEXBot2 prioritizes simplicity and operational efficiency over complex partial-handling mechanics:

1. Constant Spread: The spread zone width remains fixed at targetSpreadPercent, eliminating dynamic inflation triggers.
2. Immediate Dust Cancellation: Sub-threshold partials are cancelled on-chain as soon as detection runs; surviving non-dust partials are handled by ordinary fund-driven rebalancing without complex merge/split logic.
3. Minimal Blockchain Interaction: Fund-driven rebalancing occurs once per fill batch, not per-partial. Grid generation uses only available funds—no forced allocations.
4. Closed-Loop Market Dynamics: The boundary-crawl mechanism naturally handles price movement and fill flows without special-case logic.
5. Powerful Maintenance Tools: Periodic grid regeneration, recovery retries, and fund invariant verification keep the system healthy over long operations.

Top-Level Data Flow

The diagram below shows DEXBot2 from a data perspective: what data enters the system, how it moves through each engine, and what leaves as blockchain operations or persisted state.

graph TB
    subgraph "INPUTS"
        CFG[bots.json<br/>grid params, funds, pair]
        GS[general.settings.json<br/>timing, thresholds]
        KEYS[keys.json AES encrypted<br/>+ interactive unlock / one-shot local bootstrap]
        PERSIST[orders/botKey.json<br/>grid snapshot,<br/>feesOwed, boundaryIdx]
        FILLEV[Fill Events real-time<br/>BitShares block op-4]
        OPENORD[Open Orders polling<br/>chain open-order list]
        BALANCES[Account Balances<br/>FREE + COMMITTED assets]
        PRICE[Market Price<br/>pool or order-book]
    end

    subgraph "BOOTSTRAP - once at startup"
        AUTH[Credential Daemon<br/>Decrypt private key]
        ASSETMETA[Asset Metadata<br/>precision, fees, IDs]
        INITGRID[Initial Grid<br/>geometric price levels<br/>order sizes per side]
    end

    subgraph "CORE ENGINE - OrderManager"
        MASTERGRID[Master Grid - immutable/frozen<br/>slot-id, price, size, state<br/>orderId, blockchain, grid, proceeds]
        TWOPASS[SyncEngine<br/>2-pass: grid-to-chain then chain-to-grid<br/>match orderId, detect partials, flag stale]
        FUNDS["Accounting - SSOT for funds<br/>available, virtual, committed<br/>btsFeesOwed<br/>Avail = max 0 ChainFree minus Virtual minus Fees"]
        TARGET[Strategy Engine<br/>calculateTargetGrid<br/>boundary-crawl pivot<br/>partial correction, rotation]
        WORKGRID[WorkingGrid - COW copy<br/>all mutations here only<br/>commit to Master on confirmation]
        FILLQUEUE[Fill Queue<br/>AsyncLock + dedup 5-60 min]
        BATCHER[Fixed-Cap Batcher<br/>queue within cap: unified batch<br/>queue above cap: chunk at cap size<br/>cap = gapSlots + 1]
    end

    subgraph "OUTPUTS"
        OPS[Blockchain Operations<br/>CREATE / UPDATE / CANCEL<br/>limit orders on BitShares]
        SNAP[Grid Snapshot<br/>profiles/orders/botKey.json]
        LOGS[Logs and Metrics<br/>profiles/logs/botName.log<br/>queue depth, latency, health]
    end

    KEYS --> AUTH --> ASSETMETA
    CFG --> INITGRID
    GS --> INITGRID
    ASSETMETA --> INITGRID
    PERSIST --> MASTERGRID
    INITGRID --> MASTERGRID

    PRICE --> FUNDS
    BALANCES --> FUNDS
    BALANCES --> TWOPASS
    OPENORD --> TWOPASS
    TWOPASS --> FUNDS
    TWOPASS --> MASTERGRID

    FILLEV --> FILLQUEUE
    OPENORD --> FILLQUEUE
    FILLQUEUE --> BATCHER
    BATCHER --> WORKGRID

    MASTERGRID --> WORKGRID
    FUNDS --> TARGET
    WORKGRID --> TARGET
    TARGET --> WORKGRID
    WORKGRID --> OPS
    OPS --> MASTERGRID

    MASTERGRID --> SNAP
    FUNDS --> LOGS
    BATCHER --> LOGS
    OPS --> LOGS

Key Design Principles

Module Architecture

graph TB
    subgraph "Entry Points"
        CLI[dexbot.ts]
        BOT[bot.ts]
        PM2[pm2.ts]
        UNLOCK[unlock.ts]
        CRED_DAEMON[credential-daemon.ts]
    end

    subgraph "Core Bot"
        DEXBOT[DexBotClass<br/>modules/dexbot_class.ts]
        FILL_RUNTIME[FillRuntime<br/>modules/dexbot_fill_runtime.ts]
        MAINT_RUNTIME[MaintenanceRuntime<br/>modules/dexbot_maintenance_runtime.ts]
        CONSTANTS[Constants<br/>modules/constants.ts]
        FUND_REGISTRY[FundRegistry<br/>modules/fund_registry.ts]
        SETTINGS_MERGE[SettingsMerge<br/>modules/settings_merge.ts]
        CRED_RUNTIME[CredentialRuntime<br/>modules/credential_runtime.ts]
    end

    subgraph "Order Management System"
        MANAGER[OrderManager<br/>modules/order/manager.ts]

        subgraph "Specialized Engines"
            ACCOUNTANT[Accountant<br/>accounting.ts]
            STRATEGY[StrategyEngine<br/>strategy.ts]
            SYNC[SyncEngine<br/>sync_engine.ts]
            GRID[Grid<br/>grid.ts]
        end

        WORKGRID[WorkingGrid<br/>working_grid.ts]
        GRID_RECONCILE[GridReconcile<br/>grid_reconcile.ts]
        UTILS[Utils<br/>utils/]
        LOGGER[Logger<br/>logger.ts]
        FILL_STORE[ProcessedFillStore<br/>processed_fill_store.ts]
    end

    subgraph "Blockchain Layer"
        CHAIN_ORDERS[ChainOrders<br/>modules/chain_orders.ts]
        ACCOUNT_ORDERS[AccountOrders<br/>modules/account_orders.ts]
        ACCOUNT_BOTS[AccountBots<br/>modules/account_bots.ts]
        NODE_MGR[NodeManager<br/>modules/node_manager.ts]
        BTS_CLIENT[BitSharesClient<br/>modules/bitshares_client.ts]
    end

    subgraph "Market Adapter"
        MA[MarketAdapter<br/>market_adapter/market_adapter.ts]
        MA_SVC[AdapterService<br/>core/market_adapter_service.ts]
        AMA_RUNNER[AMA Signal Runner<br/>ama_signal_runner.ts]
        KIBANA[inputs/kibana_source.ts]
        LP_FETCH[inputs/fetch_lp_data.ts]
        STRATEGIES[core/strategies/]
    end

    CLI --> DEXBOT
    BOT --> DEXBOT
    PM2 --> DEXBOT
    UNLOCK --> CRED_DAEMON
    CRED_DAEMON --> DEXBOT

    DEXBOT --> MANAGER
    DEXBOT --> FILL_RUNTIME
    DEXBOT --> MAINT_RUNTIME
    DEXBOT --> CONSTANTS
    DEXBOT --> FUND_REGISTRY
    DEXBOT --> SETTINGS_MERGE
    DEXBOT --> CRED_RUNTIME

    MANAGER --> ACCOUNTANT
    MANAGER --> STRATEGY
    MANAGER --> SYNC
    MANAGER --> GRID
    MANAGER --> WORKGRID
    MANAGER --> GRID_RECONCILE
    MANAGER --> UTILS
    MANAGER --> LOGGER
    MANAGER --> RUNNER
    MANAGER --> FILL_STORE

    ACCOUNTANT --> FUND_REGISTRY
    STRATEGY --> UTILS
    SYNC --> UTILS
    GRID --> UTILS

    RUNNER --> CHAIN_ORDERS
    SYNC --> ACCOUNT_ORDERS
    MANAGER --> ACCOUNT_BOTS

    CHAIN_ORDERS --> BTS_CLIENT
    ACCOUNT_ORDERS --> BTS_CLIENT
    BTS_CLIENT --> NODE_MGR

    MA --> MA_SVC
    MA --> AMA_RUNNER
    MA_SVC --> KIBANA
    MA_SVC --> LP_FETCH
    MA_SVC --> STRATEGIES

    MA -.->|trigger files| MAINT_RUNTIME

Browser-Safe Surface

DEXBot2 ships both a Node CLI runtime and the building blocks for an in-browser operator UI. To keep require('fs'), process.kill, and Unix-socket calls out of the browser bundle, the codebase enforces a strict Node-vs-browser split.

Convention: everything is browser-safe unless listed as Node-only. The source of truth is the "browser" field in package.json, which maps every Node-only compiled entry (dist/*.js) to false. The list below is documentation; always check package.json before reclassifying a module.

Node-only modules (must not be reached from a browser bundle):

Environment detection must go through modules/env.ts rather than inline typeof window / typeof process checks:

import { isBrowser, hasProcess } from './env';

The previous 6+ inline ternaries that existed in bitshares-native/* and runtime.ts were consolidated into those helpers; do not reintroduce them.

Order Manager: Central Coordinator

The OrderManager is the central hub that coordinates all order operations. It delegates specialized tasks to four engine modules:

Engine Responsibilities

Copy-on-Write (COW) Grid Pattern

The OrderManager implements a Copy-on-Write (COW) pattern to protect the master grid from speculative modifications until blockchain finality is confirmed.

Core Principle

The master grid (this.orders) is immutable - it can only be replaced atomically, never mutated in place. All speculative planning operations work on isolated copies, and the master is only updated when blockchain confirms the operation.

Important: Index Sets (_ordersByState, _ordersByType) are mutable by design but must only be mutated through _applyOrderUpdate(). Direct external mutations violate the COW invariant.

Protection Mechanisms

Master Grid Update Pattern

All master grid updates follow clone-and-replace semantics:

// 1. Clone existing Map
const newMap = cloneMap(this.orders);

// 2. Apply mutation to clone
newMap.set(id, updatedOrder);

// 3. Atomically replace with frozen copy
this.orders = Object.freeze(newMap);
this._gridVersion++;

Index Sets follow the same pattern - cloned, mutated, frozen, then replaced.

WorkingGrid Class

The WorkingGrid class (modules/order/working_grid.ts) provides isolation for speculative operations:

• Deep clones the master grid on construction
• Tracks modified orders in a Set
• Supports staleness detection via baseVersion
• Never modifies the master grid

COW Rebalance Pipeline

┌─────────────────────────────────────────────────────────────┐
│  1. Create WorkingGrid from frozen master                   │
│     workingGrid = new WorkingGrid(masterGrid, {baseVersion})│
└─────────────────────────┬───────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│  2. Calculate target state (PURE - no side effects)         │
│     strategy.calculateTargetGrid() returns new Map          │
└─────────────────────────┬───────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│  3. Project target onto working grid                        │
│     Modifies working copy only                              │
└─────────────────────────┬───────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│  4. Validate funds & check staleness                        │
│     If stale: abort without committing                      │
└─────────────────────────┬───────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│  5. Submit to blockchain & wait for finality                │
│     synchronizeWithChain() confirms on-chain                │
└─────────────────────────┬───────────────────────────────────┘
                         ↓
┌─────────────────────────────────────────────────────────────┐
│  6. Commit: Replace master with working grid                │
│     this.orders = Object.freeze(workingGrid.toMap())        │
└─────────────────────────────────────────────────────────────┘

Triggers for Master Grid Updates

Only blockchain-confirmed events trigger master updates:

Defensive Measures

1. Double-check commit pattern: Staleness is checked both outside and inside the lock
2. Working grid sync: If master mutates during planning, working grid is marked stale
3. Version mismatch detection: Commits abort if baseVersion doesn't match _gridVersion

Fill Processing Pipeline

The fill pipeline handles incoming filled orders efficiently through fixed-cap batching instead of one-at-a-time processing.

Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                   Fill Event (Blockchain)                   │
│                  (Order filled at price X)                  │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│               _incomingFillQueue (FIFO Queue)               │
│             (Accumulates fills from blockchain)             │
│       Queue: [fill1, fill2, fill3, fill4, fill5, ...]       │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│             processFilledOrders() - Entry Point             │
│    Use gap-slot batch size (gapSlots+1) for deterministic batching   │
│      Rules: <=gapSlots+1 unified, >gapSlots+1 chunked      │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│            Pop Batch (up to gapSlots+1)                     │
│        Takes N fills from queue head (N = 1..gapSlots+1)     │
│   Example: pops [fill1, fill2, fill3] for batch processing  │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│            processFillAccounting() - Single Call            │
│       All fills credited to chainFree in ONE operation      │
│     chainFree += proceeds[fill1] + proceeds[fill2] + ...    │
│    Proceeds immediately available (same rebalance cycle)    │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│             calculateTargetGrid() - Single Call             │
│       Size replacement orders using combined proceeds       │
│             Apply rotations and boundary shifts             │
│ Use unallocated remainder for next allocation opportunities │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│        updateOrdersOnChainBatch() - Single Broadcast        │
│      All new orders + cancellations in single operation     │
│          Result: Atomic state update on blockchain          │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
┌─────────────────────────────────────────────────────────────┐
│                        persistGrid()                        │
│               Save grid state to disk/storage               │
└─────────────────────┬───────────────────────────────────────┘
                      ↓
                  Loop to next batch
                (or idle if queue empty)

Key Properties

• Gap-Slot Batch Sizing: Batch size is deterministic, derived from the grid gap-slot count + 1 (DEXBot._getGapSlotBatchSize)

  • 1..gapSlots+1 awaiting: single unified batch (one rebalance/broadcast cycle)
  • more than gapSlots+1 awaiting: repeated chunks of gapSlots+1 (last chunk may be smaller)

• Single Rebalance Cycle: All fills in batch processed in ONE rebalance

  • No "split across cycles" delays
  • Combined proceeds immediately available
  • Single cache fund update

• Recovery Retries: Periodic retry system replaces one-shot flag

  • Max 5 attempts per episode
  • 60s minimum interval between retries
  • Reset on fill arrival or periodic sync (10 minutes)
  • resetRecoveryState() called by Accountant

• Stale-Cleaned Order Tracking: Prevents orphan double-credit

  • Batch failure → cleanup stale order IDs
  • Delayed orphan event → check if ID in stale-cleaned map
  • Skip credit if already cleaned
  • TTL pruning (5 minute retention)

Impact vs. Legacy Sequential Processing

Scenario source: 29-fill burst during the Feb 7 market crash, modeled at roughly 3 seconds per broadcast; see FUND_MOVEMENT_AND_ACCOUNTING.md.

Spread Correction (Fund-Aware Approach)

Simplified spread maintenance that keeps the gap consistent and fund-driven, avoiding complex split/merge mechanics.

The Approach

Instead of complex partial handling, spread corrections are conservative and fund-safe:

• Target spread width stays constant at targetSpreadPercent
• Corrections scale with actual available funds, not arbitrary slot budgets
• No dynamic spread inflation based on partial consolidation flags
• Safe window-contiguous candidate ordering (with an edge-first fallback) prevents cross-side placement

Algorithm

Location: modules/order/grid.ts::prepareSpreadCorrectionOrders(), selected by modules/order/grid.ts::determineOrderSideByFunds().

The correction path is fund-only and never manufactures budget by shrinking a resting order:

1. Detect that the live window is wider than targetSpreadPercent.
2. Select up to `outOfSpread` eligible holes in safe window/gap order.
3. If the funded edge holds a non-dust PARTIAL, optionally top it up, but only
   when `ideal - current` is positive and free funds can cover the increase.
4. Create replacements from free available/chainFree funds; contiguous gap-slot
   promotion may extend the window while preserving MIN_SPREAD_ORDERS and the
   opposite placed rail.
5. If free funds cannot cover every target, place/top up the affordable prefix,
   stop without shrinking inventory, and log the fund-constrained remainder.
6. If neither side has free funds, skip correction and let the maintenance
   runtime refresh balances and open orders rather than recycle stale inventory.

Detected dust partials never enter this path: cancelDustOrders() cancels them on-chain immediately, and the five-minute health check is only a restart safety net.

Fund-Safe Constraints

Spread corrections respect these hard limits:

// In modules/constants.ts (GRID_LIMITS)
MIN_SPREAD_ORDERS: 2,           // Preserve the minimum empty gap reserve

// `outOfSpread` supplies the requested slot count. Execution is bounded by
// safe candidate selection, healthy minimum sizes, and actual free funds.
const availableFund = Math.max(0, Math.min(
    manager.funds.available[side],
    manager.accountTotals[side === 'buy' ? 'buyFree' : 'sellFree']
));

// Each target is created or topped up only while remainingBudget covers it.
// No resting ACTIVE/PARTIAL order is shrunk to fund the correction.

Benefits:

• ✅ No inventory recycling from stale-size snapshots
• ✅ Constant, predictable spread width
• ✅ Funds always respected (no forced allocation)
• ✅ Natural smoothing over multiple rebalance cycles

Periodic Market Price Refresh

Background market price updates every 4 hours to ensure grid anchoring remains accurate during long-running sessions without fills.

Purpose

If the bot hasn't seen fills for 4 hours, the startPrice might become stale if:

• Market has drifted significantly
• Liquidity pool price has shifted
• User wants grid recalculation

Configuration

Location: modules/constants.ts

BLOCKCHAIN_FETCH_INTERVAL_MIN: 240,  // 4 hours = 240 minutes

Implementation Flow

1. Interval startup — setupBlockchainFetchInterval() in modules/dexbot_maintenance_runtime.ts starts the interval during bot initialization:

bot._blockchainFetchInterval = setInterval(tick, BLOCKCHAIN_FETCH_INTERVAL_MIN * 60 * 1000);

2. Tick guards (no lock held)

• Skip if shutdown has begun or a previous tick is still in flight
• Sync market adapter watchdog config

3. Periodic work (under the fill-processing lock)

• Reset accountant recovery state
• Refresh dynamic weight distribution
• fetchAccountTotals(accountId) — refreshes balances and the valuation anchor
• Read open orders via guarded read — truncated/empty reads defer the sync; fills are still caught by subscriptions and the next cycle
• synchronizeWithChain(...) — re-aligns the grid with on-chain reality; detected fills run through the normal batched rebalance pipeline and the grid is persisted, unmatched chain orders log as surplus/divergence
• performPeriodicGridChecks(bot) — maintenance checks (see Lifecycle)

Grid placement remains fund-driven during normal operation; the periodic tick keeps balances, valuations, and open-order state fresh rather than directly moving prices. A per-instance override (_blockchainFetchIntervalMin, e.g. for shared accounts from the fund registry) can shorten or lengthen the interval; setting a non-positive value disables the loop entirely.

When startPrice is Numeric

If user set startPrice: 105.5 in bots.json:

• No auto-refresh: Numeric value is treated as fixed anchor
• Valuation uses fixed value: All calculations use 105.5
• Grid doesn't move: Orders stay where they are (fund-driven rebalancing only)

Non-Disruptive Updates

Price refresh is passive:

• ✅ Updates internal valuation
• ✅ Affects future grid resets if triggered
• ❌ Does NOT move orders on blockchain (no funds wasted on unnecessary rotations)

Out-of-Spread Metric Refinement

Refactored outOfSpread from a simple boolean flag to a numeric distance metric for more precise structural updates.

Before (Boolean)

// Old approach
mgr.outOfSpread = true;  // Binary: either in or out
if (mgr.outOfSpread) {
    // Perform spread correction
}

Problem: Doesn't distinguish between "slightly out" vs "severely out"

After (Numeric Distance)

// New approach: requested missing slots derived from the live geometry
mgr.outOfSpread = 3;  // three slots beyond the target spread

// The grid engine bounds actual work by eligible candidates, MIN_SPREAD_ORDERS,
// healthy sizes, and available free funds. There is no MAX_CORRECTION_SLOTS cap.

Benefit: Enables scaled corrections based on actual severity.

Calculation

// Calculate how many steps beyond target
const currentSpreadSteps = calculateCurrentSpreadGap();
const targetSpreadSteps = calculateTargetSpread();
const outOfSpreadDistance = Math.max(0, currentSpreadSteps - targetSpreadSteps);

mgr.outOfSpread = outOfSpreadDistance;  // 0 = in spread, 3+ = out

Pipeline Safety & Diagnostics

The bot includes a comprehensive pipeline monitoring system to prevent indefinite blocking and enable operational visibility.

Pipeline Timeout Safeguard

Problem: Pipeline checks could block indefinitely if operations hung due to network issues or stuck corrections.

Solution: 5-minute timeout with automatic, non-destructive recovery.

Configuration (modules/constants.ts):

PIPELINE_TIMING: {
    TIMEOUT_MS: 300000,  // 5 minutes
}

How It Works:

• isPipelineEmpty() tracks when pipeline operations started blocking via _pipelineBlockedSince timestamp
• If blockage exceeds 5 minutes, clearStalePipelineOperations() is called
• Non-destructive recovery: clears operation flags only, does NOT delete orders or modify grid state
• Recovery called from _executeMaintenanceLogic() during periodic maintenance checks

Location: modules/order/manager.ts (isPipelineEmpty() and clearStalePipelineOperations())

Data Flow

sequenceDiagram
    participant Bot as DexBotClass
    participant Mgr as OrderManager
    participant Sync as SyncEngine
    participant Strat as StrategyEngine
    participant Acct as Accountant
    participant Chain as Blockchain

    Bot->>Mgr: Initialize grid
    Mgr->>Acct: Reset funds
    Mgr->>Sync: Fetch account balances
    Sync->>Chain: Get balances
    Chain-->>Sync: Balance data
    Sync->>Acct: Set account totals
    Acct->>Acct: Recalculate funds

    Note over Mgr: Grid initialized, ready for trading

    Bot->>Sync: Detect fills (polling)
    Sync->>Chain: Get open orders
    Chain-->>Sync: Order data
    Sync->>Mgr: syncFromOpenOrders()
    Mgr->>Mgr: processFilledOrders()
    Strat->>Acct: Update funds (cache proceeds)
    Strat->>Strat: Identify shortages/surpluses
    Strat->>Mgr: Rotate orders
    Mgr->>Acct: Deduct funds (atomic)
    Mgr->>Chain: Place new orders

Order State Machine

Orders transition through three primary states during their lifecycle. SPREAD is an order type (like BUY or SELL), not a state — spread-zone slots always carry state: VIRTUAL.

stateDiagram-v2
    [*] --> VIRTUAL: Grid created

    VIRTUAL --> ACTIVE: Order placed on-chain
    VIRTUAL --> VIRTUAL: Rotated or rebuilt

    ACTIVE --> PARTIAL: Partial fill detected
    ACTIVE --> VIRTUAL: Order cancelled/rotated

    PARTIAL --> ACTIVE: Consolidated (size >= ideal)
    PARTIAL --> VIRTUAL: Moved/consolidated

    VIRTUAL --> [*]: Grid regenerated

    note right of VIRTUAL
        No on-chain presence
        Funds reserved in virtual pool
        Includes spread-zone placeholders
    end note

    note right of ACTIVE
        On-chain with orderId
        Funds locked/committed
    end note

    note right of PARTIAL
        Partially filled on-chain
        Dust is cancelled immediately; non-dust is corrected in rebalance
    end note

State Transition Rules

Critical: Phantom Order Prevention

A phantom order is an illegal state where an order exists as ACTIVE/PARTIAL without a corresponding blockchain orderId. This corrupts fund tracking and causes "doubled funds" warnings.

Risks the Three-Layer Defense Guards Against:

1. Resize State Forcing: Grid resize forcing VIRTUAL → ACTIVE without blockchain confirmation
2. Sync Gaps: Orders without orderId remaining ACTIVE if sync logic skips them
3. Unvalidated Assignment: Invalid state assignments without a centralized check

Prevention System (Three-Layer Defense):

Verification:

• Direct state assignment in code review: All transitions go through _updateOrder() (cannot bypass)
• Automated tests: tests/repro_phantom_orders.ts confirms all prevention layers work
• Logging: Any phantom creation attempt is logged as ERROR with context

Fund Flow Architecture

The fund tracking system uses atomic operations to prevent race conditions and overdrafts.

graph LR
    subgraph "Blockchain Balances"
        CHAIN_FREE[chainFree<br/>Unallocated funds]
        CHAIN_COMMITTED[committed.chain<br/>On-chain orders]
    end

    subgraph "Internal Tracking"
        VIRTUAL[virtual<br/>Reserved for VIRTUAL orders]
        GRID_COMMITTED[committed.grid<br/>ACTIVE order sizes]
    end

    subgraph "Calculated Values"
        AVAILABLE[available<br/>= chainFree - virtual<br/>- fees]
        TOTAL_CHAIN[total.chain<br/>= chainFree + committed.chain]
        TOTAL_GRID[total.grid<br/>= committed.grid + virtual]
    end

    CHAIN_FREE --> AVAILABLE
    VIRTUAL --> AVAILABLE

    CHAIN_FREE --> TOTAL_CHAIN
    CHAIN_COMMITTED --> TOTAL_CHAIN

    GRID_COMMITTED --> TOTAL_GRID
    VIRTUAL --> TOTAL_GRID

    style AVAILABLE fill:#90EE90
    style VIRTUAL fill:#87CEEB

Fund Components Explained

• chainFree: Unallocated funds on blockchain (from accountTotals.buyFree/sellFree)
• committed.chain: Funds locked in on-chain orders (ACTIVE orders with orderId)
• committed.grid: Internal tracking of ACTIVE order sizes
• virtual: Funds reserved for VIRTUAL orders (not yet on-chain)
• available: Free funds for new orders = max(0, chainFree - virtual - fees)

Atomic Fund Operations

sequenceDiagram
    participant Strat as StrategyEngine
    participant Mgr as OrderManager
    participant Acct as Accountant

    Note over Mgr: Want to place order<br/>size = 100

    Mgr->>Acct: tryDeductFromChainFree(type, 100)

    alt Sufficient funds (available >= 100)
        Acct->>Acct: chainFree -= 100
        Acct->>Acct: virtual += 100
        Acct-->>Mgr: true (success)
        Mgr->>Mgr: Place order
    else Insufficient funds
        Acct-->>Mgr: false (failed)
        Note over Mgr: Order not placed<br/>No fund leak
    end

Grid Topology

The grid uses a unified "Master Rail" with a dynamic boundary that shifts as fills occur.

graph LR
    subgraph "Master Rail (Price Levels)"
        direction LR
        B0[buy-0<br/>VIRTUAL]
        B1[buy-1<br/>ACTIVE]
        B2[buy-2<br/>ACTIVE]
        BOUNDARY{Boundary<br/>Index}
        S0[spread-0<br/>SPREAD]
        S1[spread-1<br/>SPREAD]
        S2[spread-2<br/>SPREAD]
        SELL0[sell-173<br/>ACTIVE]
        SELL1[sell-174<br/>ACTIVE]
        SELL2[sell-175<br/>VIRTUAL]
    end

    B0 --> B1 --> B2 --> BOUNDARY
    BOUNDARY --> S0 --> S1 --> S2
    S2 --> SELL0 --> SELL1 --> SELL2

    style B1 fill:#90EE90
    style B2 fill:#90EE90
    style S0 fill:#FFD700
    style S1 fill:#FFD700
    style S2 fill:#FFD700
    style SELL0 fill:#FF6B6B
    style SELL1 fill:#FF6B6B
    style BOUNDARY fill:#87CEEB

Boundary Movement

• Buy Fill: boundaryIdx -= 1 (shift left/down)
• Sell Fill: boundaryIdx += 1 (shift right/up)

Role Assignment

• BUY: Slots [0, boundaryIdx]
• SPREAD: Slots [boundaryIdx + 1, boundaryIdx + G] where G = spread gap size (empty slots). Actual gaps = G + 1.
• SELL: Slots [boundaryIdx + G + 1, N]

Key Operations

1. Fill Processing Flow

sequenceDiagram
    participant Chain as Blockchain
    participant Sync as SyncEngine
    participant Mgr as OrderManager
    participant Acct as Accountant
    participant Strat as StrategyEngine

    Chain->>Mgr: Order filled
    Mgr->>Mgr: Detect fill
    Mgr->>Strat: processFillsOnly([fills])

    Strat->>Acct: Add proceeds to chainFree

    Mgr->>Strat: calculateTargetGrid(params)
    Strat-->>Mgr: target grid
    Mgr->>Mgr: Apply rotations via WorkingGrid
    Mgr->>Acct: Deduct BTS fees during recalculateFunds
    Mgr->>Mgr: Cancel detected dust partials immediately

2. Order Rotation (Crawl Mechanism)

graph TB
    START[Fill Detected] --> SHIFT[Shift Boundary]
    SHIFT --> IDENTIFY[Identify Shortages<br/>Empty slots in active window]
    IDENTIFY --> CHECK{Surpluses<br/>Available?}

    CHECK -->|Yes| CRAWL[Select Crawl Candidate<br/>Furthest active order]
    CHECK -->|No| NEW[Place New Order<br/>if funds available]

    CRAWL --> COMPARE{Shortage price<br/>better than<br/>surplus price?}
    COMPARE -->|Yes| ROTATE[Rotate Order<br/>Cancel old, place new]
    COMPARE -->|No| SKIP[Skip rotation]

    ROTATE --> NEXT{More<br/>shortages?}
    SKIP --> NEXT
    NEW --> NEXT

    NEXT -->|Yes| IDENTIFY
    NEXT -->|No| DONE[Rebalance Complete]

3. Grid Divergence Detection

The grid divergence system monitors and corrects misalignment between ideal grid state and persistent blockchain state.

graph TB
    START[Grid Update Triggered] --> CALC[Calculate Ideal Grid<br/>Based on current funds]
    CALC --> RELOAD[Force Reload Persisted Grid<br/>Ensure fresh blockchain state]
    RELOAD --> COMPARE[Compare to Persisted Grid]
    COMPARE --> RMS[Calculate RMS Divergence<br/>ACTIVE + VIRTUAL orders per side]

    RMS --> CHECK{RMS > Threshold?}
    CHECK -->|Yes| UPDATE[Update Grid Sizes<br/>Trigger rebalance]
    CHECK -->|No| SKIP[Skip update]

    UPDATE --> PERSIST[Persist New Grid State]
    PERSIST --> DONE[Complete]
    SKIP --> DONE

The force reload mechanism loads fresh persisted grid data before comparison, preventing stale cache from causing false divergence detections.

Concurrency & Locking

The system uses order-level locks to prevent race conditions during async operations. For the reconciler-specific lock hierarchy and the historical _syncLock/_gridLock swap, see GRID_RECONCILE.md.

Lock Mechanism

sequenceDiagram
    participant Sync as SyncEngine
    participant Strat as StrategyEngine
    participant Mgr as OrderManager

    Note over Sync: Detected fill on order P1
    Sync->>Mgr: lockOrders([P1])
    Sync->>Sync: Process fill

    par Concurrent Strategy Check
        Strat->>Mgr: isOrderLocked(P1)?
        Mgr-->>Strat: true
        Note over Strat: Skip P1 (locked)
    end

    Sync->>Sync: Complete fill processing
    Sync->>Mgr: unlockOrders([P1])

    Note over Strat: Next cycle can now process P1

Lock Lifetime

• Default timeout: 5-10 seconds
• Auto-expiry: Prevents deadlocks from crashes
• Best practice: Always use try/finally to ensure unlock

Module Responsibilities Summary

Dynamic Configuration Refresh

The bot implementation supports runtime updates to specific configuration parameters without requiring a process restart. This is handled via a Periodic Configuration Refresh mechanism.

The Refresh Cycle

Every 4 hours (default BLOCKCHAIN_FETCH_INTERVAL_MIN), the bot performs the following safe refresh cycle:

1. Thread-Safe Load: The bot re-reads profiles/bots.json using readBotsFileWithLock to ensure it doesn't collide with manual edits or the CLI manager.
2. Memory Update: It identifies its own configuration entry and updates its internal memory state (this.config and manager.config).
3. Non-Disruptive Application: The refresh is designed to be passive. It updates valuation anchors but does not trigger on-chain order movement automatically.

Configuration Authority: startPrice / poolRef

The startPrice parameter follows a strict hierarchy of authority:

Data Persistence

graph LR
    subgraph "In-Memory State"
        ORDERS[orders Map<br/>Grid state]
        FUNDS[funds Object<br/>Fund tracking]
        INDICES[Indices<br/>_ordersByState<br/>_ordersByType]
    end

    subgraph "Persisted State"
        ORDERS_JSON[<botKey>.json<br/>Grid snapshot<br/>feesOwed, boundaryIdx, btsBalance]
        BOTS_JSON[bots.json<br/>Bot config]
    end

    ORDERS --> ORDERS_JSON
    FUNDS --> ORDERS_JSON

    ORDERS_JSON -.->|Load on startup| ORDERS
    ORDERS_JSON -.->|Load on startup| FUNDS

    BOTS_JSON -.->|Load on startup| CONFIG[Bot Config]

Persistence Strategy

• Grid state: Persisted after every rebalance to <botKey>.json in profiles/orders/
• Fund state: Available funds derived from blockchain balances at runtime (no separate persistence needed)
• Retry logic: 3 attempts with exponential backoff
• Graceful degradation: Bot continues if persistence fails (in-memory only)

Memory-Only Integer Tracking

The system has been optimized to use a "memory-driven" model for order updates, eliminating redundant blockchain API calls during normal operation.

How it works

• Raw order cache (rawOnChain): each grid slot stores the exact blockchain order integers (satoshis) — seeded from broadcast arguments on placement, updated in place on partial fills, and refreshed on updates/rotations.
• Chain-free planning: size updates and rotations build their operations from the cache; only placements and recovery syncs query the blockchain.
• buildUpdateOrderOp(cachedOrder?): accepts an optional cached order and returns finalInts alongside the operation for local tracking.
• Self-healing: a failed memory-driven transaction triggers a full state-recovery sync so the internal ledger stays consistent with the chain.

Benefits

• Faster reaction time: No waiting for blockchain queries during order updates
• Reduced API load: Fewer fetches, less network congestion
• Mathematical precision: Integer-based tracking prevents float precision errors
  • See FUND_MOVEMENT_AND_ACCOUNTING.md § 5.5 for quantization utilities and best practices
• Fallback safety: Automatic recovery if memory state becomes inconsistent

Performance Impact

• Batch operations (size updates, rotations) now run without any blockchain fetches
• Only placement operations and recovery syncs query the blockchain
• Estimated 10-20x speedup for high-frequency operations

Error Handling & Safety

Fund Invariants

The system continuously monitors three mathematical invariants:

1. Account Equality: chainTotal = chainFree + committed.chain
2. Committed Ceiling: committed.grid <= chainTotal
3. Available Leak Check: available <= chainFree

Tolerance: 0.1% (to account for fees and rounding)

Index Consistency

• Grid versioning: _gridVersion is bumped on every grid mutation; the _ordersByType / _ordersByState index caches invalidate automatically when the version changes (no manual rebuild step)
• Structural checks: tests use assertOrdersStructurallySound() (tests/helpers/order_test_helpers.ts) to verify every order in the Map is non-null with valid state and type

Performance Considerations

Optimization Strategies

1. Batch fund recalculation: pauseFundRecalc() / resumeFundRecalc()
2. Index-based lookups: O(1) access via _ordersByState and _ordersByType
3. Lock expiry: Prevents permanent blocking from crashes
4. Fee caching: Reduces blockchain API calls

Metrics Tracking

manager.getMetrics()
// Returns:
// - fundRecalcCount
// - invariantViolations
// - lockAcquisitions
// - stateTransitions
// - lastSyncDurationMs

Zero-Dependency Policy

DEXBot2 operates under a zero mandatory production dependency policy. The dependency tree is empty — every production capability (blockchain client, WebSocket transport via Node's native globalThis.WebSocket, crypto/signing, serialization, testing, price feeds, credential vault) is implemented natively within the codebase. Requires Node.js >= 22.12 for the built-in WebSocket.

Why: Trading bots handle real money. Every external dependency is a supply-chain risk surface. Keeping the dependency tree empty means no npm audit surprises, no supply-chain attacks on upstream packages, and no version-migration overhead for the core runtime.

Special case — trading bots: This level of dependency discipline is rare in open-source trading software. Established projects (Gekko, Freqtrade, Hummingbot) carry 15–20+ production dependencies. DEXBot2's empty dependency tree is a deliberate architectural choice, not an accidental outcome — it reflects the project's priority of operational safety over developer convenience.

What this means in practice:

• Blockchain connectivity (bitshares-native/) — hand-rolled from protocol primitives
• Elliptic curve crypto (secp256k1) — native JS, zero native addons
• Testing — node:assert, no Jest/Mocha/Vitest
• Persistence — JSON flat files, no SQLite/ORM
• Price sources — native candle fetching, no CCXT/CoinGecko
• Process management — PM2 ecosystem or direct Node.js, no Docker requirement

Testing Strategy & Quality Assurance

DEXBot2 uses a native Node.js assert testing strategy to ensure reliability without heavy dependencies.

Test Coverage by Module

graph LR
    A["Logic Tests<br/>(tests/test_*_logic.ts)"]
    B["Integration Tests<br/>(tests/test_*.ts)"]
    C["Signal Tests<br/>(tests/test_*_signal*.ts)"]
    D["Credit/Debt Tests<br/>(tests/test_*_credit*.ts)"]

    A -->|Manager, State Machine| A1["manager_logic"]
    A -->|Fund Tracking| A2["accounting_logic"]
    A -->|Grid Creation| A3["grid_logic"]
    A -->|Rebalancing| A4["strategy_logic"]
    A -->|Sync Logic| A5["sync_logic"]

    B -->|Multi-step Scenarios| B1["Market Scenarios"]
    B -->|Edge Cases| B2["Partial Order Tests"]
    B -->|Real-world Scenarios| B3["Fills/FEE Tests"]

    C -->|Dynamic Weight| C1["dynamic_weight"]

    D -->|CR Planner| D1["cr_planner"]
    D -->|Credit Runtime| D2["credit_runtime"]
    D -->|MPA Wiring| D3["dexbot_credit_wiring"]

Running Tests

# Run all tests (native assert)
npm test

# Specific logic area
node dist/tests/test_accounting_logic.js

# Signal tests
node dist/tests/test_market_adapter_signal_gates.js
node dist/tests/test_dynamic_weight_override_wiring.js

# Credit/debt tests
node dist/tests/test_cr_planner.js
node dist/tests/test_dexbot_credit_wiring.js

Test Quality Metrics

Coverage Goals:

• ✅ All public methods have tests
• ✅ All invariants verified automatically
• ✅ Edge cases covered (zero funds, max orders, etc.)
• ✅ Concurrent operations tested with locks
• ✅ State transitions validated end-to-end
• ✅ Signal pipelines tested (dynamic weight)
• ✅ Credit/debt runtime tested (CR planner, MPA wiring)

Test Suite Evolution:

• 50+ test cases for signal intelligence and credit runtime
• Dynamic weight override and market adapter signal gate tests
• Credit/debt tests with CR planner and MPA wiring validation

Testing Best Practices

For Developers:

1. Run tests before commits

   npm test

2. Add tests for new features

   • Follow patterns in existing tests
   • Test fund impact of new logic
   • Include edge cases

3. Verify invariants

   assertOrdersStructurallySound(manager);  // Order Map structurally valid
   expect(chainTotal === chainFree + chainCommitted).toBe(true);

4. Use debug mode for problematic scenarios

   manager.logger.level = 'debug';  // Enable detailed logging
   // ... run scenario ...
   // Check console output for detailed fund tracking

Test Documentation References

• tests/README.md - Test suite organization, categories, and running instructions

• developer_guide.md#testing-fund-calculations - Testing guide for developers

  • How to write fund tests
  • Common test patterns
  • Debugging failing tests
  • Adding tests for new features

Market Adapter Signal Pipeline

The market adapter runs as a standalone process that computes AMA-derived grid prices, trend signals, dynamic weights, and collateral-ratio advisories.

Signal Flow

price_candles -> AMA -> gridCenterPrice
                |
                +-> slope_analysis -> trend_offset (asymmetric weight shift)
                |
price_candles -> ATR -> weight_variance (symmetric shift)
                |
                +-> regime_detection (Hurst/PE) -> regime_filter
                |
                +-> Kalman_confirmation -> blended_dynamic_weight

Outputs (per cycle, per bot)

• gridCenterPrice — AMA center price, clamped to min/max bounds
• weights — { buy, sell } dynamic grid weighting
• collateralRecommendation — advisory collateral ratio hint
• trend / atr — raw regime and volatility signals
• Trigger files when the adapter accepts the first AMA center, the grid price delta exceeds threshold, or whitelisted range-scaling slope delta exceeds threshold

Integration with Bot Runtime

1. Adapter persists profiles/orders/<botKey>.dynamicgrid.json before any reset trigger.
2. Adapter writes profiles/recalculate.<botKey>.trigger for bootstrap, AMA-center delta, or whitelisted AMA-slope range reset.
3. dexbot_maintenance_runtime.ts consumes the trigger under _fillProcessingLock, with idle/dust deferral when needed.
4. Bot runtime reads accepted center, range-scaling fields, and dynamic weights from the dynamic-grid snapshot during reset and selected maintenance paths.

Credential Security Architecture

DEXBot2 uses a hardened credential daemon (credential-daemon.ts) for key management and signing.

Security Layers

• Vault v2: scrypt (N=2^17) key derivation, per-record HKDF isolation, AES-256-GCM encryption
• Daemon-backed signing: Primary bot flow uses signing tokens; all signing happens inside the daemon, raw keys never exported
• Session cache: Encrypted HKDF re-encryption with a random salt never persisted
• Runtime hardening: lstat + owner/mode/type checks on all sockets and ready files; bootstrap socket destroyed after first use
• Strict daemon policy: Memory safety and zeroing, session hardening, signing cache with time-based expiry

PM2 Integration

dexbot pm2 unlocks dexbot-cred through a one-shot local bootstrap channel instead of exporting the master password to every PM2 app. Use dexbot pm2 restart ... for DEXBot-managed PM2 actions.

Credit/Debt Runtime

Native DEXBot2 support for MPA borrowing and credit offer workflows. The logic is split between a pure-math planner and a lifecycle runtime:

Scope

• MPA borrowing: Call-order updates with debt-first CR planning
• Credit offers: Accept/repay with auto-reborrow, LP-backed collateral valuation
• Policy enforcement: Per-bot debtPolicy with hard CR floors/ceilings and max fee rates
• Grid reset coupling: Every successful CR adjustment requests a grid rebuild

Runtime Rules

• Evaluates on the dedicated credit watchdog interval
• No separate enable switch — active when debtPolicy.lending is present, non-empty, and every item declares collateralAsset
• Claw can read the same bot policy without redefining rules (via claw/modules/credit_runtime_adapter.ts)

Grid Rebalancing Robustness

The strategy engine has been significantly strengthened with improvements to fund validation, dust handling, and order constraints:

1. Pre-Flight Fund Validation

• Before executing batch order placements, available funds are validated
• Prevents insufficient fund errors during large rotation cycles
• Uses atomic check-and-deduct pattern for safety
• Located in: modules/dexbot_cow_runtime.ts - validateOperationFunds()

2. Dust Partial Handling

• Improved dust detection algorithm prevents false positives
• Detects dust as < 5% of ideal order size
• Immediate Cancellation: _cancelDustOrders() cancels dust partials on-chain when detection runs — no delay, timer map, or consolidation cycle. Detection runs after fills/sync, and a 5-minute health check is the restart safety net.

3. Strict Order Size Constraints

• Orders validated to not exceed available funds
• Maximum order size enforced during both placement and rotation
• Prevents oversized orders that fail on-chain
• Atomic validation with placement ensures consistency

4. Boundary Index Persistence

• BoundaryIdx (spread zone pivot) now correctly persisted across bot restarts
• Ensures grid rotation continues seamlessly after divergence correction
• Fixes grid instability from incorrect boundary tracking

5. Taker Fee Accounting

• Both market and blockchain taker fees now accounted for correctly
• Fee deduction uses proper isMaker parameter
• Prevents fund leaks from missing fee calculations
• Located in: modules/order/manager.ts - processFilledOrders()

6. Precision Spread Management (Logarithmic Logic)

• Discrete Step Tracking: A discrete 1-slot logarithmic buffer ensures correction triggers exactly when the market moves by one full increment.
• Center-Gap Awareness: Grid initialization math accounts for the "Center Gap" naturally created during symmetric centering, reducing the initial spread by ~0.5% (one full increment).
• Collision-Free Safety: MIN_SPREAD_FACTOR of 2.1 ensures that the security minimum (2 spread orders) never conflicts with the spread correction threshold, even at micro-spread configurations.

Related Documentation

For detailed fund calculations and test coverage, see:

• developer_guide.md#testing-fund-calculations - How fund calculations are tested
• tests/README.md - Test suite organization and running instructions

• Fund Movement Logic - Detailed mathematical formulas and algorithms
• Developer Guide - Code navigation and onboarding
• README - User documentation and setup
• WORKFLOW.md - Git branch workflow

Quick Reference

Common Code Patterns

Get orders by state and type:

const activeBuys = manager.getOrdersByTypeAndState(ORDER_TYPES.BUY, ORDER_STATES.ACTIVE);

Atomic fund deduction:

if (manager.accountant.tryDeductFromChainFree(orderType, size)) {
    // Funds deducted, safe to place order
} else {
    // Insufficient funds, skip
}

Batch order updates:

manager.pauseFundRecalc();
for (const order of orders) {
    // context parameter helps with logging/debugging the source of the update
    manager._updateOrder(order, 'batch-update', { skipAccounting: false, fee: 0 });
}
manager.resumeFundRecalc(); // Recalculates once

Lock orders during async operations:

manager.lockOrders([orderId]);
try {
    await asyncOperation();
} finally {
    manager.unlockOrders([orderId]);
}