Deep Dive into QuantMesh Core Implementation: Technical Architecture of a High-Performance Grid Trading System

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • MyrinNew
    Senior Member
    • Feb 2024
    • 5175

    #1

    Deep Dive into QuantMesh Core Implementation: Technical Architecture of a High-Performance Grid Trading System


    This article provides an in-depth exploration of the core technical implementation of QuantMesh grid trading system, covering architecture design, concurrency models, state management, and risk control mechanisms—a comprehensive guide to building a high-performance quantitative trading system.


    Introduction

    QuantMesh is a high-performance, low-latency cryptocurrency market-making system built with Go, focused on grid trading strategies for perpetual futures markets. After real-world trading validation, the system has processed over $100 million in trading volume. This article delves into the core implementation, revealing the technical secrets behind its high performance and reliability.


    I. Overall Architecture Design

    1.1 Layered Architecture

    QuantMesh employs a clear layered architecture to achieve separation of concerns:






    ┌─────────────────────────────────────────┐
    │ Application Layer (main.go) │
    │ Component orchestration, lifecycle │
    └──────────────┬──────────────────────────┘

    ┌──────────────┴──────────────────────────┐
    │ Business Logic Layer │
    │ ┌──────────┐ ┌──────────┐ │
    │ │ Position │ │ Order │ │
    │ │ (Position)│ │ (Order) │ │
    │ └──────────┘ └──────────┘ │
    │ ┌──────────┐ ┌──────────┐ │
    │ │ Monitor │ │ Safety │ │
    │ │ (Monitor)│ │ (Risk) │ │
    │ └──────────┘ └──────────┘ │
    └───┬────────────────────────────────────┘

    ┌───▼────────────────────────────────────┐
    │ Abstract Interface Layer │
    │ IExchange Unified Interface │
    └───┬────────────────────────────────────┘

    ┌───▼────────────────────────────────────┐
    │ Implementation Layer │
    │ binance/ bitget/ gate/ ... │
    └────────────────────────────────────────┘







    1.2 Core Design Principles

    Single Price Source Principle


    A key design decision is ensuring a globally unique price stream. All components obtain prices through the unified PriceMonitor, avoiding inconsistencies:






    type PriceMonitor struct {
    lastPrice atomic.Value // Atomic price storage
    lastPriceStr atomic.Value // String format (for precision detection)
    priceChangeCh chan PriceChange // Price change channel
    }







    Order Stream Priority Principle


    The system follows the principle of "start order stream first, then place orders." This ensures that when order execution notifications arrive, the system is ready to receive and process them, avoiding missed executions.


    Interface Abstraction Principle


    The IExchange interface unifies API differences across exchanges, completely decoupling business logic from specific implementations and enabling flexible extension to new exchanges.


    II. Super Position Manager System

    2.1 Slot Concept

    Slots are QuantMesh's core innovation. Each slot corresponds to a grid price point, managing order status and position status at that price:






    type InventorySlot struct {
    Price float64 // Slot price

    // Position status
    PositionStatus string // EMPTY/FILLED
    PositionQty float64

    // Order status
    OrderID int64
    ClientOID string
    OrderSide string // BUY/SELL
    OrderStatus string // NOT_PLACED/PLACED/FILLED/CANCELED

    // Lock mechanism
    SlotStatus string // FREE/PENDING/LOCKED

    mu sync.RWMutex // Slot lock
    }







    2.2 State Machine Design

    Slots manage their lifecycle through a state machine, preventing concurrency conflicts:






    FREE (idle)
    ↓ place order
    PENDING (awaiting order confirmation)
    ↓ order success
    LOCKED (locked, active order exists)
    ↓ buy order filled
    FILLED (has position, awaiting sell)
    ↓ sell order filled
    FREE (back to initial state)







    Key Design Points:

    1. PENDING State: Prevents duplicate orders during rapid price changes
    2. LOCKED State: Ensures no duplicate operations when active orders exist
    3. Fine-grained Locking: Each slot has an independent lock, supporting high concurrency


    2.3 Concurrent-Safe Slot Operations





    // Slots stored in sync.Map, supporting concurrent read/write
    type SuperPositionManager struct {
    slots sync.Map // map[float64]*InventorySlot
    mu sync.RWMutex // Global lock (for batch operations)
    }

    // Safe slot update
    func (spm *SuperPositionManager) updateSlot(price float64, updater func(*InventorySlot)) {
    slot, _ := spm.slots.LoadOrStore(price, &InventorySlot{
    Price: price,
    SlotStatus: "FREE",
    })

    s := slot.(*InventorySlot)
    s.mu.Lock()
    defer s.mu.Unlock()

    updater(s)
    }







    III. Price Monitoring System

    3.1 Atomic Operations for Consistency

    Price monitoring uses atomic.Value for lock-free reads, ensuring performance in high-concurrency scenarios:






    type PriceMonitor struct {
    lastPrice atomic.Value // float64
    lastPriceStr atomic.Value // string
    priceChangeCh chan PriceChange
    }

    func (pm *PriceMonitor) updatePrice(price float64) {
    // Atomic write, no lock needed
    pm.lastPrice.Store(price)
    pm.lastPriceStr.Store(fmt.Sprintf("%.8f", price))
    }

    func (pm *PriceMonitor) GetPrice() float64 {
    // Atomic read, no lock needed
    return pm.lastPrice.Load().(float64)
    }







    3.2 Periodic Push Mechanism

    To avoid system overload from frequent price changes, a periodic push strategy is used:






    func (pm *PriceMonitor) periodicPriceSender(ctx context.Context) {
    ticker := time.NewTicker(50 * time.Millisecond)
    defer ticker.Stop()

    for {
    select {
    case <-ctx.Done():
    return
    case <-ticker.C:
    latest := pm.latestPriceChange.Load()
    if latest != nil {
    // Non-blocking send to avoid blocking price updates
    select {
    case pm.priceChangeCh <- latest.(PriceChange):
    default: // channel full, skip
    }
    }
    }
    }
    }







    Design Considerations:
    • 50ms Push Interval: Balances real-time performance and system load
    • Non-blocking Send: Prevents blocking price updates
    • Buffered Channel: Capacity of 10, smoothing peaks


    IV. Exchange Interface Abstraction Layer

    4.1 Unified Interface Design





    type IExchange interface {
    // Order operations
    PlaceOrder(ctx context.Context, req *OrderRequest) (*Order, error)
    BatchPlaceOrders(ctx context.Context, orders []*OrderRequest) ([]*Order, bool)
    CancelOrder(ctx context.Context, symbol, orderID string) error

    // Account queries
    GetAccount(ctx context.Context) (*Account, error)
    GetPositions(ctx context.Context, symbol string) ([]*Position, error)

    // WebSocket
    StartPriceStream(ctx context.Context, symbol string, callback PriceCallback)
    StartOrderStream(ctx context.Context, callback OrderCallback)
    StartKlineStream(ctx context.Context, symbols []string, interval string, callback KlineCallback)
    }







    4.2 Adapter Pattern Implementation

    Adapter pattern encapsulates specific exchange implementations:






    // Binance adapter
    type BinanceAdapter struct {
    client *binance.Client
    // ... other fields
    }

    func (a *BinanceAdapter) PlaceOrder(ctx context.Context, req *OrderRequest) (*Order, error) {
    // Convert request format
    binanceReq := convertToBinanceRequest(req)

    // Call Binance API
    binanceOrder, err := a.client.PlaceOrder(binanceReq)
    if err != nil {
    return nil, err
    }

    // Convert response format
    return convertFromBinanceOrder(binanceOrder), nil
    }







    V. Order Execution and Rate Limiting

    5.1 Rate Limiter Implementation

    Uses golang.org/x/time/rate to implement token bucket rate limiting:






    type ExchangeOrderExecutor struct {
    exchange exchange.IExchange
    rateLimiter *rate.Limiter // 25 requests/second, burst 30
    }

    func (e *ExchangeOrderExecutor) PlaceOrder(ctx context.Context, req *OrderRequest) (*Order, error) {
    // Wait for token
    if err := e.rateLimiter.Wait(ctx); err != nil {
    return nil, err
    }

    // Execute order placement with retry mechanism
    var lastErr error
    for i := 0; i < maxRetries; i++ {
    order, err := e.exchange.PlaceOrder(ctx, req)
    if err == nil {
    return order, nil
    }

    lastErr = err
    if !isRetryableError(err) {
    break
    }

    time.Sleep(retryDelay)
    }

    return nil, lastErr
    }







    VI. Multi-Layer Risk Control System

    6.1 Pre-Start Safety Checks

    Comprehensive capital safety checks before system startup:






    func checkSafety(cfg *config.Config, account *Account) error {
    // 1. Check account balance
    requiredMargin := calculateRequiredMargin(cfg)
    if account.Balance < requiredMargin*1.2 {
    return fmt.Errorf("insufficient balance: need %v, current %v", requiredMargin, account.Balance)
    }

    // 2. Check leverage
    if cfg.Leverage > maxLeverage {
    return fmt.Errorf("leverage too high: %d > %d", cfg.Leverage, maxLeverage)
    }

    // 3. Check max positions
    maxPositions := (account.Balance * cfg.Leverage) / cfg.OrderQuantity
    if maxPositions < cfg.PositionSafetyCheck {
    return fmt.Errorf("position safety insufficient: max %d positions, require %d",
    maxPositions, cfg.PositionSafetyCheck)
    }

    return nil
    }







    6.2 Active Risk Monitoring

    Real-time monitoring of K-line trading volume across multiple major coins, detecting market anomalies:






    type RiskMonitor struct {
    klineCache map[string][]Kline
    volumeMultiplier float64
    averageWindow int
    isTriggered atomic.Bool
    }

    func (rm *RiskMonitor) checkRisk(ctx context.Context, kline Kline) {
    // Update K-line cache
    rm.updateKlineCache(kline)

    // Calculate moving average volume
    avgVolume := rm.calculateAverageVolume(kline.Symbol)

    // Detect anomalies
    if kline.Volume > avgVolume*rm.volumeMultiplier {
    rm.triggerRiskControl()
    }
    }







    VII. Concurrency Model and Performance Optimization

    7.1 Goroutine Architecture

    The system starts multiple Goroutines to handle different tasks:






    // Main Goroutines
    go priceMonitor.Start(ctx)
    go exchange.StartOrderStream(ctx, handleOrderUpdate)
    go riskMonitor.Start(ctx)
    go reconciler.Start(ctx)
    go orderCleaner.Start(ctx)

    // Price change processing
    for priceChange := range priceMonitor.Subscribe() {
    if riskMonitor.IsTriggered() {
    continue // Skip trading logic
    }
    positionManager.AdjustOrders(priceChange.Price)
    }







    7.2 Performance Optimization Techniques

    1. sync.Map instead of map+mutex: Better performance in high-concurrency read-heavy scenarios
    2. atomic.Value instead of mutex: Price reads use atomic operations, avoiding lock contention
    3. Object Pooling: Reuse order request objects, reducing GC pressure
    4. Batch Operations: Batch order placement and cancellation, reducing network round trips


    VIII. Fixed Amount Grid Strategy

    8.1 Strategy Implementation

    Unlike traditional fixed-quantity grids, QuantMesh uses a fixed-amount mode:






    func (spm *SuperPositionManager) adjustOrders(newPrice float64) {
    spm.slots.Range(func(key, value interface{}) bool {
    slot := value.(*InventorySlot)
    price := slot.Price

    // Calculate order quantity (fixed amount)
    quantity := cfg.OrderQuantity / price

    // Decide whether to place order based on price distance
    if abs(price - newPrice) <= cfg.BuyWindowSize * cfg.PriceInterval {
    if slot.SlotStatus == "FREE" {
    placeBuyOrder(slot, quantity)
    }
    }

    return true
    })
    }







    Advantages:
    • More controllable capital utilization
    • Same investment per order, uniform risk
    • Adapts to different price ranges


    IX. Challenges and Solutions in Practice

    9.1 Precision Handling

    Different exchanges have different price and quantity precisions. The system automatically obtains and handles them:






    func (a *Adapter) initialize() error {
    info, err := a.client.GetExchangeInfo()
    a.pricePrecision = info.PricePrecision
    a.quantityPrecision = info.QuantityPrecision
    return nil
    }

    func (a *Adapter) roundPrice(price float64) float64 {
    return math.Round(price*math.Pow10(a.pricePrecision)) / math.Pow10(a.pricePrecision)
    }







    9.2 Network Exception Handling

    WebSocket connections may disconnect due to network issues. The system implements automatic reconnection:






    func (ws *WebSocket) reconnect() {
    for {
    err := ws.connect()
    if err == nil {
    break
    }
    time.Sleep(exponentialBackoff(ws.reconnectAttempts ))
    ws.reconnectAttempts++
    }
    }







    X. Conclusion

    Through carefully designed architecture and implementation, QuantMesh achieves:

    1. High Performance: Go language + WebSocket + concurrency optimization, millisecond-level response
    2. High Reliability: Multi-layer risk control + automatic reconciliation + error retry
    3. Easy Extension: Interface abstraction + factory pattern + adapter pattern
    4. Concurrent Safety: Fine-grained locks + atomic operations + state machines


    The system's success lies not only in technical implementation but also in a deep understanding of quantitative trading. The slot system solves concurrency conflicts in grid trading, the fixed-amount mode achieves more controllable capital management, and multi-layer risk control ensures capital safety.





    QuantMesh is an open-source project, with code hosted on GitHub. Contributions, suggestions, and discussions are welcome.




    More...
Working...