Integration Guide

Getting Started

This guide provides comprehensive instructions for integrating Shards into your decentralized applications, protocols, and services. Whether you're building a DeFi protocol, an AI-powered trading bot, or a data analytics platform, this guide will help you leverage Shards' oracle aggregation capabilities effectively.

Prerequisites

Technical Requirements

  • Programming Knowledge: Familiarity with JavaScript/TypeScript, Python, or Rust

  • Solana Basics: Understanding of Solana accounts, transactions, and programs

  • API Experience: Basic knowledge of REST APIs and WebSocket connections

  • Development Environment: Node.js 16+ or Python 3.8+

Account Setup

  1. Register for API Access

    curl -X POST https://api.shards.tv/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{
    "email": "[email protected]",
    "organization": "Your Organization",
    "use_case": "DeFi Protocol Integration"
  }'

  2. Generate API Key

    curl -X POST https://api.shards.tv/v1/auth/api-key \
  -H "Authorization: Bearer YOUR_AUTH_TOKEN" \
  -d '{
    "name": "Production Key",
    "permissions": ["read", "stream"],
    "rate_limit_tier": "pro"
  }'

Installation

JavaScript/TypeScript

npm install @shards/sdk
# or
yarn add @shards/sdk
# or
pnpm add @shards/sdk

Python

pip install shards-sdk
# or
poetry add shards-sdk

Rust

[dependencies]
shards-sdk = "0.2.0"
tokio = { version = "1", features = ["full"] }

Basic Integration

Initialize Client

TypeScript

import { ShardsClient, Network, OracleSource } from '@shards/sdk';

const client = new ShardsClient({
  apiKey: process.env.SHARDS_API_KEY,
  network: Network.Mainnet,
  options: {
    timeout: 5000,
    retryAttempts: 3,
    cacheEnabled: true
  }
});

// Verify connection
const status = await client.getStatus();
console.log('Shards API Status:', status);

Python

import os
from shards import ShardsClient, Network
from shards.config import ClientConfig

config = ClientConfig(
    api_key=os.environ['SHARDS_API_KEY'],
    network=Network.MAINNET,
    timeout=5,
    retry_attempts=3,
    cache_enabled=True
)

client = ShardsClient(config)

# Verify connection
status = client.get_status()
print(f"Shards API Status: {status}")

Fetching Oracle Data

Simple Price Query

// Get current price with aggregation
const solPrice = await client.oracle.getPrice('SOL');
console.log(`SOL Price: $${solPrice.price}`);
console.log(`Confidence: ±${solPrice.confidence}`);
console.log(`Last Updated: ${new Date(solPrice.timestamp * 1000)}`);

// Get price from specific oracle
const pythPrice = await client.oracle.getPrice('SOL', {
  source: OracleSource.Pyth,
  includeMetadata: true
});

Batch Price Queries

// Efficient batch request
const symbols = ['SOL', 'BTC', 'ETH', 'USDC', 'RAY'];
const prices = await client.oracle.getBatchPrices(symbols);

prices.forEach(({ symbol, price, confidence }) => {
  console.log(`${symbol}: $${price} (±${confidence})`);
});

Real-time Data Streaming

WebSocket Subscription

// Create streaming client
const stream = client.createStream();

// Subscribe to price updates
stream.subscribe({
  channels: ['oracle.price'],
  symbols: ['SOL', 'BTC'],
  callback: (update) => {
    console.log(`${update.symbol} Price Update: $${update.price}`);
    console.log(`Change: ${update.change24h}%`);
  },
  errorHandler: (error) => {
    console.error('Stream error:', error);
  }
});

// Start streaming
await stream.connect();

// Graceful shutdown
process.on('SIGINT', async () => {
  await stream.disconnect();
  process.exit(0);
});

Advanced Integration Patterns

DeFi Protocol Integration

Lending Protocol Example

import { PublicKey, Connection } from '@solana/web3.js';
import { ShardsClient, PriceData } from '@shards/sdk';

class LendingProtocolOracle {
  private shards: ShardsClient;
  private priceCache: Map<string, PriceData>;
  private updateInterval: NodeJS.Timer;

  constructor(apiKey: string) {
    this.shards = new ShardsClient({ apiKey });
    this.priceCache = new Map();
    this.initializePriceUpdates();
  }

  private initializePriceUpdates() {
    // Update prices every 5 seconds
    this.updateInterval = setInterval(async () => {
      await this.refreshPrices();
    }, 5000);

    // Subscribe to real-time updates
    this.shards.stream.subscribe({
      channels: ['oracle.price'],
      symbols: this.getSupportedAssets(),
      callback: (update) => {
        this.priceCache.set(update.symbol, update);
        this.checkLiquidations(update);
      }
    });
  }

  async getCollateralValue(
    asset: string, 
    amount: number
  ): Promise<number> {
    const price = await this.getPrice(asset);
    const collateralFactor = this.getCollateralFactor(asset);
    return amount * price.price * collateralFactor;
  }

  async checkLiquidations(priceUpdate: PriceData) {
    // Get all positions for this asset
    const positions = await this.getPositionsForAsset(priceUpdate.symbol);
    
    for (const position of positions) {
      const healthFactor = await this.calculateHealthFactor(
        position,
        priceUpdate
      );
      
      if (healthFactor < 1.0) {
        await this.triggerLiquidation(position);
      }
    }
  }

  async calculateHealthFactor(
    position: LoanPosition,
    priceData: PriceData
  ): Promise<number> {
    const collateralValue = position.collateralAmount * priceData.price;
    const borrowValue = await this.getBorrowValue(position);
    
    // Include confidence interval for conservative calculation
    const adjustedCollateral = collateralValue * 
      (1 - priceData.confidence / priceData.price);
    
    return adjustedCollateral / borrowValue;
  }
}

AI/LLM Trading Bot Integration

class AITradingBot {
  private shards: ShardsClient;
  private llm: LLMInterface;

  constructor(shardsKey: string, llmKey: string) {
    this.shards = new ShardsClient({ apiKey: shardsKey });
    this.llm = new LLMInterface(llmKey);
  }

  async analyzeMarket(query: string): Promise<TradingSignal> {
    // Use Shards' LLM interface for natural language queries
    const marketData = await this.shards.llm.query(query);
    
    // Get detailed technical analysis
    const analysis = await this.performTechnicalAnalysis(
      marketData.data
    );
    
    // Generate trading signal
    return this.generateSignal(analysis);
  }

  async executeNaturalLanguageStrategy(instruction: string) {
    // Example: "Buy SOL when it drops 5% from daily high"
    const interpretation = await this.shards.llm.interpret(instruction);
    
    // Set up monitoring
    this.shards.stream.subscribe({
      channels: ['oracle.price'],
      symbols: [interpretation.asset],
      callback: async (update) => {
        if (this.meetsCondition(update, interpretation.condition)) {
          await this.executeTrade(interpretation.action);
        }
      }
    });
  }

  async getMarketContext(symbol: string): Promise<MarketContext> {
    // Parallel data fetching for comprehensive analysis
    const [
      currentPrice,
      historicalData,
      volatility,
      correlations
    ] = await Promise.all([
      this.shards.oracle.getPrice(symbol),
      this.shards.oracle.getHistory(symbol, '7d', '1h'),
      this.calculateVolatility(symbol),
      this.getCorrelations(symbol)
    ]);

    return {
      price: currentPrice,
      trend: this.analyzeTrend(historicalData),
      volatility,
      correlations,
      sentiment: await this.getSentiment(symbol)
    };
  }
}

Cross-Chain Oracle Aggregation

class CrossChainAggregator {
  private shards: ShardsClient;
  private externalOracles: Map<string, OracleInterface>;

  async getUniversalPrice(
    asset: string,
    chains: string[]
  ): Promise<UniversalPrice> {
    // Get Solana oracle data from Shards
    const solanaPrice = await this.shards.oracle.getPrice(asset, {
      source: 'aggregated',
      includeAllSources: true
    });

    // Aggregate with other chains if needed
    const crossChainPrices = await Promise.all(
      chains.map(chain => this.getChainPrice(chain, asset))
    );

    return this.calculateUniversalPrice(
      solanaPrice,
      crossChainPrices
    );
  }

  private calculateUniversalPrice(
    solanaData: PriceData,
    otherChains: ChainPrice[]
  ): UniversalPrice {
    // Weight by liquidity and update frequency
    const weights = this.calculateWeights(solanaData, otherChains);
    
    let weightedPrice = 0;
    let totalWeight = 0;
    
    // Include Solana price
    weightedPrice += solanaData.price * weights.solana;
    totalWeight += weights.solana;
    
    // Include other chain prices
    otherChains.forEach((chainPrice, i) => {
      weightedPrice += chainPrice.price * weights.chains[i];
      totalWeight += weights.chains[i];
    });

    return {
      price: weightedPrice / totalWeight,
      sources: [solanaData, ...otherChains],
      confidence: this.calculateAggregateConfidence(
        solanaData,
        otherChains
      ),
      timestamp: Date.now()
    };
  }
}

Error Handling & Resilience

Retry Logic Implementation

class ResilientShardsClient {
  private client: ShardsClient;
  private fallbackCache: Map<string, CachedPrice>;

  async getPriceWithFallback(
    symbol: string,
    maxRetries: number = 3
  ): Promise<PriceData> {
    let lastError: Error;

    for (let attempt = 0; attempt < maxRetries; attempt++) {
      try {
        const price = await this.client.oracle.getPrice(symbol);
        
        // Update cache on success
        this.fallbackCache.set(symbol, {
          data: price,
          timestamp: Date.now()
        });
        
        return price;
      } catch (error) {
        lastError = error;
        
        // Exponential backoff
        const delay = Math.min(1000 * Math.pow(2, attempt), 10000);
        await this.sleep(delay);
        
        // Check if we should use cache
        if (attempt === maxRetries - 1) {
          return this.getCachedPrice(symbol);
        }
      }
    }

    throw new Error(
      `Failed to get price after ${maxRetries} attempts: ${lastError}`
    );
  }

  private getCachedPrice(symbol: string): PriceData {
    const cached = this.fallbackCache.get(symbol);
    
    if (!cached) {
      throw new Error(`No cached price available for ${symbol}`);
    }

    const age = Date.now() - cached.timestamp;
    const maxAge = 60000; // 1 minute
    
    if (age > maxAge) {
      console.warn(`Using stale cached price for ${symbol} (${age}ms old)`);
    }

    return {
      ...cached.data,
      cached: true,
      cacheAge: age
    };
  }
}

Circuit Breaker Pattern

class CircuitBreaker {
  private failures: number = 0;
  private lastFailTime: number = 0;
  private state: 'CLOSED' | 'OPEN' | 'HALF_OPEN' = 'CLOSED';

  async execute<T>(
    operation: () => Promise<T>,
    fallback?: () => T
  ): Promise<T> {
    if (this.state === 'OPEN') {
      if (Date.now() - this.lastFailTime > 60000) {
        this.state = 'HALF_OPEN';
      } else if (fallback) {
        return fallback();
      } else {
        throw new Error('Circuit breaker is OPEN');
      }
    }

    try {
      const result = await operation();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      
      if (fallback) {
        return fallback();
      }
      throw error;
    }
  }

  private onSuccess() {
    this.failures = 0;
    this.state = 'CLOSED';
  }

  private onFailure() {
    this.failures++;
    this.lastFailTime = Date.now();
    
    if (this.failures >= 5) {
      this.state = 'OPEN';
      console.error('Circuit breaker opened due to repeated failures');
    }
  }
}

Performance Optimization

Caching Strategy

class OptimizedShardsClient {
  private cache: LRUCache<string, CacheEntry>;
  private batchQueue: Map<string, Promise<PriceData>>;

  constructor(client: ShardsClient, cacheSize: number = 1000) {
    this.cache = new LRUCache({ max: cacheSize });
    this.batchQueue = new Map();
  }

  async getPrice(
    symbol: string,
    maxAge: number = 1000
  ): Promise<PriceData> {
    // Check cache first
    const cached = this.cache.get(symbol);
    if (cached && Date.now() - cached.timestamp < maxAge) {
      return cached.data;
    }

    // Check if request is already in flight
    if (this.batchQueue.has(symbol)) {
      return this.batchQueue.get(symbol);
    }

    // Create new request
    const request = this.fetchPrice(symbol);
    this.batchQueue.set(symbol, request);

    try {
      const price = await request;
      this.cache.set(symbol, {
        data: price,
        timestamp: Date.now()
      });
      return price;
    } finally {
      this.batchQueue.delete(symbol);
    }
  }

  async getPricesBatch(symbols: string[]): Promise<Map<string, PriceData>> {
    const results = new Map();
    const toFetch = [];

    // Check cache for each symbol
    for (const symbol of symbols) {
      const cached = this.cache.get(symbol);
      if (cached && Date.now() - cached.timestamp < 1000) {
        results.set(symbol, cached.data);
      } else {
        toFetch.push(symbol);
      }
    }

    // Batch fetch missing prices
    if (toFetch.length > 0) {
      const prices = await this.client.oracle.getBatchPrices(toFetch);
      prices.forEach(price => {
        results.set(price.symbol, price);
        this.cache.set(price.symbol, {
          data: price,
          timestamp: Date.now()
        });
      });
    }

    return results;
  }
}

Testing & Development

Mock Client for Testing

import { MockShardsClient } from '@shards/sdk/testing';

describe('Trading Bot Integration', () => {
  let mockClient: MockShardsClient;
  let tradingBot: TradingBot;

  beforeEach(() => {
    mockClient = new MockShardsClient();
    tradingBot = new TradingBot(mockClient);
  });

  test('should execute trade on price threshold', async () => {
    // Set up mock price data
    mockClient.setPriceData('SOL', {
      price: 100,
      confidence: 0.1,
      timestamp: Date.now()
    });

    // Simulate price update
    mockClient.emitPriceUpdate('SOL', {
      price: 95,
      confidence: 0.1,
      timestamp: Date.now()
    });

    // Verify trade execution
    const trades = await tradingBot.getExecutedTrades();
    expect(trades).toHaveLength(1);
    expect(trades[0].action).toBe('BUY');
  });
});

Development Environment Setup

# Clone example repository
git clone https://github.com/shards-protocol/examples
cd examples

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Edit .env with your API key

# Run development server
npm run dev

# Run tests
npm test

# Build for production
npm run build

Best Practices

1. API Key Security

  • Never commit API keys to version control

  • Use environment variables for key storage

  • Implement key rotation policies

  • Use different keys for development/production

2. Rate Limit Management

  • Implement request queuing

  • Use caching to reduce API calls

  • Monitor rate limit headers

  • Implement exponential backoff

3. Data Validation

  • Always validate oracle data before use

  • Check confidence intervals

  • Verify data freshness

  • Implement sanity checks for price movements

4. Monitoring & Alerting

  • Log all oracle data requests

  • Monitor latency and error rates

  • Set up alerts for data anomalies

  • Track API usage and costs

Troubleshooting

Common Issues

  1. Connection Timeouts

    • Check network connectivity

    • Verify API endpoint status

    • Increase timeout settings

    • Implement retry logic

  2. Stale Data

    • Check oracle update frequencies

    • Verify timestamp freshness

    • Subscribe to real-time updates

    • Implement cache invalidation

  3. Rate Limiting

    • Monitor request patterns

    • Implement request batching

    • Use WebSocket for real-time data

    • Upgrade API tier if needed

Support & Resources

  • Documentation: https://docs.shards.io

  • API Status: https://status.shards.io

  • Discord Community: https://discord.gg/shards

  • GitHub Examples: https://github.com/shards-protocol/examples

  • Support Email: [email protected]

PreviousAPI Reference

Last updated