Performance Tuning

Topics

Performance Optimization Framework

ELS Data Node Performance Tuning:

Index Performance Optimization

  • Shard Sizing: Target 20-50GB per shard per GB in the heap for optimal performance

  • Replica Strategy: Configure replicas based on availability requirements

  • Refresh Intervals: Adjust refresh rates based on real-time requirements

  • Merge Policies: Optimize for write-heavy vs. read-heavy workloads

ELS Network Node Scaling:

# High-performance pipeline configuration
pipeline.workers: 8                    # Match CPU cores
pipeline.batch.size: 1000             # Optimize for throughput
pipeline.batch.delay: 50              # Balance latency vs. throughput
queue.type: persisted                 # Enable persistent queues
queue.max_bytes: 2gb                  # Size based on available storage

Capacity Planning Guidelines

Data Volume Planning:

Daily Data Volume

Recommended Architecture

ELS Data Nodes

Storage Requirements

< 10 GB/day

Single node deployment

1 node

1 TB total storage

10-100 GB/day

Small cluster

3 nodes

5 TB per node

100-500 GB/day

Medium cluster

5 nodes

10 TB per node

500GB-2TB/day

Large cluster

8+ nodes

20 TB per node

> 2TB/day

Enterprise cluster

15+ nodes

Custom sizing