Elasticsearch

  • Edit /opt/elasticsearch/bin/service/elasticsearch.conf

Memory tuning:

  • On m1.large servers please set the “set.default.ES_HEAP_SIZE” to 4096
  • Change memory MAX and MIN memory of elasticsearch do the following
  • Change set.default.ES_HEAP_SIZE value to your needs
set.default.ES_HEAP_SIZE=4096
  •  Tune Elasticsearch logs
wrapper.logfile.maxsize=10m
wrapper.logfile.maxfiles=10
  • Edit /elasticsearch/elasticsearch-0.20.5/config/logging.yml
  • Replace the “file” section with
file:
type: rollingFile
file: ${path.logs}/${cluster.name}.log
maxFileSize: 100MB
maxBackupIndex: 10
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
  •  Restart elasticsearch

Graylog

  • Edit the configuration via /etc/graylog2.conf
  • Enable tcp listener
syslog_enable_tcp = true
  •  Update the “processor_wait_strategy”
processor_wait_strategy = blocking
  •  Disk Usage, update the following parameters to limit the disk usage to about 30-40 GB
elasticsearch_max_docs_per_index = 2000000
elasticsearch_max_number_of_indices = 10
elasticsearch_shards = 1
  •  Restart graylog-server (make sure you stopped it via PS, then start again)

Streams

Avoid wild card regex filters – it kills the Graylog server performance

Lahav Savir

Founder and CTO, Cloud Platforms

Read more posts by Lahav Savir