Replace sysklogd with rsyslog on CentOS and Forward Logs to Splunk

Older CentOS servers use sysklogd, which can’t forward logs over TCP with a disk queue. Here’s how to swap it for rsyslog and send everything to Splunk.

1. Swap sysklogd for rsyslog

Use yum shell to remove and install in one transaction:

yum shell
remove sysklogd
install rsyslog
run
exit

2. Add the Splunk Forwarding Rule

Edit /etc/rsyslog.conf and paste in this block, replacing your-splunk-server with your Splunk host:

# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* @@your-splunk-server:514
# ### end of the forwarding rule ###

3. Enable and Restart rsyslog

chkconfig rsyslog on
service rsyslog restart

See also forwarding CentOS syslog to Splunk with rsyslog.