日志集中收集工具Syslog-ng+rsyslog+logrotate-白红宇

日志集中收集工具Syslog-ng+rsyslog+logrotate

阅读量：2338 次

发布时间：2019-05-10

本文共 7366 字，大约阅读时间需要 24 分钟。

syslog-ng安装

syslog-ng server IP ：10.125.192.10 Centos6.8

编辑yum源

vim /etc/yum.repo/syslog-ng.repo

[copr:copr.fedorainfracloud.org:czanik:syslog-ng37epel6]name=Copr repo for syslog-ng37epel6 owned by czanikbaseurl=https://copr-be.cloud.fedoraproject.org/results/czanik/syslog-ng37epel6/epel-6-$basearch/type=rpm-mdskip_if_unavailable=Truegpgcheck=1gpgkey=https://copr-be.cloud.fedoraproject.org/results/czanik/syslog-ng37epel6/pubkey.gpgrepo_gpgcheck=0enabled=1enabled_metadata=1

yum -y install syslog-ng

编辑syslog-ng配置文件

options {

flush_lines (0);

time_reopen (10);

log_fifo_size (1000);

chain_hostnames (off);

use_dns (no);

use_fqdn (no);

create_dirs (yes); #当指定的目标目录不存在时，是否创建该目录

keep_hostname (yes);

};

source s_network {

syslog(transport(udp) port(6666)); #定义监听UDP6666端口来作为日志来源

};

destination d_local {

file("/var/log/syslog-ng/secure_${FULLHOST_FROM}");#定义接收到日志写入位置，${FULLHOST_FROM}定义了以日志发送端HOST作为日志文件的一部分，以区分不同的主机来源

};

log { source(s_network); destination(d_local); }; #定义消息链，把日志来源和目的地关联

启动并检查

/etc/init.d/syslog-ng start

配置客户端 rsyslog

vim /etc/rsyslog.conf

authpriv.* @10.125.192.10:6666

/etc/init.d/rsyslog restart

配置完成后再客户端服务器上的/var/log/secure日志会实时传送到远程目标服务器上

在syslog-ng上查看

rsyslog简单总结

/etc/rsyslog.conf

1	# rsyslog v5 configuration file     2	# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html     3	# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html     4	#### MODULES ####  加载模块     5	$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)     6	$ModLoad imklog  模块名 # provides kernel logging support (previously done by rklogd)     7	#$ModLoad immark  # provides --MARK-- message capability     8	# Provides UDP syslog reception 允许514端口接收使用UDP协议转发过来的日志     9	#$ModLoad imudp    10	#$UDPServerRun 514    11	# Provides TCP syslog reception 允许514端口接收使用TCP协议转发过来的日志    12	#$ModLoad imtcp    13	#$InputTCPServerRun 514    14	#### GLOBAL DIRECTIVES ####  定义日志格式默认模板    15	# Use default timestamp format    16	$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat    17	# File syncing capability is disabled by default. This feature is usually not required,    18	# not useful and an extreme performance hit    19	#$ActionFileEnableSync on    20	# Include all config files in /etc/rsyslog.d/    21	$IncludeConfig /etc/rsyslog.d/*.conf    22	#### RULES ####    23	# Log all kernel messages to the console.    24	# Logging much else clutters up the screen.    25	#kern.*                                                 /dev/console #关于内核的所有级别日志都放到/dev/console(控制台)    26	# Log anything (except mail) of level info or higher.    27	# Don't log private authentication messages!    28	*.info;mail.none;authpriv.none;cron.none                /var/log/messages #记录所有日志类型的info级别以及大于info级别的信息到/var/log/messages，但是mail邮件信息，authpriv验证方面的信息和cron时间任务相关的信息除外    29	# The authpriv file has restricted access.    30	authpriv.*                                              /var/log/secure #authpriv验证相关的所有信息存放在/var/log/secure    31	authpriv.*                                              @10.125.192.10:6666    32	# Log all the mail messages in one place.    33	mail.*                                                  -/var/log/maillog #邮件的所有信息存放在/var/log/maillog; 这里有一个-符号, 表示是使用异步的方式记录, 因为日志一般会比较大    34	# Log cron stuff    35	cron.*                                                  /var/log/cron #计划任务有关的信息存放在/var/log/cron    36	# Everybody gets emergency messages    37	*.emerg                                                 * #记录所有的大于等于emerg级别信息, 方式发送给每个登录到系统的人    38	# Save news errors of level crit and higher in a special file.    39	uucp,news.crit                                          /var/log/spooler #记录uucp,news日志设备上大于等于crit级别的信息记录到/var/log/spooler    40	# Save boot messages also to boot.log    41	local7.*                                                /var/log/boot.log #将local7的所有级别的信息记录到/var/log/boot.log文件中，local0 到local7这8个是用户自定义使用的,这里的local7记录的是系统启动相关的信息    42	# ### begin forwarding rule ###    43	# The statement between the begin ... end define a SINGLE forwarding    44	# rule. They belong together, do NOT split them. If you create multiple    45	# forwarding rules, duplicate the whole block!    46	# Remote Logging (we use TCP for reliable delivery)    47	#    48	# An on-disk queue is created for this action. If the remote host is    49	# down, messages are spooled to disk and sent when it is up again.    50	#$WorkDirectory /var/lib/rsyslog # where to place spool files    51	#$ActionQueueFileName fwdRule1 # unique name prefix for spool files    52	#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)    53	#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown    54	#$ActionQueueType LinkedList   # run asynchronously    55	#$ActionResumeRetryCount -1    # infinite retries if host is down    56	# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional    57	#*.* @@remote-host:514  #所有设备所有级别的信息发送到指定主机:指定端口 @@标识通过TCP发送，@标识通过UDP发送    58	# ### end of the forwarding rule ###    59	#authpriv.* @10.125.192.10i:6666 #authpriv设备上所有级别的信息通过UDP发送到10.125.192.10i:6666

记录日志格式：

日志设备.（连接符号）日志级别日志处理方式

（多日志设备用,分割如uucp,news）

日志设备(可以理解为日志类型):

auth –pam产生的日志

authpriv –ssh,ftp等登录信息的验证信息

cron –计划任务相关

kern –内核

lpr –打印

mail –邮件

mark(syslog) –rsyslog服务内部的信息,时间标识

news –新闻组

user –用户程序产生的相关信息

uucp –unix to unix copy, unix主机之间相关的通讯

local 1~7 –自定义的日志设备

日志级别:

debug –有调式信息的，日志信息最多

info –一般信息的日志，最常用

notice –最具有重要性的普通条件的信息

warning –警告级别

err –错误级别，阻止某个功能或者模块不能正常工作的信息

crit –严重级别，阻止整个系统或者整个软件不能正常工作的信息

alert –需要立刻修改的信息

emerg –内核崩溃等严重信息

none –什么都不记录

从上到下，级别从低到高，记录的信息越来越少详细的可以查看手册: man 3 syslog

连接符号

.xxx: 表示大于等于xxx级别的信息

.=xxx：表示等于xxx级别的信息

.!xxx：表示在xxx之外的等级的信息

logrotate

logrotate是个十分有用的工具，它可以自动对日志进行截断（或轮循）、压缩以及删除旧的日志文件

vim /etc/logrotate.conf

@test ~]# cat logrotate.txt -n     1	# see "man logrotate" for details     2	# rotate log files weekly     3	weekly  #日志按周轮训     4	# keep 4 weeks worth of backlogs     5	rotate 4 #保留4份归档     6	# create new (empty) log files after rotating old ones     7	create     8	# use date as a suffix of the rotated file     9	dateext #使用日期格式作为轮转之后后缀    10	# uncomment this if you want your log files compressed    11	#compress #启用压缩    12	# RPM packages drop log rotation information into this directory    13	include /etc/logrotate.d    14	# no packages own wtmp and btmp -- we'll rotate them here    15	/var/log/wtmp {    16	    monthly    17	    create 0664 root utmp    18		minsize 1M    19	    rotate 1    20	}    21	/var/log/btmp {    22	    missingok    23	    monthly    24	    create 0600 root utmp    25	    rotate 1    26	}    27	# system-specific logs may be also be configured here.

logrotate的配置文件是/etc/logrotate.conf，通常不需要对它进行修改。日志文件的轮循设置在独立的配置文件中，放在/etc/logrotate.d/目录下

vim /etc/logrotate.d/log-file /var/log/log-file {    monthly    rotate 5    compress    delaycompress    missingok    notifempty    create 644 root root    postrotate        /usr/bin/killall -HUP rsyslogd    endscript}

参数说明