内容简介:osquery初识主要是借由如果使用启动方式就是
osquery初识主要是借由 osqueryi
的方式对 osquery
进行了一个基本的介绍。可以看到 osqueryi
是一个交互式的 shell
,我们可以很方便使用它进行测试,但是如果我们要将 osquery
投入实际使用,明显是 osqueryd
更加合适。本篇文章将详细地介绍 osqueryd
的使用。
osqueryd配置
如果使用 osqueryi
,我们可以通过 osqueryi -audit_allow_config=true --audit_allow_sockets=true --audit_persist=true
这样的方式传入设置。如果是 osqueryd
呢?其实我们安装好 osquery
之后,会以 service
的方式存在于系统中,同时可以利用 systemctl
的方式进行控制,其文件位于 /usr/lib/systemd/system/osqueryd.service
。
[Unit] Description=The osquery Daemon After=network.service syslog.service [Service] TimeoutStartSec=0 EnvironmentFile=/etc/sysconfig/osqueryd ExecStartPre=/bin/sh -c "if [ ! -f $FLAG_FILE ]; then touch $FLAG_FILE; fi" ExecStartPre=/bin/sh -c "if [ -f $LOCAL_PIDFILE ]; then mv $LOCAL_PIDFILE $PIDFILE; fi" ExecStart=/usr/bin/osqueryd \ --flagfile $FLAG_FILE \ --config_path $CONFIG_FILE Restart=on-failure KillMode=process KillSignal=SIGTERM [Install] WantedBy=multi-user.target
启动方式就是 ExecStart=/usr/bin/osqueryd --flagfile $FLAG_FILE --config_path $CONFIG_FILE
,通过 --flagfile
和 --config_path
的方式指定配置文件的路径。 $FLAG_FILE
和 $CONFIG_FILE
是在 /etc/sysconfig/osqueryd
中定义。
FLAG_FILE="/etc/osquery/osquery.flags" CONFIG_FILE="/etc/osquery/osquery.conf" LOCAL_PIDFILE="/var/osquery/osqueryd.pidfile" PIDFILE="/var/run/osqueryd.pidfile"
默认的配置文件就是位于 /etc/osquery/osquery.flags
和 /etc/osquery/osquery.conf
。当启动 osqueryd
时,如果不存在 osquery.flags
和 osquery.conf
会创建两个空文件,否则直接读取此文件的内容。其实 osquery.conf
可以认为是 osquery.flags
的超集,因为 osquery.flags
仅仅只是设置一些配置,而这些配置也同样可以在 osquery.conf
中实现,同时在 osquery.conf
中还可以配置 osqueryd
需要执行的SQL。所以接下来本文将仅仅只介绍 osquery.conf
的使用。
osquery.conf
osquery
本身提供了一个 osquery.conf
的例子,其写法是一个 JSON
格式的文件,在这里我们将其简化一下。
{ // Configure the daemon below: "options": { // Select the osquery config plugin. "config_plugin": "filesystem", // Select the osquery logging plugin. "logger_plugin": "filesystem", // The log directory stores info, warning, and errors. // If the daemon uses the 'filesystem' logging retriever then the log_dir // will also contain the query results. //"logger_path": "/var/log/osquery", // Set 'disable_logging' to true to prevent writing any info, warning, error // logs. If a logging plugin is selected it will still write query results. //"disable_logging": "false", // Splay the scheduled interval for queries. // This is very helpful to prevent system performance impact when scheduling // large numbers of queries that run a smaller or similar intervals. //"schedule_splay_percent": "10", // A filesystem path for disk-based backing storage used for events and // query results differentials. See also 'use_in_memory_database'. //"database_path": "/var/osquery/osquery.db", // Comma-delimited list of table names to be disabled. // This allows osquery to be launched without certain tables. //"disable_tables": "foo_bar,time", "utc": "true" }, // Define a schedule of queries: "schedule": { // This is a simple example query that outputs basic system information. "system_info": { // The exact query to run. "query": "SELECT hostname, cpu_brand, physical_memory FROM system_info;", // The interval in seconds to run this query, not an exact interval. "interval": 3600 } }, // Decorators are normal queries that append data to every query. "decorators": { "load": [ "SELECT uuid AS host_uuid FROM system_info;", "SELECT user AS username FROM logged_in_users ORDER BY time DESC LIMIT 1;" ] }, "packs": { // "osquery-monitoring": "/usr/share/osquery/packs/osquery-monitoring.conf", .... }, }
osquery.conf
文件大致可以分为4个部分。
-
options
,配置选项, Command Line Flags 基本上对所有的配置选项都进行了说明。其实osquery.flags
所配置也是这个部分。这也是之前说的 osquery.conf可以认为是osquery.flags的超集 的原因; -
schedule
,配置 SQL 语句。因为osqueryd
是以daemon
的方式运行,所以需要通过在schedule
中定义SQL语句使其定期执行返回结果; -
decorators
,中文意思是“装饰”。在decorators
中也是定义了一系列的SQL语句,执行得到的结果会附加在是在执行schedule
中的结果的后面;所以我们看到在decorators
我们取的是uuid
和登录的username
; -
packs
,就是一系列SQL语句的合集;
配置说明
上一节中对 osquery.conf
中的配置进了一个简单的说明,在本节中将详细说明。
options
options
就是配置。 Command Line Flags
基本上对所有的配置选项都进行了说明。我们可以进行多种配置,有兴趣的可以自行研究。本节仅仅说明几个常用的配置;
-
config_plugin
,配置选项是filesystem
。如果是通过osquery.conf
管理osquery
就是采用filesystem
,还有一种选项是tls
(这一种主要是通过API的方式来配置osquery
)。 -
logger_plugin
,配置选项是filesystem
,这也是osquery
的默认值。根据 Logger plugins ,还可以配置tls
,syslog (for POSIX
,windows_event_log (for Windows)
,kinesis
,firehose
,kafka_producer
。 -
database_path
,默认值是/var/osquery/osquery.db
。因为osquery
内部会使用到数据,所以配置此目录是osquery
的数据库文件位置。 -
disable_logging
,是配置设置osquery
的结果是否需要保存到本地,这个配置其实和logger_plugin:filesystem
有点重复。 -
hostIdentifier
,相当于表示每个主机的标识,比如可以采用hostname
作为标识。
schedule
schedule
是 osqeuryd
用于写SQL语句的标签。其中的一个示例如下所示:
"system_info": { // The exact query to run. "query": "SELECT hostname, cpu_brand, physical_memory FROM system_info;", // The interval in seconds to run this query, not an exact interval. "interval": 3600 }
其中 system_info
是定义的一个SQL任务的名字,也是一个JSON格式。在其中可以进行多项设置,包括:
-
query
,定义需要执行的SQL语句; -
interval
,定时执行的时间,示例中是3600
,表示每隔3600秒执行一次; -
snapshot
,可选选项,可以配置为snapshot:true
。osquery
默认执行的是增量模式,使用了snapshot
则是快照模式。比如执行select * from processes;
,osqeury
每次产生的结果是相比上一次变化的结果;如果采用的是snapshot
,则会显示所有的进程的,不会与之前的结果进行对比; -
removed
,可选选项,默认值是true
,用来设置是否记录action
为remove
的日志。
当然还有一些其他的不常用选项,如 platform
、 version
、 shard
、 description
等等。
更多关于 schedule
的介绍可以参考 schedule
decorators
正如其注释 Decorators are normal queries that append data to every query
所说, Decorators
会把他的执行结果添加到 schedule
中的 sql
语句执行结果中。所以根据其作用 Decorators
也不是必须存在的。。在本例中 Decorators
存在两条记录:
SELECT uuid AS host_uuid FROM system_info; SELECT user AS username FROM logged_in_users ORDER BY time DESC LIMIT 1;
-
SELECT uuid AS host_uuid FROM system_info
,从system_info
获取uuid
作为标识符1; -
SELECT user AS username FROM logged_in_users ORDER BY time DESC LIMIT 1;
,从logged_in_users
选择user
(其实查询的是用户名)的第一项作为标识符2;
当然可以在 Decorators
写多条语句作为标识符,但是感觉没有必要;
packs
packs
就是打包的 SQL
语句的合集,本示例中使用的 /usr/share/osquery/packs/osquery-monitoring.conf
,这是官方提供的一个监控系统信息的SQL语句的集合;
{ "queries": { "schedule": { "query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory, last_executed from osquery_schedule;", "interval": 7200, "removed": false, "blacklist": false, "version": "1.6.0", "description": "Report performance for every query within packs and the general schedule." }, "events": { "query": "select name, publisher, type, subscriptions, events, active from osquery_events;", "interval": 86400, "removed": false, "blacklist": false, "version": "1.5.3", "description": "Report event publisher health and track event counters." }, "osquery_info": { "query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;", "interval": 600, "removed": false, "blacklist": false, "version": "1.2.2", "description": "A heartbeat counter that reports general performance (CPU, memory) and version." } } }
packs
中的配置和 schedule
的配置方法并没有什么区别。我们在 packs
中查询到的信息包括:
-
从
osquery_schedule
拿到osqueryd
设置的schedule
的配置信息; -
从
osquery_events
中拿到osqueryd
所支持的所有的event
; -
从
processes
和osquery_info
中拿到进程相关的信息;
使用 packs
的好处是可以将一系列相同功能的SQL语句放置在同一个文件中;
运行osqueryd
当以上配置完毕之后,我们就可以通过 sudo osqueryd
的方式启动;如果我们设置 logger_plugin:filesystem
,那么日志就会落在本地 /var/log/osquery
下。此目录下包含了多个文件,每个文件分别记录不同的信息。
-
osqueryd.results.log
,osqueryd
的增量日志的信息都会写入到此文件中;保存结果的形式是JSON
形式。示例如下:{"name":"auditd_process_info","hostIdentifier":"localhost.localdomain","calendarTime":"Wed Oct 24 13:07:12 2018 UTC","unixTime":1540386432,"epoch":0,"counter":0,"decorations":{"host_uuid":"99264D56-9A4E-E593-0B4E-872FBF3CD064","username":"username"},"columns":{"atime":"1540380461","auid":"4294967295","btime":"0","cmdline":"awk { sum += $1 }; END { print 0+sum }","ctime":"1538239175","cwd":"\"/\"","egid":"0","euid":"0","gid":"0","mode":"0100755","mtime":"1498686768","owner_gid":"0","owner_uid":"0","parent":"4086","path":"/usr/bin/gawk","pid":"4090","time":"1540386418","uid":"0","uptime":"1630"},"action":"added"} {"name":"auditd_process_info","hostIdentifier":"localhost.localdomain","calendarTime":"Wed Oct 24 13:07:12 2018 UTC","unixTime":1540386432,"epoch":0,"counter":0,"decorations":{"host_uuid":"99264D56-9A4E-E593-0B4E-872FBF3CD064","username":"username"},"columns":{"atime":"1540380461","auid":"4294967295","btime":"0","cmdline":"sleep 60","ctime":"1538240835","cwd":"\"/\"","egid":"0","euid":"0","gid":"0","mode":"0100755","mtime":"1523421302","owner_gid":"0","owner_uid":"0","parent":"741","path":"/usr/bin/sleep","pid":"4091","time":"1540386418","uid":"0","uptime":"1630"},"action":"added"}
其中的
added
表示的就是相当于上一次增加的进程信息;每一次执行的结果都是一条JSON记录; -
squeryd.snapshots.log
,记录的是osqueryd
中使用snapshot:true
标记的SQL语句执行结果;{"snapshot":[{"header":"Defaults","rule_details":"!visiblepw"},{"header":"Defaults","rule_details":"always_set_home"},{"header":"Defaults","rule_details":"match_group_by_gid"},{"header":"Defaults","rule_details":"env_reset"},{"header":"Defaults","rule_details":"env_keep = \"COLORS DISPLAY HOSTNAME HISTSIZE KDEDIR LS_COLORS\""},{"header":"Defaults","rule_details":"env_keep += \"MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE\""},{"header":"Defaults","rule_details":"env_keep += \"LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES\""},{"header":"Defaults","rule_details":"env_keep += \"LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE\""},{"header":"Defaults","rule_details":"env_keep += \"LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY\""},{"header":"Defaults","rule_details":"secure_path = /sbin:/bin:/usr/sbin:/usr/bin"},{"header":"root","rule_details":"ALL=(ALL) ALL"},{"header":"%wheel","rule_details":"ALL=(ALL) ALL"}],"action":"snapshot","name":"sudoers","hostIdentifier":"localhost.localdomain","calendarTime":"Tue Oct 9 11:54:00 2018 UTC","unixTime":1539086040,"epoch":0,"counter":0,"decorations":{"host_uuid":"99264D56-9A4E-E593-0B4E-872FBF3CD064","username":"username"}} {"snapshot":[{"header":"Defaults","rule_details":"!visiblepw"},{"header":"Defaults","rule_details":"always_set_home"},{"header":"Defaults","rule_details":"match_group_by_gid"},{"header":"Defaults","rule_details":"env_reset"},{"header":"Defaults","rule_details":"env_keep = \"COLORS DISPLAY HOSTNAME HISTSIZE KDEDIR LS_COLORS\""},{"header":"Defaults","rule_details":"env_keep += \"MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE\""},{"header":"Defaults","rule_details":"env_keep += \"LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES\""},{"header":"Defaults","rule_details":"env_keep += \"LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE\""},{"header":"Defaults","rule_details":"env_keep += \"LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY\""},{"header":"Defaults","rule_details":"secure_path = /sbin:/bin:/usr/sbin:/usr/bin"},{"header":"root","rule_details":"ALL=(ALL) ALL"},{"header":"%wheel","rule_details":"ALL=(ALL) ALL"}],"action":"snapshot","name":"sudoers","hostIdentifier":"localhost.localdomain","calendarTime":"Tue Oct 9 11:54:30 2018 UTC","unixTime":1539086070,"epoch":0,"counter":0,"decorations":{"host_uuid":"99264D56-9A4E-E593-0B4E-872FBF3CD064","username":"username"}}
由于
snapshot
是快照模式,所以即使两次结果相同也会全部显示出来; -
osqueryd.INFO
,记录osqueryd
中正在运行的情况。示例如下:Log file created at: 2018/11/22 17:06:06 Running on machine: osquery.origin Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg I1122 17:06:06.729902 22686 events.cpp:862] Event publisher not enabled: auditeventpublisher: Publisher disabled via configuration I1122 17:06:06.730651 22686 events.cpp:862] Event publisher not enabled: syslog: Publisher disabled via configuration
-
osqueryd.WARNING
,记录osquery
的警告。示例如下:Log file created at: 2018/10/09 19:53:45 Running on machine: localhost.localdomain Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg E1009 19:53:45.471046 104258 events.cpp:987] Requested unknown/failed event publisher: auditeventpublisher E1009 19:53:45.471606 104259 events.cpp:987] Requested unknown/failed event publisher: inotify E1009 19:53:45.471634 104260 events.cpp:987] Requested unknown/failed event publisher: syslog E1009 19:53:45.471658 104261 events.cpp:987] Requested unknown/failed event publisher: udev
-
osqueryd.ERROR
,记录的是osquery
的错误信息。示例如下:Log file created at: 2018/10/09 19:53:45 Running on machine: localhost.localdomain Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg E1009 19:53:45.471046 104258 events.cpp:987] Requested unknown/failed event publisher: auditeventpublisher E1009 19:53:45.471606 104259 events.cpp:987] Requested unknown/failed event publisher: inotify E1009 19:53:45.471634 104260 events.cpp:987] Requested unknown/failed event publisher: syslog E1009 19:53:45.471658 104261 events.cpp:987] Requested unknown/failed event publisher: udev
在本例中错误信息和警告信息完全相同。在实际情况下,可能很多时候均不相同;
总结
本文主要是对 osqueryd
的常用配置进行了简要的说法。通过本文能够快速地利用上手 osquery
,由于篇幅的原因,有关osquery的很多东西没有介绍或者说明得很详细。官方的文档[]对osqueryd的配置已经说明得很是详尽了,如果对本文有任何的不解,可以去查阅相关的文档,也欢迎大家就相关问题与我讨论。
以上
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网
猜你喜欢:- WGCLOUD 监控系统更新,集成 ES 在线监控工具
- WGCLOUD 监控系统更新,进程监控模块 bug 修复
- 分布式监控系统 WGCLOUD,新增 docker 状态监控
- 分布式监控系统 WGCLOUD,支持进程流量指标监控
- 安全监控 划重点!机房中最重要的监控系统你了解吗?
- xrkmonitor 字符云监控系统 v2.2 发布,新增 Linux 文件目录监控插件
本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。