drep - a grep with runtime reloadable filters

栏目: IT技术 · 发布时间: 4年前

内容简介:Filter is either a regex or plain text match, provided via input file.

drep is dynamic regular expression print

drep is grep with dynamic reloadable filter expressions. This allows filtering stream of logs/lines, while changing filters on the fly.

Filter is either a regex or plain text match, provided via input file.

Here is an example usage:

tail -f /var/log/nginx/error.log | drep -f /etc/drep/filters

Demo

Filter file syntax

Each line of the filters file is an expression that starts with ~ , = , != , or !~ . The matches will be done in the order filters written in the file, and if a filter matches subsequent filters won't be executed.

  • Any line that starts with !~ implies does not match regex, e.g: !~"time": \d+.\d{0,2}
  • Any line that starts with ~ implies match regex, e.g: ~"time": \d+.\d{3,}
  • Any line that starts with != implies does not contain text, e.g: !=INFO
  • Any line that starts with = implies contain text, e.g: ="total-duration"

Everything else is ignored, as you can see from plain text. For regular expression documentation please refer to this document .

Why?

While grep --line-buffered can do something similar changing regex on the fly is not possible. Change filter regex on the fly is extremely useful in server/process environments where it's not possible to restart the process just to change the grep filter.

Building on unix philosophy drep does only one job well, given bunch of filter from an input file it can filter input lines to stdout.

Features

  • Lightweight on CPU, and memory (~3MB memory foot print, and 2 threads in total).
  • Watch and reload filters file.
  • No GC pauses and memory safe (Written in Rust).
  • Plain text & regex matching (with negation support).

Usage example

Given following simple fizzbuzz.py :

import time

i = 1
while True:
    fb = ""
    if i % 3 == 0:
        fb = "fizz"
    if i % 5 == 0:
        fb = "{}buzz".format(fb)

    if fb:
        print("{}. {}".format(i, fb), flush=True)

    i = i + 1
    time.sleep(0.1)

We can launch and pipe it's output python fizzbuzz.py | drep -f filters . Now if the contents of filters are:

~\sfizz\n

drep will only emit logs with fizz. e.g.

642. fizz
648. fizz
651. fizz
654. fizz
...

While keeping the process running without exiting you can just modify filters to:

~\sbuzz\n

This will change the drep output on the fly to only emit buzz:

805. buzz
815. buzz
820. buzz
...

Building

Just clone the repo and run cargo build --release .


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

C++编程思想

C++编程思想

埃克尔(美) / 刘宗田/等 / 机械工业出版社 / 2000-01 / 39.00

一起来看看 《C++编程思想》 这本书的介绍吧!

在线进制转换器
在线进制转换器

各进制数互转换器

图片转BASE64编码
图片转BASE64编码

在线图片转Base64编码工具