timberio/vector



Slack Chat  •  
Forums  •  
Mailing List


Vector

Vector is ahigh-performanceobservability data router. It makes
collecting,transforming, andsendinglogs, metrics,
and events easy. It decouples data collection & routing from your services, giving
you control and data ownership, amongmany other benefits.

Built inRust, Vector places high-value onperformance,correctness, and
operator friendliness. It compiles to a single static binary and is designed
to bedeployedacross your entire infrastructure, serving both as a
light-weightagentand a highly efficientservice, making the process of getting
data from A to B simple and unified.

Documentation

About

Setup

Usage

Resources

Features

  • Fast– Built inRust, Vector isfast and memory efficient. No runtime. No garbage collector.
  • Correct– Obsessed withgetting the details right.
  • Vendor Neutral– Does not favor a specific storage. Fair, open, with the user’s best interest in mind.
  • Agent Or Service– One simple tool to get data from A to B. Deploys as anagentorservice.
  • Logs, Metrics, or Events– Logs, metrics, and events. Collect, unify, and ship all observability data.
  • Clear Guarantees– Aguarantee support matrixhelps you understand your tradeoffs.
  • Easy To Deploy– Cross-compiles to a single static binary with no runtime.
  • Hot ReloadReload configuration on the fly, without skipping a beat.

Performance

TestVectorFilebeatFluentBitFluentDLogstashSplunkUFSplunkHF
TCP to Blackhole86mib/sn/a64.4mib/s27.7mib/s40.6mib/sn/an/a
File to TCP76.7mib/s7.8mib/s35mib/s26.1mib/s3.1mib/s40.1mib/s39mib/s
Regex Parsing13.2mib/sn/a20.5mib/s2.6mib/s4.6mib/sn/a7.8mib/s
TCP to HTTP26.7mib/sn/a19.6mib/s2.7mib/sn/an/a
TCP to TCP69.9mib/s5mib/s67.1mib/s3.9mib/s10mib/s70.4mib/s7.6mib/s

To learn more about our performance tests, please see theVector test harness.

Correctness

To learn more about our performance tests, please see theVector test harness.

Installation

Run the following in your terminal, then follow the on-screen instructions.

curl https://sh.vector.dev -sSf|sh

Or viewplatform specific installation instructions.

Sources

NameDescription
fileIngests data through one or more local files and outputs log events.
statsdIngests data through the StatsD UDP protocol and outputs log events.
stdinIngests data through standard input (STDIN) and outputs log events.
syslogIngests data through the Syslog 5424 protocol and outputs log events.
tcpIngests data through the TCP protocol and outputs log events.
vectorIngests data through another upstream Vector instance and outputs log events.

+ request a new source

Transforms

NameDescription
add_fieldsAccepts log events and allows you to add one or more fields.
field_filterAccepts log and metric events and allows you to filter events by a field’s value.
grok_parserAccepts log events and allows you to parse a field value with Grok.
json_parserAccepts log events and allows you to parse a field value as JSON.
luaAccepts log events and allows you to transform events with a full embedded Lua engine.
regex_parserAccepts log events and allows you to parse a field’s value with a Regular Expression.
remove_fieldsAccepts log and metric events and allows you to remove one or more event fields.
samplerAccepts log events and allows you to sample events with a configurable rate.
tokenizerAccepts log events and allows you to tokenize a field’s value by splitting on white space, ignoring special wrapping characters, and zipping the tokens into ordered field names.

+ request a new transform

Sinks

NameDescription
aws_cloudwatch_logsBatches log events to AWS CloudWatch Logs via thePutLogEventsAPI endpoint.
aws_kinesis_streamsBatches log events to AWS Kinesis Data Stream via thePutRecordsAPI endpoint.
aws_s3Batches log events to AWS S3 via thePutObjectAPI endpoint.
blackholeStreams log and metric events to a blackhole that simply discards data, designed for testing and benchmarking purposes.
consoleStreams log and metric events to the console,STDOUTorSTDERR.
elasticsearchBatches log events to Elasticsearch via the_bulkAPI endpoint.
httpBatches log events to a generic HTTP endpoint.
kafkaStreams log events to Apache Kafka.
splunk_hecBatches log events to a Splunk HTTP Event Collector.
tcpStreams log events to a TCP connection.
vectorStreams log events to another downstream Vector instance.

+ request a new sink

License

Copyright 2019, Vector Authors. All rights reserved.

Licensed under the Apache License, Version 2.0 (the “License”); you may not
use these files except in compliance with the License. You may obtain a copy
of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations under
the License.


Developed with❤️byTimber.io

Read More

LEAVE A REPLY

Please enter your comment!
Please enter your name here