Merge branch 'master' into newdirectives

1 files changed, 94 insertions, 39 deletions

diff --git a/docs/system/flow-accounting.rst b/docs/system/flow-accounting.rst
index 4f566490..df58e1f3 100644
--- a/docs/system/flow-accounting.rst
+++ b/docs/system/flow-accounting.rst
@@ -4,6 +4,20 @@
 Flow Accounting
 ###############
 
+VyOS supports flow-accounting for both IPv4 and IPv6 traffic. The system acts
+as a flow exporter, and you are free to use it with any compatible collector.
+
+Flows can be exported via two different protocols: NetFlow (versions 5, 9 and
+10/IPFIX) and sFlow. Additionally, you may save flows to an in-memory table
+internally in a router.
+
+.. warning:: You need to disable the in-memory table in production environments!
+   Using :abbr:`IMT (In-Memory Table)` may lead to heavy CPU overloading and
+   unstable flow-accounting behavior.
+
+
+NetFlow / IPFIX
+===============
 NetFlow is a feature that was introduced on Cisco routers around 1996 that
 provides the ability to collect IP network traffic as it enters or exits an
 interface. By analyzing the data provided by NetFlow, a network administrator
@@ -18,8 +32,8 @@ NetFlow) consists of three main components:
 * **application**: analyzes received flow data in the context of intrusion
   detection or traffic profiling, for example
 
-For connectionless protocols as like ICMP and UDP, a flow is considered complete
-once no more packets for this flow appear after configurable timeout.
+For connectionless protocols as like ICMP and UDP, a flow is considered
+complete once no more packets for this flow appear after configurable timeout.
 
 NetFlow is usually enabled on a per-interface basis to limit load on the router
 components involved in NetFlow, or to limit the amount of NetFlow records
@@ -31,7 +45,7 @@ Configururation
 In order for flow accounting information to be collected and displayed for an
 interface, the interface must be configured for flow accounting.
 
-.. cfgcmd:: set system flow-accounting interface '<interface>'
+.. cfgcmd:: set system flow-accounting interface <interface>
 
    Configure and enable collection of flow information for the interface
    identified by `<interface>`.
@@ -39,15 +53,41 @@ interface, the interface must be configured for flow accounting.
    You can configure multiple interfaces which whould participate in flow
    accounting.
 
+.. note:: Will be recorded only packets/flows on **incoming** direction in
+   configured interfaces.
+
+
+By default, recorded flows will be saved internally and can be listed with the
+CLI command. You may disable using the local in-memory table with the command:
+
+.. cfgcmd:: set system flow-accounting disable-imt
+
+Internally, in flow-accounting processes exist a buffer for data exchanging
+between core process and plugins (each export target is a separated plugin). If
+you have high traffic levels or noted some problems with missed records or
+stopping exporting, you may try to increase a default buffer size (10 MiB) with
+the next command:
+
+.. cfgcmd:: set system flow-accounting buffer-size <buffer size>
+
+In case, if you need to catch some logs from flow-accounting daemon, you may
+configure logging facility:
+
+.. cfgcmd:: set system flow-accounting syslog-facility <facility>
+
+
 Flow Export
 -----------
 
 In addition to displaying flow accounting information locally, one can also
 exported them to a collection server.
 
-.. cfgcmd:: set system flow-accounting netflow version '<version>'
+NetFlow
+^^^^^^^
+
+.. cfgcmd:: set system flow-accounting netflow version <version>
 
-   There are multiple versions available for the NetFlo data. The `<version>`
+   There are multiple versions available for the NetFlow data. The `<version>`
    used in the exported flow data can be configured here. The following
    versions are supported:
 
@@ -55,20 +95,20 @@ exported them to a collection server.
    * **9** - NetFlow version 9 (default)
    * **10** - :abbr:`IPFIX (IP Flow Information Export)` as per :rfc:`3917`
 
-.. cfgcmd:: set system flow-accounting netflow server '<address>'
+.. cfgcmd:: set system flow-accounting netflow server <address>
 
    Configure address of NetFlow collector. NetFlow server at `<address>` can
    be both listening on an IPv4 or IPv6 address.
 
-.. cfgcmd:: set system flow-accounting netflow source-ip '<address>'
+.. cfgcmd:: set system flow-accounting netflow source-ip <address>
 
    IPv4 or IPv6 source address of NetFlow packets
 
-.. cfgcmd:: set system flow-accounting netflow engine-id '<id>'
+.. cfgcmd:: set system flow-accounting netflow engine-id <id>
 
    NetFlow engine-id which will appear in NetFlow data. The range is 0 to 255.
 
-.. cfgcmd:: set system flow-accounting netflow sampling-rate '<rate>'
+.. cfgcmd:: set system flow-accounting netflow sampling-rate <rate>
 
    Use this command to configure the  sampling rate for flow accounting. The
    system samples one in every `<rate>` packets, where `<rate>` is the value
@@ -80,11 +120,37 @@ exported them to a collection server.
 
    Per default every packet is sampled (that is, the sampling rate is 1).
 
-.. cfgcmd:: set system flow-accounting netflow timeout expiry interval '<interval>'
+.. cfgcmd:: set system flow-accounting netflow timeout expiry interval <interval>
 
    Specifies the interval at which Netflow data will be sent to a collector. As
    per default, Netflow data will be sent every 60 seconds.
 
+   You may also additionally configure timeouts for different types of
+   connections.
+
+.. cfgcmd:: set system flow-accounting netflow max-flows <n>
+
+   If you want to change the maximum number of flows, which are tracking
+   simultaneously, you may do this with this command (default 8192).
+
+sFlow
+^^^^^
+.. cfgcmd:: set system flow-accounting sflow server <address>
+
+   Configure address of sFlow collector. sFlow server at `<address>` can
+   be an IPv4 or IPv6 address. But you cannot export to both IPv4 and
+   IPv6 collectors at the same time!
+
+.. cfgcmd:: set system flow-accounting sflow sampling-rate <rate>
+
+   Enable sampling of packets, which will be transmitted to sFlow collectors.
+
+.. cfgcmd:: set system flow-accounting sflow agent-address <address>
+
+   Configure a sFlow agent address. It can be IPv4 or IPv6 address, but you
+   must set the same protocol, which is used for sFlow collector addresses. By
+   default, using router-id from BGP or OSPF protocol, or the primary IP
+   address from the first interface.
 
 Example:
 --------
@@ -103,44 +169,33 @@ Operation
 Once flow accounting is configured on an interfaces it provides the ability to
 display captured network traffic information for all configured interfaces.
 
-.. opcmd:: show flow-accounting interface '<interface>'
+.. opcmd:: show flow-accounting interface <interface>
 
    Show flow accounting information for given `<interface>`.
 
    .. code-block:: none
 
      vyos@vyos:~$ show flow-accounting interface eth0
-     flow-accounting for [eth0]
-     Src Addr      Dst Addr     Sport Dport Proto  Packets     Bytes  Flows
-     0.0.0.0       192.0.2.50   811   811     udp     7733    591576      0
-     0.0.0.0       192.0.2.50   811   811     udp     7669    586558      1
-     192.0.2.200   192.0.2.51   56188 22      tcp      586     36504      1
-     192.0.2.99    192.0.2.51   61636 161     udp       46      6313      4
-     192.0.2.99    192.0.2.51   61638 161     udp       42      5364      9
-     192.0.2.99    192.0.2.51   61640 161     udp       42      5111      3
-     192.0.2.200   192.0.2.51   54702 22      tcp       86      4432      1
-     192.0.2.99    192.0.2.51   62509 161     udp       24      3540      1
-     192.0.2.99    192.0.2.51   0     0      icmp       49      2989      8
-     192.0.2.99    192.0.2.51   54667 161     udp       18      2658      1
-     192.0.2.99    192.0.2.51   54996 161     udp       18      2622      1
-     192.0.2.99    192.0.2.51   63708 161     udp       18      2622      1
-     192.0.2.99    192.0.2.51   62111 161     udp       18      2622      1
-     192.0.2.99    192.0.2.51   61646 161     udp       16      1977      4
-     192.0.2.99    192.0.2.51   56038 161     udp       10      1256      1
-     192.0.2.99    192.0.2.51   55570 161     udp        6      1146      1
-     192.0.2.99    192.0.2.51   54599 161     udp        6      1134      1
-     192.0.2.99    192.0.2.51   56304 161     udp        8      1029      1
-
-
-.. opcmd:: show flow-accounting interface '<interface>' host '<address>'
+     IN_IFACE    SRC_MAC            DST_MAC            SRC_IP                     DST_IP             SRC_PORT    DST_PORT  PROTOCOL      TOS    PACKETS    FLOWS    BYTES
+     ----------  -----------------  -----------------  ------------------------  ---------------  ----------  ----------  ----------  -----  ---------  -------  -------
+     eth0        00:53:01:a8:28:ac  ff:ff:ff:ff:ff:ff  192.0.2.2                 255.255.255.255        5678        5678  udp             0          1        1      178
+     eth0        00:53:01:b2:2f:34  33:33:ff:00:00:00  fe80::253:01ff:feb2:2f34  ff02::1:ff00:0            0           0  ipv6-icmp       0          2        1      144
+     eth0        00:53:01:1a:b4:53  33:33:ff:00:00:00  fe80::253:01ff:fe1a:b453  ff02::1:ff00:0            0           0  ipv6-icmp       0          1        1       72
+     eth0        00:53:01:b2:22:48  00:53:02:58:a2:92  192.0.2.100               192.0.2.14            40152          22  tcp            16         39        1     2064
+     eth0        00:53:01:c8:33:af  ff:ff:ff:ff:ff:ff  192.0.2.3                 255.255.255.255        5678        5678  udp             0          1        1      154
+     eth0        00:53:01:b2:22:48  00:53:02:58:a2:92  192.0.2.100               192.0.2.14            40006          22  tcp            16        146        1     9444
+     eth0        00:53:01:b2:22:48  00:53:02:58:a2:92  192.0.2.100               192.0.2.14                0           0  icmp          192         27        1     4455
+
+.. opcmd:: show flow-accounting interface <interface> host <address>
 
    Show flow accounting information for given `<interface>` for a specific host
    only.
 
    .. code-block:: none
 
-     vyos@vyos:~$ show flow-accounting interface eth0 host 192.0.2.200
-     flow-accounting for [eth0]
-     Src Addr      Dst Addr     Sport Dport Proto  Packets     Bytes  Flows
-     192.0.2.200   192.0.2.51   56188 22      tcp      586     36504      1
-     192.0.2.200   192.0.2.51   54702 22      tcp       86      4432      1
+     vyos@vyos:~$ show flow-accounting interface eth0 host 192.0.2.14
+     IN_IFACE    SRC_MAC            DST_MAC            SRC_IP       DST_IP        SRC_PORT    DST_PORT  PROTOCOL      TOS    PACKETS    FLOWS    BYTES
+     ----------  -----------------  -----------------  -----------  ----------  ----------  ----------  ----------  -----  ---------  -------  -------
+     eth0        00:53:01:b2:22:48  00:53:02:58:a2:92  192.0.2.100  192.0.2.14       40006          22  tcp            16        197        2    12940
+     eth0        00:53:01:b2:22:48  00:53:02:58:a2:92  192.0.2.100  192.0.2.14       40152          22  tcp            16         94        1     4924
+     eth0        00:53:01:b2:22:48  00:53:02:58:a2:92  192.0.2.100  192.0.2.14           0           0  icmp          192         36        1     5877