| Age | Commit message (Collapse) | Author |
|---|
|
This patch adds support to synchronize expectations between
firewalls. This addition aims to re-use as much as possible
of the existing infrastructure for stability reasons. The
expectation support has been tested with the FTP helper.
This extension requires libnetfilter_conntrack 1.0.0.
If this is the first time you're playing with conntrackd,
I *strongly* recommend you to get working setup of conntrackd
without expectation support before as described in the
documentation. Then, enabling expectation support is rather
easy.
To know more about expectations, if you're not familiar with them,
I suggest you to read:
"Netfilter's Connection Tracking System"
http://people.netfilter.org/pablo/docs/login.pdf
Reprinted from ;login: The Magazine of USENIX, vol. 31, no. 3
(Berkeley, CA: USENIX Association, 2006, pp40-45.)
In short, expectations allow one Linux firewall to filter multi-flow
traffic like FTP, SIP and H.323.
In my testbed, there are two firewalls in a primary-backup configuration
running keepalived. The use a couple of floating cluster IP address
(192.168.0.100 and 192.168.1.100) that are used by the client. These
firewalls protect one FTP server (192.168.1.2) that will be accessed by
one client.
In ASCII art, it looks like this:
192.168.0.100 192.168.1.100
eth1 eth2
fw-1
/ \ FTP
-- client ------ ------ server --
192.168.0.2 \ / 192.168.1.2
fw-2
This is the rule-set for the firewalls:
-A POSTROUTING -t nat -s 192.168.0.2/32 -d 192.168.1.2/32 -j SNAT --to-source 192.168.1.100
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -m state --state INVALID -j DROP
-A FORWARD -m state --state RELATED -j ACCEPT
-A FORWARD -i eth2 -m state --state ESTABLISHED -j ACCEPT
-A FORWARD -i eth1 -p tcp -m tcp --dport 21 --tcp-flags FIN,SYN,RST,ACK SYN -m state --state NEW -j ACCEPT
-A FORWARD -i eth1 -p tcp -m state --state ESTABLISHED -j ACCEPT
-A FORWARD -m state --state INVALID -j LOG --log-prefix "invalid: "
The following steps detail how to check that the expectation support
works fine for conntrackd:
1) You have to enable the expectation support in the configuration
file with the following option:
Sync {
...
Options {
ExpectationSync {
ftp
sip
h323
}
}
}
This enables expectation synchronization for the FTP, SIP and H.323 helpers.
You can alternatively use:
Sync {
...
Options {
ExpectationSync On
}
}
To enable expectation synchronization for all helpers.
2) Make sure you have loaded the FTP helper in both firewalls.
root@fw1# modprobe nf_conntrack_ftp
root@fw2# modprobe nf_conntrack_ftp
3) Switch to the client. Start one FTP control connection to one
server that is protected by the firewalls, enter passive mode:
(term-1) user@client$ nc 192.168.1.2 21
220 dummy FTP server
USER anonymous
331 Please specify the password.
PASS nothing
230 Login successful.
PASV
227 Entering Passive Mode (192,168,1,2,163,11).
This means that port 163*256+11=41739 will be used for the data
traffic. Read this if you are not familiar with the FTP protocol:
http://www.freefire.org/articles/ftpexample.php
3) Switch to fw-1 (primary) to check that the expectation is in the
internal cache.
root@fw1# conntrackd -i exp
proto=6 src=192.168.0.2 dst=192.168.1.2 sport=0 dport=41739 mask-src=255.255.255.255 mask-dst=255.255.255.255 sport=0 dport=65535 master-src=192.168.0.2 master-dst=192.168.1.2 sport=36390 dport=21 [active since 5s]
4) Switch to fw-2 (backup) to check that the expectation has been successfully
replicated.
root@fw2# conntrackd -e exp
proto=6 src=192.168.0.2 dst=192.168.1.2 sport=0 dport=41739 mask-src=255.255.255.255 mask-dst=255.255.255.255 sport=0 dport=65535 master-src=192.168.0.2 master-dst=192.168.1.2 sport=36390 dport=21 [active since 8s]
5) Make the primary firewall fw-1 fail. Now fw-2 becomes primary.
6) Switch to fw-2 (primary) to commit the external cache into the kernel.
root@fw2# conntrackd -c exp
The logs should display that the commit was successful:
root@fw2# tail -100f /var/log/conntrackd.log
[Wed Dec 7 22:16:31 2011] (pid=19195) [notice] committing external cache: expectations
[Wed Dec 7 22:16:31 2011] (pid=19195) [notice] Committed 1 new entries
[Wed Dec 7 22:16:31 2011] (pid=19195) [notice] commit has taken 0.000366 seconds
7) Switch to the client. Open a new terminal and connect to the port that
has been announced by the server:
(term-2) user@client$ nc -vvv 192.168.1.2 41739
(UNKNOWN) [192.168.1.2] 41739 (?) open
8) Switch to term-1 and ask for the file listing:
[...]
227 Entering Passive Mode (192,168,1,2,163,11).
LIST
9) Switch to term-2, it should display the listing. That means
everything has worked fine.
You may want to try disabling the expectation support and
repeating the steps to check that *it does not work* without
the state-synchronization.
You can also display expectation statistics by means of:
root@fwX# conntrackd -s exp
This update requires no changes in the primary-backup.sh script
that is used by the HA manager to interact with conntrackd. Thus,
we provide a backward compatible command line interface.
Regarding the Filter clause and expectations, we use the master
conntrack to filter expectation events. The filtering is performed
in user-space. No kernel-space filtering support for expectations
yet (this support should go in libnetfilter_conntrack at some
point).
This patch also includes support to disable caching and to allow
direct injection of expectations.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
We now include one pointer to the object in the extra section.
This is required to generalize this code for the expectation
support. We consume 4-8 bytes extra, but we will not need more
changes to support expectations which is a good idea.
|
|
This patch simplifies cache_get_extra which now takes only one
parameter that is the cache_object. With it, the extra area can be
calculated.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch generalizes the network message building and parsing
to prepare the upcoming expectation support.
Basically, it renames:
- NET_T_STATE_* by NET_T_STATE_CT_*, as I plan to add NET_T_STATE_EXP_*
- BUILD_NETMSG by BUILD_NETMSG_FROM_CT, and build_payload by ct2msg.
I plan to add exp2msg.
- parse_payload by msg2ct, since I plan to add msg2exp.
- modify object_status_to_network_type to prepare the support of
expectations.
- add prefix ct_ to all parsing functions in parse.c, as we will have
similar functions to convert messages to expectation objects.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch generalizes the caching infrastructure to store different
object types. This patch is the first in the series to prepare
support for the synchronization of expectations.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch improves the case in which we receive a commit request
but we are already performing one. This behaviour is suspicious
since the HA manager should not trigger a double master transition.
Otherwise, something probably is not configured appropriately.
This improves 98756c2608f0879a2322919c7441973216565272
"cache: close commit request if we already have one in progress".
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch reduces the number of gettimeofday syscalls by caching
the current time in a variable at the beginning of the main loop.
Based on a suggestion from Vincent Jardin.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch reworks the commit phase to avoid the forking. This is
particularly useful in active-active setups in which one node
has to commit the external cache while it is receiving new entries
to be added in the external cache. This results in really high
commit times due to the penalty of the copy-on-write that fork
performs.
The default number of steps in one run loop is limited to 64 by now.
No option to tune this parameter is still available via the
configuration file.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch adds cache_iterate_limit() and hashtable_iterate_limit()
that allows to limit the iteration to given a number of states.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch removes a reminiscent of the lifetime cache feature.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch removes the cache write-through clause. This feature
remained undocumented although some has found it looking at the
source code. This feature has remained in the tree for quite
some time although it has several limitations. Moreover, it
is specifically broken and dangerous for Linux kernels >= 2.6.29
since it generates loops in the synchronization.
We do this removal first to prepare the introduction of a feature
to bypass the external cache.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
Since Linux kernel 2.6.29, ctnetlink reports the changes that have
been done using ctnetlink. With this patch, conntrackd can recognize
who is the origin of the event messages. For example, this is
interesting to avoid a messy implicit bulk send during the commit
of entries.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
During the commit phase, the entries in the external cache entries
are inserted in the kernel conntrack table. Currently, we use a
fixed timeout that is specified in the config file. With this patch,
if you don't specify the fixed timeout value via CommitTimeout, the
daemon calculates the real timeout value during the commit phase.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
The lifetime feature is used by all working modes, it is useful to
know how long it has been an entry living in the cache. This patch
moves the lifetime feature to the main caching code.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
With this patch, the `-t' option adds an alarm that will flush the
cache after CONFIG(purge_timeout) seconds specified in the config
file. This looks much cleaner and more performance that looping
on the entire conntrack table to set the new timeout of every single
entry.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
The current commit code updates an entry it still exists in the
kernel. With this patch, we delete the entry and create a new one
to make sure that we start a clean session.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch adds the object counter to `conntrackd -s cache'. This
is useful to detect object leaks in runtime. This patch also changes
the layout of the output to fit the display in less than 24 lines
(assuming 24x80 terminal).
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
With this patch, all the states updates are enqueued in the
tx_queue. Thus, there's a single output path. This patch adds a
simple refcounting mechanism to note when an object is sitting in
the txqueue. This patch also removes the alarm that is required by
the ftfw approach.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch adds the status field to the cache object. This avoids
the (ab)use of the alarm to check if an entry is active or dead.
This is the first step to possibly move the alarm to the cache_extra
memory space of the ftfw (which is the only use by now).
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
Currently, the caching system is implemented in a two layer
architecture: hashtable (inner layer) and cache (upper layer).
This patch reworks the hash-cache infrastructure to solve some
initial design problems to make it more flexible, the main strong
points of this patch are:
* Memory handling is done in the cache layer, not in the inner
hashtable layer. This removes one of the main dependencies between
the hashtable and the cache classes.
* Remove excessive encapsulation: the former cache used to hide a
lot of details of the inner hashtable implementation.
* Fix over-hashing of some operations: lookup-delete-add required
three hash calculations. Similarly, the update-or-add operation
required two hash calculations. Now, we calculate the hash once
and re-use the value how many times as we need.
This patch simplifies the caching system. As a result, we save ~130
lines of code. Small code means and less complexity means less
chance to have bugs.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch adds cache statistics that you can check via
`conntrackd -s cache'. This information is useful for
trouble-shooting.
This patch replaces several log messages that can be triggered in
runtime. The idea behind this patch is to avoid log message flooding
under errors.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch adds the new option `-t' for conntrackd. This option shortens
the value of the timeout for the cached entries that lives in the
kernel. This option is particularly useful to remove the zombie
established entries that remain in kernel if the user tests the platform
by forcing the takeover from one to another node several times.
We currently use the value of CommitTimeout which is sane for it. Adding
a new option does not seem to add more flexibility IMO.
Once we get the patches to notify user changes via ctnetlink and the
netlink flag NLM_F_ECHO works, we may directly invoke a massive purge of
the entries, however, such solution would still need evaluation.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch hides information that may confuse users while they are
diagnosing problems in their setup. For example, we hide entries
that are schedule to expire - from the user side, they are already
destroyed entries; and we show in the counters the real active entries,
not all that are stored in the caches.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch fixes the non-timer-based cache deletion. This bug affects
the alarm-based approach since the backup replicas did not get the
deletion event, thus, delaying the deletion.
This patch introduces cache_find() to look up for a conntrack object
and __cache_del_timer() to perform direct deletions by means of the
pointer obtained with cache_find().
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
This patch introduces the TimeoutDestroy clause to determine how long a conntrack remains in the internal cache once it has been destroy from the kernel table.
|
|
|
|
import only required C headers and put local headers on top to check
|
|
use C99 integers (uint32_t instead of u_int32_t)
|
|
Fix tons of gcc warnings
|
|
use const when possible
|
|
feature is particularly useful for active-active setup without connection persistency, ie. you cannot know which firewall would filter a packet that belongs to a connection.
|
|
- empty debug_ct function if DEBUG_CT is not set
- revisit overrun handler: this is a hard battle, just try to do our best here, call Patrick :)
- explicit warning message when netlink_buffer_max_growth is reached
- fix silly bug in stats-mode when dumping in XML format
- fix UDP handler for conntrack
|
|
chain
|
