diff options
Diffstat (limited to 'netcon')
| -rw-r--r-- | netcon/README.md | 145 | ||||
| -rw-r--r-- | netcon/docker-test/README.md | 33 |
2 files changed, 145 insertions, 33 deletions
diff --git a/netcon/README.md b/netcon/README.md new file mode 100644 index 00000000..b9cca962 --- /dev/null +++ b/netcon/README.md @@ -0,0 +1,145 @@ +ZeroTier Network Containers +====== + +### Functional Overview: + +This system exists as a dynamically-linked library, and a service/IP-stack built into ZeroTier + +If you care about the technicals, + +The intercept is compiled as a shared library and installed in some user-accessible directory. When you want to intercept +a user application you dynamically link the shared library to the application during runtime. When the application starts, the +intercept's global constructor is called which sets up a hidden pipe which is used to communicate remote procedure calls (RPC) to the host Netcon service running in the background. + +When an RPC for a socket() is received by the Netcon service from the intercepted application, the Netcon service will ask the lwIP stack for a new PCB structure (used to represent a connection), if the system permits its allocation, it will be passed to Netcon where a PCB/socket table entry will be created. The table is used for mapping [callbacks from lwIP] and [RPCs from the intercept] to the correct connections. + +Upon the first call to a intercept-overriden system call, a Unix-domain socket is opened between the Netcon service and the application's intercept. This socket provides us the ability to pass file descriptors of newly-created socketpairs to the intercept (used as the read/write buffer). More specifically, after the socketpair creation, one end is kept in a table entry in Netcon and one end is sent to the intercept. + + +### Building from Source (and Installing) + +Build zerotier-intercept library: + + make -f make-intercept.mk + +Install: + + make -f make-intercept.mk install + +Build LWIP library: + + make -f make-liblwip.mk + +Run automated tests (from netcon/docker-test/ directory): + + ./build.sh + ./test.sh + + + + +### Running + +To intercept a specific application (requires an already running instance of Zerotier-One with Network Containers enabled): + + zerotier-intercept my_app + + +### Unit Tests + +To run unit tests: + +1) Set up your own network, use its network id as follows: + +2) Place a blank network config file in this directory (e.g. "e5cd7a9e1c5311ab.conf") + - This will be used to inform test-specific scripts what network to use for testing + +3) run build.sh + - Builds ZeroTier-One with Network Containers enabled + - Builds LWIP library + - Builds intercept library + - Copies all aformentioned files into unit test directory to be used for building docker files + +4) run test.sh + - Will execute each unit test's (test.sh) one at a time and populate _results/ + + +### Anatomy of a unit test + +A) Each unit test's test.sh will: + - temporarily copy all built files into local directory + - build test container + - build monitor container + - remove temporary files + - run each container and perform test and monitoring specified in netcon_entrypoint.sh and monitor_entrypoint.sh + +B) Results will be written to the 'zerotierone/docker-test/_result' directory + - Results will be a combination of raw and formatted dumps to files whose names reflect the test performed + - In the event of failure, 'FAIL.' will be appended to the result file's name + - (e.g. FAIL.my_application_1.0.2.x86_64) + - In the event of success, 'OK.' will be appended + + +### Compatibility + +Network Containers have been tested with the following: + + sshd [ WORKS as of 20151112] + ssh [ WORKS as of 20151112] + sftp [ WORKS as of 20151022] + curl [ WORKS as of 20151021] + apache (debug mode) [ WORKS as of 20150810] + apache (prefork MPM) [ WORKS as of 20151123] (2.4.6-31.x86-64 on Centos 7), (2.4.16-1.x84-64 on F22), (2.4.17-3.x86-64 on F22) + nginx [ WORKS as of 20151123] Broken on Centos 7, unreliable on Fedora 23 + nodejs [ WORKS as of 20151123] + java [ WORKS as of 20151010] + MongoDB [ WORKS as of 20151028] + Redis-server [ WORKS as of 20151123] + +Future: + + GET many different files via HTTP (web stress) + LARGE continuous transfer (e.g. /dev/urandom all night) + Open and close many TCP connections constantly + Simulate packet loss (can be done with iptables) + Many parallel TCP transfers + Multithreaded software (e.g. apache in thread mode) + UDP support + + + +### Extended Version Notes + +20151028 Added MongoDB support: + + - Added logic (RPC_MAP_REQ) to check whether a given AF_LOCAL socket is mapped to anything + inside the service instance. + + +20151027 Added Redis-server support: + + - Added extra logic to detect socket re-issuing and consequent service-side double mapping. + Redis appears to try to set its initial listen socket to IPV6 only, this currently fails. As + a result, Redis will close the socket and re-open it. The server will now test for closures + during mapping and will eliminate any mappings to broken pipes. + + +20151021 Added Node.js support: + + - syscall(long number, ...) is now intercepted and re-directs the __NR_accept4 call to our intercepted accept4() function + + - accept() now returns -EAGAIN in the case that we cannot read a signal byte from the descriptor linked to the service. This + is because the uv__server_io() function in libuv used by Node.js looks for this return value upon failure, without it we + were observing an innfinite loop in the I/O polling code in libuv. + + - accept4() now correctly sets given flags for descriptor returned by accept() + + - setsockopt() was modified to return success on any call with the following conditions: + level == IPPROTO_TCP || (level == SOL_SOCKET && option_name == SO_KEEPALIVE) + This might be unnecessary or might need a better workaround + + - Careful attention should be given to how arguments are passed in the intercepted syscall() function, this differs for + 32/64-bit systems + + + diff --git a/netcon/docker-test/README.md b/netcon/docker-test/README.md deleted file mode 100644 index a3b21e1d..00000000 --- a/netcon/docker-test/README.md +++ /dev/null @@ -1,33 +0,0 @@ -### Unit Tests - -To run unit tests: - -1) Set up your own network, use its network id as follows: - -2) Place a blank network config file in this directory (e.g. "e5cd7a9e1c5311ab.conf") - - This will be used to inform test-specific scripts what network to use for testing - -3) run build.sh - - Builds ZeroTier-One with Network Containers enabled - - Builds LWIP library - - Builds intercept library - - Copies all aformentioned files into unit test directory to be used for building docker files - -4) run test.sh - - Will execute each unit test's (test.sh) one at a time and populate _results/ - - -### Anatomy of a unit test - -A) Each unit test's test.sh will: - - temporarily copy all built files into local directory - - build test container - - build monitor container - - remove temporary files - - run each container and perform test and monitoring specified in netcon_entrypoint.sh and monitor_entrypoint.sh - -B) Results will be written to the 'zerotierone/docker-test/_result' directory - - Results will be a combination of raw and formatted dumps to files whose names reflect the test performed - - In the event of failure, 'FAIL.' will be appended to the result file's name - - (e.g. FAIL.my_application_1.0.2.x86_64) - - In the event of success, 'OK.' will be appended |