From c6a918d9962dcf2354483b709b8bf0fffbbc3983 Mon Sep 17 00:00:00 2001 From: Adam Ierymenko Date: Wed, 28 Oct 2015 12:50:48 -0700 Subject: HTTP test code. --- tests/http/README.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 tests/http/README.md (limited to 'tests/http/README.md') diff --git a/tests/http/README.md b/tests/http/README.md new file mode 100644 index 00000000..ae7f08f1 --- /dev/null +++ b/tests/http/README.md @@ -0,0 +1,5 @@ +HTTP one-to-all test +====== + +This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts. + -- cgit v1.2.3 From cabb8752cb9d9c85dc2aa2cac22a4ad101614577 Mon Sep 17 00:00:00 2001 From: Adam Ierymenko Date: Wed, 28 Oct 2015 15:28:05 -0700 Subject: docs --- tests/http/README.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'tests/http/README.md') diff --git a/tests/http/README.md b/tests/http/README.md index ae7f08f1..7852ac16 100644 --- a/tests/http/README.md +++ b/tests/http/README.md @@ -3,3 +3,6 @@ HTTP one-to-all test This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts. +A Dockerfile is also included which will build a simple Docker image that runs the agent. The image must be launched with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. + +Before using this code you will want to edit agent.js to change SERVER_HOST to the IP address of where you will run server.js. This should typically be an open Internet IP, since this makes reporting not dependent upon the thing being tested. Also note that this thing does no security of any kind. It's designed for one-off tests run over a short period of time, not to be anything that runs permanently. -- cgit v1.2.3 From e3d811b04b7fb04981d65a85d9042e2bd31798b7 Mon Sep 17 00:00:00 2001 From: Adam Ierymenko Date: Wed, 28 Oct 2015 15:55:40 -0700 Subject: docs --- tests/http/README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'tests/http/README.md') diff --git a/tests/http/README.md b/tests/http/README.md index 7852ac16..ab4827ec 100644 --- a/tests/http/README.md +++ b/tests/http/README.md @@ -3,6 +3,8 @@ HTTP one-to-all test This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts. -A Dockerfile is also included which will build a simple Docker image that runs the agent. The image must be launched with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. - Before using this code you will want to edit agent.js to change SERVER_HOST to the IP address of where you will run server.js. This should typically be an open Internet IP, since this makes reporting not dependent upon the thing being tested. Also note that this thing does no security of any kind. It's designed for one-off tests run over a short period of time, not to be anything that runs permanently. + +A Dockerfile is also included which will build a simple Docker image that runs the agent. The image must be launched with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. You can run a bunch with a command like: + + for ((n=0;n<10;n++)); do docker run --device=/dev/net/tun --privileged -d zerotier/http-test; done -- cgit v1.2.3 From 80e62ad29180800fd1669aa68515876f0e8add54 Mon Sep 17 00:00:00 2001 From: Adam Ierymenko Date: Fri, 30 Oct 2015 10:55:05 -0700 Subject: docs --- tests/http/README.md | 4 ++-- tests/http/agent.js | 2 ++ tests/http/server.js | 2 ++ 3 files changed, 6 insertions(+), 2 deletions(-) (limited to 'tests/http/README.md') diff --git a/tests/http/README.md b/tests/http/README.md index ab4827ec..58d4a989 100644 --- a/tests/http/README.md +++ b/tests/http/README.md @@ -3,8 +3,8 @@ HTTP one-to-all test This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts. -Before using this code you will want to edit agent.js to change SERVER_HOST to the IP address of where you will run server.js. This should typically be an open Internet IP, since this makes reporting not dependent upon the thing being tested. Also note that this thing does no security of any kind. It's designed for one-off tests run over a short period of time, not to be anything that runs permanently. +Before using this code you will want to edit agent.js to change SERVER_HOST to the IP address of where you will run server.js. This should typically be an open Internet IP, since this makes reporting not dependent upon the thing being tested. Also note that this thing does no security of any kind. It's designed for one-off tests run over a short period of time, not to be anything that runs permanently. You will also want to edit the Dockerfile if you want to build containers and change the network ID to the network you want to run tests over. -A Dockerfile is also included which will build a simple Docker image that runs the agent. The image must be launched with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. You can run a bunch with a command like: +The Dockerfile builds an image that launches the agent. The image must be "docker run" with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. (Unfortunately CAP_NET_ADMIN may not work due to a bug in Docker and/or Linux.) You can run a bunch with a command like: for ((n=0;n<10;n++)); do docker run --device=/dev/net/tun --privileged -d zerotier/http-test; done diff --git a/tests/http/agent.js b/tests/http/agent.js index 8b1cf512..8a8b785e 100644 --- a/tests/http/agent.js +++ b/tests/http/agent.js @@ -1,3 +1,5 @@ +// ZeroTier distributed HTTP test agent + // --------------------------------------------------------------------------- // Customizable parameters: diff --git a/tests/http/server.js b/tests/http/server.js index 69ccf527..30d8339a 100644 --- a/tests/http/server.js +++ b/tests/http/server.js @@ -1,3 +1,5 @@ +// ZeroTier distributed HTTP test coordinator and result-reporting server + // --------------------------------------------------------------------------- // Customizable parameters: -- cgit v1.2.3 From f808138a942893188537bf82a064932a2523d34d Mon Sep 17 00:00:00 2001 From: Adam Ierymenko Date: Fri, 30 Oct 2015 13:05:34 -0700 Subject: docs and stuff --- tests/http/README.md | 4 +++- tests/http/run-a-big-test.sh | 28 ++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+), 1 deletion(-) create mode 100755 tests/http/run-a-big-test.sh (limited to 'tests/http/README.md') diff --git a/tests/http/README.md b/tests/http/README.md index 58d4a989..23a95605 100644 --- a/tests/http/README.md +++ b/tests/http/README.md @@ -1,10 +1,12 @@ HTTP one-to-all test ====== -This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts. +*This is really internal use code. You're free to test it out but expect to do some editing/tweaking to make it work. We used this to run some massive scale tests of our new geo-cluster-based root server infrastructure prior to taking it live.* Before using this code you will want to edit agent.js to change SERVER_HOST to the IP address of where you will run server.js. This should typically be an open Internet IP, since this makes reporting not dependent upon the thing being tested. Also note that this thing does no security of any kind. It's designed for one-off tests run over a short period of time, not to be anything that runs permanently. You will also want to edit the Dockerfile if you want to build containers and change the network ID to the network you want to run tests over. +This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts. + The Dockerfile builds an image that launches the agent. The image must be "docker run" with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. (Unfortunately CAP_NET_ADMIN may not work due to a bug in Docker and/or Linux.) You can run a bunch with a command like: for ((n=0;n<10;n++)); do docker run --device=/dev/net/tun --privileged -d zerotier/http-test; done diff --git a/tests/http/run-a-big-test.sh b/tests/http/run-a-big-test.sh new file mode 100755 index 00000000..1c125345 --- /dev/null +++ b/tests/http/run-a-big-test.sh @@ -0,0 +1,28 @@ +#!/bin/bash + +# Edit as needed -- note that >1000 per host is likely problematic due to Linux kernel limits +NUM_CONTAINERS=100 +CONTAINER_IMAGE=zerotier/http-test + +# +# This script is designed to be run on Docker hosts to run NUM_CONTAINERS +# +# It can then be run on each Docker host via pssh or similar to run very +# large scale tests. +# + +export PATH=/bin:/usr/bin:/usr/local/bin:/usr/sbin:/sbin + +# Kill and clean up old test containers if any -- note that this kills all containers on the system! +docker ps -q | xargs -n 1 docker kill +docker ps -aq | xargs -n 1 docker rm + +# Pull latest if needed -- change this to your image name and/or where to pull it from +docker pull $CONTAINER_IMAGE + +# Run NUM_CONTAINERS +for ((n=0;n<$NUM_CONTAINERS;n++)); do + docker run --device=/dev/net/tun --privileged -d $CONTAINER_IMAGE +done + +exit 0 -- cgit v1.2.3